Modeling and validating Bayesian accrual models on clinical data and simulations using adaptive priors

The time-dependent Poisson-gamma model in practice: Recruitment forecasting in HIV trials

Despite a growing body of literature in the area of recruitment modeling for multicenter studies, in practice, statistical models to predict enrollments are rarely used and when they are, they often rely on unrealistic assumptions. The time-dependent Poisson-Gamma model (tPG) is a recently developed flexible methodology which allows analysts to predict recruitments in an ongoing multicenter trial, and its performance has been validated on data from a cohort study. In this article, we illustrate and further validate the tPG model on recruitment data from randomized controlled trials. Additionally, in the appendix, we provide a practical and easy to follow guide to its implementation via the tPG R package. To validate the model, we show the predictive performance of the proposed methodology in forecasting the recruitment process of two HIV vaccine trials conducted by the HIV Vaccine Trials Network in multiple Sub-Saharan countries.

A non-parametric approach to predict the recruitment for randomized clinical trials: an example in elderly inpatient settings

BACKGROUND

Accurate prediction of subject recruitment, which is critical to the success of a study, remains an ongoing challenge. Previous prediction models often rely on parametric assumptions which are not always met or may be difficult to implement. We aim to develop a novel method that is less sensitive to model assumptions and relatively easy to implement.

METHODS

We create a weighted resampling-based approach to predict enrollment in year two based on recruitment data from year one of the completed GRIPS and PACE clinical trials. Different weight functions accounted for a range of potential enrollment trajectory patterns. Prediction accuracy was measured by Euclidean distance for enrollment sequence in year two, total enrollment over time, and total weeks to enroll a fixed number of subjects, against the actual year two enrollment data. We compare the performance of the proposed method with an existing Bayesian method.

RESULTS

Weighted resampling using GRIPS data resulted in closer prediction evidenced by better coverage of observed enrollment with the prediction intervals and smaller Euclidean distance from actual enrollment in year 2, especially when enrollment gaps were filled prior to the weighted resampling. These scenarios also produced more accurate predictions for total enrollment and number of weeks to enroll 50 participants. These same scenarios outperformed an existing Bayesian method for all 3 accuracy measures. In PACE data, using a reduced year 1 enrollment resulted in closer prediction evidenced by better coverage of observed enrollment with the prediction intervals and smaller Euclidean distance from actual enrollment in year 2, with the weighted resampling scenarios better reflecting the seasonal variation seen in year (1) The reduced enrollment scenarios resulted in closer prediction for total enrollment over 6 and 12 months into year (2) These same scenarios also outperformed an existing Bayesian method for relevant accuracy measures.

CONCLUSION

The results demonstrate the feasibility and flexibility for a resampling-based, non-parametric approach for prediction of clinical trial recruitment with limited early enrollment data. Application to a wider setting and long-term prediction accuracy require further investigation.

Using Bayesian hierarchical modeling for performance evaluation of clinical trial accrual for a cancer center

Introduction

Slow patient accrual in cancer clinical trials is always a concern. In 2021, the University of Kansas Comprehensive Cancer Center (KUCC), an NCI-designated comprehensive cancer center, implemented the Curated Cancer Clinical Outcomes Database (C3OD) to perform trial feasibility analyses using real-time electronic medical record data. In this study, we proposed a Bayesian hierarchical model to evaluate annual cancer clinical trial accrual performance.

Methods

The Bayesian hierarchical model uses Poisson models to describe the accrual performance of individual cancer clinical trials and a hierarchical component to describe the variation in performance across studies. Additionally, this model evaluates the impacts of the C3OD and the COVID-19 pandemic using posterior probabilities across evaluation years. The performance metric is the ratio of the observed accrual rate to the target accrual rate.

Results

Posterior medians of the annual accrual performance at the KUCC from 2018 to 2023 are 0.233, 0.246, 0.197, 0.150, 0.254, and 0.340. The COVID-19 pandemic partly explains the drop in performance in 2020 and 2021. The posterior probability that annual accrual performance is better with C3OD in 2023 than pre-pandemic (2019) is 0.935.

Conclusions

This study comprehensively evaluates the annual performance of clinical trial accrual at the KUCC, revealing a negative impact of COVID-19 and an ongoing positive impact of C3OD implementation. Two sensitivity analyses further validate the robustness of our model. Evaluating annual accrual performance across clinical trials is essential for a cancer center. The performance evaluation tools described in this paper are highly recommended for monitoring clinical trial accrual.

A time-dependent Poisson-Gamma model for recruitment forecasting in multicenter studies

Forecasting recruitments is a key component of the monitoring phase of multicenter studies. One of the most popular techniques in this field is the Poisson-Gamma recruitment model, a Bayesian technique built on a doubly stochastic Poisson process. This approach is based on the modeling of enrollments as a Poisson process where the recruitment rates are assumed to be constant over time and to follow a common Gamma prior distribution. However, the constant-rate assumption is a restrictive limitation that is rarely appropriate for applications in real studies. In this paper, we illustrate a flexible generalization of this methodology which allows the enrollment rates to vary over time by modeling them through B-splines. We show the suitability of this approach for a wide range of recruitment behaviors in a simulation study and by estimating the recruitment progression of the Canadian Co-infection Cohort.

A simple and robust model for enrollment projection in clinical trials

Enrollment projection in clinical trials is a topic gaining attention in the statistics literature in recent years. A number of methods have been proposed in this area. Some approaches are sophisticated but complicated to implement. We aim to implement a simple and robust empiric Bayes Poisson Gamma model (PGM) that is suitable for practical use. We assume a constant and site-specific underlying enrollment rate over time, which comes from a common Gamma distribution. Choice of prior parameters is data driven. We tested the proposed PGM in a simulation study as well as a number of oncology trials with various enrollment patterns, which yield satisfactory results. Compared to a flexible nonparametric model (Zhang and Long, 2010), the PGM is associated with a narrower credible interval as a result of parametric assumptions. However, the model prediction may be off when the assumptions are substantially violated.

Interim recruitment prediction for multi-center clinical trials

We introduce a general framework for monitoring, modeling, and predicting the recruitment to multi-center clinical trials. The work is motivated by overly optimistic and narrow prediction intervals produced by existing time-homogeneous recruitment models for multi-center recruitment. We first present two tests for detection of decay in recruitment rates, together with a power study. We then introduce a model based on the inhomogeneous Poisson process with monotonically decaying intensity, motivated by recruitment trends observed in oncology trials. The general form of the model permits adaptation to any parametric curve-shape. A general method for constructing sensible parameter priors is provided and Bayesian model averaging is used for making predictions which account for the uncertainty in both the parameters and the model. The validity of the method and its robustness to misspecification are tested using simulated datasets. The new methodology is then applied to oncology trial data, where we make interim accrual predictions, comparing them to those obtained by existing methods, and indicate where unexpected changes in the accrual pattern occur.

Prediction of RECRUITment In randomized clinical Trials (RECRUIT-IT)-rationale and design for an international collaborative study

BACKGROUND

Poor recruitment of patients is the predominant reason for early termination of randomized clinical trials (RCTs). Systematic empirical investigations and validation studies of existing recruitment models, however, are lacking. We aim to provide evidence-based guidance on how to predict and monitor recruitment of patients into RCTs. Our specific objectives are the following: (1) to establish a large sample of RCTs (target n = 300) with individual patient recruitment data from a large variety of RCTs, (2) to investigate participant recruitment patterns and study site recruitment patterns and their association with the overall recruitment process, (3) to investigate the validity of a freely available recruitment model, and (4) to develop a user-friendly tool to assist trial investigators in the planning and monitoring of the recruitment process.

METHODS

Eligible RCTs need to have completed the recruitment process, used a parallel group design, and investigated any healthcare intervention where participants had the free choice to participate. To establish the planned sample of RCTs, we will use our contacts to national and international RCT networks, clinical trial units, and individual trial investigators. From included RCTs, we will collect patient-level information (date of randomization), site-level information (date of trial site activation), and trial-level information (target sample size). We will examine recruitment patterns using recruitment trajectories and stratifications by RCT characteristics. We will investigate associations of early recruitment patterns with overall recruitment by correlation and multivariable regression. To examine the validity of a freely available Bayesian prediction model, we will compare model predictions to collected empirical data of included RCTs. Finally, we will user-test any promising tool using qualitative methods for further tool improvement.

DISCUSSION

This research will contribute to a better understanding of participant recruitment to RCTs, which could enhance efficiency and reduce the waste of resources in clinical research with a comprehensive, concerted, international effort.

Improving the efficiency of clinical trials by standardizing processes for Investigator Initiated Trials

Early phase clinical trials are the first step in testing new medications and therapeutics developed by clinical and biomedical investigators. These trials aim to find a safe dose of a newly targeted drug (phase I) or find out more about the side effects and early signals of treatment efficacy (phase II). In a research institute, many biomedical investigators in oncology are encouraged to initiate such trials early in their careers as part of developing their research portfolio. These investigator-initiated trials (IITs) are funded internally by the University of Kansas Cancer Center or partially funded by pharmaceutical companies. As financial, administrative, and practical considerations play an essential role in the successful completion of IITs, it is imperative to efficiently allocate resources to plan, design, and execute these studies within the allotted time. This manuscript describes monitoring tools and processes to improve the efficiency, cost-effectivness, and reliability of IITs. The contributions of this team to processes such as: participant recruitment, feasibility analysis, clinical trial design, accrual monitoring, data management, interim analysis support, and final analysis and reporting are described in detail. This manuscript elucidates how, through the aid of technology and dedicated personnel support, the efficiency of IIT-related processes can be improved. Early results of these initiatives look promising, and the Biostatistics and Informatics team intends to continue fostering innovative methodologies to enhance cancer research by improving the efficiency of IITs.

Bayesian accrual modeling and prediction in multicenter clinical trials with varying center activation times

Investigators who manage multicenter clinical trials need to pay careful attention to patterns of subject accrual, and the prediction of activation time for pending centers is potentially crucial for subject accrual prediction. We propose a Bayesian hierarchical model to predict subject accrual for multicenter clinical trials in which center activation times vary. We define center activation time as the time at which a center can begin enrolling patients in the trial. The difference in activation times between centers is assumed to follow an exponential distribution, and the model of subject accrual integrates prior information for the study with actual enrollment progress. We apply our proposed Bayesian multicenter accrual model to two multicenter clinical studies. The first is the PAIN-CONTRoLS study, a multicenter clinical trial with a goal of activating 40 centers and enrolling 400 patients within 104 weeks. The second is the HOBIT trial, a multicenter clinical trial with a goal of activating 14 centers and enrolling 200 subjects within 36 months. In summary, the Bayesian multicenter accrual model provides a prediction of subject accrual while accounting for both center- and individual patient-level variation.

Accrual Prediction Program: A web-based clinical trials tool for monitoring and predicting accrual for early-phase cancer studies

BACKGROUND

Monitoring subject recruitment is key to the success of a clinical trial. Accordingly, researchers have developed accrual-monitoring tools to support the design and conduct of trials. At an institutional level, delays in identifying studies with high risk of accrual failure can lead to inefficient and costly trials with little chances of meeting study objectives. Comprehensive accrual monitoring is necessary to the success of the research enterprise.

METHODS

This article describes the design and implementation of the University of Kansas Cancer Center Accrual Prediction Program, a web-based platform was developed to support comprehensive accrual monitoring and prediction for all active clinical trials. The Accrual Prediction Program provides information on accrual, including the predicted completion date, predicted number of accrued subjects during the pre-specified accrual period, and the probability of achieving accrual targets. It relies on a Bayesian accrual prediction model to combine protocol information with real-time trial enrollment data and disseminates results via web application.

RESULTS

First released in 2016, the Accrual Prediction Program summarizes enrollment information for active studies categorized by various trial attributes. The web application supports real-time evidence-based decision making for strategic resource allocation and study management of over 120 ongoing clinical trials at the University of Kansas Cancer Center.

CONCLUSION

The Accrual Prediction Program makes accessing comprehensive accrual information manageable at an institutional level. Cancer centers or even entire institutions can reproduce the Accrual Prediction Program to achieve real-time comprehensive monitoring and prediction of subject accrual to aid investigators and administrators in the design, conduct, and management of clinical trials.

Optimizing drug development in oncology by clinical trial simulation: Why and how?

In therapeutic research, the safety and efficacy of pharmaceutical products are necessarily tested on humans via clinical trials after an extensive and expensive preclinical development period. Methodologies such as computer modeling and clinical trial simulation (CTS) might represent a valuable option to reduce animal and human assays. The relevance of these methods is well recognized in pharmacokinetics and pharmacodynamics from the preclinical phase to postmarketing. However, they are barely used and are poorly regarded for drug approval, despite Food and Drug Administration and European Medicines Agency recommendations. The generalization of CTS could be greatly facilitated by the availability of software for modeling biological systems, by clinical trial studies and hospital databases. Data sharing and data merging raise legal, policy and technical issues that will need to be addressed. Development of future molecules will have to use CTS for faster development and thus enable better patient management. Drug activity modeling coupled with disease modeling, optimal use of medical data and increased computing speed should allow this leap forward. The realization of CTS requires not only bioinformatics tools to allow interconnection and global integration of all clinical data but also a universal legal framework to protect the privacy of every patient. While recognizing that CTS can never replace 'real-life' trials, they should be implemented in future drug development schemes to provide quantitative support for decision-making. This in silico medicine opens the way to the P4 medicine: predictive, preventive, personalized and participatory.

Accrual monitoring in cardiovascular trials

Objective

To provide brief guidance on how to design accrual monitoring activities in a clinical trial protocol.

Setting

Two completed clinical trials that did not achieve the planned sample size, the Cost of Strategies After Myocardial Infarction (COSTAMI) trial and the Biventricular Pacing After Cardiac Surgery (BiPACS) trial.

Design

A Bayesian monitoring tool, the constant accrual model, is applied retrospectively to accrual data from each case study to illustrate how the tool could be used to identify problems with accrual early in the trial period and to frame the conditions in which the approach can be used in practice.

Results

After 312 days and 155 patients enrolled in the COSTAMI trial, accrual could be classified as ‘off target’ on the basis of statistical criteria outlined in the protocol. As for the BiPACS trial, after 2 years, it was already evident that the accrual was ‘considerably off target’.

Conclusions

Prompt awareness of a high risk of accrual failure could trigger different interventions to overcome protocol-related, patient-related or investigator-related barriers to recruitment or ultimately contribute to an early stopping decision due to recruitment futility.

Accrual prediction models should be included as standard tools for routine monitoring activities in cardiovascular research. Among them, methods relying on the Bayesian approach are particularly attractive, as they can naturally update past evidence when actual accrual data becomes available.

Using Poisson-gamma model to evaluate the duration of recruitment process when historical trials are available

At the design of clinical trial operation, a question of a paramount interest is how long it takes to recruit a given number of patients. Modelling the recruitment dynamics is the necessary step to answer this question. Poisson-gamma model provides very convenient, flexible and realistic approach. This model allows predicting the trial duration using data collected at an interim time with very good accuracy. A natural question arises: how to evaluate the parameters of recruitment model before the trial begins? The question is harder to handle as there are no recruitment data available for this trial. However, if there exist similar completed trials, it is appealing to use data from these trials to investigate feasibility of the recruitment process. In this paper, the authors explore the recruitment data of two similar clinical trials (Intergroupe Francais du Myélome 2005 and 2009). It is shown that the natural idea of plugging the historical rates estimated from the completed trial in the same centres of the new trial for predicting recruitment is not a relevant strategy. In contrast, using the parameters of a gamma distribution of the rates estimated from the completed trial in the recruitment dynamic model of the new trial provides reasonable predictive properties with relevant confidence intervals. Copyright © 2017 John Wiley & Sons, Ltd.

A framework for prospectively defining progression rules for internal pilot studies monitoring recruitment

Just over half of publicly funded trials recruit their target sample size within the planned study duration. When recruitment targets are missed, the funder of a trial is faced with the decision of either committing further resources to the study or risk that a worthwhile treatment effect may be missed by an underpowered final analysis. To avoid this challenging situation, when there is insufficient prior evidence to support predicted recruitment rates, funders now require feasibility assessments to be performed in the early stages of trials. Progression criteria are usually specified and agreed with the funder ahead of time. To date, however, the progression rules used are typically ad hoc. In addition, rules routinely permit adaptations to recruitment strategies but do not stipulate criteria for evaluating their effectiveness. In this paper, we develop a framework for planning and designing internal pilot studies which permit a trial to be stopped early if recruitment is disappointing or to continue to full recruitment if enrolment during the feasibility phase is adequate. This framework enables a progression rule to be pre-specified and agreed upon prior to starting a trial. The novel two-stage designs stipulate that if neither of these situations arises, adaptations to recruitment should be made and subsequently evaluated to establish whether they have been successful. We derive optimal progression rules for internal pilot studies which minimise the expected trial overrun and maintain a high probability of completing the study when the recruitment rate is adequate. The advantages of this procedure are illustrated using a real trial example.

Bayesian accrual prediction for interim review of clinical studies: open source R package and smartphone application

Background

Subject recruitment for medical research is challenging. Slow patient accrual leads to increased costs and delays in treatment advances. Researchers need reliable tools to manage and predict the accrual rate. The previously developed Bayesian method integrates researchers’ experience on former trials and data from an ongoing study, providing a reliable prediction of accrual rate for clinical studies.

Methods

In this paper, we present a user-friendly graphical user interface program developed in R. A closed-form solution for the total subjects that can be recruited within a fixed time is derived. We also present a built-in Android system using Java for web browsers and mobile devices.

Results

Using the accrual software, we re-evaluated the Veteran Affairs Cooperative Studies Program 558— ROBOTICS study. The application of the software in monitoring and management of recruitment is illustrated for different stages of the trial.

Conclusions

This developed accrual software provides a more convenient platform for estimation and prediction of the accrual process.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1457-3) contains supplementary material, which is available to authorized users.

A novel evaluation of optimality for randomized controlled trials

Balanced two-arm designs are more powerful than unbalanced designs and, consequently, Bayesian adaptive designs (BADs) are less powerful. However, when considering other subject- or community-focused design characteristics, fixed two-arm designs can be suboptimal. We use a novel approach to identify the best two-arm study design, taking into consideration both the statistical perspective and the community's perception. Data envelopment analysis (DEA) was used to estimate the relative performance of competing designs in the presence of multiple optimality criteria. The two-arm fixed design has enough deficiencies in subject- and community-specific benefit to make it the least favorable study design.