Statistical Modeling of Clinical Trials (Recruitment and Randomization)

The time-dependent Poisson-gamma model in practice: Recruitment forecasting in HIV trials

Despite a growing body of literature in the area of recruitment modeling for multicenter studies, in practice, statistical models to predict enrollments are rarely used and when they are, they often rely on unrealistic assumptions. The time-dependent Poisson-Gamma model (tPG) is a recently developed flexible methodology which allows analysts to predict recruitments in an ongoing multicenter trial, and its performance has been validated on data from a cohort study. In this article, we illustrate and further validate the tPG model on recruitment data from randomized controlled trials. Additionally, in the appendix, we provide a practical and easy to follow guide to its implementation via the tPG R package. To validate the model, we show the predictive performance of the proposed methodology in forecasting the recruitment process of two HIV vaccine trials conducted by the HIV Vaccine Trials Network in multiple Sub-Saharan countries.

Freely accessible software for recruitment prediction and recruitment monitoring of clinical trials: A systematic review

Background

The successful completion of clinical trials ultimately depends on realistic recruitment predictions. Statistical methods for recruitment prediction implemented in a free-of-charge open-source software could be routinely used by researchers worldwide to design clinical trials. However, the availability of such software implementations is currently unclear.

Methods

Two independent reviewers conducted a systematic review following PRISMA guidelines. Eligible articles included English publications focused on statistical methods for recruitment prediction and monitoring that referred to software implementations. The list of articles retrieved from well-established data bases was enriched by backtracking of references provided by eligible articles. The current software availability and open-source status were tabulated.

Results

We found 21 eligible articles, 7 of which (33 %) provide freely accessible software. Ultimately, only one article provides a link to an easy-to-comprehend, well-documented, and currently directly applicable free-of-charge open-source software. The lack of availability is mainly caused by blocked access and outdated links.

Conclusions

While several software implementations exist for recruitment prediction, only a small fraction is freely accessible. These results highlight the need for future efforts to achieve free access to well-documented software implementations supporting researchers in routinely using statistical methods to arrive at realistic recruitment predictions in clinical trials.

Enrollment forecast for clinical trials at the portfolio planning phase based on site-level historical data

An accurate forecast of a clinical trial enrollment timeline at the planning phase is of great importance to both corporate strategic planning and trial operational excellence. The naive approach often calculates an average enrollment rate from historical data and generates an inaccurate prediction based on a linear trend with the average rate. Under the traditional framework of a Poisson-Gamma model, site activation delays are often modeled with either fixed initiation time or a simple random distribution while incorporating the user-provided site planning information to achieve good forecast accuracy. However, such user-provided information is not available at the early portfolio planning stage. We present a novel statistical approach based on generalized linear mixed-effects models and the use of non-homogeneous Poisson processes through the Bayesian framework to model the country initiation, site activation, and subject enrollment sequentially in a systematic fashion. We validate the performance of our proposed enrollment modeling framework based on a set of 25 preselected studies from four therapeutic areas. Our modeling framework shows a substantial improvement in prediction accuracy in comparison to the traditional statistical approach. Furthermore, we show that our modeling and simulation approach calibrates the data variability appropriately and gives correct coverage rates for prediction intervals of various nominal levels. Finally, we demonstrate the use of our approach to generate the predicted enrollment curves through time with confidence bands overlaid.

Selecting a randomization method for a multi-center clinical trial with stochastic recruitment considerations

BACKGROUND

The design of a multi-center randomized controlled trial (RCT) involves multiple considerations, such as the choice of the sample size, the number of centers and their geographic location, the strategy for recruitment of study participants, amongst others. There are plenty of methods to sequentially randomize patients in a multi-center RCT, with or without considering stratification factors. The goal of this paper is to perform a systematic assessment of such randomization methods for a multi-center 1:1 RCT assuming a competitive policy for the patient recruitment process.

METHODS

We considered a Poisson-gamma model for the patient recruitment process with a uniform distribution of center activation times. We investigated 16 randomization methods (4 unstratified, 4 region-stratified, 4 center-stratified, 3 dynamic balancing randomization (DBR), and a complete randomization design) to sequentially randomize n = 500 patients. Statistical properties of the recruitment process and the randomization procedures were assessed using Monte Carlo simulations. The operating characteristics included time to complete recruitment, number of centers that recruited a given number of patients, several measures of treatment imbalance and estimation efficiency under a linear model for the response, the expected proportions of correct guesses under two different guessing strategies, and the expected proportion of deterministic assignments in the allocation sequence.

RESULTS

Maximum tolerated imbalance (MTI) randomization methods such as big stick design, Ehrenfest urn design, and block urn design result in a better balance-randomness tradeoff than the conventional permuted block design (PBD) with or without stratification. Unstratified randomization, region-stratified randomization, and center-stratified randomization provide control of imbalance at a chosen level (trial, region, or center) but may fail to achieve balance at the other two levels. By contrast, DBR does a very good job controlling imbalance at all 3 levels while maintaining the randomized nature of treatment allocation. Adding more centers into the study helps accelerate the recruitment process but at the expense of increasing the number of centers that recruit very few (or no) patients-which may increase center-level imbalances for center-stratified and DBR procedures. Increasing the block size or the MTI threshold(s) may help obtain designs with improved randomness-balance tradeoff.

CONCLUSIONS

The choice of a randomization method is an important component of planning a multi-center RCT. Dynamic balancing randomization with carefully chosen MTI thresholds could be a very good strategy for trials with the competitive policy for patient recruitment.

Enrollment Forecast for Clinical Trials at the Planning Phase with Study-Level Historical Data

Given progressive developments and demands on clinical trials, accurate enrollment timeline forecasting is increasingly crucial for both strategic decision-making and trial execution excellence. Naïve approach assumes flat rates on enrollment using average of historical data, while traditional statistical approach applies simple Poisson-Gamma model using time-invariant rates for site activation and subject recruitment. Both of them are lack of non-trivial factors such as time and location. We propose a novel two-segment statistical approach based on Quasi-Poisson regression for subject accrual rate and Poisson process for subject enrollment and site activation. The input study-level data are publicly accessible and it can be integrated with historical study data from user's organization to prospectively predict enrollment timeline. The new framework is neat and accurate compared to preceding works. We validate the performance of our proposed enrollment model and compare the results with other frameworks on 7 curated studies.

Designing a phase-III time-to-event clinical trial using a modified sample size formula and Poisson-Gamma model for subject accrual that accounts for the lag in site initiation using the PERT distribution

A priori estimation of sample size and subject accrual in multi-site, time-to-event clinical trials is often challenging. Such trials are powered based on the number of events needed to detect a clinically significant difference. Sample size based on number of events relates to the expected duration of observation time for each subject. Temporal patterns in site initiation and subject enrollment ultimately affect when subjects can be accrued into the study. Lag times are common as the site start-up process optimizes, resulting in delays that may curtail observational follow-up and therefore undermine power. The proposed method introduces a Program Evaluation and Review Technique (PERT) model into the sample size estimation which accounts for the lag in site start-up. Additionally, a PERT model is introduced into a Poisson-Gamma subject accrual model to predict the quantity of study sites needed. The introduction of the PERT model provides greater flexibility in both a priori power assessment and planning the number of sites, as it specifically allows for the inclusion of anticipated delays in site start-up time. This model results in minimal power loss even when PERT distribution inputs are misspecified compared to the traditional assumption of simultaneous start-up for all sites. Together these updated formulations for sample size and subject accrual models offer an improved method for designing a multi-site time-to-event clinical trial that accounts for a flexible site start-up process.

A time-dependent Poisson-Gamma model for recruitment forecasting in multicenter studies

Forecasting recruitments is a key component of the monitoring phase of multicenter studies. One of the most popular techniques in this field is the Poisson-Gamma recruitment model, a Bayesian technique built on a doubly stochastic Poisson process. This approach is based on the modeling of enrollments as a Poisson process where the recruitment rates are assumed to be constant over time and to follow a common Gamma prior distribution. However, the constant-rate assumption is a restrictive limitation that is rarely appropriate for applications in real studies. In this paper, we illustrate a flexible generalization of this methodology which allows the enrollment rates to vary over time by modeling them through B-splines. We show the suitability of this approach for a wide range of recruitment behaviors in a simulation study and by estimating the recruitment progression of the Canadian Co-infection Cohort.

Real time monitoring and prediction of time to endpoint maturation in clinical trials

In clinical trials, almost all key milestone dates can be defined in terms of time to endpoint maturation (TTEM). The real time monitoring and accurate prediction of TTEM have a significant impact on clinical trial planning and execution and can bring significant value to clinical trial practitioners. TTEM is defined as the time to achieve or observe a certain number or percentage of some endpoint of interest. It is a combination of time to site initiation, time to subject enrollment after site initiation and time to event of interest after subject enrollment. To better predict TTEM during the trial, the future site initiation and subject enrollment have to be taken into account while predicting the number of events. In this article, we propose a novel simulation-based framework combining time to site initiation, time to subject enrollment and time to event in order to predict TTEM. A nonhomogeneous Poisson process with a quadratic time-varying rate function is used to model site initiation and subject enrollment and more advanced time to event models had been explored and integrated on top of them, such as Weibull, piecewise exponential, and model averaging which is equivalent to a Bayesian model selection strategy. To evaluate the predictive performance of the proposed methodology, we conducted extensive simulations and applied the methodology to 14 randomly selected real oncology phase 2 and phase 3 studies in both solid tumor and hematology with a total 31 study-endpoint combinations. The predictive performance of the proposed methodology was then compared with popular and commonly available commercial software, for example, East (Cytel, Cambridge, MA, USA). From both simulation and real data, the proposed methodology can significantly improve the prediction accuracy by up to 54% compared to the commonly available method.

Prediction of clinical trial enrollment rates

Clinical trials represent a critical milestone of translational and clinical sciences. However, poor recruitment to clinical trials has been a long standing problem affecting institutions all over the world. One way to reduce the cost incurred by insufficient enrollment is to minimize initiating trials that are most likely to fall short of their enrollment goal. Hence, the ability to predict which proposed trials will meet enrollment goals prior to the start of the trial is highly beneficial. In the current study, we leveraged a data set extracted from ClinicalTrials.gov that consists of 46,724 U.S. based clinical trials from 1990 to 2020. We constructed 4,636 candidate predictors based on data collected by ClinicalTrials.gov and external sources for enrollment rate prediction using various state-of-the-art machine learning methods. Taking advantage of a nested time series cross-validation design, our models resulted in good predictive performance that is generalizable to future data and stable over time. Moreover, information content analysis revealed the study design related features to be the most informative feature type regarding enrollment. Compared to the performance of models built with all features, the performance of models built with study design related features is only marginally worse (AUC = 0.78 ± 0.03 vs. AUC = 0.76 ± 0.02). The results presented can form the basis for data-driven decision support systems to assess whether proposed clinical trials would likely meet their enrollment goal.

A Bayesian approach for event predictions in clinical trials with time-to-event outcomes

In clinical trials with time-to-event outcome as the primary endpoint, the end of study date is often based on the number of observed events, which drives the statistical power and the sample size calculation. It is of great value for study sponsors to have a good understanding of the recruitment process and the event milestones to manage the logistical tasks, which require a considerable amount of resources. The objective of the proposed statistical approach is to predict, as accurately as possible, the timing of an analysis planned once a target number of events is collected. The method takes into account the enrollment, the time to event, and the time to censor processes, using Weibull models in a Bayesian framework. We also consider a possible delay in the event reporting by the investigators, and covariates may also be included. Several metrics can be obtained, such as the probability of study completion at specific timepoints or the credible interval of the date of study completion. The approach was applied to oncology trials, with progression-free survival as primary outcome. A retrospective analysis shows the accuracy of the approach on these examples, as well as the benefit of updating the predictive probability of study completion as data are accumulating or new information becomes available. We also evaluated the performances of the proposed method in a comprehensive simulation study.

Concept and development of an interactive tool for trial recruitment planning and management

Background

Predicting and monitoring recruitment in large, complex trials is essential to ensure appropriate resource management and budgeting. In a novel partnership between clinical trial investigators of the South African Medical Research Council and industrial engineers from the Stellenbosch University Health Systems Engineering and Innovation Hub, we developed a trial recruitment tool (TRT). The objective of the tool is to serve as a computerised decisions-support system to aid the planning and management phases of the trial recruitment process.

Method

The specific requirements of the TRT were determined in several workshops between the partners. A Poisson process simulation model was formulated and incorporated in the TRT to predict the recruitment duration. The assumptions underlying the model were made in consultation with the trial team at the start of the project and were deemed reasonable. Real-world data extracted from a current cluster trial, Project MIND, based in 24 sites in South Africa was used to verify the simulation model and to develop the monitoring component of the TRT.

Results

The TRT comprises a planning and monitoring component. The planning component generates different trial scenarios for predicted trial recruitment duration based on user inputs, e.g. number of sites, initiation delays. The monitoring component uses and analyses the data retrieved from the trial management information system to generate different levels of information, displayed visually on an interactive, user-friendly dashboard. Users can analyse the results at trial or site level, changing input parameters to see the resultant effect on the duration of trial recruitment.

Conclusion

This TRT is an easy-to-use tool that assists in the management of the trial recruitment process. The TRT has potential to expedite improved management of clinical trials by providing the appropriate information needed for the planning and monitoring of the trial recruitment phase. This TRT extends prior tools describing historic recruitment only to using historic data to predict future recruitment. The broader project demonstrates the value of collaboration between clinicians and engineers to optimise their respective skillsets.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05112-z.

Recruitment prediction for multicenter clinical trials based on a hierarchical Poisson-gamma model: Asymptotic analysis and improved intervals

We analyze predictions of future recruitment to a multicenter clinical trial based on a maximum-likelihood fitting of a commonly used hierarchical Poisson-gamma model for recruitments at individual centers. We consider the asymptotic accuracy of quantile predictions in the limit as the number of recruitment centers grows large and find that, in an important sense, the accuracy of the quantiles does not improve as the number of centers increases. When predicting the number of further recruits in an additional time period, the accuracy degrades as the ratio of the additional time to the census time increases, whereas when predicting the amount of additional time to recruit a further n • + patients, the accuracy degrades as the ratio of n • + to the number recruited up to the census period increases. Our analysis suggests an improved quantile predictor. Simulation studies verify that the predicted pattern holds for typical recruitment scenarios in clinical trials and verify the much improved coverage properties of prediction intervals obtained from our quantile predictor. In the process of extending the applicability of our methodology, we show that in terms of the accuracy of all integer moments it is always better to approximate the sum of independent gamma random variables by a single gamma random variable matched on the first two moments than by the moment-matched Gaussian available from the central limit theorem.

Predicting study duration in clinical trials with a time-to-event endpoint

In event-driven clinical trials comparing the survival functions of two groups, the number of events required to achieve the desired power is usually calculated using the Freedman formula or the Schoenfeld formula. Then, the sample size and the study duration derived from the required number of events are considered; however, their combination is not uniquely determined. In practice, various combinations are examined considering the enrollment speed, study duration, and the cost of enrollment. However, effective methods for visually representing their relationships and evaluating the uncertainty in study duration are insufficient. We developed a graphical approach for examining the relationship between sample size and study duration. To evaluate the uncertainty in study duration under a given sample size, we also derived the probability density function of the study duration and a method for updating the probability density function according to the observed number of events (ie, information time). The proposed methods are expected to improve the operation and management of clinical trials with a time-to-event endpoint.

Does asymmetry in patient recruitment in large critical care trials follow the Pareto principle?

BACKGROUND

Randomised controlled trials (RCT) may be hindered by slow recruitment rates, particularly in critically ill patients. While statistical models to predict recruitment rates have been described, no systematic assessment has been conducted of the distribution of recruitment across sites, temporal trends in site participation and impact of competing trials on patient recruitment.

METHODS

We used recruitment and screening logs from the SAFE, NICE-SUGAR, RENAL, CHEST and ADRENAL trials, five of the largest critical care RCTs. We quantified the extent of recruitment asymmetry between sites using Lorenz curves and Gini coefficients and assessed whether the recruitment distribution across sites follow the Pareto principle, which states that 80% of effects come from 20% of causes. Peak recruitment rates and growth in participating sites were calculated.

RESULTS

In total, 25,412 patients were randomised in 99 intensive care units (ICUs) for the five trials. Distribution of recruitment was asymmetric, with a small number of ICUs recruiting a large proportion of the patients. The Gini coefficients ranged from 0.14 to 0.52. The time to peak recruitment rate ranged from 7 to 41 months and was variable (7, 31, 41, 10 and 40 months). Over time, the proportion of recruitment at non-tertiary ICUs increased from 15% to 34%.

CONCLUSIONS

There is asymmetry of recruitment with a small proportion of ICUs recruiting a large proportion of patients. The distributions of recruitment were not consistent with the Pareto principle. There has been increasing participation of non-tertiary ICUs in clinical trials.

Bayesian accrual modeling and prediction in multicenter clinical trials with varying center activation times

Investigators who manage multicenter clinical trials need to pay careful attention to patterns of subject accrual, and the prediction of activation time for pending centers is potentially crucial for subject accrual prediction. We propose a Bayesian hierarchical model to predict subject accrual for multicenter clinical trials in which center activation times vary. We define center activation time as the time at which a center can begin enrolling patients in the trial. The difference in activation times between centers is assumed to follow an exponential distribution, and the model of subject accrual integrates prior information for the study with actual enrollment progress. We apply our proposed Bayesian multicenter accrual model to two multicenter clinical studies. The first is the PAIN-CONTRoLS study, a multicenter clinical trial with a goal of activating 40 centers and enrolling 400 patients within 104 weeks. The second is the HOBIT trial, a multicenter clinical trial with a goal of activating 14 centers and enrolling 200 subjects within 36 months. In summary, the Bayesian multicenter accrual model provides a prediction of subject accrual while accounting for both center- and individual patient-level variation.

Milestone prediction for time-to-event endpoint monitoring in clinical trials

Predicting the times of milestone events, ie, interim and final analyses in clinical trials, helps resource planning. This manuscript presents and compares several easily implemented methods for predicting when a milestone event is achieved. We show that it is beneficial to combine the predictions from different models to craft a better predictor through prediction synthesis. Furthermore, a Bayesian approach provides a better measure of the uncertainty involved in prediction of milestone events. We compare the methods through two simulations where the model has been correctly specified and where the models are a mixture of three incorrectly specified model classes. We then apply the methods on two real clinical trial data, North Central Cancer Treatment Group (NCCTG) N0147 and N9841. In summary, the Bayesian prediction synthesis methods automatically perform well even when the data collection is far from homogeneous. An R shiny app is under development to carry out the prediction in a user-friendly fashion.

Using Poisson-gamma model to evaluate the duration of recruitment process when historical trials are available

At the design of clinical trial operation, a question of a paramount interest is how long it takes to recruit a given number of patients. Modelling the recruitment dynamics is the necessary step to answer this question. Poisson-gamma model provides very convenient, flexible and realistic approach. This model allows predicting the trial duration using data collected at an interim time with very good accuracy. A natural question arises: how to evaluate the parameters of recruitment model before the trial begins? The question is harder to handle as there are no recruitment data available for this trial. However, if there exist similar completed trials, it is appealing to use data from these trials to investigate feasibility of the recruitment process. In this paper, the authors explore the recruitment data of two similar clinical trials (Intergroupe Francais du Myélome 2005 and 2009). It is shown that the natural idea of plugging the historical rates estimated from the completed trial in the same centres of the new trial for predicting recruitment is not a relevant strategy. In contrast, using the parameters of a gamma distribution of the rates estimated from the completed trial in the recruitment dynamic model of the new trial provides reasonable predictive properties with relevant confidence intervals. Copyright © 2017 John Wiley & Sons, Ltd.

A framework for prospectively defining progression rules for internal pilot studies monitoring recruitment

Just over half of publicly funded trials recruit their target sample size within the planned study duration. When recruitment targets are missed, the funder of a trial is faced with the decision of either committing further resources to the study or risk that a worthwhile treatment effect may be missed by an underpowered final analysis. To avoid this challenging situation, when there is insufficient prior evidence to support predicted recruitment rates, funders now require feasibility assessments to be performed in the early stages of trials. Progression criteria are usually specified and agreed with the funder ahead of time. To date, however, the progression rules used are typically ad hoc. In addition, rules routinely permit adaptations to recruitment strategies but do not stipulate criteria for evaluating their effectiveness. In this paper, we develop a framework for planning and designing internal pilot studies which permit a trial to be stopped early if recruitment is disappointing or to continue to full recruitment if enrolment during the feasibility phase is adequate. This framework enables a progression rule to be pre-specified and agreed upon prior to starting a trial. The novel two-stage designs stipulate that if neither of these situations arises, adaptations to recruitment should be made and subsequently evaluated to establish whether they have been successful. We derive optimal progression rules for internal pilot studies which minimise the expected trial overrun and maintain a high probability of completing the study when the recruitment rate is adequate. The advantages of this procedure are illustrated using a real trial example.

Conditional moving linear regression: modeling the recruitment process for ALLHAT

Effective recruitment is a prerequisite for successful execution of a clinical trial. ALLHAT, a large hypertension treatment trial (N = 42, 418), provided an opportunity to evaluate adaptive modeling of recruitment processes using conditional moving linear regression. Our statistical modeling of recruitment, comparing Brownian and fractional Brownian motion, indicates that fractional Brownian motion combined with moving linear regression is better than classic Brownian motion in terms of higher conditional probability of achieving a global recruitment goal in four week ahead projections. Further research is needed to evaluate how recruitment modeling can assist clinical trialists in planning and executing clinical trials. Clinical Trial Registration: www.clinicaltrials.gov NCT00000542.

How to deal with the Poisson-gamma model to forecast patients' recruitment in clinical trials when there are pauses in recruitment dynamic?

Recruiting patients is a crucial step of a clinical trial. Estimation of the trial duration is a question of paramount interest. Most techniques are based on deterministic models and various ad hoc methods neglecting the variability in the recruitment process. To overpass this difficulty the so-called Poisson-gamma model has been introduced involving, for each centre, a recruitment process modelled by a Poisson process whose rate is assumed constant in time and gamma-distributed. The relevancy of this model has been widely investigated. In practice, rates are rarely constant in time, there are breaks in recruitment (for instance week-ends or holidays). Such information can be collected and included in a model considering piecewise constant rate functions yielding to an inhomogeneous Cox model. The estimation of the trial duration is much more difficult. Three strategies of computation of the expected trial duration are proposed considering all the breaks, considering only large breaks and without considering breaks. The bias of these estimations procedure are assessed by means of simulation studies considering three scenarios of breaks simulation. These strategies yield to estimations with a very small bias. Moreover, the strategy with the best performances in terms of prediction and with the smallest bias is the one which does not take into account of breaks. This result is important as, in practice, collecting breaks data is pretty hard to manage.