Some issues in predicting patient recruitment in multi-centre clinical trials

Enhancing site selection strategies in clinical trial recruitment using real-world data modeling

Slow patient enrollment or failing to enroll the required number of patients is a disruptor of clinical trial timelines. To meet the planned trial recruitment, site selection strategies are used during clinical trial planning to identify research sites that are most likely to recruit a sufficiently high number of subjects within trial timelines. We developed a machine learning approach that outperforms baseline methods to rank research sites based on their expected recruitment in future studies. Indication level historical recruitment and real-world data are used in the machine learning approach to predict patient enrollment at site level. We define covariates based on published recruitment hypotheses and examine the effect of these covariates in predicting patient enrollment. We compare model performance of a linear and a non-linear machine learning model with common industry baselines that are constructed from historical recruitment data. Performance of the methodology is evaluated and reported for two disease indications, inflammatory bowel disease and multiple myeloma, both of which are actively being pursued in clinical development. We validate recruitment hypotheses by reviewing the covariates relationship with patient recruitment. For both indications, the non-linear model significantly outperforms the baselines and the linear model on the test set. In this paper, we present a machine learning approach to site selection that incorporates site-level recruitment and real-world patient data. The model ranks research sites by predicting the number of recruited patients and our results suggest that the model can improve site ranking compared to common industry baselines.

A time-dependent Poisson-Gamma model for recruitment forecasting in multicenter studies

Forecasting recruitments is a key component of the monitoring phase of multicenter studies. One of the most popular techniques in this field is the Poisson-Gamma recruitment model, a Bayesian technique built on a doubly stochastic Poisson process. This approach is based on the modeling of enrollments as a Poisson process where the recruitment rates are assumed to be constant over time and to follow a common Gamma prior distribution. However, the constant-rate assumption is a restrictive limitation that is rarely appropriate for applications in real studies. In this paper, we illustrate a flexible generalization of this methodology which allows the enrollment rates to vary over time by modeling them through B-splines. We show the suitability of this approach for a wide range of recruitment behaviors in a simulation study and by estimating the recruitment progression of the Canadian Co-infection Cohort.

Modeling time-varying recruitment rates in multicenter clinical trials

Multicenter phase II/III clinical trials are large-scale operations that often include hundreds of recruiting centers in several countries. Therefore, the operational aspects of a trial must be thoroughly planned and closely monitored to ensure better oversight and study conduct. Predicting patient recruitment plays a pivotal role in trial monitoring as it informs how many people are expected to be recruited on a given day. As such, study teams may rely on predictions to assess progress and detect any deviations from the original plan that might put the study and potentially, patients at risk. Recruitment predictions are often based on a Poisson-Gamma model that assumes that centers have a constant recruitment rate throughout the trial. The model has useful features and has extensively been used, yet its main assumption is often unrealistic. A nonhomogeneous Poisson process has been recently proposed that can incorporate time-varying recruitment rates. In this work, we predict patient recruitment using both approaches and assess the impact of said assumption in a real-world setting where studies may not necessarily have constant center-specific recruitment rates. The paper showcases experience from modeling recruitment in trials sponsored by AstraZeneca between 2005 and 2018. In these data, there is evidence of time-varying recruitment rates. The predictive performance of models that account for both constant and time-varying recruitment effects is presented. Following a descriptive analysis of data, we assess model performance and investigate the impact of model misspecification.

Recruitment prediction for multicenter clinical trials based on a hierarchical Poisson-gamma model: Asymptotic analysis and improved intervals

We analyze predictions of future recruitment to a multicenter clinical trial based on a maximum-likelihood fitting of a commonly used hierarchical Poisson-gamma model for recruitments at individual centers. We consider the asymptotic accuracy of quantile predictions in the limit as the number of recruitment centers grows large and find that, in an important sense, the accuracy of the quantiles does not improve as the number of centers increases. When predicting the number of further recruits in an additional time period, the accuracy degrades as the ratio of the additional time to the census time increases, whereas when predicting the amount of additional time to recruit a further n • + patients, the accuracy degrades as the ratio of n • + to the number recruited up to the census period increases. Our analysis suggests an improved quantile predictor. Simulation studies verify that the predicted pattern holds for typical recruitment scenarios in clinical trials and verify the much improved coverage properties of prediction intervals obtained from our quantile predictor. In the process of extending the applicability of our methodology, we show that in terms of the accuracy of all integer moments it is always better to approximate the sum of independent gamma random variables by a single gamma random variable matched on the first two moments than by the moment-matched Gaussian available from the central limit theorem.

Accrual Prediction Program: A web-based clinical trials tool for monitoring and predicting accrual for early-phase cancer studies

BACKGROUND

Monitoring subject recruitment is key to the success of a clinical trial. Accordingly, researchers have developed accrual-monitoring tools to support the design and conduct of trials. At an institutional level, delays in identifying studies with high risk of accrual failure can lead to inefficient and costly trials with little chances of meeting study objectives. Comprehensive accrual monitoring is necessary to the success of the research enterprise.

METHODS

This article describes the design and implementation of the University of Kansas Cancer Center Accrual Prediction Program, a web-based platform was developed to support comprehensive accrual monitoring and prediction for all active clinical trials. The Accrual Prediction Program provides information on accrual, including the predicted completion date, predicted number of accrued subjects during the pre-specified accrual period, and the probability of achieving accrual targets. It relies on a Bayesian accrual prediction model to combine protocol information with real-time trial enrollment data and disseminates results via web application.

RESULTS

First released in 2016, the Accrual Prediction Program summarizes enrollment information for active studies categorized by various trial attributes. The web application supports real-time evidence-based decision making for strategic resource allocation and study management of over 120 ongoing clinical trials at the University of Kansas Cancer Center.

CONCLUSION

The Accrual Prediction Program makes accessing comprehensive accrual information manageable at an institutional level. Cancer centers or even entire institutions can reproduce the Accrual Prediction Program to achieve real-time comprehensive monitoring and prediction of subject accrual to aid investigators and administrators in the design, conduct, and management of clinical trials.

Using Poisson-gamma model to evaluate the duration of recruitment process when historical trials are available

At the design of clinical trial operation, a question of a paramount interest is how long it takes to recruit a given number of patients. Modelling the recruitment dynamics is the necessary step to answer this question. Poisson-gamma model provides very convenient, flexible and realistic approach. This model allows predicting the trial duration using data collected at an interim time with very good accuracy. A natural question arises: how to evaluate the parameters of recruitment model before the trial begins? The question is harder to handle as there are no recruitment data available for this trial. However, if there exist similar completed trials, it is appealing to use data from these trials to investigate feasibility of the recruitment process. In this paper, the authors explore the recruitment data of two similar clinical trials (Intergroupe Francais du Myélome 2005 and 2009). It is shown that the natural idea of plugging the historical rates estimated from the completed trial in the same centres of the new trial for predicting recruitment is not a relevant strategy. In contrast, using the parameters of a gamma distribution of the rates estimated from the completed trial in the recruitment dynamic model of the new trial provides reasonable predictive properties with relevant confidence intervals. Copyright © 2017 John Wiley & Sons, Ltd.

A framework for prospectively defining progression rules for internal pilot studies monitoring recruitment

Just over half of publicly funded trials recruit their target sample size within the planned study duration. When recruitment targets are missed, the funder of a trial is faced with the decision of either committing further resources to the study or risk that a worthwhile treatment effect may be missed by an underpowered final analysis. To avoid this challenging situation, when there is insufficient prior evidence to support predicted recruitment rates, funders now require feasibility assessments to be performed in the early stages of trials. Progression criteria are usually specified and agreed with the funder ahead of time. To date, however, the progression rules used are typically ad hoc. In addition, rules routinely permit adaptations to recruitment strategies but do not stipulate criteria for evaluating their effectiveness. In this paper, we develop a framework for planning and designing internal pilot studies which permit a trial to be stopped early if recruitment is disappointing or to continue to full recruitment if enrolment during the feasibility phase is adequate. This framework enables a progression rule to be pre-specified and agreed upon prior to starting a trial. The novel two-stage designs stipulate that if neither of these situations arises, adaptations to recruitment should be made and subsequently evaluated to establish whether they have been successful. We derive optimal progression rules for internal pilot studies which minimise the expected trial overrun and maintain a high probability of completing the study when the recruitment rate is adequate. The advantages of this procedure are illustrated using a real trial example.

Conditional moving linear regression: modeling the recruitment process for ALLHAT

Effective recruitment is a prerequisite for successful execution of a clinical trial. ALLHAT, a large hypertension treatment trial (N = 42, 418), provided an opportunity to evaluate adaptive modeling of recruitment processes using conditional moving linear regression. Our statistical modeling of recruitment, comparing Brownian and fractional Brownian motion, indicates that fractional Brownian motion combined with moving linear regression is better than classic Brownian motion in terms of higher conditional probability of achieving a global recruitment goal in four week ahead projections. Further research is needed to evaluate how recruitment modeling can assist clinical trialists in planning and executing clinical trials. Clinical Trial Registration: www.clinicaltrials.gov NCT00000542.

How to deal with the Poisson-gamma model to forecast patients' recruitment in clinical trials when there are pauses in recruitment dynamic?

Recruiting patients is a crucial step of a clinical trial. Estimation of the trial duration is a question of paramount interest. Most techniques are based on deterministic models and various ad hoc methods neglecting the variability in the recruitment process. To overpass this difficulty the so-called Poisson-gamma model has been introduced involving, for each centre, a recruitment process modelled by a Poisson process whose rate is assumed constant in time and gamma-distributed. The relevancy of this model has been widely investigated. In practice, rates are rarely constant in time, there are breaks in recruitment (for instance week-ends or holidays). Such information can be collected and included in a model considering piecewise constant rate functions yielding to an inhomogeneous Cox model. The estimation of the trial duration is much more difficult. Three strategies of computation of the expected trial duration are proposed considering all the breaks, considering only large breaks and without considering breaks. The bias of these estimations procedure are assessed by means of simulation studies considering three scenarios of breaks simulation. These strategies yield to estimations with a very small bias. Moreover, the strategy with the best performances in terms of prediction and with the smallest bias is the one which does not take into account of breaks. This result is important as, in practice, collecting breaks data is pretty hard to manage.

Modeling and validating Bayesian accrual models on clinical data and simulations using adaptive priors

Slow recruitment in clinical trials leads to increased costs and resource utilization, which includes both the clinic staff and patient volunteers. Careful planning and monitoring of the accrual process can prevent the unnecessary loss of these resources. We propose two hierarchical extensions to the existing Bayesian constant accrual model: the accelerated prior and the hedging prior. The new proposed priors are able to adaptively utilize the researcher's previous experience and current accrual data to produce the estimation of trial completion time. The performance of these models, including prediction precision, coverage probability, and correct decision-making ability, is evaluated using actual studies from our cancer center and simulation. The results showed that a constant accrual model with strongly informative priors is very accurate when accrual is on target or slightly off, producing smaller mean squared error, high percentage of coverage, and a high number of correct decisions as to whether or not continue the trial, but it is strongly biased when off target. Flat or weakly informative priors provide protection against an off target prior but are less efficient when the accrual is on target. The accelerated prior performs similar to a strong prior. The hedging prior performs much like the weak priors when the accrual is extremely off target but closer to the strong priors when the accrual is on target or only slightly off target. We suggest improvements in these models and propose new models for future research.