Inference in response-adaptive clinical trials when the enrolled population varies over time

A common assumption of data analysis in clinical trials is that the patient population, as well as treatment effects, do not vary during the course of the study. However, when trials enroll patients over several years, this hypothesis may be violated. Ignoring variations of the outcome distributions over time, under the control and experimental treatments, can lead to biased treatment effect estimates and poor control of false positive results. We propose and compare two procedures that account for possible variations of the outcome distributions over time, to correct treatment effect estimates, and to control type-I error rates. The first procedure models trends of patient outcomes with splines. The second leverages conditional inference principles, which have been introduced to analyze randomized trials when patient prognostic profiles are unbalanced across arms. These two procedures are applicable in response-adaptive clinical trials. We illustrate the consequences of trends in the outcome distributions in response-adaptive designs and in platform trials, and investigate the proposed methods in the analysis of a glioblastoma study.

Integrating non-concurrent controls in the analyses of late-entry experimental arms in multi-arm trials with a shared control group in the presence of parameter drift

BACKGROUND

Under a master protocol, open platform trials allow new experimental treatments to enter an existing clinical trial. Whether late-entry experimental treatments should be compared to all available or concurrently randomized controls is not well established. Using all available data can increase power and precision; however, drift in population parameters can yield biased estimates and impact type I error rate.

METHODS

We explored the application of methods developed to incorporate historical controls in two-arm trials to the analysis of a late-entry arm in a simulated open platform trial under varying scenarios of parameter drift. Methods explored include test-then-pool, fixed power prior, dynamic power prior, and multi-source exchangeability model approaches.

RESULTS/CONCLUSIONS

Simulated trial results confirm that in the presence of no drift, naively pooling all controls increases power and produces more precise, unbiased estimates when compared to using concurrent controls only. However, under drift, pooling can result in type I error rate inflation or deflation and biased estimates. In the presence of parameter drift, methods that partially borrow non-concurrent data, either through a static weighting mechanism or through methods that allow the heterogeneity between non-concurrent and concurrent data to determine the degree of borrowing, are superior to naively pooling the data. However, compared to using concurrent controls only, these approaches cannot guarantee type I error control or unbiased estimates. Thus, concurrent controls should be used as comparators in confirmatory studies.

Novel Clinical Trial Designs in Neuro-Oncology

Scientific and technologic advances have led to a boon of candidate therapeutics for patients with malignancies of the central nervous system. The path from drug development to clinical use has generally followed a regimented order of sequential clinical trial phases. The recent increase in novel therapies, however, has strained the regulatory process and unearthed limitations of the current system, including significant cost, prolonged development time, and difficulties in testing therapies for rarer tumors. Novel clinical trial designs have emerged to increase efficiencies in clinical trial conduct to better evaluate and bring impactful drugs to patients in a timely manner. In order to better capture meaningful benefits for brain tumor patients, new endpoints to complement or replace traditional endpoints are also an increasingly important consideration. This review will explore the current challenges in the current clinical trial landscape and discuss novel clinical trial concepts, including consideration of limitations and risks of novel trial designs, within the context of neuro-oncology.

Economic Evaluation of Cost and Time Required for a Platform Trial vs Conventional Trials

IMPORTANCE

Platform trial design allows the introduction of new interventions after the trial is initiated and offers efficiencies to clinical research. However, limited guidance exists on the economic resources required to establish and maintain platform trials.

OBJECTIVE

To compare cost (US dollars) and time requirements of conducting a platform trial vs a series of conventional (nonplatform) trials using a real-life example.

DESIGN, SETTING, AND PARTICIPANTS

For this economic evaluation, an online survey was administered to a group of international experts (146 participants) with publication records of platform trials to elicit their opinions on cost and time to set up and conduct platform, multigroup, and 2-group trials. Using the reported entry dates of 10 interventions into Systemic Therapy in Advancing Metastatic Prostate Cancer: Evaluation of Drug Efficacy, the longest ongoing platform trial, 3 scenarios were designed involving a single platform trial (scenario 1), 1 multigroup followed by 5 2-group trials (scenario 2), and a series of 10 2-group trials (scenario 3). All scenarios started with 5 interventions, then 5 more interventions were either added to the platform or evaluated independently. Simulations with the survey results as inputs were used to compare the platform vs conventional trial designs. Data were analyzed from July to September 2021.

EXPOSURE

Platform trial design.

MAIN OUTCOMES AND MEASURES

Total trial setup and conduct cost and cumulative duration.

RESULTS

Although setup time and cost requirements of a single trial were highest for the platform trial, cumulative requirements of setting up a series of multiple trials in scenarios 2 and 3 were larger. Compared with the platform trial, there was a median (IQR) increase of 216.7% (202.2%-242.5%) in cumulative setup costs for scenario 2 and 391.1% (365.3%-437.9%) for scenario 3. In terms of total cost, there was a median (IQR) increase of 17.4% (12.1%-22.5%) for scenario 2 and 57.5% (43.1%-69.9%) for scenario 3. There was a median (IQR) increase in cumulative trial duration of 171.1% (158.3%-184.3%) for scenario 2 and 311.9% (282.0%-349.1%) for scenario 3. Cost and time reductions in the platform trial were observed in both the initial and subsequently evaluated interventions.

CONCLUSIONS AND RELEVANCE

Although setting up platform trials can take longer and be costly, the findings of this study suggest that having a single infrastructure can improve efficiencies with respect to costs and efforts.

Combining factorial and multi-arm multi-stage platform designs to evaluate multiple interventions efficiently

BACKGROUND

Factorial-MAMS design platform designs have many advantages, but the practical advantages and disadvantages of combining the two designs have not been explored.

METHODS

We propose practical methods for a combined design within the platform trial paradigm where some interventions are not expected to interact and could be given together.

RESULTS

We describe the combined design and suggest diagrams that can be used to represent it. Many properties are common both to standard factorial designs, including the need to consider interactions between interventions and the impact of intervention efficacy on power of other comparisons, and to standard multi-arm multi-stage designs, including the need to pre-specify procedures for starting and stopping intervention comparisons. We also identify some specific features of the factorial-MAMS design: timing of interim and final analyses should be determined by calendar time or total observed events; some non-factorial modifications may be useful; eligibility criteria should be broad enough to include any patient eligible for any part of the randomisation; stratified randomisation may conveniently be performed sequentially; and analysis requires special care to use only concurrent controls.

CONCLUSION

A combined factorial-MAMS design can combine the efficiencies of factorial trials and multi-arm multi-stage platform trials. It allows us to address multiple research questions under one protocol and to test multiple new treatment options, which is particularly important when facing a new emergent infection such as COVID-19.

Decision rules for identifying combination therapies in open-entry, randomized controlled platform trials

Platform trials have become increasingly popular for drug development programs, attracting interest from statisticians, clinicians and regulatory agencies. Many statistical questions related to designing platform trials-such as the impact of decision rules, sharing of information across cohorts, and allocation ratios on operating characteristics and error rates-remain unanswered. In many platform trials, the definition of error rates is not straightforward as classical error rate concepts are not applicable. For an open-entry, exploratory platform trial design comparing combination therapies to the respective monotherapies and standard-of-care, we define a set of error rates and operating characteristics and then use these to compare a set of design parameters under a range of simulation assumptions. When setting up the simulations, we aimed for realistic trial trajectories, such that for example, a priori we do not know the exact number of treatments that will be included over time in a specific simulation run as this follows a stochastic mechanism. Our results indicate that the method of data sharing, exact specification of decision rules and a priori assumptions regarding the treatment efficacy all strongly contribute to the operating characteristics of the platform trial. Furthermore, different operating characteristics might be of importance to different stakeholders. Together with the potential flexibility and complexity of a platform trial, which also impact the achieved operating characteristics via, for example, the degree of efficiency of data sharing this implies that utmost care needs to be given to evaluation of different assumptions and design parameters at the design stage.

Stratified randomization for platform trials with differing experimental arm eligibility

BACKGROUND

Platform trials facilitate efficient use of resources by comparing multiple experimental agents to a common standard of care arm. They can accommodate a changing scientific paradigm within a single trial protocol by adding or dropping experimental arms-critical features for trials in rapidly developing disease areas such as COVID-19 or cancer therapeutics. However, in these trials, efficacy and safety issues may render certain participant subgroups ineligible to some experimental arms, and methods for stratified randomization do not readily apply to this setting.

METHODS

We propose extensions for conventional methods of stratified randomization for platform trials whose experimental arms may differ in eligibility criteria. These methods balance on a prespecified set of stratification variables observable prior to arm assignment that remains the same across experimental arms. To do so, we suggest modifying block randomization by including experimental arm eligibility as a stratifying variable, and we suggest modifying the imbalance score calculation in dynamic balancing by performing pairwise comparisons between each eligible experimental arm and standard of care arm participants eligible to that experimental arm.

RESULTS

We provide worked examples to illustrate the proposed extensions. In addition, we also provide a formula to quantify the relative efficiency loss of platform trials with varying eligibility compared with trials with non-varying eligibility as measured by the size of the common standard of care arm.

CONCLUSIONS

This article presents important extensions to conventional methods for stratified randomization in order to facilitate the implementation of platform trials with differing experimental arm eligibility.

Emulating Control Arms for Cancer Clinical Trials Using External Cohorts Created From Electronic Health Record-Derived Real-World Data

Electronic health record (EHR)-derived real-world data (RWD) can be sourced to create external comparator cohorts to oncology clinical trials. This exploratory study assessed whether EHR-derived patient cohorts could emulate select clinical trial control arms across multiple tumor types. The impact of analytic decisions on emulation results was also evaluated. By digitizing Kaplan-Meier curves, we reconstructed published control arm results from 15 trials that supported drug approvals from January 1, 2016, to April 30, 2018. RWD cohorts were constructed using a nationwide EHR-derived de-identified database by aligning eligibility criteria and weighting to trial baseline characteristics. Trial data and RWD cohorts were compared using Kaplan-Meier and Cox proportional hazards regression models for progression-free survival (PFS) and overall survival (OS; individual cohorts) and multitumor random effects models of hazard ratios (HRs) for median endpoint correlations (across cohorts). Post hoc, the impact of specific analytic decisions on endpoints was assessed using a case study. Comparing trial data and weighted RWD cohorts, PFS results were more similar (HR range = 0.63-1.18, pooled HR = 0.84, correlation of median = 0.91) compared to OS (HR range = 0.36-1.09, pooled HR = 0.76, correlation of median = 0.85). OS HRs were more variable and trended toward worse for RWD cohorts. The post hoc case study had OS HR ranging from 0.67 (95% confidence interval (CI): 0.56-0.79) to 0.92 (95% CI: 0.78-1.09) depending on specific analytic decisions. EHR-derived RWD can emulate oncology clinical trial control arm results, although with variability. Visibility into clinical trial cohort characteristics may shape and refine analytic approaches.

Randomised trials at the level of the individual

In global health research, short-term, small-scale clinical trials with fixed, two-arm trial designs that generally do not allow for major changes throughout the trial are the most common study design. Building on the introductory paper of this Series, this paper discusses data-driven approaches to clinical trial research across several adaptive trial designs, as well as the master protocol framework that can help to harmonise clinical trial research efforts in global health research. We provide a general framework for more efficient trial research, and we discuss the importance of considering different study designs in the planning stage with statistical simulations. We conclude this second Series paper by discussing the methodological and operational complexity of adaptive trial designs and master protocols and the current funding challenges that could limit uptake of these approaches in global health research.

The effects of releasing early results from ongoing clinical trials

Most trials do not release interim summaries on efficacy and toxicity of the experimental treatments being tested, with this information only released to the public after the trial has ended. While early release of clinical trial data to physicians and patients can inform enrollment decision making, it may also affect key operating characteristics of the trial, statistical validity and trial duration. We investigate the public release of early efficacy and toxicity results, during ongoing clinical studies, to better inform patients about their enrollment options. We use simulation models of phase II glioblastoma (GBM) clinical trials in which early efficacy and toxicity estimates are periodically released accordingly to a pre-specified protocol. Patients can use the reported interim efficacy and toxicity information, with the support of physicians, to decide which trial to enroll in. We describe potential effects on various operating characteristics, including the study duration, selection bias and power.

The vitamin D for COVID-19 (VIVID) trial: A pragmatic cluster-randomized design

OBJECTIVES

To determine the effect of vitamin D supplementation on disease progression and post-exposure prophylaxis for COVID-19 infection. We hypothesize that high-dose vitamin D3 supplementation will reduce risk of hospitalization/death among those with recently diagnosed COVID-19 infection and will reduce risk of COVID-19 infection among their close household contacts.

METHODS

We report the rationale and design of a planned pragmatic, cluster randomized, double-blinded trial (N = 2700 in total nationwide), with 1500 newly diagnosed individuals with COVID-19 infection, together with up to one close household contact each (~1200 contacts), randomized to either vitamin D3 (loading dose, then 3200 IU/day) or placebo in a 1:1 ratio and a household cluster design. The study duration is 4 weeks. The primary outcome for newly diagnosed individuals is the occurrence of hospitalization and/or mortality. Key secondary outcomes include symptom severity scores among cases and changes in the infection (seroconversion) status for their close household contacts. Changes in vitamin D 25(OH)D levels will be assessed and their relation to study outcomes will be explored.

CONCLUSIONS

The proposed pragmatic trial will allow parallel testing of vitamin D3 supplementation for early treatment and post-exposure prophylaxis of COVID-19. The household cluster design provides a cost-efficient approach to testing an intervention for reducing rates of hospitalization and/or mortality in newly diagnosed cases and preventing infection among their close household contacts.

An overview of platform trials with a checklist for clinical readers

OBJECTIVES

The objective of the study was to outline key considerations for general clinical readers when critically evaluating publications on platform trials and for researchers when designing these types of clinical trials.

STUDY DESIGN AND SETTING

In this review, we describe key concepts of platform trials with case study discussion of two hallmark platform trials in STAMPEDE and I-SPY2. We provide reader's guide to platform trials with a critical appraisal checklist.

RESULTS

Platform trials offer flexibilities of dropping ineffective arms early based on interim data and introducing new arms into the trial. For platform trials, it is important to consider how interventions are compared and evaluated throughout and how new interventions are introduced. For intervention comparisons, it is important to consider what the primary analysis is, what and how many interventions are active simultaneously, and allocation between different arms. Interim evaluation considerations should include the number and timing of interim evaluations and outcomes and statistical rules used to drop interventions. New interventions are usually introduced based on scientific merits, so consideration of these merits is important, together with the timing and mechanisms in which new interventions are added.

CONCLUSION

More efforts are needed to improve the scientific literacy of platform trials. Our review provides an overview of the important concepts of platform trials.

Adding new experimental arms to randomised clinical trials: Impact on error rates

BACKGROUND

Experimental treatments pass through various stages of development. If a treatment passes through early-phase experiments, the investigators may want to assess it in a late-phase randomised controlled trial. An efficient way to do this is adding it as a new research arm to an ongoing trial while the existing research arms continue, a so-called multi-arm platform trial. The familywise type I error rate is often a key quantity of interest in any multi-arm platform trial. We set out to clarify how it should be calculated when new arms are added to a trial some time after it has started.

METHODS

We show how the familywise type I error rate, any-pair and all-pairs powers can be calculated when a new arm is added to a platform trial. We extend the Dunnett probability and derive analytical formulae for the correlation between the test statistics of the existing pairwise comparison and that of the newly added arm. We also verify our analytical derivation via simulations.

RESULTS

Our results indicate that the familywise type I error rate depends on the shared control arm information (i.e. individuals in continuous and binary outcomes and primary outcome events in time-to-event outcomes) from the common control arm patients and the allocation ratio. The familywise type I error rate is driven more by the number of pairwise comparisons and the corresponding (pairwise) type I error rates than by the timing of the addition of the new arms. The familywise type I error rate can be estimated using Šidák's correction if the correlation between the test statistics of pairwise comparisons is less than 0.30.

CONCLUSIONS

The findings we present in this article can be used to design trials with pre-planned deferred arms or to add new pairwise comparisons within an ongoing platform trial where control of the pairwise error rate or familywise type I error rate (for a subset of pairwise comparisons) is required.

To randomize, or not to randomize, that is the question: using data from prior clinical trials to guide future designs

BACKGROUND

Understanding the value of randomization is critical in designing clinical trials. Here, we introduce a simple and interpretable quantitative method to compare randomized designs versus single-arm designs using indication-specific parameters derived from the literature. We demonstrate the approach through application to phase II trials in newly diagnosed glioblastoma (ndGBM).

METHODS

We abstracted data from prior ndGBM trials and derived relevant parameters to compare phase II randomized controlled trials (RCTs) and single-arm designs within a quantitative framework. Parameters included in our model were (i) the variability of the primary endpoint distributions across studies, (ii) potential for incorrectly specifying the single-arm trial's benchmark, and (iii) the hypothesized effect size. Strengths and weaknesses of RCT and single-arm designs were quantified by various metrics, including power and false positive error rates.

RESULTS

We applied our method to show that RCTs should be preferred to single-arm trials for evaluating overall survival in ndGBM patients based on parameters estimated from prior trials. More generally, for a given effect size, the utility of randomization compared with single-arm designs is highly dependent on (i) interstudy variability of the outcome distributions and (ii) potential errors in selecting standard of care efficacy estimates for single-arm studies.

CONCLUSIONS

A quantitative framework using historical data is useful in understanding the utility of randomization in designing prospective trials. For typical phase II ndGBM trials using overall survival as the primary endpoint, randomization should be preferred over single-arm designs.

The clinical trials landscape for glioblastoma: is it adequate to develop new treatments?

Background

There have been few treatment advances for patients with glioblastoma (GBM) despite increasing scientific understanding of the disease. While factors such as intrinsic tumor biology and drug delivery are challenges to developing efficacious therapies, it is unclear whether the current clinical trial landscape is optimally evaluating new therapies and biomarkers.

Methods

We queried ClinicalTrials.gov for interventional clinical trials for patients with GBM initiated between January 2005 and December 2016 and abstracted data regarding phase, status, start and end dates, testing locations, endpoints, experimental interventions, sample size, clinical presentation/indication, and design to better understand the clinical trials landscape.

Results

Only approximately 8%-11% of patients with newly diagnosed GBM enroll on clinical trials with a similar estimate for all patients with GBM. Trial duration was similar across phases with median time to completion between 3 and 4 years. While 93% of clinical trials were in phases I-II, 26% of the overall clinical trial patient population was enrolled on phase III studies. Of the 8 completed phase III trials, only 1 reported positive results. Although 58% of the phase III trials were supported by phase II data with a similar endpoint, only 25% of these phase II trials were randomized.

Conclusions

The clinical trials landscape for GBM is characterized by long development times, inadequate dissemination of information, suboptimal go/no-go decision making, and low patient participation.

The Statistical Curriculum Within Randomized Controlled Trials in Critical Illness

OBJECTIVES

Incomplete biostatistical knowledge among clinicians is widely described. This study aimed to categorize and summarize the statistical methodology within recent critical care randomized controlled trials.

DESIGN

Descriptive analysis, with comparison of findings to previous work.

SETTING

Ten high-impact clinical journals publishing trials in critical illness.

SUBJECTS

Randomized controlled trials published between 2011 and 2015 inclusive.

INTERVENTIONS

Data extraction from published reports.

MEASUREMENTS AND MAIN RESULTS

The frequency and overall proportion of each statistical method encountered, grouped according to those used to generate each trial's primary outcome and separately according to underlying statistical methodology. Subsequent analysis compared these proportions with previously published reports. A total of 580 statistical tests or methods were identified within 116 original randomized controlled trials published between 2011 and 2015. Overall, the chi-square test was the most commonly encountered (70/116; 60%), followed by the Cox proportional hazards model (63/116; 54%) and logistic regression (53/116; 46%). When classified according to underlying statistical assumptions, the most common types of analyses were tests of 2 × 2 contingency tables and nonparametric tests of rank order. A greater proportion of more complex methodology was observed compared with trial reports from previous work.

CONCLUSIONS

Physicians assessing recent randomized controlled trials in critical illness encounter results derived from a substantial and potentially expanding range of biostatistical methods. In-depth training in the assumptions and limitations of these current and emerging biostatistical methods may not be practically achievable for most clinicians, making accessible specialist biostatistical support an asset to evidence-based clinical practice.

Integrating molecular nuclear imaging in clinical research to improve anticancer therapy

Effective patient selection before or early during treatment is important to increasing the therapeutic benefits of anticancer treatments. This selection process is often predicated on biomarkers, predominantly biospecimen biomarkers derived from blood or tumour tissue; however, such biomarkers provide limited information about the true extent of disease or about the characteristics of different, potentially heterogeneous tumours present in an individual patient. Molecular imaging can also produce quantitative outputs; such imaging biomarkers can help to fill these knowledge gaps by providing complementary information on tumour characteristics, including heterogeneity and the microenvironment, as well as on pharmacokinetic parameters, drug-target engagement and responses to treatment. This integrative approach could therefore streamline biomarker and drug development, although a range of issues need to be overcome in order to enable a broader use of molecular imaging in clinical trials. In this Perspective article, we outline the multistage process of developing novel molecular imaging biomarkers. We discuss the challenges that have restricted the use of molecular imaging in clinical oncology research to date and outline future opportunities in this area.

To add or not to add a new treatment arm to a multiarm study: A decision-theoretic framework

Multiarm clinical trials, which compare several experimental treatments against control, are frequently recommended due to their efficiency gain. In practise, all potential treatments may not be ready to be tested in a phase II/III trial at the same time. It has become appealing to allow new treatment arms to be added into on‐going clinical trials using a “platform” trial approach. To the best of our knowledge, many aspects of when to add arms to an existing trial have not been explored in the literature. Most works on adding arm(s) assume that a new arm is opened whenever a new treatment becomes available. This strategy may prolong the overall duration of a study or cause reduction in marginal power for each hypothesis if the adaptation is not well accommodated. Within a two‐stage trial setting, we propose a decision‐theoretic framework to investigate when to add or not to add a new treatment arm based on the observed stage one treatment responses. To account for different prospect of multiarm studies, we define utility in two different ways; one for a trial that aims to maximise the number of rejected hypotheses; the other for a trial that would declare a success when at least one hypothesis is rejected from the study. Our framework shows that it is not always optimal to add a new treatment arm to an existing trial. We illustrate a case study by considering a completed trial on knee osteoarthritis.

Individualized Screening Trial of Innovative Glioblastoma Therapy (INSIGhT): A Bayesian Adaptive Platform Trial to Develop Precision Medicines for Patients With Glioblastoma

PURPOSE

Adequately prioritizing the numerous therapies and biomarkers available in late-stage testing for patients with glioblastoma (GBM) requires an efficient clinical testing platform. We developed and implemented INSIGhT (Individualized Screening Trial of Innovative Glioblastoma Therapy) as a novel adaptive platform trial (APT) to develop precision medicine approaches in GBM.

METHODS

INSIGhT compares experimental arms with a common control of standard concurrent temozolomide and radiation therapy followed by adjuvant temozolomide. The primary end point is overall survival. Patients with newly diagnosed unmethylated GBM who are IDH R132H mutation negative and with genomic data available for biomarker grouping are eligible. At the initiation of INSIGhT, three experimental arms (neratinib, abemaciclib, and CC-115), each with a proposed genomic biomarker, are tested simultaneously. Initial randomization is equal across arms. As the trial progresses, randomization probabilities adapt on the basis of accumulating results using Bayesian estimation of the biomarker-specific probability of treatment impact on progression-free survival. Treatment arms may drop because of low probability of treatment impact on overall survival, and new arms may be added. Detailed information on the statistical model and randomization algorithm is provided to stimulate discussion on trial design choices more generally and provide an example for other investigators developing APTs.

CONCLUSION

INSIGhT (NCT02977780) is an ongoing novel biomarker-based, Bayesian APT for patients with newly diagnosed unmethylated GBM. Our goal is to dramatically shorten trial execution timelines while increasing scientific power of results and biomarker discovery using adaptive randomization. We anticipate that trial execution efficiency will also be improved by using the APT format, which allows for the collaborative addition of new experimental arms while retaining the overall trial structure.