Statistical consideration when adding new arms to ongoing clinical trials: the potentials and the caveats

Statistical documentation for multi-disease, multi-domain platform trials: our experience with the Staphylococcus aureus Network Adaptive Platform trial

Platform trials have become widely adopted across multiple disease areas over recent years, however, guidelines for operationalising these trials have not kept pace. We outline a series of documents that summarise the statistical components, and implicit processes, of the Staphylococcus aureus Network Adaptive Platform (SNAP) trial to provide an informal template for other researchers and reviewers of platform trials. We briefly summarise the content and role of the core protocol, statistical appendix, domain-specific appendices, simulation report, statistical implementation guides, data safety and monitoring committee (DSMC) reports, and domain-specific statistical analysis plans and final reports, and a transparent governance structure that ensures separate blinded and unblinded statistical teams. In the absence of guidelines or checklists for platform trial statistical documents, we hope to provide useful guidance to others in terms of what has worked so far for the SNAP trial, stimulate discussion, and inform a future consensus.Trial registration NCT05137119 . Registered on 30 November 2021.

Examining the bias-efficiency tradeoff from incorporation of nonconcurrent controls in platform trials: A simulation study example from the adaptive COVID-19 treatment trial

BACKGROUND

Platform trials typically feature a shared control arm and multiple experimental treatment arms. Staggered entry and exit of arms splits the control group into two cohorts: those randomized during the same period in which the experimental arm was open (concurrent controls) and those randomized outside that period (nonconcurrent controls). Combining these control groups may offer increased statistical power but can lead to bias if analyses do not account for time trends in the response variable. Proposed methods of adjustment for time may increase type I error rates when time trends impact arms unequally or when large, sudden changes to the response rate occur. However, there has been limited exploration of the degree of type I error inflation one can plausibly expect in real-world scenarios.

METHODS

We use data from the Adaptive COVID-19 Treatment Trial (ACTT) to mimic a realistic platform trial with a remdesivir control arm. We compare four strategies for estimating the effect of interferon beta-1a (the ACTT-3 experimental arm) relative to remdesivir (data from ACTT-1, ACTT-2, and ACTT-3) on recovery and death by day 29: utilizing concurrent controls only (the prespecified analysis), pooling all remdesivir arm data without adjustment (the "unadjusted-pooled" analysis), adjusting for time as a categorical variable, and a Bayesian hierarchical model implementation which adjusts for time trends using smoothing techniques (the "Bayesian time machine"). We compare type I error rates and relative efficiency of each method in simulation settings based on observed ACTT remdesivir arm data.

RESULTS

The unadjusted-pooled approach provided substantially different estimates of the effect of interferon beta-1a relative to remdesivir compared with the concurrent-only and model-based approaches, indicating that changes in recovery and death rates over time were not ignorable across different stages of ACTT. The model-based approaches rely on an assumption of constant treatment effects for each arm in the platform relative to control; error rates more than doubled in settings where this was not satisfied. Relative efficiency of the model-based approaches compared with the concurrent-only analysis was moderate.

CONCLUSIONS

In simulation settings where key model assumptions were not met, potential efficiency gains from incorporation of nonconcurrent controls were outweighed by the risk of substantial type I error rate inflation. This leads us to advise against these strategies for primary analyses in confirmatory clinical trials, aligning with current FDA guidance advising against comparisons to nonconcurrent controls in COVID-19 settings. The model-based adjustment methods may be useful in other settings, but we recommend performing the concurrent-only analysis as a reference for assessing the degree to which nonconcurrent controls drive results.

A Preplanned Multi-Stage Platform Trial for Discovering Multiple Superior Treatments With Control of FWER and Power

There is a growing interest in the implementation of platform trials, which provide the flexibility to incorporate new treatment arms during the trial and the ability to halt treatments early based on lack of benefit or observed superiority. In such trials, it can be important to ensure that error rates are controlled. This paper introduces a multi-stage design that enables the addition of new treatment arms, at any point, in a preplanned manner within a platform trial, while still maintaining control over the family-wise error rate. This paper focuses on finding the required sample size to achieve a desired level of statistical power when treatments are continued to be tested even after a superior treatment has already been found. This may be of interest if there are treatments from different sponsors which are also superior to the current control or multiple doses being tested. The calculations to determine the expected sample size is given. A motivating trial is presented in which the sample size of different configurations is studied. In addition, the approach is compared to running multiple separate trials and it is shown that in many scenarios if family-wise error rate control is needed there may not be benefit in using a platform trial when comparing the sample size of the trial.

Analysis of Nonconcurrent Controls in Adaptive Platform Trials: Separating Randomized and Nonrandomized Information

Adaptive platform trials allow treatments to be added or dropped during the study, meaning that the control arm may be active for longer than the experimental arms. This leads to nonconcurrent controls, which provide nonrandomized information that may increase efficiency but may introduce bias from temporal confounding and other factors. Various methods have been proposed to control confounding from nonconcurrent controls, based on adjusting for time period. We demonstrate that time adjustment is insufficient to prevent bias in some circumstances where nonconcurrent controls are present in adaptive platform trials, and we propose a more general analytical framework that accounts for nonconcurrent controls in such circumstances. We begin by defining nonconcurrent controls using the concept of a concurrently randomized cohort, which is a subgroup of participants all subject to the same randomized design. We then use cohort adjustment rather than time adjustment. Due to flexibilities in platform trials, more than one randomized design may be in force at any time, meaning that cohort-adjusted and time-adjusted analyses may be quite different. Using simulation studies, we demonstrate that time-adjusted analyses may be biased while cohort-adjusted analyses remove this bias. We also demonstrate that the cohort-adjusted analysis may be interpreted as a synthesis of randomized and indirect comparisons analogous to mixed treatment comparisons in network meta-analysis. This allows the use of network meta-analysis methodology to separate the randomized and nonrandomized components and to assess their consistency. Whenever nonconcurrent controls are used in platform trials, the separate randomized and indirect contributions to the treatment effect should be presented.

The use of master protocols for efficient trial design to evaluate radiotherapy interventions: a systematic review

The aim of this review was to highlight why the use of master protocols trial design is particularly useful for radiotherapy intervention trials where complex setup pathways (including quality assurance, user training, and integrating multiple modalities of treatment) may hinder clinical advances. We carried out a systematic review according to Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, reviewing the findings using a landscape analysis. Results were summarized descriptively, reporting on trial characteristics highlighting the benefits, limitations, and challenges of developing and implementing radiotherapy master protocols, with three case studies selected to explore these issues in more detail. Twelve studies were suitable for inclusion (4 platform trials, 3 umbrella trials, and 5 basket trials), evaluating a mix of solid tumor sites in both curative and palliative settings. The interventions were categorized into 1) novel agent and radiotherapy combinations; 2) radiotherapy dose personalization; and 3) device evaluation, with a case study provided for each intervention. Benefits of master protocol trials for radiotherapy intervention include protocol efficiency for implementation of novel radiotherapy techniques; accelerating the evaluation of novel agent drug and radiotherapy combinations; and more efficient translational research opportunities, leading to cost savings and research efficiency to improve patient outcomes. Master protocols offer an innovative platform under which multiple clinical questions can be addressed within a single trial. Due to the complexity of radiotherapy trial setup, cost and research efficiency savings may be more apparent than in systemic treatment trials. Use of this research approach may be the change needed to push forward oncological innovation within radiation oncology.

The impact of heterogeneity on the analysis of platform trials with normally distributed outcomes

BACKGROUND

A platform trial approach allows adding arms to on-going trials to speed up intervention discovery programs. A control arm remains open for recruitment in a platform trial while intervention arms may be added after the onset of the study and could be terminated early for efficacy and/or futility when early stopping is allowed. The topic of utilising non-concurrent control data in the analysis of platform trials has been explored and discussed extensively. A less familiar issue is the presence of heterogeneity, which may exist for example due to modification of enrolment criteria and recruitment strategy.

METHOD

We conduct a simulation study to explore the impact of heterogeneity on the analysis of a two-stage platform trial design. We consider heterogeneity in treatment effects and heteroscedasticity in outcome data across stages for a normally distributed endpoint. We examine the performance of some hypothesis testing procedures and modelling strategies. The use of non-concurrent control data is also considered accordingly. Alongside standard regression analysis, we examine the performance of a novel method that was known as the pairwise trials analysis. It is similar to a network meta-analysis approach but adjusts for treatment comparisons instead of individual studies using fixed effects.

RESULTS

Several testing strategies with concurrent control data seem to control the type I error rate at the required level when there is heteroscedasticity in outcome data across stages and/or a random cohort effect. The main parameter of treatment effects in some analysis models correspond to overall treatment effects weighted by stage wise sample sizes; while others correspond to the effect observed within a single stage. The characteristics of the estimates are not affected significantly by the presence of a random cohort effect and/ or heteroscedasticity.

CONCLUSION

In view of heterogeneity in treatment effect across stages, the specification of null hypotheses in platform trials may need to be more subtle. We suggest employing testing procedure of adaptive design as opposed to testing the statistics from regression models; comparing the estimates from the pairwise trials analysis method and the regression model with interaction terms may indicate if heterogeneity is negligible.

Optimal allocation strategies in platform trials with continuous endpoints

Platform trials are randomized clinical trials that allow simultaneous comparison of multiple interventions, usually against a common control. Arms to test experimental interventions may enter and leave the platform over time. This implies that the number of experimental intervention arms in the trial may change as the trial progresses. Determining optimal allocation rates to allocate patients to the treatment and control arms in platform trials is challenging because the optimal allocation depends on the number of arms in the platform and the latter typically varies over time. In addition, the optimal allocation depends on the analysis strategy used and the optimality criteria considered. In this article, we derive optimal treatment allocation rates for platform trials with shared controls, assuming that a stratified estimation and a testing procedure based on a regression model are used to adjust for time trends. We consider both, analysis using concurrent controls only as well as analysis methods using concurrent and non-concurrent controls and assume that the total sample size is fixed. The objective function to be minimized is the maximum of the variances of the effect estimators. We show that the optimal solution depends on the entry time of the arms in the trial and, in general, does not correspond to the square root of k allocation rule used in classical multi-arm trials. We illustrate the optimal allocation and evaluate the power and type 1 error rate compared to trials using one-to-one and square root of k allocations by means of a case study.

Improved efficiency for cross-arm comparisons via platform designs

Though platform trials have been touted for their flexibility and streamlined use of trial resources, their statistical efficiency is not well understood. We fill this gap by establishing their greater efficiency for comparing the relative efficacy of multiple interventions over using several separate, 2-arm trials, where the relative efficacy of an arbitrary pair of interventions is evaluated by contrasting their relative risks as compared to control. In theoretical and numerical studies, we demonstrate that the inference of such a contrast using data from a platform trial enjoys identical or better precision than using data from separate trials, even when the former enrolls substantially fewer participants. This benefit is attributed to the sharing of controls among interventions under contemporaneous randomization. We further provide a novel procedure for establishing the noninferiority of a given intervention relative to the most efficacious of the other interventions under evaluation, where this procedure is adaptive in the sense that it need not be a priori known which of these other interventions is most efficacious. Our numerical studies show that this testing procedure can attain substantially better power when the data arise from a platform trial rather than multiple separate trials. Our results are illustrated using data from two monoclonal antibody trials for the prevention of HIV.

Allocation in platform trials to maintain comparability across time and eligibility

Platform trials, with arms entering and leaving the trial over time, are complex. In addition to trial changes over time, certain arms in a platform may come with patient restrictions. Both of these issues (time and eligibility) can create biases in comparing active arms to control. The largest of these biases, using non-concurrent controls or including control patients that were ineligible for an active arm, have been extensively discussed in the literature. Here we show that even restricting to concurrent, eligible controls can induce biases if proper allocation ratios are not maintained throughout the platform. We also build on results in Ventz et al. Biostat., 19:199-215, 2018 to describe an algorithm that guarantees comparability between active and control groups in arm analyses in both time and eligibility, and allows for both re-randomization of patients and two-stage randomization procedures. The resulting method is both flexible and easily implemented, allowing robust comparisons when assumptions that underlie alternative randomization methods are in doubt.

Estimands and Complex Innovative Designs

Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.

Design and analysis of umbrella trials: Where do we stand?

Background

The efficiencies that master protocol designs can bring to modern drug development have seen their increased utilization in oncology. Growing interest has also resulted in their consideration in non-oncology settings. Umbrella trials are one class of master protocol design that evaluates multiple targeted therapies in a single disease setting. Despite the existence of several reviews of master protocols, the statistical considerations of umbrella trials have received more limited attention.

Methods

We conduct a systematic review of the literature on umbrella trials, examining both the statistical methods that are available for their design and analysis, and also their use in practice. We pay particular attention to considerations for umbrella designs applied outside of oncology.

Findings

We identified 38 umbrella trials. To date, most umbrella trials have been conducted in early phase settings (73.7%, 28/38) and in oncology (92.1%, 35/38). The quality of statistical information available about conducted umbrella trials to date is poor; for example, it was impossible to ascertain how sample size was determined in the majority of trials (55.3%, 21/38). The literature on statistical methods for umbrella trials is currently sparse.

Conclusions

Umbrella trials have potentially great utility to expedite drug development, including outside of oncology. However, to enable lessons to be effectively learned from early use of such designs, there is a need for higher-quality reporting of umbrella trials. Furthermore, if the potential of umbrella trials is to be realized, further methodological research is required.

On model-based time trend adjustments in platform trials with non-concurrent controls

BACKGROUND

Platform trials can evaluate the efficacy of several experimental treatments compared to a control. The number of experimental treatments is not fixed, as arms may be added or removed as the trial progresses. Platform trials are more efficient than independent parallel group trials because of using shared control groups. However, for a treatment entering the trial at a later time point, the control group is divided into concurrent controls, consisting of patients randomised to control when that treatment arm is in the platform, and non-concurrent controls, patients randomised before. Using non-concurrent controls in addition to concurrent controls can improve the trial's efficiency by increasing power and reducing the required sample size, but can introduce bias due to time trends.

METHODS

We focus on a platform trial with two treatment arms and a common control arm. Assuming that the second treatment arm is added at a later time, we assess the robustness of recently proposed model-based approaches to adjust for time trends when utilizing non-concurrent controls. In particular, we consider approaches where time trends are modeled either as linear in time or as a step function, with steps at time points where treatments enter or leave the platform trial. For trials with continuous or binary outcomes, we investigate the type 1 error rate and power of testing the efficacy of the newly added arm, as well as the bias and root mean squared error of treatment effect estimates under a range of scenarios. In addition to scenarios where time trends are equal across arms, we investigate settings with different time trends or time trends that are not additive in the scale of the model.

RESULTS

A step function model, fitted on data from all treatment arms, gives increased power while controlling the type 1 error, as long as the time trends are equal for the different arms and additive on the model scale. This holds even if the shape of the time trend deviates from a step function when patients are allocated to arms by block randomisation. However, if time trends differ between arms or are not additive to treatment effects in the scale of the model, the type 1 error rate may be inflated.

CONCLUSIONS

The efficiency gained by using step function models to incorporate non-concurrent controls can outweigh potential risks of biases, especially in settings with small sample sizes. Such biases may arise if the model assumptions of equality and additivity of time trends are not satisfied. However, the specifics of the trial, scientific plausibility of different time trends, and robustness of results should be carefully considered.

Resting state electroencephalographic rhythms are affected by immediately preceding memory demands in cognitively unimpaired elderly and patients with mild cognitive impairment

Experiments on event-related electroencephalographic oscillations in aged people typically include blocks of cognitive tasks with a few minutes of interval between them. The present exploratory study tested the effect of being engaged on cognitive tasks over the resting state cortical arousal after task completion, and whether it differs according to the level of the participant’s cognitive decline. To investigate this issue, we used a local database including data in 30 healthy cognitively unimpaired (CU) persons and 40 matched patients with amnestic mild cognitive impairment (aMCI). They had been involved in 2 memory tasks for about 40 min and underwent resting-state electroencephalographic (rsEEG) recording after 5 min from the task end. eLORETA freeware estimated rsEEG alpha source activity as an index of general cortical arousal. In the CU but not aMCI group, there was a negative correlation between memory tasks performance and posterior rsEEG alpha source activity. The better the memory tasks performance, the lower the posterior alpha activity (i.e., higher cortical arousal). There was also a negative correlation between neuropsychological test scores of global cognitive status and alpha source activity. These results suggest that engagement in memory tasks may perturb background brain arousal for more than 5 min after the tasks end, and that this effect are dependent on participants global cognitive status. Future studies in CU and aMCI groups may cross-validate and extend these results with experiments including (1) rsEEG recordings before memory tasks and (2) post-tasks rsEEG recordings after 5, 15, and 30 min.

Combining factorial and multi-arm multi-stage platform designs to evaluate multiple interventions efficiently

BACKGROUND

Factorial designs and multi-arm multi-stage (MAMS) platform designs have many advantages, but the practical advantages and disadvantages of combining the two designs have not been explored.

METHODS

We propose practical methods for a combined design within the platform trial paradigm where some interventions are not expected to interact and could be given together.

RESULTS

We describe the combined design and suggest diagrams that can be used to represent it. Many properties are common both to standard factorial designs, including the need to consider interactions between interventions and the impact of intervention efficacy on power of other comparisons, and to standard multi-arm multi-stage designs, including the need to pre-specify procedures for starting and stopping intervention comparisons. We also identify some specific features of the factorial-MAMS design: timing of interim and final analyses should be determined by calendar time or total observed events; some non-factorial modifications may be useful; eligibility criteria should be broad enough to include any patient eligible for any part of the randomisation; stratified randomisation may conveniently be performed sequentially; and analysis requires special care to use only concurrent controls.

CONCLUSION

A combined factorial-MAMS design can combine the efficiencies of factorial trials and multi-arm multi-stage platform trials. It allows us to address multiple research questions under one protocol and to test multiple new treatment options, which is particularly important when facing a new emergent infection such as COVID-19.

Uptake of the multi-arm multi-stage (MAMS) adaptive platform approach: a trial-registry review of late-phase randomised clinical trials

BACKGROUND

For medical conditions with numerous interventions worthy of investigation, there are many advantages of a multi-arm multi-stage (MAMS) platform trial approach. However, there is currently limited knowledge on uptake of the MAMS design, especially in the late-phase setting. We sought to examine uptake and characteristics of late-phase MAMS platform trials, to enable better planning for teams considering future use of this approach.

DESIGN

We examined uptake of registered, late-phase MAMS platforms in the EU clinical trials register, Australian New Zealand Clinical Trials Registry, International Standard Randomised Controlled Trial Number registry, Pan African Clinical Trials Registry, WHO International Clinical Trial Registry Platform and databases: PubMed, Medline, Cochrane Library, Global Health Library and EMBASE. Searching was performed and review data frozen on 1 April 2021. MAMS platforms were defined as requiring two or more comparison arms, with two or more trial stages, with an interim analysis allowing for stopping of recruitment to arms and typically the ability to add new intervention arms.

RESULTS

62 late-phase clinical trials using an MAMS approach were included. Overall, the number of late-phase trials using the MAMS design has been increasing since 2001 and been accelerated by COVID-19. The majority of current MAMS platforms were either targeting infectious diseases (52%) or cancers (29%) and all identified trials were for treatment interventions. 89% (55/62) of MAMS platforms were evaluating medications, with 45% (28/62) of the MAMS platforms having at least one or more repurposed medication as a comparison arm.

CONCLUSIONS

Historically, late-phase trials have adhered to long-established standard (two-arm) designs. However, the number of late-phase MAMS platform trials is increasing, across a range of different disease areas. This study highlights the potential scope of MAMS platform trials and may assist research teams considering use of this approach in the late-phase randomised clinical trial setting.

PROSPERO REGISTRATION NUMBER

CRD42019153910.

The design of a Bayesian adaptive clinical trial of tranexamic acid in severely injured children

BACKGROUND

Trauma is the leading cause of death and disability in children in the USA. Tranexamic acid (TXA) reduces the blood transfusion requirements in adults and children during surgery. Several studies have evaluated TXA in adults with hemorrhagic trauma, but no randomized controlled trials have occurred in children with trauma. We propose a Bayesian adaptive clinical trial to investigate TXA in children with brain and/or torso hemorrhagic trauma.

METHODS/DESIGN

We designed a double-blind, Bayesian adaptive clinical trial that will enroll up to 2000 patients. We extend the traditional Emax dose-response model to incorporate a hierarchical structure so multiple doses of TXA can be evaluated in different injury populations (isolated head injury, isolated torso injury, or both head and torso injury). Up to 3 doses of TXA (15 mg/kg, 30 mg/kg, and 45 mg/kg bolus doses) will be compared to placebo. Equal allocation between placebo, 15 mg/kg, and 30 mg/kg will be used for an initial period within each injury group. Depending on the dose-response curve, the 45 mg/kg arm may open in an injury group if there is a trend towards increasing efficacy based on the observed relationship using the data from the lower doses. Response-adaptive randomization allows each injury group to differ in allocation proportions of TXA so an optimal dose can be identified for each injury group. Frequent interim stopping periods are included to evaluate efficacy and futility. The statistical design is evaluated through extensive simulations to determine the operating characteristics in several plausible scenarios. This trial achieves adequate power in each injury group.

DISCUSSION

This trial design evaluating TXA in pediatric hemorrhagic trauma allows for three separate injury populations to be analyzed and compared within a single study framework. Individual conclusions regarding optimal dosing of TXA can be made within each injury group. Identifying the optimal dose of TXA, if any, for various injury types in childhood may reduce death and disability.

Moving forward in clinical research with master protocols

With billions of dollars in research and development (R&D) funding continuing to be invested, the novel coronavirus disease 2019 (COVID-19) has become into a singular focus for the scientific community. However, the collective response from the scientific communities have seen poor return on investment, particularly for therapeutic research for COVID-19, revealing the existing weaknesses and inefficiencies of the clinical trial enterprise. In this article, we argue for the importance of structural changes to existing research programs for clinical trials in light of the lessons learned from COVID-19.