Testing and estimation in flexible group sequential designs with adaptive treatment selection

SaLudo: a randomized phase IIb/III study of lurbinectedin plus doxorubicin as first-line treatment in leiomyosarcoma

Previous phase I/II trials indicate promising activity of lurbinectedin plus doxorubicin (DOX) in leiomyosarcoma (LMS). We describe here the rationale and design of SaLuDo, an open label, randomized, multicenter, seamless phase IIb/III study to evaluate the antitumor activity and safety of lurbinectedin plus DOX versus DOX alone in the first-line setting of metastatic LMS. The phase IIb stage will evaluate two schedules of the combination for the phase III stage given every 3 weeks (q3wk): DOX 50 mg/m2 plus lurbinectedin 2.2 mg/m2, and DOX 25 mg/m2 plus lurbinectedin 3.2 mg/m2. The control arm will be DOX 75 mg/m2 q3wk. The primary endpoint is progression-free survival by independent review; overall survival is the key secondary endpoint. Clinical trial registration: www.clinicaltrials.gov identifier is NCT06088290.

Examining the bias-efficiency tradeoff from incorporation of nonconcurrent controls in platform trials: A simulation study example from the adaptive COVID-19 treatment trial

BACKGROUND

Platform trials typically feature a shared control arm and multiple experimental treatment arms. Staggered entry and exit of arms splits the control group into two cohorts: those randomized during the same period in which the experimental arm was open (concurrent controls) and those randomized outside that period (nonconcurrent controls). Combining these control groups may offer increased statistical power but can lead to bias if analyses do not account for time trends in the response variable. Proposed methods of adjustment for time may increase type I error rates when time trends impact arms unequally or when large, sudden changes to the response rate occur. However, there has been limited exploration of the degree of type I error inflation one can plausibly expect in real-world scenarios.

METHODS

We use data from the Adaptive COVID-19 Treatment Trial (ACTT) to mimic a realistic platform trial with a remdesivir control arm. We compare four strategies for estimating the effect of interferon beta-1a (the ACTT-3 experimental arm) relative to remdesivir (data from ACTT-1, ACTT-2, and ACTT-3) on recovery and death by day 29: utilizing concurrent controls only (the prespecified analysis), pooling all remdesivir arm data without adjustment (the "unadjusted-pooled" analysis), adjusting for time as a categorical variable, and a Bayesian hierarchical model implementation which adjusts for time trends using smoothing techniques (the "Bayesian time machine"). We compare type I error rates and relative efficiency of each method in simulation settings based on observed ACTT remdesivir arm data.

RESULTS

The unadjusted-pooled approach provided substantially different estimates of the effect of interferon beta-1a relative to remdesivir compared with the concurrent-only and model-based approaches, indicating that changes in recovery and death rates over time were not ignorable across different stages of ACTT. The model-based approaches rely on an assumption of constant treatment effects for each arm in the platform relative to control; error rates more than doubled in settings where this was not satisfied. Relative efficiency of the model-based approaches compared with the concurrent-only analysis was moderate.

CONCLUSIONS

In simulation settings where key model assumptions were not met, potential efficiency gains from incorporation of nonconcurrent controls were outweighed by the risk of substantial type I error rate inflation. This leads us to advise against these strategies for primary analyses in confirmatory clinical trials, aligning with current FDA guidance advising against comparisons to nonconcurrent controls in COVID-19 settings. The model-based adjustment methods may be useful in other settings, but we recommend performing the concurrent-only analysis as a reference for assessing the degree to which nonconcurrent controls drive results.

Optimal sample size division in two-stage seamless designs

Inferentially seamless 2/3 designs are increasingly popular in clinical trials. It is important to understand their relative advantages compared with separate phase 2 and phase 3 trials, and to understand the consequences of design choices such as the proportion of patients included in the phase 2 portion of the design. Extending previous work in this area, we perform a simulation study across multiple numbers of arms and efficacy response curves. We consider a design space crossing the choice of a separate versus seamless design with the choice of allocating 0%-100% of available patients in phase 2, with the remainder in phase 3. The seamless designs achieve greater power than their separate trial counterparts. Importantly, the optimal seamless design is more robust than the optimal separate program, meaning that one range of values for the proportion of patients used in phase 2 (30%-50% of the total phase 2/3 sample size) is nearly optimal for a wide range of response scenarios. In contrast, a percentage of patients used in phase 2 for separate trials may be optimal for some alternative scenarios but decidedly inferior for other alternative scenarios. When operationally and scientifically viable, seamless trials provide superior performance compared with separate phase 2 and phase 3 trials. The results also provide guidance for the implementation of these trials in practice.

Adding experimental treatment arms to multi-arm multi-stage platform trials in progress

Multi-arm multi-stage (MAMS) platform trials efficiently compare several treatments with a common control arm. Crucially MAMS designs allow for adjustment for multiplicity if required. If for example, the active treatment arms in a clinical trial relate to different dose levels or different routes of administration of a drug, the strict control of the family-wise error rate (FWER) is paramount. Suppose a further treatment becomes available, it is desirable to add this to the trial already in progress; to access both the practical and statistical benefits of the MAMS design. In any setting where control of the error rate is required, we must add corresponding hypotheses without compromising the validity of the testing procedure.To strongly control the FWER, MAMS designs use pre-planned decision rules that determine the recruitment of the next stage of the trial based on the available data. The addition of a treatment arm presents an unplanned change to the design that we must account for in the testing procedure. We demonstrate the use of the conditional error approach to add hypotheses to any testing procedure that strongly controls the FWER. We use this framework to add treatments to a MAMS trial in progress. Simulations illustrate the possible characteristics of such procedures.

Strategies for successful dose optimization in oncology drug development: a practical guide

Dose optimization is a critical challenge in drug development. Historically, dose determination in oncology has followed a divergent path from other non-oncology therapeutic areas due to the unique characteristics and requirements in Oncology. However, with the emergence of new drug modalities and mechanisms of drugs in oncology, such as immune therapies, radiopharmaceuticals, targeted therapies, cytostatic agents, and others, the dose-response relationship for efficacy and toxicity could be vastly varied compared to the cytotoxic chemotherapies. The doses below the MTD may demonstrate similar efficacy to the MTD with an improved tolerability profile, resembling what is commonly observed in non-oncology treatments. Hence, alternate strategies for dose optimization are required for new modalities in oncology drug development. This paper delves into the historical evolution of dose finding methods from non-oncology to oncology, highlighting examples and summarizing the underlying drivers of change. Subsequently, a practical framework and guidance are provided to illustrate how dose optimization can be incorporated into various stages of the development program. We provide the following general recommendations: 1) The objective for phase I is to identify a dose range rather than a single MTD dose for subsequent development to better characterize the safety and tolerability profile within the dose range. 2) At least two doses separable by PK are recommended for dose optimization in phase II. 3) Ideally, dose optimization should be performed before launching the confirmatory study. Nevertheless, innovative designs such as seamless II/III design can be implemented for dose selection and may accelerate the drug development program.

The impact of heterogeneity on the analysis of platform trials with normally distributed outcomes

BACKGROUND

A platform trial approach allows adding arms to on-going trials to speed up intervention discovery programs. A control arm remains open for recruitment in a platform trial while intervention arms may be added after the onset of the study and could be terminated early for efficacy and/or futility when early stopping is allowed. The topic of utilising non-concurrent control data in the analysis of platform trials has been explored and discussed extensively. A less familiar issue is the presence of heterogeneity, which may exist for example due to modification of enrolment criteria and recruitment strategy.

METHOD

We conduct a simulation study to explore the impact of heterogeneity on the analysis of a two-stage platform trial design. We consider heterogeneity in treatment effects and heteroscedasticity in outcome data across stages for a normally distributed endpoint. We examine the performance of some hypothesis testing procedures and modelling strategies. The use of non-concurrent control data is also considered accordingly. Alongside standard regression analysis, we examine the performance of a novel method that was known as the pairwise trials analysis. It is similar to a network meta-analysis approach but adjusts for treatment comparisons instead of individual studies using fixed effects.

RESULTS

Several testing strategies with concurrent control data seem to control the type I error rate at the required level when there is heteroscedasticity in outcome data across stages and/or a random cohort effect. The main parameter of treatment effects in some analysis models correspond to overall treatment effects weighted by stage wise sample sizes; while others correspond to the effect observed within a single stage. The characteristics of the estimates are not affected significantly by the presence of a random cohort effect and/ or heteroscedasticity.

CONCLUSION

In view of heterogeneity in treatment effect across stages, the specification of null hypotheses in platform trials may need to be more subtle. We suggest employing testing procedure of adaptive design as opposed to testing the statistics from regression models; comparing the estimates from the pairwise trials analysis method and the regression model with interaction terms may indicate if heterogeneity is negligible.

Conditional bias adjusted estimator of treatment effect in 2-in-1 adaptive design

For implementation of adaptive design, the adjustment of bias in treatment effect estimation becomes an increasingly important topic in recent years. While adaptive design literature traditionally focuses on the control of type I error rate and the adjustment of overall unconditional bias, the research on adjusting conditional bias has been limited. This paper proposes a conditional bias adjustment estimator of treatment effect under the context of 2-in-1 adaptive design and aims to provide a comprehensive investigation on their statistical properties including bias, mean squared error and coverage probability of confidence intervals. It demonstrated that conditional bias adjusted estimators greatly reduce the conditional bias and have similarly negligible unconditional bias compared with mean and median (unconditional) unbiased estimators. In addition, the test statistics is constructed based on the conditional bias adjustment estimators and compared with the naive unadjusted test.

Seamless phase II/III design: a useful strategy to reduce the sample size for dose optimization

BACKGROUND

The traditional more-is-better dose selection paradigm, originally developed for cytotoxic chemotherapeutics, can be problematic when applied to the development of novel molecularly targeted agents. Recognizing this issue, the US Food and Drug Administration initiated Project Optimus to reform the dose optimization and selection paradigm in oncology drug development, emphasizing the need for greater attention to benefit-risk considerations.

METHODS

We identify different types of phase II/III dose-optimization designs, classified according to trial objectives and endpoint types. Through computer simulations, we examine their operating characteristics and discuss the relevant statistical and design considerations for effective dose optimization.

RESULTS

Phase II/III dose-optimization designs are capable of controlling family-wise type I error rates and achieving appropriate statistical power with substantially smaller sample sizes than the conventional approach while also reducing the number of patients who experience toxicity. Depending on the design and scenario, the sample size savings range from 16.6% to 27.3%, with a mean savings of 22.1%.

CONCLUSIONS

Phase II/III dose-optimization designs offer an efficient way to reduce sample sizes for dose optimization and accelerate the development of targeted agents. However, because of interim dose selection, the phase II/III dose-optimization design presents logistical and operational challenges and requires careful planning and implementation to ensure trial integrity.

Adjusting for treatment selection in phase II/III clinical trials with time to event data

Phase II/III clinical trials are efficient two-stage designs that test multiple experimental treatments. In stage 1, patients are allocated to the control and all experimental treatments, with the data collected from them used to select experimental treatments to continue to stage 2. Patients recruited in stage 2 are allocated to the selected treatments and the control. Combined data of stage 1 and stage 2 are used for a confirmatory phase III analysis. Appropriate analysis needs to adjust for selection bias of the stage 1 data. Point estimators exist for normally distributed outcome data. Extending these estimators to time to event data is not straightforward because treatment selection is based on correlated treatment effects and stage 1 patients who do not get events in stage 1 are followed-up in stage 2. We have derived an approximately uniformly minimum variance conditional unbiased estimator (UMVCUE) and compared its biases and mean squared errors to existing bias adjusted estimators. In simulations, one existing bias adjusted estimator has similar properties as the practically unbiased UMVCUE while the others can have noticeable biases but they are less variable than the UMVCUE. For confirmatory phase II/III clinical trials where unbiased estimators are desired, we recommend the UMVCUE or the existing estimator with which it has similar properties.

Estimands and Complex Innovative Designs

Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.

The benefits of covariate adjustment for adaptive multi-arm designs

Covariate adjustment via a regression approach is known to increase the precision of statistical inference when fixed trial designs are employed in randomized controlled studies. When an adaptive multi-arm design is employed with the ability to select treatments, it is unclear how covariate adjustment affects various aspects of the study. Consider the design framework that relies on pre-specified treatment selection rule(s) and a combination test approach for hypothesis testing. It is our primary goal to evaluate the impact of covariate adjustment on adaptive multi-arm designs with treatment selection. Our secondary goal is to show how the Uniformly Minimum Variance Conditionally Unbiased Estimator can be extended to account for covariate adjustment analytically. We find that adjustment with different sets of covariates can lead to different treatment selection outcomes and hence probabilities of rejecting hypotheses. Nevertheless, we do not see any negative impact on the control of the familywise error rate when covariates are included in the analysis model. When adjusting for covariates that are moderately or highly correlated with the outcome, we see various benefits to the analysis of the design. Conversely, there is negligible impact when including covariates that are uncorrelated with the outcome. Overall, pre-specification of covariate adjustment is recommended for the analysis of adaptive multi-arm design with treatment selection. Having the statistical analysis plan in place prior to the interim and final analyses is crucial, especially when a non-collapsible measure of treatment effect is considered in the trial.

A simulation-based comparison of estimation methods for adaptive and classical group sequential clinical trials

Statistical methods for controlling the type-I error of hypothesis tests in adaptive group sequential clinical trials are well established and well understood. However, methods for obtaining statistically valid point estimates and confidence intervals for adaptive designs are not as well established or as well understood. At the end of an adaptive trial, one may calculate the repeated confidence interval (RCI), which provides conservative coverage of δ , or the backward image confidence interval (BWCI), which provides exact coverage of δ and is an extension of the stagewise adjusted confidence interval (SWCI, used in classical group sequential designs). The BWCI can also provide a median unbiased estimate (MUE) of δ . There is a need to better understand the coverage and possible biases associated with these methods. We conducted a simulation study exploring parameter estimation following sample size reestimation based on testing methods with strong control of type-I error. Generally, the BWCI provided exact coverage, the naïve CI provided inconsistent coverage, and the RCI provided conservative coverage. Additionally, we note considerable asymmetry in the coverage from above/from below for the RCI, although we did not see any instance where the 95% RCI excluded the true parameter more than 2.5% on either side. At the end of an adaptive group sequential trial, we strongly recommend the use of the BWCI (and associated MUE), with the RCI computed during interim looks; the naïve CI should be avoided. These results and conclusions also hold true for classical group sequential designs.

Design of the National Adaptive Trial for PTSD-related Insomnia (NAP Study), VA Cooperative Study Program (CSP) #2016

There are currently no validated pharmacotherapies for posttraumatic stress disorder (PTSD)-related insomnia. The purpose of the National Adaptive Trial for PTSD-Related Insomnia (NAP Study) is to efficiently compare to placebo the effects of three insomnia medications with different mechanisms of action that are already prescribed widely to veterans diagnosed with PTSD within U.S. Department of Veterans Affairs (VA) Medical Centers. This study plans to enroll 1224 patients from 34 VA Medical Centers into a 12- week prospective, randomized placebo-controlled clinical trial comparing trazodone, eszopiclone, and gabapentin. The primary outcome measure is insomnia, assessed with the Insomnia Severity Index. A novel aspect of this study is its adaptive design. At the recruitment midpoint, an interim analysis will be conducted to inform a decision to close recruitment to any "futile" arms (i.e. arms where further recruitment is very unlikely to yield a significant result) while maintaining the overall study recruitment target. This step could result in the enrichment of the remaining study arms, enhancing statistical power for the remaining comparisons to placebo. This study will also explore clinical, actigraphic, and biochemical predictors of treatment response that may guide future biomarker development. Lastly, due to the COVID-19 pandemic, this study will allow the consenting process and follow-up visits to be conducted via video or phone contact if in-person meetings are not possible. Overall, this study aims to identify at least one effective pharmacotherapy for PTSD-related insomnia, and, perhaps, to generate definitive negative data to reduce the use of ineffective insomnia medications. NATIONAL CLINICAL TRIAL (NCT) IDENTIFIED NUMBER: NCT03668041.

Collaborative Platform Trials to Fight COVID-19: Methodological and Regulatory Considerations for a Better Societal Outcome

For the development of coronavirus disease 2019 (COVID-19) drugs during the ongoing pandemic, speed is of essence whereas quality of evidence is of paramount importance. Although thousands of COVID-19 trials were rapidly started, many are unlikely to provide robust statistical evidence and meet regulatory standards (e.g., because of lack of randomization or insufficient power). This has led to an inefficient use of time and resources. With more coordination, the sheer number of patients in these trials might have generated convincing data for several investigational treatments. Collaborative platform trials, comparing several drugs to a shared control arm, are an attractive solution. Those trials can utilize a variety of adaptive design features in order to accelerate the finding of life-saving treatments. In this paper, we discuss several possible designs, illustrate them via simulations, and also discuss challenges, such as the heterogeneity of the target population, time-varying standard of care, and the potentially high number of false hypothesis rejections in phase II and phase III trials. We provide corresponding regulatory perspectives on approval and reimbursement, and note that the optimal design of a platform trial will differ with our societal objective and by stakeholder. Hasty approvals may delay the development of better alternatives, whereas searching relentlessly for the single most efficacious treatment may indirectly diminish the number of lives saved as time is lost. We point out the need for incentivizing developers to participate in collaborative evidence-generation initiatives when a positive return on investment is not met.

Design and analysis of drop-the-losers studies using binary endpoints in the rare disease setting

The drop-the-losers design combines a phase 2 trial of k treatments and a confirmatory phase 3 trial under a single adaptive protocol, thereby gaining efficiency over a traditional clinical development approach. Such designs may be particularly useful in the rare disease setting, where conserving sample size is paramount, and control arms may not be feasible. We propose an unconditional exact likelihood (UEL) testing and inference procedure for these designs for a binary endpoint using small sample sizes, comparing its operating characteristics to existing methods. Additional practical considerations are evaluated, including the choice of stagewise sample sizes and effect of ties.

Optimizing subgroup selection in two-stage adaptive enrichment and umbrella designs

We design two‐stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision‐theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per‐comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.

Systematic review of available software for multi-arm multi-stage and platform clinical trial design

BACKGROUND

In recent years, the popularity of multi-arm multi-stage, seamless adaptive, and platform trials has increased. However, many design-related questions and questions regarding which operating characteristics should be evaluated to determine the potential performance of a specific trial design remain and are often further complicated by the complexity of such trial designs.

METHODS

A systematic search was conducted to review existing software for the design of platform trials, whereby multi-arm multi-stage trials were also included. The results of this search are reported both on the literature level and the software level, highlighting the software judged to be particularly useful.

RESULTS

In recent years, many highly specialized software packages targeting single design elements on platform studies have been released. Only a few of the developed software packages provide extensive design flexibility, at the cost of limited access due to being commercial or not being usable as out-of-the-box solutions.

CONCLUSIONS

We believe that both an open-source modular software similar to OCTOPUS and a collaborative effort will be necessary to create software that takes advantage of and investigates the impact of all the flexibility that platform trials potentially provide.

An automation-based adaptive seamless design for dose selection and confirmation with improved power and efficiency

In a drug development program, the efficacy and safety of multiple doses can be evaluated in patients through a phase 2b dose ranging study. With a demonstrated dose response in the trial, promising doses are identified. Their effectiveness then is further investigated and confirmed in phase 3 studies. Although this two-step approach serves the purpose of the program, in general, it is inefficient because of its prolonged development duration and the exclusion of the phase 2b data in the final efficacy evaluation and confirmation which are only based on phase 3 data. To address the issue, we propose a new adaptive design, which seamlessly integrates the dose finding and confirmation steps under one pivotal study. Unlike existing adaptive seamless phase 2b/3 designs, the proposed design combines the response adaptive randomization, sample size modification, and multiple testing techniques to achieve better efficiency. The design can be easily implemented through an automated randomization process. At the end, a number of targeted doses are selected and their effectiveness is confirmed with guaranteed control of family-wise error rate.

Adaptive design implementation in confirmatory trials: methods, practical considerations and case studies

The rapidly changing drug development landscapes have brought unique challenges to sponsors in designing clinical trials in a faster and more efficient way. With the ability to accelerate development timeline, reduce redundant sample size, and select the right dose and patient population during the clinical trial, adaptive designs help to increase the probability of success of clinical trials and eventually contribute to bringing the promising drugs to patients earlier and fulfilling their unmet medical needs. Although extensive adaptive design methods have been proposed in recent years, a comprehensive review of how to implement adaptive design in the practical confirmatory trials is still lacking. In this paper, we will review the evolving history of adaptive designs, updates of newly released regulatory guidance and emerging practical adaptive designs, including but not limited to sample size re-estimation, seamless design and surrogate endpoint used in the interim analysis. Furthermore, we will discuss the current practice of adaptive design implementation by demonstrating a complex oncology seamless phase 2/3 adaptive design case study. Through this example, we will introduce the critical roles of each cross disciplinary function, communication process and important documents when adaptive designs are implemented in real-world setting.

The Evolution of Master Protocol Clinical Trial Designs: A Systematic Literature Review

PURPOSE

Recent years have seen a change in the way that clinical trials are being conducted. There has been a rise of designs more flexible than traditional adaptive and group sequential trials which allow the investigation of multiple substudies with possibly different objectives, interventions, and subgroups conducted within an overall trial structure, summarized by the term master protocol. This review aims to identify existing master protocol studies and summarize their characteristics. The review also identifies articles relevant to the design of master protocol trials, such as proposed trial designs and related methods.

METHODS

We conducted a comprehensive systematic search to review current literature on master protocol trials from a design and analysis perspective, focusing on platform trials and considering basket and umbrella trials. Articles were included regardless of statistical complexity and classified as reviews related to planned or conducted trials, trial designs, or statistical methods. The results of the literature search are reported, and some features of the identified articles are summarized.

FINDINGS

Most of the trials using master protocols were designed as single-arm (n = 29/50), Phase II trials (n = 32/50) in oncology (n = 42/50) using a binary endpoint (n = 26/50) and frequentist decision rules (n = 37/50). We observed an exponential increase in publications in this domain during the last few years in both planned and conducted trials, as well as relevant methods, which we assume has not yet reached its peak. Although many operational and statistical challenges associated with such trials remain, the general consensus seems to be that master protocols provide potentially enormous advantages in efficiency and flexibility of clinical drug development.

IMPLICATIONS

Master protocol trials and especially platform trials have the potential to revolutionize clinical drug development if the methodologic and operational challenges can be overcome.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

A model-based approach for simulating adaptive clinical studies with surrogate endpoints used for interim decision-making

In clinical trials, when exploring multiple dose groups to establish efficacy and safety on one or more selected doses, adaptive designs with interim dose selection are often used for dropping less effective dose groups. When it takes a long time to observe primary outcomes, utilizing information on a surrogate endpoint available at an earlier interim may be preferred for selecting which dose to continue. We propose a Bayesian model-based approach where historical data can be leveraged to incorporate a correlation model for investigating the design's operating characteristics. Simulation studies were conducted and the method can be readily applied for power and sample size calculations.

Seamless phase 2/3 oncology trial design with flexible sample size determination

Conventional seamless phase 2/3 design with fixed sample size determination (SSD) has gained its popularity in oncology drug development due to attractive features such as significantly shortening the development timeline, minimizing sample size, as well as early decision making. However, this design is not immune to inaccurate treatment effect assumption when only limited efficacy data are available at study design stage. We propose an innovative seamless phase 2/3 study design with flexible SSD for oncology trials, in which the trial is designed under a distribution of treatment effect instead of one single assumption due to huge uncertainty of treatment effect at design stage and the sample size for end of phase 3 analysis is not predetermined at design stage, but rather dynamically determined based on observed treatment effect at phase 2 portion. Some practical sample size determination rules for end of phase 3 analysis will be discussed. The proposed design can lead to reduced sample size or/and improved power compared with conventional seamless phase 2/3 design with fixed SSD. This innovative study design can be especially useful for programs with aggressive development strategy to expedite the process in delivering efficacious treatment to patients.

Adaptive multiarm multistage clinical trials

Two methods for designing adaptive multiarm multistage (MAMS) clinical trials, originating from conceptually different group sequential frameworks are presented, and their operating characteristics are compared. In both methods pairwise comparisons are made, stage-by-stage, between each treatment arm and a common control arm with the goal of identifying active treatments and dropping inactive ones. At any stage one may alter the future course of the trial through adaptive changes to the prespecified decision rules for treatment selection and sample size reestimation, and notwithstanding such changes, both methods guarantee strong control of the family-wise error rate. The stage-wise MAMS approach was historically the first to be developed and remains the standard method for designing inferentially seamless phase 2-3 clinical trials. In this approach, at each stage, the data from each treatment comparison are summarized by a single multiplicity adjusted P-value. These stage-wise P-values are combined by a prespecified combination function and the resultant test statistic is monitored with respect to the classical two-arm group sequential efficacy boundaries. The cumulative MAMS approach is a more recent development in which a separate test statistic is constructed for each treatment comparison from the cumulative data at each stage. These statistics are then monitored with respect to multiplicity adjusted group sequential efficacy boundaries. We compared the powers of the two methods for designs with two and three active treatment arms, under commonly utilized decision rules for treatment selection, sample size reestimation and early stopping. In our investigations, which were carried out over a reasonably exhaustive exploration of the parameter space, the cumulative MAMS designs were more powerful than the stage-wise MAMS designs, except for the homogeneous case of equal treatment effects, where a small power advantage was discernable for the stage-wise MAMS designs.

Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods, simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so‐called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked‐up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.

Response adaptive randomization procedures in seamless phase II/III clinical trials

It is desirable to work efficiently and cost effectively to evaluate new therapies in a time-sensitive and ethical manner without compromising the integrity and validity of the development process. The seamless phase II/III clinical trial has been proposed to meet this need, and its efficient, ethical and economic advantages can be strengthened by its combination with innovative response adaptive randomization (RAR) procedures. In particular, well-designed frequentist RAR procedures can target theoretically optimal allocation proportions, and there are explicit asymptotic results. However, there has been little research into seamless phase II/III clinical trials with frequentist RAR because of the difficulty in performing valid statistical inference and controlling the type I error rate. In this paper, we propose the framework for a family of frequentist RAR designs for seamless phase II/III trials, derive the asymptotic distribution of the parameter estimators using martingale processes and offer solutions to control the type I error rate. The numerical studies demonstrate our theoretical findings and the advantages of the proposed methods.

Group sequential adaptive designs in series of time-to-event randomised trials in rare diseases: A simulation study

In rare diseases, fully powered large trials may not be doable in a reasonable time frame even with international collaborations. In a previous work, we proposed an approach based on a series of smaller parallel group two-arm randomised controlled trials (RCT) performed over a long research horizon. Within the series of trials, the treatment selected after each trial becomes the control treatment of the next one. We concluded that running more trials with smaller sample sizes and relaxed α-levels leads in the long term and under reasonable assumptions to larger survival benefits with a moderate increase of risk as compared to traditional designs based on larger but fewer trials designed to meet stringent evidence criteria. We now extend this quantitative framework with more 'flexible' designs including interim analyses for futility and/or efficacy, and three-arm adaptive designs with treatment selection at interim. In the simulation study, we considered different disease severities, accrual rates, and hypotheses of how treatments improve over time. For each design, we estimated the long-term survival benefit as the relative difference in hazard rates between the end and the start of the research horizon, and the risk defined as the probability of selecting at the end of the research horizon a treatment inferior to the initial control. We assessed the impact of the α-level and the choice of the stopping rule on the operating characteristics. We also compared the performance of series based on two- vs. three-arm trials. We show that relaxing α-levels within the limit of 0.1 is associated with larger survival gains and moderate increase of risk which remains within acceptable ranges. Including an interim analysis with a futility rule is associated with an additional survival gain and a better risk control as compared to series with no interim analysis, when the α-level is below or equal to 0.1, whereas the benefit of including an interim analysis is rather small for higher α-levels. Including an interim analysis for efficacy yields almost no additional gain. Series based on three-arm trials are associated with a systematic improvement in terms of survival gain and risk control as compared to series of two-arm trials.

Comparing the MAMS framework with the combination method in multi-arm adaptive trials with binary outcomes

In multi-arm adaptive trials, several treatments are assessed simultaneously and accumulating data are used to inform decisions about the trial, such as whether treatments are dropped or continued. Different methodological approaches have been developed for such trials and research has compared the performance of different subsets of these. One particular approach, for which we use the acronym MAMS(R), has generally not been included in these comparisons because control of the family-wise error rate (FWER) could not be guaranteed. Recently, the MAMS(R) approach has been extended to facilitate the generation of efficient designs which strongly control the FWER. We consider multi-arm two-stage trials with binary outcomes and propose parameterising treatment effects using the log odds ratio. We conduct a simulation study comparing the extended MAMS(R) framework with the well-established combination method both for trials where a different outcome is used for mid-trial analysis and for trials where the same outcome is used throughout. We show how the MAMS(R) framework compares favourably only in scenarios where the same outcome is used. We propose a hybrid selection rule within MAMS(R) methodology and demonstrate that this makes it possible to use the MAMS(R) framework in trials incorporating comparative treatment selection.

Nested combination tests with a time-to-event endpoint using a short-term endpoint for design adaptations

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family-wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short-term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short-term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.

Adaptive designs in clinical trials: from scientific advice to marketing authorisation to the European Medicine Agency

Background

In recent years, experience on the application of adaptive designs in confirmatory clinical trials has accumulated. Although planning such trials comes at the cost of additional operational complexity, adaptive designs offer the benefit of flexibility to update trial design and objectives as data accrue. In 2007, the European Medicines Agency (EMA) provided guidance on confirmatory clinical trials with adaptive (or flexible) designs. In order to better understand how adaptive trials are implemented in practice and how they may impact medicine approval within the EMA centralised procedure, we followed on 59 medicines for which an adaptive clinical trial had been submitted to the EMA Scientific Advice (SA) and analysed previously in a dedicated EMA survey of scientific advice letters. We scrutinized in particular the submission of the corresponding medicines for a marketing authorisation application (MAA). We also discuss the current regulatory perspective as regards the implementation of adaptive designs in confirmatory clinical trials.

Methods

Using the internal EMA MAA database, the AdisInsight database and related trial registries, we analysed how many of these 59 trials actually started, the completion status, results, the time to trial start, the adaptive elements finally implemented after SA, their possible influence on the success of the trial and corresponding product approval.

Results

Overall 31 trials out of 59 (53%) were retrieved. Thirty of them (97%) have been started and 23 (74%) concluded. Nine of these trials (39% out of 23) demonstrated a significant treatment effect on their primary endpoint and 4 (17% out of 23) supported a marketing authorisation (MA). An additional two trials were stopped using pre-defined criteria for futility, efficiently identifying trials on which further resources should not be spent. Median time to trial start after SA letter was given by EMA was 5 months. In the investigated trial registries, at least 18 trial (58% of 31 retrieved trials) designs were implemented with adaptive elements, which were predominantly dose selection, sample size reassessment (SSR) and stopping for futility (SFF). Among the 11 completed trials including adaptive elements, 6 demonstrated a significant treatment effect on their primary endpoint (55%).

Conclusions

Adaptive designs are now well established in the drug development landscape. If properly pre-planned, adaptations can play a key role in the success of some of these trials, for example to help successfully select the most promising dose regimens for phase II/III trials. Interim analyses can also enable stopping of trials for futility when they do not hold their promises. Type I error rate control, trial integrity and results consistency between the different stages of the analyses are fundamental aspects to be discussed thoroughly. Engaging early dialogue with regulators and implementing the scientific advice received is strongly recommended, since much experience in discussing adaptive designs and assessing their results has been accumulated.

Nonparametric adaptive enrichment designs using categorical surrogate data

Adaptive survival trials are particularly important for enrichment designs in oncology and other life-threatening diseases. Current statistical methodology for adaptive survival trials provide type I error rate control only under restrictions. For instance, if we use stage-wise P values based on increments of the log-rank test, then the information used for the interim decisions need to be restricted to the primary survival endpoint. However, it is often desirable to base interim decisions also on correlated short-term endpoints like tumor response. Alternative statistical approaches based on a patient-wise splitting of the data require unnatural restrictions on the follow-up times and do not permit to efficiently account for an early rejection of the primary null hypothesis. We therefore suggest new approaches that enable us to use discrete surrogate endpoints (like tumor response status) and also to incorporate interim rejection boundaries. The new approaches are based on weighted Kaplan-Meier estimates and thereby have additional advantages. They permit us to account for nonproportional hazards and are robust against informative censoring based on the surrogate endpoint. We will show that nonproportionality is an intrinsic and relevant issue in enrichment designs. Moreover, informative censoring based on the surrogate endpoint is likely because of withdrawals and treatment switches after insufficient treatment response. It is shown and illustrated how nonparametric tests based on weighted Kaplan-Meier estimates can be used in closed combination tests for adaptive enrichment designs, such that type I error rate control is achieved and justified asymptotically.

Interval and point estimation in adaptive Phase II trials with binary endpoint

Phase II clinical trials are concerned with making decision of whether a treatment is sufficiently efficacious to be worth further investigations in late large scale Phase III trials. In oncology Phase II trials, frequentist single-arm two-stage group-sequential designs with a binary endpoint are commonly used. To allow for more flexibility, adaptive versions of these designs have been proposed. In this paper, we propose point and interval estimation for adaptive designs in which the second stage sample size is a pre-specified function of first stage's number of responses. Our approach is based on sample space orderings, from which we derive p-values, and point and interval estimates. Simulation studies show that our proposed methods perform better, in terms of bias and root mean square error, than the fixed-sample maximum likelihood estimator.

Estimation after blinded sample size reassessment

Blinded sample size reassessment is a popular means to control the power in clinical trials if no reliable information on nuisance parameters is available in the planning phase. We investigate how sample size reassessment based on blinded interim data affects the properties of point estimates and confidence intervals for parallel group superiority trials comparing the means of a normal endpoint. We evaluate the properties of two standard reassessment rules that are based on the sample size formula of the z-test, derive the worst case reassessment rule that maximizes the absolute mean bias and obtain an upper bound for the mean bias of the treatment effect estimate.

Adaptive designs in clinical trials: why use them, and how to run and report them

Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial's course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented.We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.

Practical characteristics of adaptive design in phase 2 and 3 clinical trials

WHAT IS KNOWN AND OBJECTIVE

Adaptive design methods are expected to be ethical, reflect real medical practice, increase the likelihood of research and development success and reduce the allocation of patients into ineffective treatment groups by the early termination of clinical trials. However, the comprehensive details regarding which types of clinical trials will include adaptive designs remain unclear. We examined the practical characteristics of adaptive design used in clinical trials.

METHODS

We conducted a literature search of adaptive design clinical trials published from 2012 to 2015 using PubMed, EMBASE, and the Cochrane Central Register of Controlled Trials, with common search terms related to adaptive design. We systematically assessed the types and characteristics of adaptive designs and disease areas employed in the adaptive design trials.

RESULTS AND DISCUSSION

Our survey identified 245 adaptive design clinical trials. The number of trials by the publication year increased from 2012 to 2013 and did not greatly change afterwards. The most frequently used adaptive design was group sequential design (n = 222, 90.6%), especially for neoplasm or cardiovascular disease trials. Among the other types of adaptive design, adaptive dose/treatment group selection (n = 21, 8.6%) and adaptive sample-size adjustment (n = 19, 7.8%) were frequently used. The adaptive randomization (n = 8, 3.3%) and adaptive seamless design (n = 6, 2.4%) were less frequent. Adaptive dose/treatment group selection and adaptive sample-size adjustment were frequently used (up to 23%) in "certain infectious and parasitic diseases," "diseases of nervous system," and "mental and behavioural disorders" in comparison with "neoplasms" (<6.6%). For "mental and behavioural disorders," adaptive randomization was used in two trials of eight trials in total (25%). Group sequential design and adaptive sample-size adjustment were used frequently in phase 3 trials or in trials where study phase was not specified, whereas the other types of adaptive designs were used more in phase 2 trials. Approximately 82% (202 of 245 trials) resulted in early termination at the interim analysis. Among the 202 trials, 132 (54% of 245 trials) had fewer randomized patients than initially planned. This result supports the motive to use adaptive design to make study durations shorter and include a smaller number of subjects.

WHAT IS NEW AND CONCLUSION

We found that adaptive designs have been applied to clinical trials in various therapeutic areas and interventions. The applications were frequently reported in neoplasm or cardiovascular clinical trials. The adaptive dose/treatment group selection and sample-size adjustment are increasingly common, and these adaptations generally follow the Food and Drug Administration's (FDA's) recommendations.

Advanced Methods for Dose and Regimen Finding During Drug Development: Summary of the EMA/EFPIA Workshop on Dose Finding (London 4-5 December 2014)

Inadequate dose selection for confirmatory trials is currently still one of the most challenging issues in drug development, as illustrated by high rates of late‐stage attritions in clinical development and postmarketing commitments required by regulatory institutions. In an effort to shift the current paradigm in dose and regimen selection and highlight the availability and usefulness of well‐established and regulatory‐acceptable methods, the European Medicines Agency (EMA) in collaboration with the European Federation of Pharmaceutical Industries Association (EFPIA) hosted a multistakeholder workshop on dose finding (London 4–5 December 2014). Some methodologies that could constitute a toolkit for drug developers and regulators were presented. These methods are described in the present report: they include five advanced methods for data analysis (empirical regression models, pharmacometrics models, quantitative systems pharmacology models, MCP‐Mod, and model averaging) and three methods for study design optimization (Fisher information matrix (FIM)‐based methods, clinical trial simulations, and adaptive studies). Pairwise comparisons were also discussed during the workshop; however, mostly for historical reasons. This paper discusses the added value and limitations of these methods as well as challenges for their implementation. Some applications in different therapeutic areas are also summarized, in line with the discussions at the workshop. There was agreement at the workshop on the fact that selection of dose for phase III is an estimation problem and should not be addressed via hypothesis testing. Dose selection for phase III trials should be informed by well‐designed dose‐finding studies; however, the specific choice of method(s) will depend on several aspects and it is not possible to recommend a generalized decision tree. There are many valuable methods available, the methods are not mutually exclusive, and they should be used in conjunction to ensure a scientifically rigorous understanding of the dosing rationale.

Accurate p-values for adaptive designs with binary endpoints

Adaptive designs encompass all trials allowing various types of design modifications over the course of the trial. A key requirement for confirmatory adaptive designs to be accepted by regulators is the strong control of the family-wise error rate. This can be achieved by combining the p-values for each arm and stage to account for adaptations (including but not limited to treatment selection), sample size adaptation and multiple stages. While the theory for this is novel and well-established, in practice, these methods can perform poorly, especially for unbalanced designs and for small to moderate sample sizes. The problem is that standard stagewise tests have inflated type I error rate, especially but not only when the baseline success rate is close to the boundary and this is carried over to the adaptive tests, seriously inflating the family-wise error rate. We propose to fix this problem by feeding the adaptive test with second-order accurate p-values, in particular bootstrap p-values. Secondly, an adjusted version of the Simes procedure for testing intersection hypotheses that reduces the built-in conservatism is suggested. Numerical work and simulations show that unlike their standard counterparts the new approach preserves the overall error rate, at or below the nominal level across the board, irrespective of the baseline rate, stagewise sample sizes or allocation ratio. Copyright © 2017 John Wiley & Sons, Ltd.

Design and monitoring of multi-arm multi-stage clinical trials

Two-arm group sequential designs have been widely used for over 40 years, especially for studies with mortality endpoints. The natural generalization of such designs to trials with multiple treatment arms and a common control (MAMS designs) has, however, been implemented rarely. While the statistical methodology for this extension is clear, the main limitation has been an efficient way to perform the computations. Past efforts were hampered by algorithms that were computationally explosive. With the increasing interest in adaptive designs, platform designs, and other innovative designs that involve multiple comparisons over multiple stages, the importance of MAMS designs is growing rapidly. This article provides break-through algorithms that can compute MAMS boundaries rapidly thereby making such designs practical. For designs with efficacy-only boundaries the computational effort increases linearly with number of arms and number of stages. For designs with both efficacy and futility boundaries the computational effort doubles with successive increases in number of stages.