The reassessment of trial perspectives from interim data—a critical view

Sample Size Reestimation in Stochastic Curtailment Tests With Time-to-Events Outcome in the Case of Nonproportional Hazards Utilizing Two Weibull Distributions With Unknown Shape Parameters

Stochastic curtailment tests for Phase II two-arm trials with time-to-event end points are traditionally performed using the log-rank test. Recent advances in designing time-to-event trials have utilized the Weibull distribution with a known shape parameter estimated from historical studies. As sample size calculations depend on the value of this shape parameter, these methods either cannot be used or likely underperform/overperform when the natural variation around the point estimate is ignored. We demonstrate that when the magnitude of the Weibull shape parameters changes, unblinded interim information on the shape of the survival curves can be useful to enrich the final analysis for reestimation of the sample size. For such scenarios, we propose two Bayesian solutions to estimate the natural variations of the Weibull shape parameter. We implement these approaches under the framework of the newly proposed relative time method that allows nonproportional hazards and nonproportional time. We also demonstrate the sample size reestimation for the relative time method using three different approaches (internal pilot study approach, conditional power, and predictive power approach) at the interim stage of the trial. We demonstrate our methods using a hypothetical example and provide insights regarding the practical constraints for the proposed methods.

Flexible seamless 2-in-1 design with sample size adaptation

The 2-in-1 design is becoming popular in oncology drug development, with the flexibility in using different endpoints at different decision time. Based on the observed interim data, sponsors can choose to seamlessly advance a small phase 2 trial to a full-scale confirmatory phase 3 trial with a pre-determined maximum sample size or remain in a phase 2 trial. While this approach may increase efficiency in drug development, it is rigid and requires a pre-specified fixed sample size. In this paper, we propose a flexible 2-in-1 design with sample size adaptation, while retaining the advantage of allowing an intermediate endpoint for interim decision-making. The proposed design reflects the needs of the recent FDA's Project FrontRunner initiative, which encourages the use of an earlier surrogate endpoint to potentially support accelerated approval with conversion to standard approval with long-term endpoints from the same randomized study. Additionally, we identify the interim decision cut-off to allow a conventional test procedure at the final analysis. Extensive simulation studies showed that the proposed design requires much a smaller sample size and shorter timeline than the simple 2-in-1 design, while achieving similar power. We present a case study in multiple myeloma to demonstrate the benefits of the proposed design.

Two-stage randomized clinical trials with a right-censored endpoint: Comparison of frequentist and Bayesian adaptive designs

Adaptive randomized clinical trials are of major interest when dealing with a time-to-event outcome in a prolonged observation window. No consensus exists either to define stopping boundaries or to combinep $$ p $$ values or test statistics in the terminal analysis in the case of a frequentist design and sample size adaptation. In a one-sided setting, we compared three frequentist approaches using stopping boundaries relying onα $$ \alpha $$ -spending functions and a Bayesian monitoring setting with boundaries based on the posterior distribution of the log-hazard ratio. All designs comprised a single interim analysis with an efficacy stopping rule and the possibility of sample size adaptation at this interim step. Three frequentist approaches were defined based on the terminal analysis: combination of stagewise statistics (Wassmer) or ofp $$ p $$ values (Desseaux), or on patientwise splitting (Jörgens), and we compared the results with those of the Bayesian monitoring approach (Freedman). These different approaches were evaluated in a simulation study and then illustrated on a real dataset from a randomized clinical trial conducted in elderly patients with chronic lymphocytic leukemia. All approaches controlled for the type I error rate, except for the Bayesian monitoring approach, and yielded satisfactory power. It appears that the frequentist approaches are the best in underpowered trials. The power of all the approaches was affected by the violation of the proportional hazards (PH) assumption. For adaptive designs with a survival endpoint and a one-sided alternative hypothesis, the Wassmer and Jörgens approaches after sample size adaptation should be preferred, unless violation of PH is suspected.

RNA Polymerase Inhibitor Enisamium for Treatment of Moderate COVID-19 Patients: A Randomized, Placebo-Controlled, Multicenter, Double-Blind Phase 3 Clinical Trial

Enisamium is an orally available therapeutic that inhibits influenza A virus and SARS-CoV-2 replication. We evaluated the clinical efficacy of enisamium treatment combined with standard care in adult, hospitalized patients with moderate COVID-19 requiring external oxygen. Hospitalized patients with laboratory-confirmed SARS-CoV-2 infection were randomly assigned to receive either enisamium (500 mg per dose, four times a day) or a placebo. The primary outcome was an improvement of at least two points on an eight-point severity rating (SR) scale within 29 days of randomization. We initially set out to study the effect of enisamium on patients with a baseline SR of 4 or 5. However, because the study was started early in the COVID-19 pandemic, and COVID-19 had been insufficiently studied at the start of our study, an interim analysis was performed alongside a conditional power analysis in order to ensure patient safety and assess whether the treatment was likely to be beneficial for one or both groups. Following this analysis, a beneficial effect was observed for patients with an SR of 4 only, i.e., patients with moderate COVID-19 requiring supplementary oxygen. The study was continued for these COVID-19 patients. Overall, a total of 592 patients were enrolled and randomized between May 2020 and March 2021. Patients with a baseline SR of 4 were divided into two groups: 142 (49.8%) were assigned to the enisamium group and 143 (50.2%) to the placebo group. An analysis of the population showed that if patients were treated within 4 days of the onset of COVID-19 symptoms (n = 33), the median time to improvement was 8 days for the enisamium group and 13 days for the placebo group (p = 0.005). For patients treated within 10 days of the onset of COVID-19 symptoms (n = 154), the median time to improvement was 10 days for the enisamium group and 12 days for the placebo group (p = 0.002). Our findings suggest that enisamium is safe to use with COVID-19 patients, and that the observed clinical benefit of enisamium is worth reporting and studying in detail.

Beta spending function based on conditional power in group sequential design

Conditional power (CP) serves as a widely utilized approach for futility monitoring in group sequential designs. However, adopting the CP methods may lead to inadequate control of the type II error rate at the desired level. In this study, we introduce a flexible beta spending function tailored to regulate the type II error rate while employing CP based on a predetermined standardized effect size for futility monitoring (a so-called CP-beta spending function). This function delineates the expenditure of type II error rate across the entirety of the trial. Unlike other existing beta spending functions, the CP-beta spending function seamlessly incorporates beta spending concept into the CP framework, facilitating precise stagewise control of the type II error rate during futility monitoring. In addition, the stopping boundaries derived from the CP-beta spending function can be calculated via integration akin to other traditional beta spending function methods. Furthermore, the proposed CP-beta spending function accommodates various thresholds on the CP-scale at different stages of the trial, ensuring its adaptability across different information time scenarios. These attributes render the CP-beta spending function competitive among other forms of beta spending functions, making it applicable to any trials in group sequential designs with straightforward implementation. Both simulation study and example from an acute ischemic stroke trial demonstrate that the proposed method accurately captures expected power, even when the initially determined sample size does not consider futility stopping, and exhibits a good performance in maintaining overall type I error rates for evident futility.

Combining evidence from clinical trials in conditional or accelerated approval

Conditional (European Medicines Agency) or accelerated (U.S. Food and Drug Administration) approval of drugs allows earlier access to promising new treatments that address unmet medical needs. Certain post-marketing requirements must typically be met in order to obtain full approval, such as conducting a new post-market clinical trial. We study the applicability of the recently developed harmonic mean χ 2 $$ {\chi}^2 $$ -test to this conditional or accelerated approval framework. The proposed approach can be used both to support the design of the post-market trial and the analysis of the combined evidence provided by both trials. Other methods considered are the two-trials rule, Fisher's criterion and Stouffer's method. In contrast to some of the traditional methods, the harmonic mean χ 2 $$ {\chi}^2 $$ -test always requires a post-market clinical trial. If the p $$ p $$ -value from the pre-market clinical trial is ≪ 0.025 $$ \ll 0.025 $$ , a smaller sample size for the post-market clinical trial is needed than with the two-trials rule. For illustration, we apply the harmonic mean χ 2 $$ {\chi}^2 $$ -test to a drug which received conditional (and later full) market licensing by the EMA. A simulation study is conducted to study the operating characteristics of the harmonic mean χ 2 $$ {\chi}^2 $$ -test and two-trials rule in more detail. We finally investigate the applicability of these two methods to compute the power at interim of an ongoing post-market trial. These results are expected to aid in the design and assessment of the required post-market studies in terms of the level of evidence required for full approval.

A retrospective analysis of conditional power assumptions in clinical trials with continuous or binary endpoints

BACKGROUND

Adaptive clinical trials may use conditional power (CP) to make decisions at interim analyses, requiring assumptions about the treatment effect for remaining patients. It is critical that these assumptions are understood by those using CP in decision-making, as well as timings of these decisions.

METHODS

Data for 21 outcomes from 14 published clinical trials were made available for re-analysis. CP curves for accruing outcome information were calculated using and compared with a pre-specified objective criteria for original and transformed versions of the trial data using four future treatment effect assumptions: (i) observed current trend, (ii) hypothesised effect, (iii) 80% optimistic confidence limit, (iv) 90% optimistic confidence limit.

RESULTS

The hypothesised effect assumption met objective criteria when the true effect was close to that planned, but not when smaller than planned. The opposite was seen using the current trend assumption. Optimistic confidence limit assumptions appeared to offer a compromise between the two, performing well against objective criteria when the end observed effect was as planned or smaller.

CONCLUSION

The current trend assumption could be the preferable assumption when there is a wish to stop early for futility. Interim analyses could be undertaken as early as 30% of patients have data available. Optimistic confidence limit assumptions should be considered when using CP to make trial decisions, although later interim timings should be considered where logistically feasible.

Online control of the False Discovery Rate in group-sequential platform trials

When testing multiple hypotheses, a suitable error rate should be controlled even in exploratory trials. Conventional methods to control the False Discovery Rate assume that all p-values are available at the time point of test decision. In platform trials, however, treatment arms enter and leave the trial at different times during its conduct. Therefore, the actual number of treatments and hypothesis tests is not fixed in advance and hypotheses are not tested at once, but sequentially. Recently, for such a setting the concept of online control of the False Discovery Rate was introduced. We propose several heuristic variations of the LOND procedure (significance Levels based On Number of Discoveries) that incorporate interim analyses for platform trials, and study their online False Discovery Rate via simulations. To adjust for the interim looks spending functions are applied with O'Brien-Fleming or Pocock type group-sequential boundaries. The power depends on the prior distribution of effect sizes, for example, whether true alternatives are uniformly distributed over time or not. We consider the choice of design parameters for the LOND procedure to maximize the overall power and investigate the impact on the False Discovery Rate by including both concurrent and non-concurrent control data.

Conditional power in vaccine trials with seasonal variations

Conditional power (CP) is widely used in clinical trial monitoring to quantify the evidence for futility stopping or sample size adaptation during the trial. When planning an interim analysis in vaccine trials for seasonal infectious diseases, CPs calculated under the hypothesized or currently estimated effect sizes may not truly reflect future data due to seasonal variations in disease incidence and/or vaccine efficacy (VE). Relying on these estimates alone could lead to erroneous decisions. Therefore, we carried out simulation studies to investigate the use of seven different choices for the drift parameter in computing CP or predictive power (PP) in end-of-season interim analysis. Our simulations showed that, when used to inform futility stopping, CP under the hypothesized effect and a weighted PP under a normal prior distribution appear to outperform others in terms of the overall type II error rate. All CPs and PPs considered in this study resulted in comparable powers and expected sample sizes when used to inform sample size adaptation. The performance of either CP or PP largely depends on the extent to which the chosen drift parameter or the prior distribution of the drift parameter matches the remainder of the trial. Weighted CP/PP tends to be less sensitive to settings where observed data and emerging data in future seasons differ substantially as they incorporate both current estimate and future variations. Therefore, weighted strategies deserve further exploration and perhaps increased usage in guiding trial operations because they are more robust to inaccuracies in prediction. In summary, for vaccine trials with seasonal variations, a decision on trial operations should be guided by a careful consideration of plausible CPs and PPs calculated under reasonable assumptions leveraging the data, prior hypotheses, and new evidence on clinical relevance.

Optimal unplanned design modification in adaptive two-stage trials

Adaptive planning of clinical trials allows modifying the entire trial design at any time point mid-course. In this paper, we consider the case when a trial-external update of the planning assumptions during the ongoing trial makes an unforeseen design adaptation necessary. We take up the idea to construct adaptive designs with defined features by solving an optimization problem and apply it to the situation of unplanned design reassessment. By using the conditional error principle, we present an approach on how to optimally modify the trial design at an unplanned interim analysis while at the same time strictly protecting the type I error rate. This linking of optimal design planning and the conditional error principle allows sound reactions to unforeseen events that make a design reassessment necessary.

Optimization of adaptive designs with respect to a performance score

Adaptive designs are an increasingly popular method for the adaptation of design aspects in clinical trials, such as the sample size. Scoring different adaptive designs helps to make an appropriate choice among the numerous existing adaptive design methods. Several scores have been proposed to evaluate adaptive designs. Moreover, it is possible to determine optimal two-stage adaptive designs with respect to a customized objective score by solving a constrained optimization problem. In this paper, we use the conditional performance score by Herrmann et al. (2020) as the optimization criterion to derive optimal adaptive two-stage designs. We investigate variations of the original performance score, for example, by assigning different weights to the score components and by incorporating prior assumptions on the effect size. We further investigate a setting where the optimization framework is extended by a global power constraint, and additional optimization of the critical value function next to the stage-two sample size is performed. Those evaluations with respect to the sample size curves and the resulting design's performance can contribute to facilitate the score's usage in practice.

Optimal adaptive promising zone designs

We develop optimal decision rules for sample size re-estimation in two-stage adaptive group sequential clinical trials. It is usual for the initial sample size specification of such trials to be adequate to detect a realistic treatment effect δ a with good power, but not sufficient to detect the smallest clinically meaningful treatment effect δ min . Moreover it is difficult for the sponsors of such trials to make the up-front commitment needed to adequately power a study to detect δ min . It is easier to justify increasing the sample size if the interim data enter a so-called "promising zone" that ensures with high probability that the trial will succeed. We have considered promising zone designs that optimize unconditional power and promising zone designs that optimize conditional power and have discussed the tension that exists between these two objectives. Where there is reluctance to base the sample size re-estimation rule on the parameter δ min we propose a Bayesian option whereby a prior distribution is assigned to the unknown treatment effect δ , which is then integrated out of the objective function with respect to its posterior distribution at the interim analysis.

Conditional power and friends: The why and how of (un)planned, unblinded sample size recalculations in confirmatory trials

Adapting the final sample size of a trial to the evidence accruing during the trial is a natural way to address planning uncertainty. Since the sample size is usually determined by an argument based on the power of the trial, an interim analysis raises the question of how the final sample size should be determined conditional on the accrued information. To this end, we first review and compare common approaches to estimating conditional power, which is often used in heuristic sample size recalculation rules. We then discuss the connection of heuristic sample size recalculation and optimal two-stage designs, demonstrating that the latter is the superior approach in a fully preplanned setting. Hence, unplanned design adaptations should only be conducted as reaction to trial-external new evidence, operational needs to violate the originally chosen design, or post hoc changes in the optimality criterion but not as a reaction to trial-internal data. We are able to show that commonly discussed sample size recalculation rules lead to paradoxical adaptations where an initially planned optimal design is not invariant under the adaptation rule even if the planning assumptions do not change. Finally, we propose two alternative ways of reacting to newly emerging trial-external evidence in ways that are consistent with the originally planned design to avoid such inconsistencies.

Interim data monitoring in cluster randomised trials: Practical issues and a case study

BACKGROUND

There is an abundance of guidance for the interim monitoring of individually randomised trials. While methodological literature exists on how to extend these methods to cluster randomised trials, there is little guidance on practical implementation. Cluster trials have many features which make their monitoring needs different. We outline the methodological and practical challenges of interim monitoring of cluster trials; and apply these considerations to a case study.

CASE STUDY

The E-MOTIVE study is an 80-cluster randomised trial of a bundle of interventions to treat postpartum haemorrhage. The proposed data monitoring plan includes (1) monitor sample size assumptions, (2) monitor for evidence of selection bias, and (3) an interim assessment of the primary outcome, as well as monitoring data completeness. The timing of the sample size monitoring is chosen with both consideration of statistical precision and to allow time to recruit more clusters. Monitoring for selection bias involves comparing individual-level characteristics and numbers recruited between study arms to identify any post-randomisation participant identification bias. An interim analysis of outcomes presented with 99.9% confidence intervals using the Haybittle-Peto approach should mitigate any concern regarding the inflation of type-I error. The pragmatic nature of the trial means monitoring for adherence is not relevant, as it is built into a process evaluation.

CONCLUSIONS

The interim analyses of cluster trials have a number of important differences to monitoring individually randomised trials. In cluster trials, there will often be a greater need to monitor nuisance parameters, yet there will often be considerable uncertainty in their estimation. This means the utility of sample size re-estimation can be questionable particularly when there are practical or funding difficulties associated with making any changes to planned sample sizes. Perhaps most importantly interim monitoring has the potential to identify selection bias, particularly in trials with post-randomisation identification or recruitment. Finally, the pragmatic nature of cluster trials might mean that the utility of methods to allow for interim monitoring of outcomes based on statistical testing, or monitoring for adherence to study interventions, are less relevant. Our intention is to facilitate the planning of future cluster randomised trials and to promote discussion and debate to improve monitoring of these studies.

Randomised trials at the level of the individual

In global health research, short-term, small-scale clinical trials with fixed, two-arm trial designs that generally do not allow for major changes throughout the trial are the most common study design. Building on the introductory paper of this Series, this paper discusses data-driven approaches to clinical trial research across several adaptive trial designs, as well as the master protocol framework that can help to harmonise clinical trial research efforts in global health research. We provide a general framework for more efficient trial research, and we discuss the importance of considering different study designs in the planning stage with statistical simulations. We conclude this second Series paper by discussing the methodological and operational complexity of adaptive trial designs and master protocols and the current funding challenges that could limit uptake of these approaches in global health research.

Optimal planning of adaptive two-stage designs

Adaptive designs are playing an increasingly important role in the planning of clinical trials. While there exists various research on the optimal determination of a two-stage design, non-optimal versions still are frequently applied in clinical research. In this article, we strive to motivate the application of optimal adaptive designs and give guidance on how to determine them. It is demonstrated that optimizing a trial design with respect to particular objective criteria can have a substantial benefit over the application of conventional adaptive sample size recalculation rules. Furthermore, we show that in many practical situations, optimal group-sequential designs show an almost negligible performance loss compared to optimal adaptive designs. Finally, we illustrate how optimal designs can be tailored to specific operational requirements by customizing the underlying optimization problem.

Sample size re-estimation for pivotal clinical trials

It is well known that if the hypothesis test is left unchanged, the Type I error rate may be inflated for sample size re-estimation (SSR) designs. To address this issue, three main approaches have been proposed in the literature: combination test, conditional error and conventional test with sample size increase in the allowable region (AR) only. These three seemingly different approaches are in fact connected. For each combination test, there is a corresponding conditional error function and AR. Designing adaptation rules in this AR with conventional test guarantees the Type I error rate control but at the same time always leads to smaller power comparing to the corresponding combination test (or conditional error) approach. In cases where conventional test is still preferable, step-wise type adaptation rules that do not fully reside in the AR can be alternatively considered. We believe controversies in the statistical community on the efficiency comparisons between group sequential (GS) and SSR design stem partially from the misalignment of performance metrics and conditional versus unconditional evaluations. We advocate summary metrics, such as median, variance or tail probabilities of the sample size in addition to expectation and personalizing efficiency definition for each trial sponsor. Conditional metrics by favorable, promising and unfavorable zones of the interim results provide additional insights and should always be incorporated into the decision-making process.

Assessing the Impact of COVID-19 on the Clinical Trial Objective and Analysis of Oncology Clinical Trials-Application of the Estimand Framework

Abstract-Coronavirus disease 2019 (COVID-19) outbreak has rapidly evolved into a global pandemic. The impact of COVID-19 on patient journeys in oncology represents a new risk to interpretation of trial results and its broad applicability for future clinical practice. We identify key intercurrent events (ICEs) that may occur due to COVID-19 in oncology clinical trials with a focus on time-to-event endpoints and discuss considerations pertaining to the other estimand attributes introduced in the ICH E9 addendum. We propose strategies to handle COVID-19 related ICEs, depending on their relationship with malignancy and treatment and the interpretability of data after them. We argue that the clinical trial objective from a world without COVID-19 pandemic remains valid. The estimand framework provides a common language to discuss the impact of COVID-19 in a structured and transparent manner. This demonstrates that the applicability of the framework may even go beyond what it was initially intended for.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

Improving interim decisions in randomized trials by exploiting information on short-term endpoints and prognostic baseline covariates

Conditional power calculations are frequently used to guide the decision whether or not to stop a trial for futility or to modify planned sample size. These ignore the information in short-term endpoints and baseline covariates, and thereby do not make fully efficient use of the information in the data. We therefore propose an interim decision procedure based on the conditional power approach which exploits the information contained in baseline covariates and short-term endpoints. We will realize this by considering the estimation of the treatment effect at the interim analysis as a missing data problem. This problem is addressed by employing specific prediction models for the long-term endpoint which enable the incorporation of baseline covariates and multiple short-term endpoints. We show that the proposed procedure leads to an efficiency gain and a reduced sample size, without compromising the Type I error rate of the procedure, even when the adopted prediction models are misspecified. In particular, implementing our proposal in the conditional power approach enables earlier decisions relative to standard approaches, whilst controlling the probability of an incorrect decision. This time gain results in a lower expected number of recruited patients in case of stopping for futility, such that fewer patients receive the futile regimen. We explain how these methods can be used in adaptive designs with unblinded sample size re-assessment based on the inverse normal P-value combination method to control Type I error. We support the proposal by Monte Carlo simulations based on data from a real clinical trial.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

A conditional error function approach for adaptive enrichment designs with continuous endpoints

Adaptive enrichment designs offer an efficient and flexible way to demonstrate the efficacy of a treatment in a clinically defined full population or in, eg, biomarker-defined subpopulations while controlling the family-wise Type I error rate in the strong sense. Frequently used testing strategies in designs with two or more stages include the combination test and the conditional error function approach. Here, we focus on the latter and present some extensions. In contrast to previous work, we allow for multiple subgroups rather than one subgroup only. For nested as well as nonoverlapping subgroups with normally distributed endpoints, we explore the effect of estimating the variances in the subpopulations. Instead of using a normal approximation, we derive new t-distribution-based methods for two different scenarios. First, in the case of equal variances across the subpopulations, we present exact results using a multivariate t-distribution. Second, in the case of potentially varying variances across subgroups, we provide some improved approximations compared to the normal approximation. The performance of the proposed conditional error function approaches is assessed and compared to the combination test in a simulation study. The proposed methods are motivated by an example in pulmonary arterial hypertension.

Interim analysis incorporating short- and long-term binary endpoints

Designs incorporating more than one endpoint have become popular in drug development. One of such designs allows for incorporation of short‐term information in an interim analysis if the long‐term primary endpoint has not been yet observed for some of the patients. At first we consider a two‐stage design with binary endpoints allowing for futility stopping only based on conditional power under both fixed and observed effects. Design characteristics of three estimators: using primary long‐term endpoint only, short‐term endpoint only, and combining data from both are compared. For each approach, equivalent cut‐off point values for fixed and observed effect conditional power calculations can be derived resulting in the same overall power. While in trials stopping for futility the type I error rate cannot get inflated (it usually decreases), there is loss of power. In this study, we consider different scenarios, including different thresholds for conditional power, different amount of information available at the interim, different correlations and probabilities of success. We further extend the methods to adaptive designs with unblinded sample size reassessments based on conditional power with inverse normal method as the combination function. Two different futility stopping rules are considered: one based on the conditional power, and one from P‐values based on Z‐statistics of the estimators. Average sample size, probability to stop for futility and overall power of the trial are compared and the influence of the choice of weights is investigated.

Critical concepts in adaptive clinical trials

Adaptive clinical trials are an innovative trial design aimed at reducing resources, decreasing time to completion and number of patients exposed to inferior interventions, and improving the likelihood of detecting treatment effects. The last decade has seen an increasing use of adaptive designs, particularly in drug development. They frequently differ importantly from conventional clinical trials as they allow modifications to key trial design components during the trial, as data is being collected, using preplanned decision rules. Adaptive designs have increased likelihood of complexity and also potential bias, so it is important to understand the common types of adaptive designs. Many clinicians and investigators may be unfamiliar with the design considerations for adaptive designs. Given their complexities, adaptive trials require an understanding of design features and sources of bias. Herein, we introduce some common adaptive design elements and biases and specifically address response adaptive randomization, sample size reassessment, Bayesian methods for adaptive trials, seamless trials, and adaptive enrichment using real examples.

An evaluation of increasing sample size based on conditional power

We evaluate properties of sample size re-estimation (SSR) designs similar to the promising zone design considered by Mehta and Pocock (2011). We evaluate these designs under the assumption of a true effect size of 1.1 down to 0.4 of the protocol-specified effect size by six measures: 1. The probability of a sample size increase, 2. The mean proportional increase in sample size given an increase; 3 and 4. The mean true conditional power with and without a sample size increase; 5 and 6. The expected increase in sample size and power due to the SSR procedure. These measures show the probability of a sample size increase and the cost/benefit for given true effect sizes, particularly when the SSR may either be pursuing a small effect size of little clinical importance or be unnecessary when the true effect size is close to the protocol-specified effect size. The results show the clear superiority of conducting the SSR late in the study and the inefficiency of a mid-study SSR. The results indicate that waiting until late in the study for the SSR yields a smaller, better targeted set of studies with a greater increase in overall power than a mid-study SSR.

CBER's Experience With Adaptive Design Clinical Trials

There is considerable interest among pharmaceutical and other medical product developers in adaptive clinical trials, in which knowledge learned during the course of a trial affects ongoing conduct or analysis of the trial. When the FDA released a draft Guidance document on adaptive design clinical trials in early 2010, expectations were high that it would lead to an increase in regulatory submissions involving adaptive design features, particularly for confirmatory trials. A 6-year (2008-2013) retrospective survey was performed within the Center for Biologics Evaluation and Research (CBER) at the FDA to gather information regarding the submission and evaluation of adaptive design trial proposals. We present an up-to-date summary of adaptive design proposals seen in CBER and provide an overview of our experiences. We share our concerns regarding the statistical issues and operational challenges raised during the review process for adaptive design trials. We also provide general recommendations for developing proposals for such trials. Our motivation in writing this paper was to encourage the best study design proposals to be submitted to CBER. Sometimes these can be adaptive, and sometimes a simpler design is most efficient.

Bayesian predictive power: choice of prior and some recommendations for its use as probability of success in drug development

Bayesian predictive power, the expectation of the power function with respect to a prior distribution for the true underlying effect size, is routinely used in drug development to quantify the probability of success of a clinical trial. Choosing the prior is crucial for the properties and interpretability of Bayesian predictive power. We review recommendations on the choice of prior for Bayesian predictive power and explore its features as a function of the prior. The density of power values induced by a given prior is derived analytically and its shape characterized. We find that for a typical clinical trial scenario, this density has a u-shape very similar, but not equal, to a β-distribution. Alternative priors are discussed, and practical recommendations to assess the sensitivity of Bayesian predictive power to its input parameters are provided. Copyright © 2016 John Wiley & Sons, Ltd.

Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials

Mid-study design modifications are becoming increasingly accepted in confirmatory clinical trials, so long as appropriate methods are applied such that error rates are controlled. It is therefore unfortunate that the important case of time-to-event endpoints is not easily handled by the standard theory. We analyze current methods that allow design modifications to be based on the full interim data, i.e., not only the observed event times but also secondary endpoint and safety data from patients who are yet to have an event. We show that the final test statistic may ignore a substantial subset of the observed event times. An alternative test incorporating all event times is found, where a conservative assumption must be made in order to guarantee type I error control. We examine the power of this approach using the example of a clinical trial comparing two cancer therapies.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Sample size adjustment based on promising interim results and its application in confirmatory clinical trials

BACKGROUND

For a carefully planned and well-designed Phase 3 confirmatory trial, there is still a potential risk of failing to meet the study objective due to possible differences between Phase 2 and Phase 3 studies. As illustrated by the ENGAGE trial, potential sample size increase at an interim analysis can mitigate the risk for an otherwise underpowered study. Many approaches for sample size adjustment (SSA) require certain modifications to the conventional statistical method, such as changing critical values or using a weighted Z-statistic for final hypothesis testing. Without modification, the type I error rate can be inflated, primarily caused by sample size increase for nonpromising interim observation that is close to null or no treatment effect. As illustrated by the TOPICAL trial, increasing sample size for nonpromising interim result could waste limited resource on ineffective treatment. The modifications in these approaches are therefore unnecessary costs of flexibility/interpretability for unnecessary scenarios of sample size increase.

PURPOSE

To discuss and illustrate the appropriateness of SSA based on promising interim results, that is, conditional power being greater than 50% (or CDL approach), in a carefully planned and well-designed Phase 3 confirmatory trial.

METHODS

Two clinical trials are used to illustrate the clinical setting for the CDL approach and appropriateness of its application. Operating characteristics are assessed and compared to other methods using numeric computation. Hypothetical trials based on real clinical data are used to illustrate the approach.

RESULTS

The CDL approach for SSA leads to a small increase in expected sample size resulting in a small power gain versus the fixed design. This indicates that adding SSA will not on average substantially affect the budget at the portfolio level. However, when the interim result is promising, the CDL approach can dramatically increase the conditional power therefore mitigating the risk of an otherwise underpowered study.

LIMITATIONS

Implementation challenges of the SSA methods are not in the scope of this paper. SSA is not intended to replace careful design of a confirmatory trial; instead, it can mitigate the risk for a well-designed trial.

CONCLUSIONS

The CDL approach for SSA based on promising interim results, that is, conditional power being greater than 50%, is particularly useful in mitigating the risk for a carefully planned and well-designed Phase 3 confirmatory trial. No modification to the conventional statistical procedure is necessary while the type I error rate is controlled. Such a feature of ''no interference,'' or no change to the conventional statistical procedure with or without sample size adjustment, is important for the interpretation of a confirmatory trial. Similar to the fixed design, carefully planned and well-designed group sequential studies can also benefit from SSA to mitigate the risk of failing to meet the study objective.

Some challenges with statistical inference in adaptive designs

Adaptive designs have generated a great deal of attention to clinical trial communities. The literature contains many statistical methods to deal with added statistical uncertainties concerning the adaptations. Increasingly encountered in regulatory applications are adaptive statistical information designs that allow modification of sample size or related statistical information and adaptive selection designs that allow selection of doses or patient populations during the course of a clinical trial. For adaptive statistical information designs, a few statistical testing methods are mathematically equivalent, as a number of articles have stipulated, but arguably there are large differences in their practical ramifications. We pinpoint some undesirable features of these methods in this work. For adaptive selection designs, the selection based on biomarker data for testing the correlated clinical endpoints may increase statistical uncertainty in terms of type I error probability, and most importantly the increased statistical uncertainty may be impossible to assess.

Two-stage designs for cross-over bioequivalence trials

The topic of applying two-stage designs in the field of bioequivalence studies has recently gained attention in the literature and in regulatory guidelines. While there exists some methodological research on the application of group sequential designs in bioequivalence studies, implementation of adaptive approaches has focused up to now on superiority and non-inferiority trials. Especially, no comparison of the features and performance characteristics of these designs has been performed, and therefore, the question of which design to employ in this setting remains open. In this paper, we discuss and compare 'classical' group sequential designs and three types of adaptive designs that offer the option of mid-course sample size recalculation. A comprehensive simulation study demonstrates that group sequential designs can be identified, which show power characteristics that are similar to those of the adaptive designs but require a lower average sample size. The methods are illustrated with a real bioequivalence study example.

Adaptive graph-based multiple testing procedures

Multiple testing procedures defined by directed, weighted graphs have recently been proposed as an intuitive visual tool for constructing multiple testing strategies that reflect the often complex contextual relations between hypotheses in clinical trials. Many well-known sequentially rejective tests, such as (parallel) gatekeeping tests or hierarchical testing procedures are special cases of the graph based tests. We generalize these graph-based multiple testing procedures to adaptive trial designs with an interim analysis. These designs permit mid-trial design modifications based on unblinded interim data as well as external information, while providing strong family wise error rate control. To maintain the familywise error rate, it is not required to prespecify the adaption rule in detail. Because the adaptive test does not require knowledge of the multivariate distribution of test statistics, it is applicable in a wide range of scenarios including trials with multiple treatment comparisons, endpoints or subgroups, or combinations thereof. Examples of adaptations are dropping of treatment arms, selection of subpopulations, and sample size reassessment. If, in the interim analysis, it is decided to continue the trial as planned, the adaptive test reduces to the originally planned multiple testing procedure. Only if adaptations are actually implemented, an adjusted test needs to be applied. The procedure is illustrated with a case study and its operating characteristics are investigated by simulations.

Performance of adaptive designs for single-armed phase II oncology trials

When planning a single-armed clinical trial with binary endpoint, the sample size is determined such that the desired power is achieved for a single value of the target rate. However, there is usually some uncertainty with respect to the true treatment effect. It is therefore more realistic to specify an interval for the possible true rate to accommodate this uncertainty. For this situation, we examine comprehensively the overall performance of various Phase II oncology designs and sample size recalculation strategies. The methods and results of our investigations can be used to identify the most appropriate approach for a specific clinical trial situation at hand. Application is illustrated with a clinical trial in rectal cancer.

A review and re-interpretation of a group-sequential approach to sample size re-estimation in two-stage trials

In this paper, we review the adaptive design methodology of Li et al. (Biostatistics 3:277-287) for two-stage trials with mid-trial sample size adjustment. We argue that it is closer in principle to a group sequential design, in spite of its obvious adaptive element. Several extensions are proposed that aim to make it even more attractive and transparent alternative to a standard (fixed sample size) trial for funding bodies to consider. These enable a cap to be put on the maximum sample size and for the trial data to be analysed using standard methods at its conclusion. The regulatory view of trials incorporating unblinded sample size re-estimation is also discussed.

Blinded versus unblinded covariate selection in confirmatory survival trials

Adjustment for covariates and specification of the correct covariate set are important issues in the analysis of clinical trials. Edwards (1999) proposes a model selection approach where the model is chosen on the final data set, which remains blinded for treatment group allocation. We investigate this method for time-to-event endpoints and compare its performance to variable selection within an adaptive design. This adaptive design integrates the methods of Schäfer and Müller (2001) and Keiding et al. (1987) and allows variable selection on the unblinded data during an interim analysis. Monte Carlo simulation shows that Edwards' method-though blinded-outperforms the adaptive method in terms of ability to select the survival relevant covariates and power. The application of the methods is illustrated by a clinical trial example.

Adaptive Budgets in Clinical Trials

We consider situations where a drug developer gets access to additional financial resources when a promising result has been observed in a pre-planned interim analysis during a clinical trial which should lead to the registration of the drug. First the option that the drug developer completely puts the additional resources into increasing the second stage sample size has been investigated. If investors invest the more the larger the observed interim effect, this may not be a reasonable strategy: Then additional sample sizes are applied when the conditional power is already very large and hardly any impact on the overall power can be expected. Nevertheless, further reducing the type II error rate in promising situations may be of interest for a drug developer. In a second step, sample size was based on a utility function including the reward of registration (which was allowed to depend on the observed effect size at the end of the trial) and sampling costs. Utility as a function of the sample size may have more than one local maximum, one of them at the lowest per group sample size. For small effects an optimal strategy could be to apply the smallest sample size accepted by regulators.

Practical guidelines for adaptive seamless phase II/III clinical trials that use Bayesian methods

Hommel (Biometrical Journal; 45:581-589) proposed a flexible testing procedure for seamless phase II/III clinical trials. Schmidli et al. (Statistics in Medicine; 26:4925-4938), Kimani et al. (Statistics in Medicine; 28:917-936) and Brannath et al. (Statistics in Medicine; 28:1445-1463) exploited the flexible testing of Hommel to propose adaptation in seamless phase II/III clinical trials that incorporate prior knowledge by using Bayesian methods. In this paper, we show that adaptation incorporating prior knowledge may lead to higher power. Other important issues to consider in such adaptive designs are the gain in power (or saving in patients) over traditional testing and how utility values used to make the adaptation may be used to stop a trial early. In contrast to the aforementioned authors, we discuss these issues in detail and propose a unified approach to address them so that implementing the aforementioned designs and proposing similar designs is clearer.

Adaptive designs for single-arm phase II trials in oncology

Clinical phase II trials in oncology are conducted to determine whether the activity of a new anticancer treatment is promising enough to merit further investigation. Two-stage designs are commonly used for this situation to allow for early termination. Designs proposed in the literature so far have the common drawback that the sample sizes for the two stages have to be specified in the protocol and have to be adhered to strictly during the course of the trial. As a consequence, designs that allow a higher extent of flexibility are desirable. In this article, we propose a new adaptive method that allows an arbitrary modification of the sample size of the second stage using the results of the interim analysis or external information while controlling the type I error rate. If the sample size is not changed during the trial, the proposed design shows very similar characteristics to the optimal two-stage design proposed by Chang et al. (Biometrics 1987; 43:865-874). However, the new design allows the use of mid-course information for the planning of the second stage, thus meeting practical requirements when performing clinical phase II trials in oncology.

Regulatory perspectives on multiplicity in adaptive design clinical trials throughout a drug development program

A clinical research program for drug development often consists of a sequence of clinical trials that may begin with uncontrolled and nonrandomized trials, followed by randomized trials or randomized controlled trials. Adaptive designs are not infrequently proposed for use. In the regulatory setting, the success of a drug development program can be defined to be that the experimental treatment at a specific dose level including regimen and frequency is approved based on replicated evidence from at least two confirmatory trials. In the early stage of clinical research, multiplicity issues are very broad. What is the maximum tolerable dose in an adaptive dose escalation trial? What should the dose range be to consider in an adaptive dose-ranging trial? What is the minimum effective dose in an adaptive dose-response study given the tolerability and the toxicity observable in short term or premarketing trials? Is establishing the dose-response relationship important or the ability to select a superior treatment with high probability more important? In the later stage of clinical research, multiplicity problems can be formulated with better focus, depending on whether the study is for exploration to estimate or select design elements or for labeling consideration. What is the study objective for an early-phase versus a later phase adaptive clinical trial? How many doses are to be studied in the early exploratory adaptive trial versus in the confirmatory adaptive trial? Is the intended patient population well defined or is the applicable patient population yet to be adaptively selected in the trial due to the potential patient and/or disease heterogeneity? Is the primary efficacy endpoint well defined or still under discussion providing room for adaptation? What are the potential treatment indications that may adaptively lead to an intended-to-treat patient population and the primary efficacy endpoint? In this work we stipulate the multiplicity issues with adaptive designs encountered in regulatory applications. For confirmatory adaptive design clinical trials, controlling studywise type I error and type II error is of paramount importance. For exploratory adaptive trials, we define the probability of correct selection of design features, e.g., dose, effect size, and the probability of correct decision for drug development. We assert that maximizing these probabilities would be critical to determine whether the drug development program continues or how to plan the confirmatory trials if the development continues.