Recursive Combination Tests

The impact of heterogeneity on the analysis of platform trials with normally distributed outcomes

BACKGROUND

A platform trial approach allows adding arms to on-going trials to speed up intervention discovery programs. A control arm remains open for recruitment in a platform trial while intervention arms may be added after the onset of the study and could be terminated early for efficacy and/or futility when early stopping is allowed. The topic of utilising non-concurrent control data in the analysis of platform trials has been explored and discussed extensively. A less familiar issue is the presence of heterogeneity, which may exist for example due to modification of enrolment criteria and recruitment strategy.

METHOD

We conduct a simulation study to explore the impact of heterogeneity on the analysis of a two-stage platform trial design. We consider heterogeneity in treatment effects and heteroscedasticity in outcome data across stages for a normally distributed endpoint. We examine the performance of some hypothesis testing procedures and modelling strategies. The use of non-concurrent control data is also considered accordingly. Alongside standard regression analysis, we examine the performance of a novel method that was known as the pairwise trials analysis. It is similar to a network meta-analysis approach but adjusts for treatment comparisons instead of individual studies using fixed effects.

RESULTS

Several testing strategies with concurrent control data seem to control the type I error rate at the required level when there is heteroscedasticity in outcome data across stages and/or a random cohort effect. The main parameter of treatment effects in some analysis models correspond to overall treatment effects weighted by stage wise sample sizes; while others correspond to the effect observed within a single stage. The characteristics of the estimates are not affected significantly by the presence of a random cohort effect and/ or heteroscedasticity.

CONCLUSION

In view of heterogeneity in treatment effect across stages, the specification of null hypotheses in platform trials may need to be more subtle. We suggest employing testing procedure of adaptive design as opposed to testing the statistics from regression models; comparing the estimates from the pairwise trials analysis method and the regression model with interaction terms may indicate if heterogeneity is negligible.

Point estimation, confidence intervals, and P-values for optimal adaptive two-stage designs with normal endpoints

Due to the dependency structure in the sampling process, adaptive trial designs create challenges in point and interval estimation and in the calculation of P-values. Optimal adaptive designs, which are designs where the parameters governing the adaptivity are chosen to maximize some performance criterion, suffer from the same problem. Various analysis methods which are able to handle this dependency structure have already been developed. In this work, we aim to give a comprehensive summary of these methods and show how they can be applied to the class of designs with planned adaptivity, of which optimal adaptive designs are an important member. The defining feature of these kinds of designs is that the adaptive elements are completely prespecified. This allows for explicit descriptions of the calculations involved, which makes it possible to evaluate different methods in a fast and accurate manner. We will explain how to do so, and present an extensive comparison of the performance characteristics of various estimators between an optimal adaptive design and its group-sequential counterpart.

The use of real-world data for clinical investigation of effectiveness in drug development

With the growing interest in leveraging real-world data (RWD) to support effectiveness evaluations for new indications, new target populations, and post-market performance, the United States Food and Drug Administration has published several guidance documents on RWD sources and real-world studies (RWS) to assist sponsors in generating credible real-world evidence (RWE). Meanwhile, the randomized controlled trial (RCT) remains the gold standard in drug evaluation. Along this line, we propose a hybrid two-stage adaptive design to evaluate effectiveness based on evidence from both RCT and RWS. At the first stage, a typical non-inferiority test is conducted using RCT data to test for not-ineffectiveness. Once not-ineffectiveness is established, the study proceeds to the second stage to conduct an RWS and test for effectiveness using integrated information from RCT and RWD. The composite likelihood approach is implemented as a down-weighing strategy to account for the impact of high variability in RWS population. An optimal sample size determination procedure for RCT and RWS is introduced, aiming to achieve the minimal expected sample size. Through extensive numerical study, the proposed design demonstrates the ability to control type I error inflation in most cases and consistently maintain statistical power above the desired level. In general, this RCT/RWS hybrid two-stage adaptive design is beneficial for effectiveness evaluations in drug development, especially for oncology and rare diseases.

Familywise error rate control for block response-adaptive randomization

Response-adaptive randomization allows the probabilities of allocating patients to treatments in a clinical trial to change based on the previously observed response data, in order to achieve different experimental goals. One concern over the use of such designs in practice, particularly from a regulatory viewpoint, is controlling the type I error rate. To address this, Robertson and Wason (Biometrics, 2019) proposed methodology that guarantees familywise error rate control for a large class of response-adaptive designs by re-weighting the usual z -test statistic. In this article, we propose an improvement of their method that is conceptually simpler, in the context where patients are allocated to the experimental treatment arms in a trial in blocks (i.e. groups) using response-adaptive randomization. We show the modified method guarantees that there will never be negative weights for the contribution of each block of data to the adjusted test statistics, and can also provide a substantial power advantage in practice.

Adaptive enrichment designs with a continuous biomarker

A popular design for clinical trials assessing targeted therapies is the two-stage adaptive enrichment design with recruitment in stage 2 limited to a biomarker-defined subgroup chosen based on data from stage 1. The data-dependent selection leads to statistical challenges if data from both stages are used to draw inference on treatment effects in the selected subgroup. If subgroups considered are nested, as when defined by a continuous biomarker, treatment effect estimates in different subgroups follow the same distribution as estimates in a group-sequential trial. This result is used to obtain tests controlling the familywise type I error rate (FWER) for six simple subgroup selection rules, one of which also controls the FWER for any selection rule. Two approaches are proposed: one based on multivariate normal distributions suitable if the number of possible subgroups, k, is small, and one based on Brownian motion approximations suitable for large k. The methods, applicable in the wide range of settings with asymptotically normal test statistics, are illustrated using survival data from a breast cancer trial.

Point estimation for adaptive trial designs I: A methodological review

Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value," and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realized trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This article is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this article can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.

A two-stage adaptive clinical trial design with data-driven subgroup identification at interim analysis

In this paper, we consider randomized controlled clinical trials comparing two treatments in efficacy assessment using a time to event outcome. We assume a relatively small number of candidate biomarkers available in the beginning of the trial, which may help define an efficacy subgroup which shows differential treatment effect. The efficacy subgroup is to be defined by one or two biomarkers and cut-offs that are unknown to the investigator and must be learned from the data. We propose a two-stage adaptive design with a pre-planned interim analysis and a final analysis. At the interim, several subgroup-finding algorithms are evaluated to search for a subgroup with enhanced survival for treated versus placebo. Conditional powers computed based on the subgroup and the overall population are used to make decision at the interim to terminate the study for futility, continue the study as planned, or conduct sample size recalculation for the subgroup or the overall population. At the final analysis, combination tests together with closed testing procedures are used to determine efficacy in the subgroup or the overall population. We conducted simulation studies to compare our proposed procedures with several subgroup-identification methods in terms of a novel utility function and several other measures. This research demonstrated the benefit of incorporating data-driven subgroup selection into adaptive clinical trial designs.

Digital breast tomosynthesis plus synthesised mammography versus digital screening mammography for the detection of invasive breast cancer (TOSYMA): a multicentre, open-label, randomised, controlled, superiority trial

BACKGROUND

Two dimensional (2D) full-field digital mammography is the current standard of breast cancer screening. Digital breast tomosynthesis generates pseudo-three dimensional datasets of the breast from which synthesised 2D (s2D) mammograms can be reconstructed. This innovative approach reduces the likelihood of overlapping breast tissues that can conceal features of malignancy. We aimed to compare digital breast tomosynthesis plus s2D mammography with digital screening mammography for the detection of invasive breast cancer.

METHODS

TOSYMA was a randomised, open-label, superiority trial done at 17 screening units in two federal states of Germany. Eligible participants were women aged 50-69 years who had been invited to participate in a population-wide, quality-controlled mammography screening programme. Women were randomly assigned (1:1) to digital breast tomosynthesis plus s2D mammography or digital mammography alone using block randomisation (block size of 32), stratified by site. The primary endpoints were the detection rate of invasive breast cancer and invasive interval cancer rate at 24 months, analysed in the modified full analysis set, which included all randomly assigned participants who underwent either type of screening examination. Ten examinations, corresponding to a second study participation, were excluded. Analyses were done according to the intention-to-treat principle. Interval cancer rates will be reported in the follow-up study. Safety was assessed in the as-treated population, which included all participants who were randomly assigned. This trial is registered with ClinicalTrials.gov, NCT03377036, and is closed to accrual.

FINDINGS

Between July 5, 2018, and Dec 30, 2020, 99 689 women were randomly assigned to digital breast tomosynthesis plus s2D mammography (n=49 804) or digital mammography (n=49 830). Invasive breast cancers were detected in 354 of 49 715 women with evaluable primary endpoint data in the digital breast tomosynthesis plus s2D group (detection rate 7·1 cases per 1000 women screened) and in 240 of 49 762 women in the digital mammography group (4·8 cases per 1000 women screened; odds ratio 1·48 [95% CI 1·25-1·75]; p<0·0001). Adverse events and device deficiencies were rare (six adverse events in each group; 23 device deficiencies in the digital breast tomosynthesis plus s2D group vs five device deficiencies in the digital mammography group) and no serious adverse events were reported.

INTERPRETATION

The results from this study indicate that the detection rate for invasive breast cancer was significantly higher with digital breast tomosynthesis plus s2D mammography than digital mammography alone. Evaluation of interval cancer rates in the follow-up study will further help to investigate incremental long-term benefits of digital breast tomosynthesis screening.

FUNDING

Deutsche Forschungsgemeinschaft (German Research Foundation).

Adaptive dose-response studies to establish proof-of-concept in learning-phase clinical trials

In learning-phase clinical trials in drug development, adaptive designs can be efficient and highly informative when used appropriately. In this article, we extend the multiple comparison procedures with modeling techniques (MCP-Mod) procedure with generalized multiple contrast tests (GMCTs) to two-stage adaptive designs for establishing proof-of-concept. The results of an interim analysis of first-stage data are used to adapt the candidate dose-response models and the dosages studied in the second stage. GMCTs are used in both stages to obtain stage-wise p -values, which are then combined to determine an overall p -value. An alternative approach is also considered that combines the t -statistics across stages, employing the conditional rejection probability principle to preserve the Type I error probability. Simulation studies demonstrate that the adaptive designs are advantageous compared to the corresponding tests in a nonadaptive design if the selection of the candidate set of dose-response models is not well informed by evidence from preclinical and early-phase studies.

A practical response adaptive block randomization (RABR) design with analytic type I error protection

Response adaptive randomization (RAR) is appealing from methodological, ethical, and pragmatic perspectives in the sense that subjects are more likely to be randomized to better performing treatment groups based on accumulating data. However, applications of RAR in confirmatory drug clinical trials with multiple active arms are limited largely due to its complexity, and lack of control of randomization ratios to different treatment groups. To address the aforementioned issues, we propose a Response Adaptive Block Randomization (RABR) design allowing arbitrarily prespecified randomization ratios for the control and high-performing groups to meet clinical trial objectives. We show the validity of the conventional unweighted test in RABR with a controlled type I error rate based on the weighted combination test for sample size adaptive design invoking no large sample approximation. The advantages of the proposed RABR in terms of robustly reaching target final sample size to meet regulatory requirements and increasing statistical power as compared with the popular Doubly Adaptive Biased Coin Design are demonstrated by statistical simulations and a practical clinical trial design example.

Sample size adaptations and evaluation of pediatric study interpretability

To extend marketing exclusivity, a drug manufacture is required by the Food and Drug Administration to conduct an interpretable study of approved drugs that may benefit pediatric patients. In return for conducting an interpretable study, the drug manufacturer is rewarded with an additional 6 months of marketing exclusivity. If the study design uses a fixed sample size and nothing is changed during the study, it is relatively easy to determine whether an interpretable study was conducted. However, if the sample size can be changed based on an unblinded interim analysis, it is difficult to answer the question of whether the study was interpretable. We investigate the optimal design from the manufacturers perspective and find the strategy for sample size adaptation that optimizes the utility including the chance of an interpretable study.

ANALYSIS OF "LEARN-AS-YOU-GO" (LAGO) STUDIES

In Learn-As-you-GO (LAGO) adaptive studies, the intervention is a complex multicomponent package, and is adapted in stages during the study based on past outcome data. This design formalizes standard practice in public health intervention studies. An effective intervention package is sought, while minimizing intervention package cost. In LAGO study data, the interventions in later stages depend upon the outcomes in the previous stages, violating standard statistical theory. We develop an estimator for the intervention effects, and prove consistency and asymptotic normality using a novel coupling argument, ensuring the validity of the test for the hypothesis of no overall intervention effect. We develop a confidence set for the optimal intervention package and confidence bands for the success probabilities under alternative package compositions. We illustrate our methods in the BetterBirth Study, which aimed to improve maternal and neonatal outcomes among 157,689 births in Uttar Pradesh, India through a multicomponent intervention package.

Peritoneal flap for lymphocele prophylaxis following robotic-assisted laparoscopic radical prostatectomy with pelvic lymph node dissection: study protocol and trial update for the randomized controlled PELYCAN study

Background

Data from interventional studies suggest that a peritoneal flap after pelvic lymph node dissection (LND) during laparoscopic, robotic-assisted radical prostatectomy (RARP) may reduce the rate of symptomatic lymphoceles in transperitoneal approach. However, most of these studies are not conducted in a randomized controlled fashion, thus limiting their scientific value. A recent prospective, randomized, controlled trial (RCT) did not show superiority of a peritoneal flap while further trials are lacking. Therefore, the aim of the presented RCT will be to show that creating a peritoneal flap decreases the rate of symptomatic lymphoceles compared to the current standard procedure without creation of a flap.

Methods/design

PELYCAN is a parallel-group, patient- and assessor-blinded, phase III, adaptive randomized controlled superiority trial. Men with histologically confirmed prostate cancer who undergo transperitoneal RARP with pelvic LND will be randomly assigned in a 1:1 ratio to two groups—either with creating a peritoneal flap (PELYCAN) or without creating a peritoneal flap (control). Sample size calculation yielded a sample size of 300 with a planned interim analysis after 120 patients, which will be performed by an independent statistician. This provides a possibility for early stopping or sample size recalculation.

Patients will be stratified for contributing factors for the development of postoperative lymphoceles. The primary outcome measure will be the rate of symptomatic lymphoceles in both groups within 6 months postoperatively. Patients and assessors will be blinded for the intervention until the end of the follow-up period of 6 months. The surgeon will be informed about the randomization result after performance of vesicourethral anastomosis. Secondary outcome measures include asymptomatic lymphoceles at the time of discharge and within 6 months of follow-up, postoperative complications, mortality, re-admission rate, and quality of life assessed by the EORTC QLQ-C30 questionnaire.

Discussion

The PELYCAN study is designed to assess whether the application of a peritoneal flap during RARP reduces the rate of symptomatic lymphoceles, as compared with the standard operation technique. In case of superiority of the intervention, this peritoneal flap may be suggested as a new standard of care.

Trial registration

German Clinical Trials Register DRKS00016794. Registered on 14 May 2019.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05168-x.

Sample size re-estimation for pivotal clinical trials

It is well known that if the hypothesis test is left unchanged, the Type I error rate may be inflated for sample size re-estimation (SSR) designs. To address this issue, three main approaches have been proposed in the literature: combination test, conditional error and conventional test with sample size increase in the allowable region (AR) only. These three seemingly different approaches are in fact connected. For each combination test, there is a corresponding conditional error function and AR. Designing adaptation rules in this AR with conventional test guarantees the Type I error rate control but at the same time always leads to smaller power comparing to the corresponding combination test (or conditional error) approach. In cases where conventional test is still preferable, step-wise type adaptation rules that do not fully reside in the AR can be alternatively considered. We believe controversies in the statistical community on the efficiency comparisons between group sequential (GS) and SSR design stem partially from the misalignment of performance metrics and conditional versus unconditional evaluations. We advocate summary metrics, such as median, variance or tail probabilities of the sample size in addition to expectation and personalizing efficiency definition for each trial sponsor. Conditional metrics by favorable, promising and unfavorable zones of the interim results provide additional insights and should always be incorporated into the decision-making process.

Exact change point detection with improved power in small-sample binomial sequences

To detect a change in the probability of a sequence of independent binomial random variables, a variety of asymptotic and exact testing procedures have been proposed. Whenever the sample size or the event rate is small, asymptotic approximations of maximally selected test statistics have been shown to be inaccurate. Although exact methods control the type I error rate, they can be overly conservative due to the discreteness of the test statistics in these situations. We extend approaches by Worsley and Halpern to develop a test that is less discrete to increase the power. Building on ideas from binary segmentation, the proposed test utilizes unused information in the binomial sequences to add a new ordering to test statistics that are of equal value. The exact distributions are derived under side conditions that arise in hypothetical segmentation steps and do not depend on the type of test statistic used (e.g., log likelihood ratio, cumulative sum, or Fisher's exact test). Using the proposed exact segmentation procedure, we construct a change point test and prove that it controls the type-I-error rate at any given nominal level. Furthermore, we prove that the new test is uniformly at least as powerful as Worsley's exact test. In a Monte Carlo simulation study, the gain in power can be remarkable, especially in scenarios with small sample size. Giving a clinical database example about pin site infections and an example assessing publication bias in neuropsychiatric drug research, we demonstrate the wide-ranging applicability of the test.

Clinical Trials Impacted by the COVID-19 Pandemic: Adaptive Designs to the Rescue?

Very recently the new pathogen severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified and the coronavirus disease 2019 (COVID-19) declared a pandemic by the World Health Organization. The pandemic has a number of consequences for ongoing clinical trials in non-COVID-19 conditions. Motivated by four current clinical trials in a variety of disease areas we illustrate the challenges faced by the pandemic and sketch out possible solutions including adaptive designs. Guidance is provided on (i) where blinded adaptations can help; (ii) how to achieve Type I error rate control, if required; (iii) how to deal with potential treatment effect heterogeneity; (iv) how to use early read-outs; and (v) how to use Bayesian techniques. In more detail approaches to resizing a trial affected by the pandemic are developed including considerations to stop a trial early, the use of group-sequential designs or sample size adjustment. All methods considered are implemented in a freely available R shiny app. Furthermore, regulatory and operational issues including the role of data monitoring committees are discussed.

Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods, simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so‐called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked‐up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.

Symmetric graphs for equally weighted tests, with application to the Hochberg procedure

The graphical approach to multiple testing provides a convenient tool for designing, visualizing, and performing multiplicity adjustments in confirmatory clinical trials while controlling the familywise error rate. It assigns a set of weights to each intersection null hypothesis within the closed test framework. These weights form the basis for intersection tests using weighted individual p-values, such as the weighted Bonferroni test. In this paper, we extend the graphical approach to intersection tests that assume equal weights for the elementary null hypotheses associated with any intersection hypothesis, including the Hochberg procedure as well as omnibus tests such as Fisher's combination, O'Brien's, and F tests. More specifically, we introduce symmetric graphs that generate sets of equal weights so that the aforementioned tests can be applied with the graphical approach. In addition, we visualize the Hochberg and the truncated Hochberg procedures in serial and parallel gatekeeping settings using symmetric component graphs. We illustrate the method with two clinical trial examples.

Adaptive trial designs in diagnostic accuracy research

The aim of diagnostic accuracy studies is to evaluate how accurately a diagnostic test can distinguish diseased from nondiseased individuals. Depending on the research question, different study designs and accuracy measures are appropriate. As the prior knowledge in the planning phase is often very limited, modifications of design aspects such as the sample size during the ongoing trial could increase the efficiency of diagnostic trials. In intervention studies, group sequential and adaptive designs are well established. Such designs are characterized by preplanned interim analyses, giving the opportunity to stop early for efficacy or futility or to modify elements of the study design. In contrast, in diagnostic accuracy studies, such flexible designs are less common, even if they are as important as for intervention studies. However, diagnostic accuracy studies have specific features, which may require adaptations of the statistical methods or may lead to specific advantages or limitations of sequential and adaptive designs. In this article, we summarize the current status of methodological research and applications of flexible designs in diagnostic accuracy research. Furthermore, we indicate and advocate future development of adaptive design methodology and their use in diagnostic accuracy trials from an interdisciplinary viewpoint. The term "interdisciplinary viewpoint" describes the collaboration of experts of the academic and nonacademic research.

A conditional error function approach for adaptive enrichment designs with continuous endpoints

Adaptive enrichment designs offer an efficient and flexible way to demonstrate the efficacy of a treatment in a clinically defined full population or in, eg, biomarker-defined subpopulations while controlling the family-wise Type I error rate in the strong sense. Frequently used testing strategies in designs with two or more stages include the combination test and the conditional error function approach. Here, we focus on the latter and present some extensions. In contrast to previous work, we allow for multiple subgroups rather than one subgroup only. For nested as well as nonoverlapping subgroups with normally distributed endpoints, we explore the effect of estimating the variances in the subpopulations. Instead of using a normal approximation, we derive new t-distribution-based methods for two different scenarios. First, in the case of equal variances across the subpopulations, we present exact results using a multivariate t-distribution. Second, in the case of potentially varying variances across subgroups, we provide some improved approximations compared to the normal approximation. The performance of the proposed conditional error function approaches is assessed and compared to the combination test in a simulation study. The proposed methods are motivated by an example in pulmonary arterial hypertension.

Group sequential adaptive designs in series of time-to-event randomised trials in rare diseases: A simulation study

In rare diseases, fully powered large trials may not be doable in a reasonable time frame even with international collaborations. In a previous work, we proposed an approach based on a series of smaller parallel group two-arm randomised controlled trials (RCT) performed over a long research horizon. Within the series of trials, the treatment selected after each trial becomes the control treatment of the next one. We concluded that running more trials with smaller sample sizes and relaxed α-levels leads in the long term and under reasonable assumptions to larger survival benefits with a moderate increase of risk as compared to traditional designs based on larger but fewer trials designed to meet stringent evidence criteria. We now extend this quantitative framework with more 'flexible' designs including interim analyses for futility and/or efficacy, and three-arm adaptive designs with treatment selection at interim. In the simulation study, we considered different disease severities, accrual rates, and hypotheses of how treatments improve over time. For each design, we estimated the long-term survival benefit as the relative difference in hazard rates between the end and the start of the research horizon, and the risk defined as the probability of selecting at the end of the research horizon a treatment inferior to the initial control. We assessed the impact of the α-level and the choice of the stopping rule on the operating characteristics. We also compared the performance of series based on two- vs. three-arm trials. We show that relaxing α-levels within the limit of 0.1 is associated with larger survival gains and moderate increase of risk which remains within acceptable ranges. Including an interim analysis with a futility rule is associated with an additional survival gain and a better risk control as compared to series with no interim analysis, when the α-level is below or equal to 0.1, whereas the benefit of including an interim analysis is rather small for higher α-levels. Including an interim analysis for efficacy yields almost no additional gain. Series based on three-arm trials are associated with a systematic improvement in terms of survival gain and risk control as compared to series of two-arm trials.

A group sequential test for ABR detection

Objective: To detect the auditory brainstem response (ABR) automatically using an innovative sequentially applied Hotelling's T ² test, with the overall goal of optimising test time whilst controlling the false-positive rate (FPR). Design: The stage-wise critical decision boundaries for accepting or rejecting the null hypothesis were found using a new approach called the Convolutional Group Sequential Test (CGST). Specificity, sensitivity, and test time were evaluated using simulations and subject recorded data. Study sample: Data consists of click-evoked ABR threshold series from 12 normal hearing adults, and recordings of EEG background activity from 17 normal hearing adults. Results: Reductions in mean test time of up to 40-45% were observed for the sequential test, relative to a conventional "single shot" test where the statistical test is applied to the data just once. To obtain these results, it will occasionally be necessary to run the test to a higher number of stimuli, i.e. the maximum test time needs to be increased. Conclusions: The CGST can be used to control the specificity of a sequentially applied ABR detection method. Doing so can reduce test time, relative to the "single shot" test, when considered across a cohort of test subjects.

The Convolutional Group Sequential Test: Reducing Test Time for Evoked Potentials

When using a statistical test for automatically detecting evoked potentials, the number of stimuli presented to the subject (the sample size for the statistical test) should be specified at the outset. For evoked response detection, this may be inefficient, i.e., because the signal-to-noise ratio (SNR) of the response is not known in advance, the user would usually err on the cautious side and use a relatively high number of stimuli to ensure adequate statistical power. A more efficient approach is to apply the statistical test repeatedly to the accumulating data over time, as this allows the test to be stopped early for the high SNR responses (thus reducing test time), or later for the low SNR responses. The caveat is that the critical decision boundaries for rejecting the null hypothesis need to be adjusted if the intended type-I error rate is to be obtained. This study presents an intuitive and flexible method for controlling the type-I error rate for sequentially applied statistical tests. The method is built around the discrete convolution of truncated probability density functions, which allows the null distribution for the test statistic to be constructed at each stage of the sequential analysis. Because the null distribution remains tractable, the procedure for finding the stage-wise critical decision boundaries is greatly simplified. The method also permits data-driven adaptations (using data from previous stages) to both the sample size and the statistical test, which offers new opportunities to speed up testing for evoked response detection.

Two-stage adaptive enrichment design for testing an active factor

We propose an adaptive enrichment approach to test an active factor, which is a factor whose effect is non-zero in at least one subpopulation. We implement a two-stage play-the-winner design where all subjects in the second stage are enrolled from the subpopulation that has the highest observed effect in the first stage. We recommend a weighted Fisher's combination of the most powerful test for each stage, respectively: the first stage Hotelling's test and the second stage noncentral chi-square test. The test is further extended to cover binary outcomes and time-to-event outcomes.

To add or not to add a new treatment arm to a multiarm study: A decision-theoretic framework

Multiarm clinical trials, which compare several experimental treatments against control, are frequently recommended due to their efficiency gain. In practise, all potential treatments may not be ready to be tested in a phase II/III trial at the same time. It has become appealing to allow new treatment arms to be added into on‐going clinical trials using a “platform” trial approach. To the best of our knowledge, many aspects of when to add arms to an existing trial have not been explored in the literature. Most works on adding arm(s) assume that a new arm is opened whenever a new treatment becomes available. This strategy may prolong the overall duration of a study or cause reduction in marginal power for each hypothesis if the adaptation is not well accommodated. Within a two‐stage trial setting, we propose a decision‐theoretic framework to investigate when to add or not to add a new treatment arm based on the observed stage one treatment responses. To account for different prospect of multiarm studies, we define utility in two different ways; one for a trial that aims to maximise the number of rejected hypotheses; the other for a trial that would declare a success when at least one hypothesis is rejected from the study. Our framework shows that it is not always optimal to add a new treatment arm to an existing trial. We illustrate a case study by considering a completed trial on knee osteoarthritis.

Nonparametric adaptive enrichment designs using categorical surrogate data

Adaptive survival trials are particularly important for enrichment designs in oncology and other life-threatening diseases. Current statistical methodology for adaptive survival trials provide type I error rate control only under restrictions. For instance, if we use stage-wise P values based on increments of the log-rank test, then the information used for the interim decisions need to be restricted to the primary survival endpoint. However, it is often desirable to base interim decisions also on correlated short-term endpoints like tumor response. Alternative statistical approaches based on a patient-wise splitting of the data require unnatural restrictions on the follow-up times and do not permit to efficiently account for an early rejection of the primary null hypothesis. We therefore suggest new approaches that enable us to use discrete surrogate endpoints (like tumor response status) and also to incorporate interim rejection boundaries. The new approaches are based on weighted Kaplan-Meier estimates and thereby have additional advantages. They permit us to account for nonproportional hazards and are robust against informative censoring based on the surrogate endpoint. We will show that nonproportionality is an intrinsic and relevant issue in enrichment designs. Moreover, informative censoring based on the surrogate endpoint is likely because of withdrawals and treatment switches after insufficient treatment response. It is shown and illustrated how nonparametric tests based on weighted Kaplan-Meier estimates can be used in closed combination tests for adaptive enrichment designs, such that type I error rate control is achieved and justified asymptotically.

Estimation after blinded sample size reassessment

Blinded sample size reassessment is a popular means to control the power in clinical trials if no reliable information on nuisance parameters is available in the planning phase. We investigate how sample size reassessment based on blinded interim data affects the properties of point estimates and confidence intervals for parallel group superiority trials comparing the means of a normal endpoint. We evaluate the properties of two standard reassessment rules that are based on the sample size formula of the z-test, derive the worst case reassessment rule that maximizes the absolute mean bias and obtain an upper bound for the mean bias of the treatment effect estimate.

Comparing the MAMS framework with the combination method in multi-arm adaptive trials with binary outcomes

In multi-arm adaptive trials, several treatments are assessed simultaneously and accumulating data are used to inform decisions about the trial, such as whether treatments are dropped or continued. Different methodological approaches have been developed for such trials and research has compared the performance of different subsets of these. One particular approach, for which we use the acronym MAMS(R), has generally not been included in these comparisons because control of the family-wise error rate (FWER) could not be guaranteed. Recently, the MAMS(R) approach has been extended to facilitate the generation of efficient designs which strongly control the FWER. We consider multi-arm two-stage trials with binary outcomes and propose parameterising treatment effects using the log odds ratio. We conduct a simulation study comparing the extended MAMS(R) framework with the well-established combination method both for trials where a different outcome is used for mid-trial analysis and for trials where the same outcome is used throughout. We show how the MAMS(R) framework compares favourably only in scenarios where the same outcome is used. We propose a hybrid selection rule within MAMS(R) methodology and demonstrate that this makes it possible to use the MAMS(R) framework in trials incorporating comparative treatment selection.

Controlling the type I error rate in two-stage sequential adaptive designs when testing for average bioequivalence

In a 2×2 crossover trial for establishing average bioequivalence (ABE) of a generic agent and a currently marketed drug, the recommended approach to hypothesis testing is the two one-sided test (TOST) procedure, which depends, among other things, on the estimated within-subject variability. The power of this procedure, and therefore the sample size required to achieve a minimum power, depends on having a good estimate of this variability. When there is uncertainty, it is advisable to plan the design in two stages, with an interim sample size reestimation after the first stage, using an interim estimate of the within-subject variability. One method and 3 variations of doing this were proposed by Potvin et al. Using simulation, the operating characteristics, including the empirical type I error rate, of the 4 variations (called Methods A, B, C, and D) were assessed by Potvin et al and Methods B and C were recommended. However, none of these 4 variations formally controls the type I error rate of falsely claiming ABE, even though the amount of inflation produced by Method C was considered acceptable. A major disadvantage of assessing type I error rate inflation using simulation is that unless all possible scenarios for the intended design and analysis are investigated, it is impossible to be sure that the type I error rate is controlled. Here, we propose an alternative, principled method of sample size reestimation that is guaranteed to control the type I error rate at any given significance level. This method uses a new version of the inverse-normal combination of p-values test, in conjunction with standard group sequential techniques, that is more robust to large deviations in initial assumptions regarding the variability of the pharmacokinetic endpoints. The sample size reestimation step is based on significance levels and power requirements that are conditional on the first-stage results. This necessitates a discussion and exploitation of the peculiar properties of the power curve of the TOST testing procedure. We illustrate our approach with an example based on a real ABE study and compare the operating characteristics of our proposed method with those of Method B of Povin et al.

Adaptive Designs with Discrete Test Statistics and Consideration of Overrunning

Background: Interim analyses are used in clinical trials in order to enable early decisions for medical, ethical, and economic reasons. However, it appears unfeasible to stop a trial during such an interim analysis. New patients will thus enter the trial while the interim analysis is ongoing. Moreover, depending on the event kinetics of the specific disease, the trial design, and the corresponding endpoints, some patients might still be unevaluable at the interim analysis due to not yet completed follow-up. Occur-rence of these types of patients is characteristic for sequentially analyzed trials. Such patients are referred to as interim patients. In trials with multiple primary endpoints, another type of interim patients occurs. If some but not all null hypotheses can be rejected at the interim analysis, the trial might be continued to a second stage in order to answer the remaining questions. These second stage patients, however, provide new data to all trial questions including the already rejected ones and thus formally act as interim patients regarding the already rejected null hypotheses. Although all kinds of interim patients are not part of the interim analysis, the data collected on those patients have to be sent to the office of regulatory affairs and will be analyzed. If a smaller or contrasting treatment effect is observed in interim patients, this might lead to a withdrawal of an earlier superiority proof.

Objectives: Presently, interim patients and their data are usually not considered in the confirmatory test. We offer a strategy to deal with interim patients in sequentially analyzed trials with discrete test statistics. The method covers sequentially analyzed single-and multi-arm trials with one or multiple primary endpoints.

Methods: When planning adaptive designs, it is common practice to assume that the stage-wise p-values are independent and standard uniformly distributed under the null hypothesis. In the context of discrete test statistics, this implies conservative tests. We provide an algorithm which iteratively optimizes an initially given design while adjusting for both discreteness of test statistics and interim patients. The algorithm is described verbally, graphically and formally to facilitate immediate implementation in computer software.

Results: The optimized design exploits the aspired significance level better and is more powerful than the initial one. The algorithm applies to fixed sample and planned flexible adaptive designs for single- and multi-arm trials with one or multiple primary endpoints. The benefit increases with the number of interim patients.

Conclusions: When planning a trial with interim analyses, the rules for decisions must be adjusted to interim patients. Otherwise, the test procedure is conservative resulting in loss of power. This is essential in situations where the number of interim patients is important compared to the first stage, particularly in trials with multiple primary endpoints.

Mixture-based gatekeeping procedures in adaptive clinical trials

Clinical trials with data-driven decision rules often pursue multiple clinical objectives such as the evaluation of several endpoints or several doses of an experimental treatment. These complex analysis strategies give rise to "multivariate" multiplicity problems with several components or sources of multiplicity. A general framework for defining gatekeeping procedures in clinical trials with adaptive multistage designs is proposed in this paper. The mixture method is applied to build a gatekeeping procedure at each stage and inferences at each decision point (interim or final analysis) are performed using the combination function approach. An advantage of utilizing the mixture method is that it enables powerful gatekeeping procedures applicable to a broad class of settings with complex logical relationships among the hypotheses of interest. Further, the combination function approach supports flexible data-driven decisions such as a decision to increase the sample size or remove a treatment arm. The paper concludes with a clinical trial example that illustrates the methodology by applying it to develop an adaptive two-stage design with a mixture-based gatekeeping procedure.

Accurate p-values for adaptive designs with binary endpoints

Adaptive designs encompass all trials allowing various types of design modifications over the course of the trial. A key requirement for confirmatory adaptive designs to be accepted by regulators is the strong control of the family-wise error rate. This can be achieved by combining the p-values for each arm and stage to account for adaptations (including but not limited to treatment selection), sample size adaptation and multiple stages. While the theory for this is novel and well-established, in practice, these methods can perform poorly, especially for unbalanced designs and for small to moderate sample sizes. The problem is that standard stagewise tests have inflated type I error rate, especially but not only when the baseline success rate is close to the boundary and this is carried over to the adaptive tests, seriously inflating the family-wise error rate. We propose to fix this problem by feeding the adaptive test with second-order accurate p-values, in particular bootstrap p-values. Secondly, an adjusted version of the Simes procedure for testing intersection hypotheses that reduces the built-in conservatism is suggested. Numerical work and simulations show that unlike their standard counterparts the new approach preserves the overall error rate, at or below the nominal level across the board, irrespective of the baseline rate, stagewise sample sizes or allocation ratio. Copyright © 2017 John Wiley & Sons, Ltd.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Maximum type I error rate inflation from sample size reassessment when investigators are blind to treatment labels

Consider a parallel group trial for the comparison of an experimental treatment to a control, where the second‐stage sample size may depend on the blinded primary endpoint data as well as on additional blinded data from a secondary endpoint. For the setting of normally distributed endpoints, we demonstrate that this may lead to an inflation of the type I error rate if the null hypothesis holds for the primary but not the secondary endpoint.

We derive upper bounds for the inflation of the type I error rate, both for trials that employ random allocation and for those that use block randomization. We illustrate the worst‐case sample size reassessment rule in a case study. For both randomization strategies, the maximum type I error rate increases with the effect size in the secondary endpoint and the correlation between endpoints. The maximum inflation increases with smaller block sizes if information on the block size is used in the reassessment rule. Based on our findings, we do not question the well‐established use of blinded sample size reassessment methods with nuisance parameter estimates computed from the blinded interim data of the primary endpoint. However, we demonstrate that the type I error rate control of these methods relies on the application of specific, binding, pre‐planned and fully algorithmic sample size reassessment rules and does not extend to general or unplanned sample size adjustments based on blinded data. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Sample size reassessment for a two-stage design controlling the false discovery rate

Sample size calculations for gene expression microarray and NGS-RNA-Seq experiments are challenging because the overall power depends on unknown quantities as the proportion of true null hypotheses and the distribution of the effect sizes under the alternative. We propose a two-stage design with an adaptive interim analysis where these quantities are estimated from the interim data. The second stage sample size is chosen based on these estimates to achieve a specific overall power. The proposed procedure controls the power in all considered scenarios except for very low first stage sample sizes. The false discovery rate (FDR) is controlled despite of the data dependent choice of sample size. The two-stage design can be a useful tool to determine the sample size of high-dimensional studies if in the planning phase there is high uncertainty regarding the expected effect sizes and variability.

Flexible selection of a single treatment incorporating short-term endpoint information in a phase II/III clinical trial

Seamless phase II/III clinical trials in which an experimental treatment is selected at an interim analysis have been the focus of much recent research interest. Many of the methods proposed are based on the group sequential approach. This paper considers designs of this type in which the treatment selection can be based on short-term endpoint information for more patients than have primary endpoint data available. We show that in such a case, the familywise type I error rate may be inflated if previously proposed group sequential methods are used and the treatment selection rule is not specified in advance. A method is proposed to avoid this inflation by considering the treatment selection that maximises the conditional error given the data available at the interim analysis. A simulation study is reported that illustrates the type I error rate inflation and compares the power of the new approach with two other methods: a combination testing approach and a group sequential method that does not use the short-term endpoint data, both of which also strongly control the type I error rate. The new method is also illustrated through application to a study in Alzheimer's disease.

A comparison of methods for treatment selection in seamless phase II/III clinical trials incorporating information on short-term endpoints

In an adaptive seamless phase II/III clinical trial interim analysis, data are used for treatment selection, enabling resources to be focused on comparison of more effective treatment(s) with a control. In this paper, we compare two methods recently proposed to enable use of short-term endpoint data for decision-making at the interim analysis. The comparison focuses on the power and the probability of correctly identifying the most promising treatment. We show that the choice of method depends on how well short-term data predict the best treatment, which may be measured by the correlation between treatment effects on short- and long-term endpoints.

Some challenges with statistical inference in adaptive designs

Adaptive designs have generated a great deal of attention to clinical trial communities. The literature contains many statistical methods to deal with added statistical uncertainties concerning the adaptations. Increasingly encountered in regulatory applications are adaptive statistical information designs that allow modification of sample size or related statistical information and adaptive selection designs that allow selection of doses or patient populations during the course of a clinical trial. For adaptive statistical information designs, a few statistical testing methods are mathematically equivalent, as a number of articles have stipulated, but arguably there are large differences in their practical ramifications. We pinpoint some undesirable features of these methods in this work. For adaptive selection designs, the selection based on biomarker data for testing the correlated clinical endpoints may increase statistical uncertainty in terms of type I error probability, and most importantly the increased statistical uncertainty may be impossible to assess.

Closed Testing in Pharmaceutical Research: Historical and Recent Developments

In pharmaceutical research, making multiple statistical inferences is standard practice. Unless adjustments are made for multiple testing, the probability of making erroneous determinations of significance increases with the number of inferences. Closed testing is a flexible and easily explained approach to controlling the overall error rate that has seen wide use in pharmaceutical research, particularly in clinical trials settings. In this article, we first give a general review of the uses of multiple testing in pharmaceutical research, with particular emphasis on the benefits and pitfalls of closed testing procedures. We then provide a more technical examination of a class of closed tests that use additive-combination-based and minimum-based p-value statistics, both of which are commonly used in pharmaceutical research. We show that, while the additive combination tests are generally far superior to minimum p-value tests for composite hypotheses, the reverse is true for multiple comparisons using closure-based testing. The loss of power of additive combination tests is explained in terms worst-case "hurdles" that must be cleared before significance can be determined via closed testing. We prove mathematically that this problem can result in the power of a closure-based minimum p-value test approaching 1, while the power of an closure-based additive combination test approaches 0. Finally, implications of these results to pharmaceutical researchers are given.

[Modern study designs and analysis methods in clinical research]

BACKGROUND

Progress in the field of medical research requires further development of clinical trial methodology to overcome the challenges resulting from small patient populations and restricted resources.

METHODS

Classical single-stage designs with fixed sample sizes do not allow for interim analyses or design modifications. In contrast, adaptive designs adhere to established quality criteria while providing flexibility when conducting a clinical trial. In the face of new discoveries or information collected in the course of a trial, sample size adjustment, the selection of the target population and further design modifications can be performed. This enhances the chance of success of a clinical trial. Besides adaptive designs, classical approaches may be replaced or complemented by Bayesian methods. In a Bayesian approach prior knowledge can be efficiently included and hence the amount of information utilized in statistical analyses is increased. Furthermore, Bayes procedures allow the results of a statistical evaluation to be displayed very clearly.

CONCLUSION

Modern approaches, such as adaptive designs and Bayesian designs overcome the challenges in clinical research due to enhanced flexibility and efficiency. In addition, both approaches can be combined.