Use of Seamless Study Designs in Oncology Clinical Development- A Survey Conducted by IDSWG Oncology Sub-team

Seamless study designs have the potential to accelerate clinical development. The use of innovative seamless designs has been increasing in the oncology area; however, while the concept of seamless designs becomes more popular and accepted, many challenges remain in both the design and conduct of these trials. This may be especially true when seamless designs are used in late phase development supporting regulatory decision-making. The Innovative Design Scientific Working Group (IDSWG) Oncology team conducted a survey to understand the current use of seamless study designs for registration purposes in oncology clinical development. The survey was designed to provide insights into the benefits and to identify the roadblocks. A total of 16 questions were included in the survey that was distributed using the ASA Biopharmaceutical Section and IDSWG email listings from August to September 2022. A total of 51 responses were received, with 39 (76%) respondents indicating that their organizations had seamless oncology studies in planning or implementation for registration purposes. Detailed survey results are presented in the manuscript. Overall, while seamless designs offer advantages in terms of timeline reduction and cost saving, they also present challenges related to additional complexity and the need for efficient surrogate clinical endpoints in oncology drug development.

Point estimation, confidence intervals, and P-values for optimal adaptive two-stage designs with normal endpoints

Due to the dependency structure in the sampling process, adaptive trial designs create challenges in point and interval estimation and in the calculation of P-values. Optimal adaptive designs, which are designs where the parameters governing the adaptivity are chosen to maximize some performance criterion, suffer from the same problem. Various analysis methods which are able to handle this dependency structure have already been developed. In this work, we aim to give a comprehensive summary of these methods and show how they can be applied to the class of designs with planned adaptivity, of which optimal adaptive designs are an important member. The defining feature of these kinds of designs is that the adaptive elements are completely prespecified. This allows for explicit descriptions of the calculations involved, which makes it possible to evaluate different methods in a fast and accurate manner. We will explain how to do so, and present an extensive comparison of the performance characteristics of various estimators between an optimal adaptive design and its group-sequential counterpart.

A simulation-based comparison of estimation methods for adaptive and classical group sequential clinical trials

Statistical methods for controlling the type-I error of hypothesis tests in adaptive group sequential clinical trials are well established and well understood. However, methods for obtaining statistically valid point estimates and confidence intervals for adaptive designs are not as well established or as well understood. At the end of an adaptive trial, one may calculate the repeated confidence interval (RCI), which provides conservative coverage of δ , or the backward image confidence interval (BWCI), which provides exact coverage of δ and is an extension of the stagewise adjusted confidence interval (SWCI, used in classical group sequential designs). The BWCI can also provide a median unbiased estimate (MUE) of δ . There is a need to better understand the coverage and possible biases associated with these methods. We conducted a simulation study exploring parameter estimation following sample size reestimation based on testing methods with strong control of type-I error. Generally, the BWCI provided exact coverage, the naïve CI provided inconsistent coverage, and the RCI provided conservative coverage. Additionally, we note considerable asymmetry in the coverage from above/from below for the RCI, although we did not see any instance where the 95% RCI excluded the true parameter more than 2.5% on either side. At the end of an adaptive group sequential trial, we strongly recommend the use of the BWCI (and associated MUE), with the RCI computed during interim looks; the naïve CI should be avoided. These results and conclusions also hold true for classical group sequential designs.

Clinical Trial Designs for Neuromodulation in Chronic Spinal Cord Injury Using Epidural Stimulation

STUDY DESIGN

This is a narrative review focused on specific challenges related to adequate controls that arise in neuromodulation clinical trials involving perceptible stimulation and physiological effects of stimulation activation.

OBJECTIVES

1) To present the strengths and limitations of available clinical trial research designs for the testing of epidural stimulation to improve recovery after spinal cord injury. 2) To describe how studies can control for the placebo effects that arise due to surgical implantation, the physical presence of the battery, generator, control interfaces, and rehabilitative activity aimed to promote use-dependent plasticity. 3) To mitigate Hawthorne effects that may occur in clinical trials with intensive supervised participation, including rehabilitation.

MATERIALS AND METHODS

Focused literature review of neuromodulation clinical trials with integration to the specific context of epidural stimulation for persons with chronic spinal cord injury.

CONCLUSIONS

Standard of care control groups fail to control for the multiple effects of knowledge of having undergone surgical procedures, having implanted stimulation systems, and being observed in a clinical trial. The irreducible effects that have been identified as "placebo" require sham controls or comparison groups in which both are implanted with potentially active devices and undergo similar rehabilitative training.

Optimizing subgroup selection in two-stage adaptive enrichment and umbrella designs

We design two‐stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision‐theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per‐comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.

Sample size re-estimation for pivotal clinical trials

It is well known that if the hypothesis test is left unchanged, the Type I error rate may be inflated for sample size re-estimation (SSR) designs. To address this issue, three main approaches have been proposed in the literature: combination test, conditional error and conventional test with sample size increase in the allowable region (AR) only. These three seemingly different approaches are in fact connected. For each combination test, there is a corresponding conditional error function and AR. Designing adaptation rules in this AR with conventional test guarantees the Type I error rate control but at the same time always leads to smaller power comparing to the corresponding combination test (or conditional error) approach. In cases where conventional test is still preferable, step-wise type adaptation rules that do not fully reside in the AR can be alternatively considered. We believe controversies in the statistical community on the efficiency comparisons between group sequential (GS) and SSR design stem partially from the misalignment of performance metrics and conditional versus unconditional evaluations. We advocate summary metrics, such as median, variance or tail probabilities of the sample size in addition to expectation and personalizing efficiency definition for each trial sponsor. Conditional metrics by favorable, promising and unfavorable zones of the interim results provide additional insights and should always be incorporated into the decision-making process.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

Adaptive multiarm multistage clinical trials

Two methods for designing adaptive multiarm multistage (MAMS) clinical trials, originating from conceptually different group sequential frameworks are presented, and their operating characteristics are compared. In both methods pairwise comparisons are made, stage-by-stage, between each treatment arm and a common control arm with the goal of identifying active treatments and dropping inactive ones. At any stage one may alter the future course of the trial through adaptive changes to the prespecified decision rules for treatment selection and sample size reestimation, and notwithstanding such changes, both methods guarantee strong control of the family-wise error rate. The stage-wise MAMS approach was historically the first to be developed and remains the standard method for designing inferentially seamless phase 2-3 clinical trials. In this approach, at each stage, the data from each treatment comparison are summarized by a single multiplicity adjusted P-value. These stage-wise P-values are combined by a prespecified combination function and the resultant test statistic is monitored with respect to the classical two-arm group sequential efficacy boundaries. The cumulative MAMS approach is a more recent development in which a separate test statistic is constructed for each treatment comparison from the cumulative data at each stage. These statistics are then monitored with respect to multiplicity adjusted group sequential efficacy boundaries. We compared the powers of the two methods for designs with two and three active treatment arms, under commonly utilized decision rules for treatment selection, sample size reestimation and early stopping. In our investigations, which were carried out over a reasonably exhaustive exploration of the parameter space, the cumulative MAMS designs were more powerful than the stage-wise MAMS designs, except for the homogeneous case of equal treatment effects, where a small power advantage was discernable for the stage-wise MAMS designs.

Estimation after blinded sample size reassessment

Blinded sample size reassessment is a popular means to control the power in clinical trials if no reliable information on nuisance parameters is available in the planning phase. We investigate how sample size reassessment based on blinded interim data affects the properties of point estimates and confidence intervals for parallel group superiority trials comparing the means of a normal endpoint. We evaluate the properties of two standard reassessment rules that are based on the sample size formula of the z-test, derive the worst case reassessment rule that maximizes the absolute mean bias and obtain an upper bound for the mean bias of the treatment effect estimate.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Exact inference for adaptive group sequential designs

Methods for controlling the type-1 error of an adaptive group sequential trial were developed in seminal papers by Cui, Hung, and Wang (Biometrics, 1999), Lehmacher and Wassmer (Biometrics, 1999), and Müller and Schäfer (Biometrics, 2001). However, corresponding solutions for the equally important and related problem of parameter estimation at the end of the adaptive trial have not been completely satisfactory. In this paper, a method is provided for computing a two-sided confidence interval having exact coverage, along with a point estimate that is median unbiased for the primary efficacy parameter in a two-arm adaptive group sequential design. The possible adaptations are not only confined to sample size alterations but also include data-dependent changes in the number and spacing of interim looks and changes in the error spending function. The procedure is based on mapping the final test statistic obtained in the modified trial into a corresponding backward image in the original trial. This is an advance on previously available methods, which either produced conservative coverage and no point estimates or provided exact coverage for one-sided intervals only.

Adaptive increase in sample size when interim results are promising: a practical guide with examples

This paper discusses the benefits and limitations of adaptive sample size re-estimation for phase 3 confirmatory clinical trials. Comparisons are made with more traditional fixed sample and group sequential designs. It is seen that the real benefit of the adaptive approach arises through the ability to invest sample size resources into the trial in stages. The trial starts with a small up-front sample size commitment. Additional sample size resources are committed to the trial only if promising results are obtained at an interim analysis. This strategy is shown through examples of actual trials, one in neurology and one in cardiology, to be more advantageous than the fixed sample or group sequential approaches in certain settings. A major factor that has generated controversy and inhibited more widespread use of these methods has been their reliance on non-standard tests and p-values for preserving the type-1 error. If, however, the sample size is only increased when interim results are promising, one can dispense with these non-standard methods of inference. Therefore, in the spirit of making adaptive increases in trial size more widely appealing and readily implementable we here define those promising circumstances in which a conventional final inference can be performed while preserving the overall type-1 error. Methodological, regulatory and operational issues are examined.