Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: general concepts

CBER's Experience With Adaptive Design Clinical Trials

There is considerable interest among pharmaceutical and other medical product developers in adaptive clinical trials, in which knowledge learned during the course of a trial affects ongoing conduct or analysis of the trial. When the FDA released a draft Guidance document on adaptive design clinical trials in early 2010, expectations were high that it would lead to an increase in regulatory submissions involving adaptive design features, particularly for confirmatory trials. A 6-year (2008-2013) retrospective survey was performed within the Center for Biologics Evaluation and Research (CBER) at the FDA to gather information regarding the submission and evaluation of adaptive design trial proposals. We present an up-to-date summary of adaptive design proposals seen in CBER and provide an overview of our experiences. We share our concerns regarding the statistical issues and operational challenges raised during the review process for adaptive design trials. We also provide general recommendations for developing proposals for such trials. Our motivation in writing this paper was to encourage the best study design proposals to be submitted to CBER. Sometimes these can be adaptive, and sometimes a simpler design is most efficient.

PIPELINEs: Creating Comparable Clinical Knowledge Efficiently by Linking Trial Platforms

Adaptive, seamless, multisponsor, multitherapy clinical trial designs executed as large scale platforms, could create superior evidence more efficiently than single-sponsor, single-drug trials. These trial PIPELINEs also could diminish barriers to trial participation, increase the representation of real-world populations, and create systematic evidence development for learning throughout a therapeutic life cycle, to continually refine its use. Comparable evidence could arise from multiarm design, shared comparator arms, and standardized endpoints-aiding sponsors in demonstrating the distinct value of their innovative medicines; facilitating providers and patients in selecting the most appropriate treatments; assisting regulators in efficacy and safety determinations; helping payers make coverage and reimbursement decisions; and spurring scientists with translational insights. Reduced trial times and costs could enable more indications, reduced development cycle times, and improved system financial sustainability. Challenges to overcome range from statistical to operational to collaborative governance and data exchange.

Comparison of design strategies for a three-arm clinical trial with time-to-event endpoint: Power, time-to-analysis, and operational aspects

To optimize resources, randomized clinical trials with multiple arms can be an attractive option to simultaneously test various treatment regimens in pharmaceutical drug development. The motivation for this work was the successful conduct and positive final outcome of a three-arm randomized clinical trial primarily assessing whether obinutuzumab plus chlorambucil in patients with chronic lympocytic lymphoma and coexisting conditions is superior to chlorambucil alone based on a time-to-event endpoint. The inference strategy of this trial was based on a closed testing procedure. We compare this strategy to three potential alternatives to run a three-arm clinical trial with a time-to-event endpoint. The primary goal is to quantify the differences between these strategies in terms of the time it takes until the first analysis and thus potential approval of a new drug, number of required events, and power. Operational aspects of implementing the various strategies are discussed. In conclusion, using a closed testing procedure results in the shortest time to the first analysis with a minimal loss in power. Therefore, closed testing procedures should be part of the statistician's standard clinical trials toolbox when planning multiarm clinical trials.

A multi-stage drop-the-losers design for multi-arm clinical trials

Multi-arm multi-stage trials can improve the efficiency of the drug development process when multiple new treatments are available for testing. A group-sequential approach can be used in order to design multi-arm multi-stage trials, using an extension to Dunnett’s multiple-testing procedure. The actual sample size used in such a trial is a random variable that has high variability. This can cause problems when applying for funding as the cost will also be generally highly variable. This motivates a type of design that provides the efficiency advantages of a group-sequential multi-arm multi-stage design, but has a fixed sample size. One such design is the two-stage drop-the-losers design, in which a number of experimental treatments, and a control treatment, are assessed at a prescheduled interim analysis. The best-performing experimental treatment and the control treatment then continue to a second stage. In this paper, we discuss extending this design to have more than two stages, which is shown to considerably reduce the sample size required. We also compare the resulting sample size requirements to the sample size distribution of analogous group-sequential multi-arm multi-stage designs. The sample size required for a multi-stage drop-the-losers design is usually higher than, but close to, the median sample size of a group-sequential multi-arm multi-stage trial. In many practical scenarios, the disadvantage of a slight loss in average efficiency would be overcome by the huge advantage of a fixed sample size. We assess the impact of delay between recruitment and assessment as well as unknown variance on the drop-the-losers designs.

Adaptive Clinical Trial Design: An Overview and Potential Applications in Dermatology

The challenges of drug development, including increasing costs, late-stage drug failures, and the decline in the number of drugs being approved by the US Food and Drug Administration over time, have generated interest in adaptive study designs that have the potential to address these problems. Adaptive trial designs use interim data analysis to amend trials, and have been recognized for more than a decade as a way to increase trial efficiency, partly by the increased probability of demonstrating a drug effect if one exists. In this article, we define adaptive trials; give examples of the most common types; highlight the pros, cons, and ethical considerations of these designs; and illustrate how these tools can be applied to drug development in dermatology.

Unbiased estimation in seamless phase II/III trials with unequal treatment effect variances and hypothesis-driven selection rules

Seamless phase II/III clinical trials offer an efficient way to select an experimental treatment and perform confirmatory analysis within a single trial. However, combining the data from both stages in the final analysis can induce bias into the estimates of treatment effects. Methods for bias adjustment developed thus far have made restrictive assumptions about the design and selection rules followed. In order to address these shortcomings, we apply recent methodological advances to derive the uniformly minimum variance conditionally unbiased estimator for two‐stage seamless phase II/III trials. Our framework allows for the precision of the treatment arm estimates to take arbitrary values, can be utilised for all treatments that are taken forward to phase III and is applicable when the decision to select or drop treatment arms is driven by a multiplicity‐adjusted hypothesis testing procedure. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Group sequential designs with prospectively planned rules for subpopulation enrichment

We propose a class of randomized trial designs aimed at gaining the advantages of wider generalizability and faster recruitment while mitigating the risks of including a population for which there is greater a priori uncertainty. We focus on testing null hypotheses for the overall population and a predefined subpopulation. Our designs have preplanned rules for modifying enrollment criteria based on data accrued at interim analyses. For example, enrollment can be restricted if the participants from a predefined subpopulation are not benefiting from the new treatment. Our designs have the following features: the multiple testing procedure fully leverages the correlation among statistics for different populations; the asymptotic familywise Type I error rate is strongly controlled; for outcomes that are binary or normally distributed, the decision rule and multiple testing procedure are functions of the data only through minimal sufficient statistics. Our designs incorporate standard group sequential boundaries for each population of interest; this may be helpful in communicating the designs, because many clinical investigators are familiar with such boundaries, which can be summarized succinctly in a single table or graph. We demonstrate these designs through simulations of a Phase III trial of a new treatment for stroke. User-friendly, free software implementing these designs is described. Copyright © 2016 John Wiley & Sons, Ltd.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Testing multiple primary endpoints in clinical trials with sample size adaptation

In this paper, we propose a design that uses a short-term endpoint for accelerated approval at interim analysis and a long-term endpoint for full approval at final analysis with sample size adaptation based on the long-term endpoint. Two sample size adaptation rules are compared: an adaptation rule to maintain the conditional power at a prespecified level and a step function type adaptation rule to better address the bias issue. Three testing procedures are proposed: alpha splitting between the two endpoints; alpha exhaustive between the endpoints; and alpha exhaustive with improved critical value based on correlation. Family-wise error rate is proved to be strongly controlled for the two endpoints, sample size adaptation, and two analysis time points with the proposed designs. We show that using alpha exhaustive designs greatly improve the power when both endpoints are effective, and the power difference between the two adaptation rules is minimal. The proposed design can be extended to more general settings.

Flexible selection of a single treatment incorporating short-term endpoint information in a phase II/III clinical trial

Seamless phase II/III clinical trials in which an experimental treatment is selected at an interim analysis have been the focus of much recent research interest. Many of the methods proposed are based on the group sequential approach. This paper considers designs of this type in which the treatment selection can be based on short-term endpoint information for more patients than have primary endpoint data available. We show that in such a case, the familywise type I error rate may be inflated if previously proposed group sequential methods are used and the treatment selection rule is not specified in advance. A method is proposed to avoid this inflation by considering the treatment selection that maximises the conditional error given the data available at the interim analysis. A simulation study is reported that illustrates the type I error rate inflation and compares the power of the new approach with two other methods: a combination testing approach and a group sequential method that does not use the short-term endpoint data, both of which also strongly control the type I error rate. The new method is also illustrated through application to a study in Alzheimer's disease.

Seamless Phase IIa/IIb and enhanced dose-finding adaptive design

In drug development, when the drug class has a relatively well-defined path to regulatory approval and the enrollment is slow with certain patient populations, one may want to consider combining studies of different phases. This article considers combining a proof of concept (POC) study and a dose-finding (DF) study with a control treatment. Conventional DF study designs sometimes are not efficient, or do not have a high probability to find the optimal dose(s) for Phase III trials. This article seeks more efficient DF strategies that allow the economical testing of more doses. Hypothetical examples are simulated to compare the proposed adaptive design vs. the conventional design based on different models of the overall quantitative representation of efficacy, safety, and tolerability. The results show that the proposed adaptive design tests more active doses with higher power and comparable or smaller sample size in a shorter overall study duration for POC and DF, compared with a conventional design.

Estimation after subpopulation selection in adaptive seamless trials

During the development of new therapies, it is not uncommon to test whether a new treatment works better than the existing treatment for all patients who suffer from a condition (full population) or for a subset of the full population (subpopulation). One approach that may be used for this objective is to have two separate trials, where in the first trial, data are collected to determine if the new treatment benefits the full population or the subpopulation. The second trial is a confirmatory trial to test the new treatment in the population selected in the first trial. In this paper, we consider the more efficient two-stage adaptive seamless designs (ASDs), where in stage 1, data are collected to select the population to test in stage 2. In stage 2, additional data are collected to perform confirmatory analysis for the selected population. Unlike the approach that uses two separate trials, for ASDs, stage 1 data are also used in the confirmatory analysis. Although ASDs are efficient, using stage 1 data both for selection and confirmatory analysis introduces selection bias and consequently statistical challenges in making inference. We will focus on point estimation for such trials. In this paper, we describe the extent of bias for estimators that ignore multiple hypotheses and selecting the population that is most likely to give positive trial results based on observed stage 1 data. We then derive conditionally unbiased estimators and examine their mean squared errors for different scenarios.

A comparison of methods for treatment selection in seamless phase II/III clinical trials incorporating information on short-term endpoints

In an adaptive seamless phase II/III clinical trial interim analysis, data are used for treatment selection, enabling resources to be focused on comparison of more effective treatment(s) with a control. In this paper, we compare two methods recently proposed to enable use of short-term endpoint data for decision-making at the interim analysis. The comparison focuses on the power and the probability of correctly identifying the most promising treatment. We show that the choice of method depends on how well short-term data predict the best treatment, which may be measured by the correlation between treatment effects on short- and long-term endpoints.

Closed Testing in Pharmaceutical Research: Historical and Recent Developments

In pharmaceutical research, making multiple statistical inferences is standard practice. Unless adjustments are made for multiple testing, the probability of making erroneous determinations of significance increases with the number of inferences. Closed testing is a flexible and easily explained approach to controlling the overall error rate that has seen wide use in pharmaceutical research, particularly in clinical trials settings. In this article, we first give a general review of the uses of multiple testing in pharmaceutical research, with particular emphasis on the benefits and pitfalls of closed testing procedures. We then provide a more technical examination of a class of closed tests that use additive-combination-based and minimum-based p-value statistics, both of which are commonly used in pharmaceutical research. We show that, while the additive combination tests are generally far superior to minimum p-value tests for composite hypotheses, the reverse is true for multiple comparisons using closure-based testing. The loss of power of additive combination tests is explained in terms worst-case "hurdles" that must be cleared before significance can be determined via closed testing. We prove mathematically that this problem can result in the power of a closure-based minimum p-value test approaching 1, while the power of an closure-based additive combination test approaches 0. Finally, implications of these results to pharmaceutical researchers are given.

Adaptive design of confirmatory trials: Advances and challenges

The past decade witnessed major developments in innovative designs of confirmatory clinical trials, and adaptive designs represent the most active area of these developments. We give an overview of the developments and associated statistical methods in several classes of adaptive designs of confirmatory trials. We also discuss their statistical difficulties and implementation challenges, and show how these problems are connected to other branches of mainstream Statistics, which we then apply to resolve the difficulties and bypass the bottlenecks in the development of adaptive designs for the next decade.

Adaptive Clinical Trials: Overview of Phase III Designs and Challenges

Adaptive designs use accruing data to make changes in an ongoing trial according to a prespecified plan and potentially offer great efficiencies for clinical development. There are many types of adaptive designs and many trial aspects that could in theory be adapted. However, the scope of adaptive designs with relevance in confirmatory trials is narrower, and in addition, extensive pre-planning is needed and various types of challenges need to be addressed in order to use these designs in this stage of development. Nevertheless, with careful planning, there are opportunities for these designs to offer important benefits even in the confirmatory stage of development. We provide an overview of adaptive designs that have relevance for confirmatory trials and discuss considerations that may affect whether they should or should not be used in particular trials or programs as well as the challenges that need to be addressed.

Advances in Statistical Approaches Oncology Drug Development

We describe some recent developments in statistical methodology and practice in oncology drug development from an academic and an industry perspective. Many adaptive designs were pioneered in oncology, and oncology is still at the forefront of novel methods to enable better and faster Go/No-Go decision making while controlling the cost.

A simple and flexible graphical approach for adaptive group-sequential clinical trials

In this article, we introduce a graphical approach to testing multiple hypotheses in group-sequential clinical trials allowing for midterm design modifications. It is intended for structured study objectives in adaptive clinical trials and extends the graphical group-sequential designs from Maurer and Bretz (Statistics in Biopharmaceutical Research 2013; 5: 311-320) to adaptive trial designs. The resulting test strategies can be visualized graphically and performed iteratively. We illustrate the methodology with two examples from our clinical trial practice. First, we consider a three-armed gold-standard trial with the option to reallocate patients to either the test drug or the active control group, while stopping the recruitment of patients to placebo, after having demonstrated superiority of the test drug over placebo at an interim analysis. Second, we consider a confirmatory two-stage adaptive design with treatment selection at interim.

Optimizing the data combination rule for seamless phase II/III clinical trials

We consider seamless phase II/III clinical trials that compare K treatments with a common control in phase II then test the most promising treatment against control in phase III. The final hypothesis test for the selected treatment can use data from both phases, subject to controlling the familywise type I error rate. We show that the choice of method for conducting the final hypothesis test has a substantial impact on the power to demonstrate that an effective treatment is superior to control. To understand these differences in power, we derive decision rules maximizing power for particular configurations of treatment effects. A rule with such an optimal frequentist property is found as the solution to a multivariate Bayes decision problem. The optimal rules that we derive depend on the assumed configuration of treatment means. However, we are able to identify two decision rules with robust efficiency: a rule using a weighted average of the phase II and phase III data on the selected treatment and control, and a closed testing procedure using an inverse normal combination rule and a Dunnett test for intersection hypotheses. For the first of these rules, we find the optimal division of a given total sample size between phases II and III. We also assess the value of using phase II data in the final analysis and find that for many plausible scenarios, between 50% and 70% of the phase II numbers on the selected treatment and control would need to be added to the phase III sample size in order to achieve the same increase in power. © 2014 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

A seamless phase IIB/III adaptive outcome trial: design rationale and implementation challenges

BACKGROUND

The licensed four-valent prophylactic human papillomavirus vaccine is highly efficacious in preventing cervical, vulvar, vaginal, and anal cancers and related precancers caused by human papillomavirus types 6, 11, 16, and 18. These four types account for approximately 70% of cervical cancers. A nine-valent human papillomavirus vaccine, including the four original types (6, 11, 16, and 18) plus the next five most prevalent types in cervical cancer (31, 33, 45, 52, and 58) could provide approximately 90% overall cervical cancer coverage. To expedite the nine-valent human papillomavirus vaccine clinical development, an adaptive, seamless Phase IIB/III outcome trial with ∼ 15,000 subjects was conducted to facilitate dose formulation selection and provide pivotal evidence of safety and efficacy for regulatory registrations.

PURPOSE

We discuss the design rationale and implementation challenges of the outcome trial, focusing on the adaptive feature of the seamless Phase IIB/III design.

METHODS

Subjects were enrolled in two parts (Part A and Part B). Approximately 1240 women, 16-26 years of age, were enrolled in Part A for Phase IIB evaluation and equally randomized to one of three dose formulations of the nine-valent human papillomavirus vaccine or the four-valent human papillomavirus vaccine (active control). Based on an interim analysis of immunogenicity and safety, one dose formulation of the nine-valent human papillomavirus vaccine was selected for evaluation in the Phase III part of the study. Subjects enrolled in Part A who received the selected dose formulation of the nine-valent human papillomavirus vaccine or four-valent human papillomavirus vaccine continued to be followed up and contributed to the final efficacy and safety analyses. In addition, ∼ 13,400 women 16-26 years of age were enrolled in Part B, randomized to nine-valent human papillomavirus vaccine at the selected dose formulation or four-valent human papillomavirus vaccine, and followed for immunogenicity, efficacy, and safety.

RESULTS

A seamless Phase IIB/III design was justified by the extensive pre-existing knowledge of the licensed four-valent human papillomavirus vaccine and the development objectives for the nine-valent human papillomavirus vaccine. Subjects enrolled in Part A who received either the selected nine-valent human papillomavirus formulation or four-valent human papillomavirus vaccine contributed ∼ 10% of person-years of follow-up due to its earlier start-thereby maximizing the overall efficiency of the trial. Some of the challenges encountered in the implementation of the adaptive design included practical considerations during Phase IIB formulation selection by internal and external committees, End-of-Phase II discussion with health authorities and managing changes in the assay for immunological endpoints.

LIMITATIONS

Application of the experience and lesson learned from this seamless adaptive design to other clinical programs may depend on case-by-case consideration.

CONCLUSION

A seamless Phase IIB/III adaptive design was successfully implemented in this large outcome study. The development time of the second-generation nine-valent human papillomavirus vaccine was shortened due to improved statistical efficiency.

Design of telehealth trials--introducing adaptive approaches

BACKGROUND

The field of telehealth and telemedicine is expanding as the need to improve efficiency of health care becomes more pressing. The decision to implement a telehealth system is generally an expensive undertaking that impacts a large number of patients and other stakeholders. It is therefore extremely important that the decision is fully supported by accurate evaluation of telehealth interventions.

OBJECTIVE

Numerous reviews of telehealth have described the evidence base as inconsistent. In response they call for larger, more rigorously controlled trials, and trials which go beyond evaluation of clinical effectiveness alone. The aim of this paper is to discuss various ways in which evaluation of telehealth could be improved by the use of adaptive trial designs.

RESULTS

We discuss various adaptive design options, such as sample size reviews and changing the study hypothesis to address uncertain parameters, group sequential trials and multi-arm multi-stage trials to improve efficiency, and enrichment designs to maximise the chances of obtaining clear evidence about the telehealth intervention.

CONCLUSION

There is potential to address the flaws discussed in the telehealth literature through the adoption of adaptive approaches to trial design. Such designs could lead to improvements in efficiency, allow the evaluation of multiple telehealth interventions in a cost-effective way, or accurately assess a range of endpoints that are important in the overall success of a telehealth programme.

Data-driven treatment selection for seamless phase II/III trials incorporating early-outcome data

Seamless phase II/III clinical trials are conducted in two stages with treatment selection at the first stage. In the first stage, patients are randomized to a control or one of k > 1 experimental treatments. At the end of this stage, interim data are analysed, and a decision is made concerning which experimental treatment should continue to the second stage. If the primary endpoint is observable only after some period of follow-up, at the interim analysis data may be available on some early outcome on a larger number of patients than those for whom the primary endpoint is available. These early endpoint data can thus be used for treatment selection. For two previously proposed approaches, the power has been shown to be greater for one or other method depending on the true treatment effects and correlations. We propose a new approach that builds on the previously proposed approaches and uses data available at the interim analysis to estimate these parameters and then, on the basis of these estimates, chooses the treatment selection method with the highest probability of correctly selecting the most effective treatment. This method is shown to perform well compared with the two previously described methods for a wide range of true parameter values. In most cases, the performance of the new method is either similar to or, in some cases, better than either of the two previously proposed methods.

An adaptive two-stage dose-response design method for establishing proof of concept

We propose an adaptive two-stage dose-response design where a prespecified adaptation rule is used to add and/or drop treatment arms between the stages. We extend the multiple comparison procedures-modeling (MCP-Mod) approach into a two-stage design. In each stage, we use the same set of candidate dose-response models and test for a dose-response relationship or proof of concept (PoC) via model-associated statistics. The stage-wise test results are then combined to establish "global" PoC using a conditional error function. Our simulation studies showed good and more robust power in our design method compared to conventional and fixed designs.

Physiotherapy Rehabilitation for Osteoporotic Vertebral Fracture (PROVE): study protocol for a randomised controlled trial

Background

Osteoporosis and vertebral fracture can have a considerable impact on an individual’s quality of life. There is increasing evidence that physiotherapy including manual techniques and exercise interventions may have an important treatment role. This pragmatic randomised controlled trial will investigate the clinical and cost-effectiveness of two different physiotherapy approaches for people with osteoporosis and vertebral fracture, in comparison to usual care.

Methods/Design

Six hundred people with osteoporosis and a clinically diagnosed vertebral fracture will be recruited and randomly allocated to one of three management strategies, usual care (control - A), an exercise-based physiotherapy intervention (B) or a manual therapy-based physiotherapy intervention (C). Those in the usual care arm will receive a single session of education and advice, those in the active treatment arms (B + C) will be offered seven individual physiotherapy sessions over 12 weeks. The trial is designed as a prospective, adaptive single-blinded randomised controlled trial. An interim analysis will be completed and if one intervention is clearly superior the trial will be adapted at this point to continue with just one intervention and the control. The primary outcomes are quality of life measured by the disease specific QUALLEFO 41 and the Timed Loaded Standing test measured at 1 year.

Discussion

There are a variety of different physiotherapy packages used to treat patients with osteoporotic vertebral fracture. At present, the indication for each different therapy is not well defined, and the effectiveness of different modalities is unknown.

Trial registration

Reference number ISRCTN49117867.