Adaptive Methods: Telling “The Rest of the Story”

The Future Ain't What It Used to Be…Out With the Old…In With the Better: Antibacterial Resistance Leadership Group Innovations

Clinical research networks conduct important studies that would not otherwise be performed by other entities. In the case of the Antibacterial Resistance Leadership Group (ARLG), such studies include diagnostic studies using master protocols, controlled phage intervention trials, and studies that evaluate treatment strategies or dynamic interventions, such as sequences of empiric and definitive therapies. However, the value of a clinical research network lies not only in the results from these important studies but in the creation of new approaches derived from collaborative thinking, carefully examining and defining the most important research questions for clinical practice, recognizing and addressing common but suboptimal approaches, and anticipating that the standard approaches of today may be insufficient for tomorrow. This results in the development and implementation of new methodologies and tools for the design, conduct, analyses, and reporting of research studies. These new methodologies directly impact the studies conducted within the network and have a broad and long-lasting impact on the field, enhancing the scientific value and efficiency of generations of research studies. This article describes innovations from the ARLG in diagnostic studies, observational studies, and clinical trials evaluating interventions for the prevention and treatment of antibiotic-resistant bacterial infections.

Evolocumab's Long-Term Mortality Risk Unclear Due to Shortened Follow-Up of FOURIER

The FOURIER (Further Cardiovascular Outcomes Research with PCSK9 inhibition in Subjects with Elevated Risk) trial was conducted to study cardiovascular outcomes of treatment with evolocumab. The trial was terminated after a median follow-up of 2.2 years instead of the planned 3.6 years. We question this decision. According to the investigators, the event rate was 50% higher than expected. However, the accrued number of key secondary events (1829) was only 12% higher than the targeted number (1630). Also, around one-third of the events consisted of non-atherosclerotic myocardial infarctions, hemorrhagic strokes, and cardiovascular deaths unrelated to myocardial infarction or stroke. Moreover, halfway through the trial, the sample size changed from 22,500 to 27,500, even though the accrual of the targeted number of events was on track. Finally, the rate of all-cause mortality had started to diverge in favor of placebo after 2 years of follow-up. It was 4.8% for evolocumab and 4.3% for placebo in participants with > 2.5 years of follow-up. A long-term follow-up would have yielded more events and thus more power to evaluate the effect of evolocumab on all-cause mortality. We conclude that adaptive designs carry a recognized risk of false-positive efficacy results, but the risk of false-negative safety results is underappreciated.

Adaptive Designs for Clinical Trials: Application to Healthcare Epidemiology Research

Clinical trials with adaptive designs use data that accumulate during the course of the study to modify study elements in a prespecified manner. The goal is to provide flexibility such that a trial can serve as a definitive test of its primary hypothesis, preferably in a shorter time period, involving fewer human subjects, and at lower cost. Elements that may be modified include the sample size, end points, eligible population, randomization ratio, and interventions. Accumulating data used to drive these modifications include the outcomes, subject enrollment (including factors associated with the outcomes), and information about the application of the interventions. This review discusses the types of adaptive designs for clinical trials, emphasizing their advantages and limitations in comparison with conventional designs, and opportunities for applying these designs to healthcare epidemiology research, including studies of interventions to prevent healthcare-associated infections, combat antimicrobial resistance, and improve antimicrobial stewardship.

Adaptive design clinical trials: a review of the literature and ClinicalTrials.gov

OBJECTIVES

This review investigates characteristics of implemented adaptive design clinical trials and provides examples of regulatory experience with such trials.

DESIGN

Review of adaptive design clinical trials in EMBASE, PubMed, Cochrane Registry of Controlled Clinical Trials, Web of Science and ClinicalTrials.gov. Phase I and seamless Phase I/II trials were excluded. Variables extracted from trials included basic study characteristics, adaptive design features, size and use of independent data monitoring committees (DMCs) and blinded interim analyses. We also examined use of the adaptive trials in new drug submissions to the Food and Drug Administration (FDA) and European Medicines Agency (EMA) and recorded regulators' experiences with adaptive designs.

RESULTS

142 studies met inclusion criteria. There has been a recent growth in publicly reported use of adaptive designs among researchers around the world. The most frequently appearing types of adaptations were seamless Phase II/III (57%), group sequential (21%), biomarker adaptive (20%), and adaptive dose-finding designs (16%). About one-third (32%) of trials reported an independent DMC, while 6% reported blinded interim analysis. We found that 9% of adaptive trials were used for FDA product approval consideration, and 12% were used for EMA product approval consideration. International regulators had mixed experiences with adaptive trials. Many product applications with adaptive trials had extensive correspondence between drug sponsors and regulators regarding the adaptive designs, in some cases with regulators requiring revisions or alterations to research designs.

CONCLUSIONS

Wider use of adaptive designs will necessitate new drug application sponsors to engage with regulatory scientists during planning and conduct of the trials. Investigators need to more consistently report protections intended to preserve confidentiality and minimise potential operational bias during interim analysis.

Biomarker-Guided Adaptive Trial Designs in Phase II and Phase III: A Methodological Review

BACKGROUND

Personalized medicine is a growing area of research which aims to tailor the treatment given to a patient according to one or more personal characteristics. These characteristics can be demographic such as age or gender, or biological such as a genetic or other biomarker. Prior to utilizing a patient's biomarker information in clinical practice, robust testing in terms of analytical validity, clinical validity and clinical utility is necessary. A number of clinical trial designs have been proposed for testing a biomarker's clinical utility, including Phase II and Phase III clinical trials which aim to test the effectiveness of a biomarker-guided approach to treatment; these designs can be broadly classified into adaptive and non-adaptive. While adaptive designs allow planned modifications based on accumulating information during a trial, non-adaptive designs are typically simpler but less flexible.

METHODS AND FINDINGS

We have undertaken a comprehensive review of biomarker-guided adaptive trial designs proposed in the past decade. We have identified eight distinct biomarker-guided adaptive designs and nine variations from 107 studies. Substantial variability has been observed in terms of how trial designs are described and particularly in the terminology used by different authors. We have graphically displayed the current biomarker-guided adaptive trial designs and summarised the characteristics of each design.

CONCLUSIONS

Our in-depth overview provides future researchers with clarity in definition, methodology and terminology for biomarker-guided adaptive trial designs.

Design of a multi-arm randomized clinical trial with no control arm

BACKGROUND

Clinical trial designs that include multiple treatments are currently limited to those that perform pairwise comparisons of each investigational treatment to a single control. However, there are settings, such as the recent Ebola outbreak, in which no treatment has been demonstrated to be effective; and therefore, no standard of care exists which would serve as an appropriate control.

METHODS/DESIGN

For illustrative purposes, we focused on the care of patients presenting in austere settings with critically ill 'sepsis-like' syndromes. Our approach involves a novel algorithm for comparing mortality among arms without requiring a single fixed control. The algorithm allows poorly-performing arms to be dropped during interim analyses. Consequently, the study may be completed earlier than planned. We used simulation to determine operating characteristics for the trial and to estimate the required sample size.

RESULTS

We present a potential study design targeting a minimal effect size of a 23% relative reduction in mortality between any pair of arms. Using estimated power and spurious significance rates from the simulated scenarios, we show that such a trial would require 2550 participants. Over a range of scenarios, our study has 80 to 99% power to select the optimal treatment. Using a fixed control design, if the control arm is least efficacious, 640 subjects would be enrolled into the least efficacious arm, while our algorithm would enroll between 170 and 430. This simulation method can be easily extended to other settings or other binary outcomes.

CONCLUSION

Early dropping of arms is efficient and ethical when conducting clinical trials with multiple arms.

Improving clinical trials for cardiovascular diseases: a position paper from the Cardiovascular Round Table of the European Society of Cardiology

AIMS

Cardiovascular disease is the most common cause of mortality and morbidity in the world, but the pharmaceutical industry's willingness to invest in this field has declined because of the many challenges involved with bringing new cardiovascular drugs to market, including late-stage failures, escalating regulatory requirements, bureaucracy of the clinical trial business enterprise, and limited patient access after approval. This contrasts with the remaining burden of cardiovascular disease in Europe and in the world. Thus, clinical cardiovascular research needs to adapt to address the impact of these challenges in order to ensure development of new cardiovascular medicines.

METHODS AND RESULTS

The present paper is the outcome of a two-day workshop held by the Cardiovascular Round Table of the European Society of Cardiology. We propose strategies to improve development of effective new cardiovascular therapies. These can include (i) the use of biomarkers to describe patients who will benefit from new therapies more precisely, achieving better human target validation; (ii) targeted, mechanism-based approaches to drug development for defined populations; (iii) the use of information technology to simplify data collection and follow-up in clinical trials; (iv) streamlining adverse event collection and reducing monitoring; (v) extended patent protection or limited rapid approval of new agents to motivate investment in early phase development; and (vi) collecting data needed for health technology assessment continuously throughout the drug development process (before and after approval) to minimize delays in patient access. Collaboration across industry, academia, regulators, and payers will be necessary to enact change and to unlock the existing potential for cardiovascular clinical drug development.

CONCLUSIONS

A coordinated effort involving academia, regulators, industry, and payors will help to foster better and more effective conduct of clinical cardiovascular trials, supporting earlier availability of innovative therapies and better management of cardiovascular diseases.

Adaptive design of confirmatory trials: Advances and challenges

The past decade witnessed major developments in innovative designs of confirmatory clinical trials, and adaptive designs represent the most active area of these developments. We give an overview of the developments and associated statistical methods in several classes of adaptive designs of confirmatory trials. We also discuss their statistical difficulties and implementation challenges, and show how these problems are connected to other branches of mainstream Statistics, which we then apply to resolve the difficulties and bypass the bottlenecks in the development of adaptive designs for the next decade.

When is a seamless study desirable? Case studies from different pharmaceutical sponsors

BACKGROUND

Inferentially seamless studies are one of the best-known adaptive trial designs. Statistical inference for these studies is a well-studied problem. Regulatory guidance suggests that statistical issues associated with study conduct are not as well understood. Some of these issues are caused by the need for early pre-specification of the phase III design and the absence of sponsor access to unblinded data. Before statisticians decide to choose a seamless IIb/III design for their programme, they should consider whether these pitfalls will be an issue for their programme.

METHODS

We consider four case studies. Each design met with varying degrees of success. We explore the reasons for this variation to identify characteristics of drug development programmes that lend themselves well to inferentially seamless trials and other characteristics that warn of difficulties.

RESULTS

Seamless studies require increased upfront investment and planning to enable the phase III design to be specified at the outset of phase II. Pivotal, inferentially seamless studies are unlikely to allow meaningful sponsor access to unblinded data before study completion. This limits a sponsor's ability to reflect new information in the phase III portion.

CONCLUSIONS

When few clinical data have been gathered about a drug, phase II data will answer many unresolved questions. Committing to phase III plans and study designs before phase II begins introduces extra risk to drug development. However, seamless pivotal studies may be an attractive option when the clinical setting and development programme allow, for example, when revisiting dose selection.

Overview, hurdles, and future work in adaptive designs: perspectives from a National Institutes of Health-funded workshop

BACKGROUND

The clinical trials community has a never-ending search for dependable and reliable ways to improve clinical research. This exploration has led to considerable interest in adaptive clinical trial designs, which provide the flexibility to adjust trial characteristics on the basis of data reviewed at interim stages. Statisticians and clinical investigators have proposed or implemented a wide variety of adaptations in clinical trials, but specific approaches have met with differing levels of support. Within industry, investigators are actively exploring the benefits and pitfalls associated with adaptive designs (ADs). For example, a Drug Information Association (DIA) working group on ADs has engaged regulatory agencies in discussions. Many researchers working on publicly funded clinical trials, however, are not yet fully engaged in this discussion. We organized the Scientific Advances in Adaptive Clinical Trial Designs Workshop to begin a conversation about using ADs in publicly funded research. Held in November of 2009, the 1½-day workshop brought together representatives from the National Institutes of Health (NIH), the Food and Drug Administration (FDA), the European Medicines Agency (EMA), the pharmaceutical industry, nonprofit foundations, the patient advocacy community, and academia. The workshop offered a forum for participants to address issues of ADs that arise at the planning, designing, and execution stages of clinical trials, and to hear the perspectives of influential members of the clinical trials community. The participants also set forth recommendations for guiding action to promote the appropriate use of ADs. These recommendations have since been presented, discussed, and vetted in a number of venues including the University of Pennsylvania Conference on Statistical Issues in Clinical Trials and the Society for Clinical Trials annual meeting.

PURPOSE

To provide a brief overview of ADs, describe the rationale behind conducting the workshop, and summarize the main recommendations that were produced as a result of this workshop.

CONCLUSIONS

There is a growing interest in the use of adaptive clinical trial designs. However, a number of logistical barriers need to be addressed in order to obtain the potential advantages of an AD. Currently, the pharmaceutical industry is well ahead of academic trialists with respect to addressing these barriers. Academic trialists will need to address important issues such as education, infrastructure, modifications to existing funding models, and the impact on Data and Safety Monitoring Boards (DSMB) in order to achieve the possible benefits of adaptive clinical trial designs.

An insight into the properties of a two-stage design in bioequivalence studies

PURPOSE

Unveil the properties of a two-stage design (TSD) for bioequivalence (BE) studies.

METHODS

A TSD with an upper sample size limit (UL) is described and analyzed under different conditions using Monte Carlo simulations. TSD was split into three branches: A, B1, and B2. The first stage included branches A and B1, while stage two referred to branch B2. Sample size re-estimation at B2 relies on the observed GMR and variability of stage 1. The properties studied were % BE acceptance, % uses and % efficiency of each branch, as well as the reason of BE failure.

RESULTS

No inflation of type I error was observed. Each TSD branch exhibits different performance. Stage two exhibits the greatest % BE acceptances when highly variable drugs are assessed with a low starting number of subjects (N₁) or when formulations differ significantly. Branch A is more frequently used when variability is low, drug products are similar, and a large N₁ is included. BE assessment at branch A is very efficient.

CONCLUSIONS

The overall acceptance profile of TSD resembles the typical pattern observed in single-stage studies, but it is actually different. Inclusion of a UL is necessary to avoid inflation of type I error.

Adaptive trial designs: a review of barriers and opportunities

Adaptive designs allow planned modifications based on data accumulating within a study. The promise of greater flexibility and efficiency stimulates increasing interest in adaptive designs from clinical, academic, and regulatory parties. When adaptive designs are used properly, efficiencies can include a smaller sample size, a more efficient treatment development process, and an increased chance of correctly answering the clinical question of interest. However, improper adaptations can lead to biased studies. A broad definition of adaptive designs allows for countless variations, which creates confusion as to the statistical validity and practical feasibility of many designs. Determining properties of a particular adaptive design requires careful consideration of the scientific context and statistical assumptions. We first review several adaptive designs that garner the most current interest. We focus on the design principles and research issues that lead to particular designs being appealing or unappealing in particular applications. We separately discuss exploratory and confirmatory stage designs in order to account for the differences in regulatory concerns. We include adaptive seamless designs, which combine stages in a unified approach. We also highlight a number of applied areas, such as comparative effectiveness research, that would benefit from the use of adaptive designs. Finally, we describe a number of current barriers and provide initial suggestions for overcoming them in order to promote wider use of appropriate adaptive designs. Given the breadth of the coverage all mathematical and most implementation details are omitted for the sake of brevity. However, the interested reader will find that we provide current references to focused reviews and original theoretical sources which lead to details of the current state of the art in theory and practice.

Design issues in randomized phase II/III trials

Phase II trials are used to show sufficient preliminary activity of a new treatment (in single-arm designs or randomized screening designs) or to select among treatments with demonstrated activity (in randomized selection designs). The treatments prioritized in a phase II trial are then tested definitively against a control treatment in a randomized phase III trial. Randomized phase II/III trials use an adaptive trial design that combines these two types of trials in one, with potential gains in time and reduced numbers of patients required to be treated. Two key considerations in designing a phase II/III trial are whether to suspend accrual while the phase II data mature and the choice of phase II target treatment effect. We discuss these phase II/III design parameters, give examples of phase II/III trials, and provide recommendations concerning efficient phase II/III trial designs.

HIV treatment as prevention: considerations in the design, conduct, and analysis of cluster randomized controlled trials of combination HIV prevention

The rigorous evaluation of the impact of combination HIV prevention packages at the population level will be critical for the future of HIV prevention. In this review, we discuss important considerations for the design and interpretation of cluster randomized controlled trials (C-RCTs) of combination prevention interventions. We focus on three large C-RCTs that will start soon and are designed to test the hypothesis that combination prevention packages, including expanded access to antiretroviral therapy, can substantially reduce HIV incidence. Using a general framework to integrate mathematical modelling analysis into the design, conduct, and analysis of C-RCTs will complement traditional statistical analyses and strengthen the evaluation of the interventions. Importantly, even with combination interventions, it may be challenging to substantially reduce HIV incidence over the 2- to 3-y duration of a C-RCT, unless interventions are scaled up rapidly and key populations are reached. Thus, we propose the innovative use of mathematical modelling to conduct interim analyses, when interim HIV incidence data are not available, to allow the ongoing trials to be modified or adapted to reduce the likelihood of inconclusive outcomes. The preplanned, interactive use of mathematical models during C-RCTs will also provide a valuable opportunity to validate and refine model projections.

How to improve the clinical development paradigm and its division into phases I, II and III

Based on the observation that over the last 30 years the cost of development has risen regularly as the number of new chemical entities reaching the market has fallen, how can "savings" be made in terms of clinical development, the objective being more rapid access to a drug for medical needs that are not covered? Several instruments exist to enable innovative products to be made available more quickly: temporary use authorisations, which are not concerned by this work (ATUs), conditional marketing authorisations (MAs) and MAs under exceptional circumstances. These aspects have been taken up in the European medicines agency (EMA)'s "Road Map", which states "A key issue for Regulators will be if a more "staggered" approval should be envisaged, characterised by a better defined/more restricted population of good responders, followed by a broadening of the population post-authorisation when more "real life" data are available. In addition, maximising the value of information generated in the post-authorisation phase should be developed through the use of cohorts and other prospectively collected use data, especially in the case of conditional marketing authorisations." The rules of procedure of the Transparency Commission for their part provide for the notion of preliminary examination: in order to prepare as best as possible the examination of dossiers of products assumed to be innovative and to limit delays, the office can undertake a preliminary study as soon as the dossier has been filed at the Committee for medicinal products for human use (CHMP). It may, at this time, request the firm to provide further information and may call on external experts. The implementation of this preliminary study does not exonerate the firm of the obligation of filing a complete dossier. The post inscription studies requested by the Transparency Commission (ISPEP - public health benefit and post-marketing studies) are usually requested in the case of hesitations regarding the level of improvement of the medical benefit (ASMR) [level II/III or IV/V]. Such requests mainly concern uncertainties regarding the transposability, the patient profile or correct usage in real life. Among the studies whose results were provided, in 15 cases the results were in line with expectations, in 6 cases they resulted in downward re-evaluations and the final 3 cases were inconclusive. The final recommendations of the round table were: Defining the medical need that is not covered by working in consultation (Industry and Health Authorities); Providing a Complementary Investigations Plan (PIC) after the MA at a very early stage to reinforce the early MA, and/or HTA (health technology assessment) preparation and monitoring (possible constraining actions); Enhanced use of modelling techniques and their transposability; "Intussusception" of phases to optimise the development of a complete dossier; Early "scientific opinions" (EMA, French Health Products Safety Agency [Afssaps], French Health Authority [HAS]); Raising the awareness of the authorities, industry, doctors and patients with regard to controlled observational studies; Developing the use of public data bases.

A Sequential Phase 2b Trial Design for Evaluating Vaccine Efficacy and Immune Correlates for Multiple HIV Vaccine Regimens

Five preventative HIV vaccine efficacy trials have been conducted over the last 12 years, all of which evaluated vaccine efficacy (VE) to prevent HIV infection for a single vaccine regimen versus placebo. Now that one of these trials has supported partial VE of a prime-boost vaccine regimen, there is interest in conducting efficacy trials that simultaneously evaluate multiple prime-boost vaccine regimens against a shared placebo group in the same geographic region, for accelerating the pace of vaccine development. This article proposes such a design, which has main objectives (1) to evaluate VE of each regimen versus placebo against HIV exposures occurring near the time of the immunizations; (2) to evaluate durability of VE for each vaccine regimen showing reliable evidence for positive VE; (3) to expeditiously evaluate the immune correlates of protection if any vaccine regimen shows reliable evidence for positive VE; and (4) to compare VE among the vaccine regimens. The design uses sequential monitoring for the events of vaccine harm, non-efficacy, and high efficacy, selected to weed out poor vaccines as rapidly as possible while guarding against prematurely weeding out a vaccine that does not confer efficacy until most of the immunizations are received. The evaluation of the design shows that testing multiple vaccine regimens is important for providing a well-powered assessment of the correlation of vaccine-induced immune responses with HIV infection, and is critically important for providing a reasonably powered assessment of the value of identified correlates as surrogate endpoints for HIV infection.