An overview of the adaptive designs accelerating promising trials into treatments (ADAPT-IT) project

Bayesian approach for design and analysis of medical device trials in the era of modern clinical studies

Medical device technology develops rapidly, and the life cycle of a medical device is much shorter than drugs. It is necessary to evaluate the safety and effectiveness of a medical device in a timely manner to keep up with technology flux. Bayesian methods provides an efficient approach to addressing this challenge. In this article, we review the characteristics of the Bayesian approach and some Bayesian designs that were commonly used in medical device regulatory setting, including Bayesian adaptive design, Bayesian diagnostic design, Bayesian multiregional design, and Bayesian label expansion study. We illustrate these designs with medical devices approved by the US Food and Drug Administration (FDA). We also review several innovative Bayesian information borrowing methods, and briefly discuss the challenges and future directions of the Bayesian application in medical device trials. Our objective is to promote the use of the Bayesian approach to accelerate the development of innovative medical devices and their accessibility to patients for effective disease diagnoses and treatments.

Bayesian Statistics for Medical Devices: Progress Since 2010

The use of Bayesian statistics to support regulatory evaluation of medical devices began in the late 1990s. We review the literature, focusing on recent developments of Bayesian methods, including hierarchical modeling of studies and subgroups, borrowing strength from prior data, effective sample size, Bayesian adaptive designs, pediatric extrapolation, benefit-risk decision analysis, use of real-world evidence, and diagnostic device evaluation. We illustrate how these developments were utilized in recent medical device evaluations. In Supplementary Material, we provide a list of medical devices for which Bayesian statistics were used to support approval by the US Food and Drug Administration (FDA), including those since 2010, the year the FDA published their guidance on Bayesian statistics for medical devices. We conclude with a discussion of current and future challenges and opportunities for Bayesian statistics, including artificial intelligence/machine learning (AI/ML) Bayesian modeling, uncertainty quantification, Bayesian approaches using propensity scores, and computational challenges for high dimensional data and models.

Fighting for recovery on multiple fronts: The past, present, and future of clinical trials for spinal cord injury

Through many decades of preclinical research, great progress has been achieved in understanding the complex nature of spinal cord injury (SCI). Preclinical research efforts have guided and shaped clinical trials, which are growing in number by the year. Currently, 1,149 clinical trials focused on improving outcomes after SCI are registered in the U.S. National Library of Medicine at ClinicalTrials.gov. We conducted a systematic analysis of these SCI clinical trials, using publicly accessible data downloaded from ClinicalTrials.gov. After extracting all available data for these trials, we categorized each trial according to the types of interventions being tested and the types of outcomes assessed. We then evaluated clinical trial characteristics, both globally and by year, in order to understand the areas of growth and change over time. With regard to clinical trial attributes, we found that most trials have low enrollment, only test single interventions, and have limited numbers of primary outcomes. Some gaps in reporting are apparent; for instance, over 75% of clinical trials with "Completed" status do not have results posted, and the Phase of some trials is incorrectly classified as "Not applicable" despite testing a drug or biological compound. When analyzing trials based on types of interventions assessed, we identified the largest representation in trials testing rehab/training/exercise, neuromodulation, and behavioral modifications. Most highly represented primary outcomes include motor function of the upper and lower extremities, safety, and pain. The most highly represented secondary outcomes include quality of life and pain. Over the past 15 years, we identified increased representation of neuromodulation and rehabilitation trials, and decreased representation of drug trials. Overall, the number of new clinical trials initiated each year continues to grow, signifying a hopeful future for the clinical treatment of SCI. Together, our work provides a comprehensive glimpse into the past, present, and future of SCI clinical trials, and suggests areas for improvement in clinical trial reporting.

Comparison of Bayesian vs Frequentist Adaptive Trial Design in the Stroke Hyperglycemia Insulin Network Effort Trial

IMPORTANCE

Bayesian adaptive trial design has the potential to create more efficient clinical trials. However, a barrier to the uptake of bayesian adaptive designs for confirmatory trials is limited experience with how they may perform compared with a frequentist design.

OBJECTIVE

To compare the performance of a bayesian and a frequentist adaptive clinical trial design.

DESIGN, SETTING, AND PARTICIPANTS

This prospective cohort study compared 2 trial designs for a completed multicenter acute stroke trial conducted within a National Institutes of Health neurologic emergencies clinical trials network, with individual patient-level data, including the timing and order of enrollments and outcome ascertainment, from 1151 patients with acute stroke and hyperglycemia randomized to receive intensive or standard insulin therapy. The implemented frequentist design had group sequential boundaries for efficacy and futility interim analyses at 90 days after randomization for 500, 700, 900, and 1100 patients. The bayesian alternative used predictive probability of trial success to govern early termination for efficacy and futility with a first interim analysis at 500 randomized patients and subsequent interims after every 100 randomizations.

MAIN OUTCOMES AND MEASURES

The main outcome was the sample size at end of study, which was defined as the sample size at which each of the studies stopped accrual of patients.

RESULTS

Data were collected from 1151 patients. As conducted, the frequentist design passed the futility boundary after 936 participants were randomized. Using the same sequence and timing of randomization and outcome data, the bayesian alternative crossed the futility boundary approximately 3 months earlier after 800 participants were randomized.

CONCLUSIONS AND RELEVANCE

Both trial designs stopped for futility before reaching the planned maximum sample size. In both cases, the clinical community and patients would benefit from learning the answer to the trial's primary question earlier. The common feature across the 2 designs was frequent interim analyses to stop early for efficacy or for futility. Differences between how these analyses were implemented between the 2 trials resulted in the differences in early stopping.

The ORVAC trial: a phase IV, double-blind, randomised, placebo-controlled clinical trial of a third scheduled dose of Rotarix rotavirus vaccine in Australian Indigenous infants to improve protection against gastroenteritis: a statistical analysis plan

Objective

The purpose of this double-blind, randomised, placebo-controlled, adaptive design trial with frequent interim analyses is to determine if Australian Indigenous children, who receive an additional (third) dose of human rotavirus vaccine (Rotarix, GlaxoSmithKline) for children aged 6 to < 12 months, would improve protection against clinically significant all-cause gastroenteritis.

Participants

Up to 1000 Australian Aboriginal and Torres Strait Islander (hereafter Indigenous) infants aged 6 to < 12 months will be recruited from all regions of the Northern Territory.

Interventions

The intervention is the addition of a third scheduled dose of human monovalent rotavirus vaccine.

Co-primary and secondary outcome measures

ORVAC has two co-primary outcomes: (1) anti-rotavirus IgA seroconversion, defined as serum anti-rotavirus IgA ≥ 20 U/ml 28 to 55 days post Rotarix/placebo, and (2) time from randomisation to medical attendance for which the primary reason for presentation is acute gastroenteritis or acute diarrhoea illness before age 36 months. Secondary outcomes include (1) change in anti-rotavirus IgA log titre, (2) time from randomisation to hospitalisation with primary admission code presumed or confirmed acute diarrhoea illness before age 36 months, (3) time from randomisation to hospitalisation for which the admission is rotavirus confirmed diarrhoea illness before age 36 months and (4) time from randomisation to rotavirus infection (not necessarily requiring hospitalisation) meeting the jurisdictional definition before age 36 months.

Discussion

A detailed, prospective statistical analysis plan is presented for this Bayesian adaptive design. The plan was written by the trial statistician and details the study design, pre-specified adaptative elements, decision thresholds, statistical methods and the simulations used to evaluate the operating characteristics of the trial. As at August 2020, four interim analyses have been run, but no stopping rules have been triggered. Application of this SAP will minimise bias and supports transparent and reproducible research.

Trial registration

Clinicaltrials.gov NCT02941107. Registered on 21 October 2016

Original protocol for the study

10.1136/bmjopen-2019-032549

Oseltamivir plus usual care versus usual care for influenza-like illness in primary care: an open-label, pragmatic, randomised controlled trial

BACKGROUND

Antivirals are infrequently prescribed in European primary care for influenza-like illness, mostly because of perceived ineffectiveness in real world primary care and because individuals who will especially benefit have not been identified in independent trials. We aimed to determine whether adding antiviral treatment to usual primary care for patients with influenza-like illness reduces time to recovery overall and in key subgroups.

METHODS

We did an open-label, pragmatic, adaptive, randomised controlled trial of adding oseltamivir to usual care in patients aged 1 year and older presenting with influenza-like illness in primary care. The primary endpoint was time to recovery, defined as return to usual activities, with fever, headache, and muscle ache minor or absent. The trial was designed and powered to assess oseltamivir benefit overall and in 36 prespecified subgroups defined by age, comorbidity, previous symptom duration, and symptom severity, using a Bayesian piece-wise exponential primary analysis model. The trial is registered with the ISRCTN Registry, number ISRCTN 27908921.

FINDINGS

Between Jan 15, 2016, and April 12, 2018, we recruited 3266 participants in 15 European countries during three seasonal influenza seasons, allocated 1629 to usual care plus oseltamivir and 1637 to usual care, and ascertained the primary outcome in 1533 (94%) and 1526 (93%). 1590 (52%) of 3059 participants had PCR-confirmed influenza infection. Time to recovery was shorter in participants randomly assigned to oseltamivir (hazard ratio 1·29, 95% Bayesian credible interval [BCrI] 1·20-1·39) overall and in 30 of the 36 prespecified subgroups, with estimated hazard ratios ranging from 1·13 to 1·72. The estimated absolute mean benefit from oseltamivir was 1·02 days (95% [BCrI] 0·74-1·31) overall, and in the prespecified subgroups, ranged from 0·70 (95% BCrI 0·30-1·20) in patients younger than 12 years, with less severe symptoms, no comorbidities, and shorter previous illness duration to 3·20 (95% BCrI 1·00-5·50) in patients aged 65 years or older who had more severe illness, comorbidities, and longer previous illness duration. Regarding harms, an increased burden of vomiting or nausea was observed in the oseltamivir group.

INTERPRETATION

Primary care patients with influenza-like illness treated with oseltamivir recovered one day sooner on average than those managed by usual care alone. Older, sicker patients with comorbidities and longer previous symptom duration recovered 2-3 days sooner.

FUNDING

European Commission's Seventh Framework Programme.

Bayesian clinical trials at The University of Texas MD Anderson Cancer Center: An update

BACKGROUND/AIMS

In our 2009 article, we showed that Bayesian methods had established a foothold in developing therapies in our institutional oncology trials. In this article, we will document what has happened since that time. In addition, we will describe barriers to implementing Bayesian clinical trials, as well as our experience overcoming them.

METHODS

We reviewed MD Anderson Cancer Center clinical trials submitted to the institutional protocol office for scientific and ethical review between January 2009 and December 2013, the same length time period as the previous article. We tabulated Bayesian methods implemented for design or analyses for each trial and then compared these to our previous findings.

RESULTS

Overall, we identified 1020 trials and found that 283 (28%) had Bayesian components so we designated them as Bayesian trials. Among MD Anderson-only and multicenter trials, 56% and 14%, respectively, were Bayesian, higher rates than our previous study. Bayesian trials were more common in phase I/II trials (34%) than in phase III/IV (6%) trials. Among Bayesian trials, the most commonly used features were for toxicity monitoring (65%), efficacy monitoring (36%), and dose finding (22%). The majority (86%) of Bayesian trials used non-informative priors. A total of 75 (27%) trials applied Bayesian methods for trial design and primary endpoint analysis. Among this latter group, the most commonly used methods were the Bayesian logistic regression model (N = 22), the continual reassessment method (N = 20), and adaptive randomization (N = 16). Median institutional review board approval time from protocol submission was the same 1.4 months for Bayesian and non-Bayesian trials. Since the previous publication, the Biomarker-Integrated Approaches of Targeted Therapy for Lung Cancer Elimination (BATTLE) trial was the first large-scale decision trial combining multiple treatments in a single trial. Since then, two regimens in breast cancer therapy have been identified and published from the cooperative Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging and Molecular Analysis (I-SPY 2), enhancing cooperation among investigators and drug developers across the nation, as well as advancing information needed for personalized medicine. Many software programs and Shiny applications for Bayesian trial design and calculations are available from our website which has had more than 21,000 downloads worldwide since 2004.

CONCLUSION

Bayesian trials have the increased flexibility in trial design needed for personalized medicine, resulting in more cooperation among researchers working to fight against cancer. Some disadvantages of Bayesian trials remain, but new methods and software are available to improve their function and incorporation into cancer clinical research.

Adaptive crossover designs for assessment of symptomatic treatments targeting behaviour in neurodegenerative disease: a phase 2 clinical trial of intranasal oxytocin for frontotemporal dementia (FOXY)

Background

There are currently no treatments for empathy deficits in neuropsychiatric disorders. Acute administration of the hormone oxytocin has been associated with symptomatic improvements across animal models and several neuropsychiatric disorders, but results of the majority of oxytocin randomised controlled trials (RCTs) of longer duration have been negative or inconclusive. This lack of efficacy of may be due to rapid habituation to oxytocin with chronic dosing. The objective of the present study is to describe the design of a phase 2 adaptive randomised controlled crossover trial of intranasal oxytocin in frontotemporal dementia (FOXY) as an efficient model for future investigations of symptomatic treatments in neuropsychiatric and neurodegenerative disorders.

Methods

Stage 1 will identify which of three dose schedules is most promising based on change in the primary outcome measure, the Neuropsychiatric Inventory apathy/indifference domain score, over 6 weeks of treatment. In stage 2, additional patients are enrolled at the most promising dose for preliminary efficacy analysis when combined with stage 1 to determine if a phase 3 trial is warranted. Objective measures include facial expression recognition, cerebrospinal fluid (CSF) oxytocin levels, and behavioural ratings of videotaped interactions.

Results

A total of 20 patients per arm will be entered into stage 1 for a total of 60 patients. In stage 2, an additional 40 patients will be enrolled in the most promising dose arm.

Conclusions

The use of adaptive, crossover designs and inclusion of objective measures of change in CSF oxytocin levels and social behaviour will improve the efficiency and conclusiveness of RCTs of oxytocin and other symptomatic treatments in neuropsychiatric disorders.

Trial registration

ClinicalTrials.gov, NCT03260920. Registered on August 24, 2017.

Electronic supplementary material

The online version of this article (10.1186/s13195-018-0427-2) contains supplementary material, which is available to authorized users.

Adaptive Designs for Clinical Trials: Application to Healthcare Epidemiology Research

Clinical trials with adaptive designs use data that accumulate during the course of the study to modify study elements in a prespecified manner. The goal is to provide flexibility such that a trial can serve as a definitive test of its primary hypothesis, preferably in a shorter time period, involving fewer human subjects, and at lower cost. Elements that may be modified include the sample size, end points, eligible population, randomization ratio, and interventions. Accumulating data used to drive these modifications include the outcomes, subject enrollment (including factors associated with the outcomes), and information about the application of the interventions. This review discusses the types of adaptive designs for clinical trials, emphasizing their advantages and limitations in comparison with conventional designs, and opportunities for applying these designs to healthcare epidemiology research, including studies of interventions to prevent healthcare-associated infections, combat antimicrobial resistance, and improve antimicrobial stewardship.

Adaptive design clinical trials: a review of the literature and ClinicalTrials.gov

OBJECTIVES

This review investigates characteristics of implemented adaptive design clinical trials and provides examples of regulatory experience with such trials.

DESIGN

Review of adaptive design clinical trials in EMBASE, PubMed, Cochrane Registry of Controlled Clinical Trials, Web of Science and ClinicalTrials.gov. Phase I and seamless Phase I/II trials were excluded. Variables extracted from trials included basic study characteristics, adaptive design features, size and use of independent data monitoring committees (DMCs) and blinded interim analyses. We also examined use of the adaptive trials in new drug submissions to the Food and Drug Administration (FDA) and European Medicines Agency (EMA) and recorded regulators' experiences with adaptive designs.

RESULTS

142 studies met inclusion criteria. There has been a recent growth in publicly reported use of adaptive designs among researchers around the world. The most frequently appearing types of adaptations were seamless Phase II/III (57%), group sequential (21%), biomarker adaptive (20%), and adaptive dose-finding designs (16%). About one-third (32%) of trials reported an independent DMC, while 6% reported blinded interim analysis. We found that 9% of adaptive trials were used for FDA product approval consideration, and 12% were used for EMA product approval consideration. International regulators had mixed experiences with adaptive trials. Many product applications with adaptive trials had extensive correspondence between drug sponsors and regulators regarding the adaptive designs, in some cases with regulators requiring revisions or alterations to research designs.

CONCLUSIONS

Wider use of adaptive designs will necessitate new drug application sponsors to engage with regulatory scientists during planning and conduct of the trials. Investigators need to more consistently report protections intended to preserve confidentiality and minimise potential operational bias during interim analysis.

Precision medicine for all? Challenges and opportunities for a precision medicine approach to critical illness

All of medicine aspires to be precise, where a greater understanding of individual data will lead to personalized treatment and improved outcomes. Prompted by specific examples in oncology, the field of critical care may be tempted to envision that complex, acute syndromes could bend to a similar reductionist philosophy-where single mutations could identify and target our critically ill patients for treatment. However, precision medicine faces many challenges in critical care. These include confusion about terminology, uncertainty about how to divide patients into discrete groups, the challenges of multi-morbidity, scale, and the need for timely interventions. This review addresses these challenges and provides a translational roadmap spanning preclinical work to identify putative treatment targets, novel designs for clinical trials, and the integration of the electronic health record to implement precision critical care for all.

The life cycles of six multi-center adaptive clinical trials focused on neurological emergencies developed for the Advancing Regulatory Science initiative of the National Institutes of Health and US Food and Drug Administration: Case studies from the Adaptive Designs Accelerating Promising Treatments Into Trials Project

OBJECTIVES

Clinical trials are complicated, expensive, time-consuming, and frequently do not lead to discoveries that improve the health of patients with disease. Adaptive clinical trials have emerged as a methodology to provide more flexibility in design elements to better answer scientific questions regarding whether new treatments are efficacious. Limited observational data exist that describe the complex process of designing adaptive clinical trials. To address these issues, the Adaptive Designs Accelerating Promising Treatments Into Trials project developed six, tailored, flexible, adaptive, phase-III clinical trials for neurological emergencies, and investigators prospectively monitored and observed the processes. The objective of this work is to describe the adaptive design development process, the final design, and the current status of the adaptive trial designs that were developed.

METHODS

To observe and reflect upon the trial development process, we employed a rich, mixed methods evaluation that combined quantitative data from visual analog scale to assess attitudes about adaptive trials, along with in-depth qualitative data about the development process gathered from observations.

RESULTS

The Adaptive Designs Accelerating Promising Treatments Into Trials team developed six adaptive clinical trial designs. Across the six designs, 53 attitude surveys were completed at baseline and after the trial planning process completed. Compared to baseline, the participants believed significantly more strongly that the adaptive designs would be accepted by National Institutes of Health review panels and non-researcher clinicians. In addition, after the trial planning process, the participants more strongly believed that the adaptive design would meet the scientific and medical goals of the studies.

CONCLUSION

Introducing the adaptive design at early conceptualization proved critical to successful adoption and implementation of that trial. Involving key stakeholders from several scientific domains early in the process appears to be associated with improved attitudes towards adaptive designs over the life cycle of clinical trial development.

Bayesian dose selection design for a binary outcome using restricted response adaptive randomization

Background

In phase II trials, the most efficacious dose is usually not known. Moreover, given limited resources, it is difficult to robustly identify a dose while also testing for a signal of efficacy that would support a phase III trial. Recent designs have sought to be more efficient by exploring multiple doses through the use of adaptive strategies. However, the added flexibility may potentially increase the risk of making incorrect assumptions and reduce the total amount of information available across the dose range as a function of imbalanced sample size.

Methods

To balance these challenges, a novel placebo-controlled design is presented in which a restricted Bayesian response adaptive randomization (RAR) is used to allocate a majority of subjects to the optimal dose of active drug, defined as the dose with the lowest probability of poor outcome. However, the allocation between subjects who receive active drug or placebo is held constant to retain the maximum possible power for a hypothesis test of overall efficacy comparing the optimal dose to placebo. The design properties and optimization of the design are presented in the context of a phase II trial for subarachnoid hemorrhage.

Results

For a fixed total sample size, a trade-off exists between the ability to select the optimal dose and the probability of rejecting the null hypothesis. This relationship is modified by the allocation ratio between active and control subjects, the choice of RAR algorithm, and the number of subjects allocated to an initial fixed allocation period. While a responsive RAR algorithm improves the ability to select the correct dose, there is an increased risk of assigning more subjects to a worse arm as a function of ephemeral trends in the data. A subarachnoid treatment trial is used to illustrate how this design can be customized for specific objectives and available data.

Conclusions

Bayesian adaptive designs are a flexible approach to addressing multiple questions surrounding the optimal dose for treatment efficacy within the context of limited resources. While the design is general enough to apply to many situations, future work is needed to address interim analyses and the incorporation of models for dose response.

A conceptual model for the development process of confirmatory adaptive clinical trials within an emergency research network

BACKGROUND

Adaptive clinical trials use accumulating data from enrolled subjects to alter trial conduct in pre-specified ways based on quantitative decision rules. In this research, we sought to characterize the perspectives of key stakeholders during the development process of confirmatory-phase adaptive clinical trials within an emergency clinical trials network and to build a model to guide future development of adaptive clinical trials.

METHODS

We used an ethnographic, qualitative approach to evaluate key stakeholders' views about the adaptive clinical trial development process. Stakeholders participated in a series of multidisciplinary meetings during the development of five adaptive clinical trials and completed a Strengths-Weaknesses-Opportunities-Threats questionnaire. In the analysis, we elucidated overarching themes across the stakeholders' responses to develop a conceptual model.

RESULTS

Four major overarching themes emerged during the analysis of stakeholders' responses to questioning: the perceived statistical complexity of adaptive clinical trials and the roles of collaboration, communication, and time during the development process. Frequent and open communication and collaboration were viewed by stakeholders as critical during the development process, as were the careful management of time and logistical issues related to the complexity of planning adaptive clinical trials.

CONCLUSION

The Adaptive Design Development Model illustrates how statistical complexity, time, communication, and collaboration are moderating factors in the adaptive design development process. The intensity and iterative nature of this process underscores the need for funding mechanisms for the development of novel trial proposals in academic settings.

Attitudes and opinions regarding confirmatory adaptive clinical trials: a mixed methods analysis from the Adaptive Designs Accelerating Promising Trials into Treatments (ADAPT-IT) project

Background

Adaptive designs have been increasingly used in the pharmaceutical and device industries, but adoption within the academic setting has been less widespread — particularly for confirmatory phase trials. We sought to understand perceptions about understanding, acceptability, and scientific validity of adaptive clinical trials (ACTs).

Methods

We used a convergent mixed methods design using survey and mini-focus group data collection procedures to elucidate attitudes and opinions among “trial community” stakeholders regarding understanding, acceptability, efficiency, scientific validity, and speed of discovery with adaptive designs. Data were collected about various aspects of ACTs using self-administered surveys (paper or Web-based) with visual analog scales (VASs) with free text responses and with mini-focus groups of key stakeholders. Participants were recruited as part of an ongoing NIH/FDA-funded research project exploring the incorporation of ACTs into an existing NIH network that focuses on confirmatory phase clinical trials in neurological emergencies. “Trial community” representatives, namely, clinical investigators, biostatisticians, NIH officials, and FDA scientists involved in the planning of four clinical trials, were eligible to participate. In addition, recent and current members of a clinical trial-oriented NIH study section were also eligible.

Results

A total of 76 stakeholders completed the survey (out of 91 who were offered it, response rate 84 %). While the VAS attitudinal data showed substantial variability across respondents about acceptability and understanding of ACTs by various constituencies, respondents perceived clinicians to be less likely to understand ACTs and that ACTs probably would increase the efficiency of discovery. Textual and focus group responses emerged into several themes that enhanced understanding of VAS attitudinal data including the following: acceptability of adaptive designs depends on constituency and situation; there is variable understanding of ACTs (limited among clinicians, perceived to be higher at FDA); views about the potential for efficiency depend on the situation and implementation. Participants also frequently mentioned a need for greater education within the academic community. Finally, the empiric, non-quantitative selection of treatments for phase III trials based on limited phase II trials was highlighted as an opportunity for improvement and a potential explanation for the high number of neutral confirmatory trials.

Conclusions

These data show considerable variations in attitudes and beliefs about ACTs among trial community representatives. For adaptive trials to be fully considered when appropriate and for the research enterprise to realize the full potential of adaptive designs will likely require extensive experience and trust building within the trial community.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1493-z) contains supplementary material, which is available to authorized users.

Reflections on the Adaptive Designs Accelerating Promising Trials Into Treatments (ADAPT-IT) Process-Findings from a Qualitative Study

CONTEXT

The context for this study was the Adaptive Designs Advancing Promising Treatments Into Trials (ADAPT-IT) project, which aimed to incorporate flexible adaptive designs into pivotal clinical trials and to conduct an assessment of the trial development process. Little research provides guidance to academic institutions in planning adaptive trials.

OBJECTIVES

The purpose of this qualitative study was to explore the perspectives and experiences of stakeholders as they reflected back about the interactive ADAPT-IT adaptive design development process, and to understand their perspectives regarding lessons learned about the design of the trials and trial development.

MATERIALS AND METHODS

We conducted semi-structured interviews with ten key stakeholders and observations of the process. We employed qualitative thematic text data analysis to reduce the data into themes about the ADAPT-IT project and adaptive clinical trials.

RESULTS

The qualitative analysis revealed four themes: education of the project participants, how the process evolved with participant feedback, procedures that could enhance the development of other trials, and education of the broader research community.

DISCUSSION AND CONCLUSIONS

While participants became more likely to consider flexible adaptive designs, additional education is needed to both understand the adaptive methodology and articulate it when planning trials.

Clinical Trial Adaptation by Matching Evidence in Complementary Patient Sub-groups of Auxiliary Blinding Questionnaire Responses

Clinical trial adaptation refers to any adjustment of the trial protocol after the onset of the trial. Such adjustment may take on various forms, including the change in the dose of administered medicines, the frequency of administering an intervention, the number of trial participants, or the duration of the trial, to name just some possibilities. The main goal is to make the process of introducing new medical interventions to patients more efficient, either by reducing the cost or the time associated with evaluating their safety and efficacy. The principal challenge, which is an outstanding research problem, is to be found in the question of how adaptation should be performed so as to minimize the chance of distorting the outcome of the trial. In this paper we propose a novel method for achieving this. Unlike most of the previously published work, our approach focuses on trial adaptation by sample size adjustment i.e. by reducing the number of trial participants in a statistically informed manner. We adopt a stratification framework recently proposed for the analysis of trial outcomes in the presence of imperfect blinding and based on the administration of a generic auxiliary questionnaire that allows the participants to express their belief concerning the assigned intervention (treatment or control). We show that this data, together with the primary measured variables, can be used to make the probabilistically optimal choice of the particular sub-group a participant should be removed from if trial size reduction is desired. Extensive experiments on a series of simulated trials are used to illustrate the effectiveness of our method.

Clinical trialist perspectives on the ethics of adaptive clinical trials: a mixed-methods analysis

BACKGROUND

In an adaptive clinical trial (ACT), key trial characteristics may be altered during the course of the trial according to predefined rules in response to information that accumulates within the trial itself. In addition to having distinguishing scientific features, adaptive trials also may involve ethical considerations that differ from more traditional randomized trials. Better understanding of clinical trial experts' views about the ethical aspects of adaptive designs could assist those planning ACTs. Our aim was to elucidate the opinions of clinical trial experts regarding their beliefs about ethical aspects of ACTs.

METHODS

We used a convergent, mixed-methods design employing a 22-item ACTs beliefs survey with visual analog scales and open-ended questions and mini-focus groups. We developed a coding scheme to conduct thematic searches of textual data, depicted responses to visual analog scales on box-plot diagrams, and integrated findings thematically. Fifty-three clinical trial experts from four constituent groups participated: academic biostatisticians (n = 5); consultant biostatisticians (n = 6); academic clinicians (n = 22); and other stakeholders including patient advocacy, National Institutes of Health, and U.S. Food and Drug Administration representatives (n = 20).

RESULTS

The respondents recognized potential ethical benefits of ACTs, including a higher probability of receiving an effective intervention for participants, optimizing resource utilization, and accelerating treatment discovery. Ethical challenges voiced include developing procedures so trial participants can make informed decisions about taking part in ACTs and plausible, though unlikely risks of research personnel altering enrollment patterns.

CONCLUSIONS

Clinical trial experts recognize ethical advantages but also pose potential ethical challenges of ACTs. The four constituencies differ in their weighing of ACT ethical considerations based on their professional vantage points. These data suggest further discussion about the ethics of ACTs is needed to facilitate ACT planning, design and conduct, and ultimately better allow planners to weigh ethical implications of competing trial designs.

The Stroke Hyperglycemia Insulin Network Effort (SHINE) trial: an adaptive trial design case study

Background

The ‘Adaptive Designs Accelerating Promising Trials into Treatments (ADAPT-IT)’ project is a collaborative effort supported by the National Institutes of Health (NIH) and United States Food & Drug Administration (FDA) to explore how adaptive clinical trial design might improve the evaluation of drugs and medical devices. ADAPT-IT uses the National Institute of Neurologic Disorders & Stroke-supported Neurological Emergencies Treatment Trials (NETT) network as a ‘laboratory’ in which to study the development of adaptive clinical trial designs in the confirmatory setting. The Stroke Hyperglycemia Insulin Network Effort (SHINE) trial was selected for funding by the NIH-NINDS at the start of ADAPT-IT and is currently an ongoing phase III trial of tight glucose control in hyperglycemic acute ischemic stroke patients. Within ADAPT-IT, a Bayesian adaptive Goldilocks trial design alternative was developed.

Methods

The SHINE design includes response adaptive randomization, a sample size re-estimation, and monitoring for early efficacy and futility according to a group sequential design. The Goldilocks design includes more frequent monitoring for predicted success or futility and a longitudinal model of the primary endpoint. Both trial designs were simulated and compared in terms of their mean sample size and power across a range of treatment effects and success rates for the control group.

Results

As simulated, the SHINE design tends to have slightly higher power and the Goldilocks design has a lower mean sample size. Both designs were tuned to have approximately 80% power to detect a difference of 25% versus 32% between control and treatment, respectively. In this scenario, mean sample sizes are 1,114 and 979 for the SHINE and Goldilocks designs, respectively.

Conclusions

Two designs were brought forward, and both were evaluated, revised, and improved based on the input of all parties involved in the ADAPT-IT process. However, the SHINE investigators were tasked with choosing only a single design to implement and ultimately elected not to implement the Goldilocks design. The Goldilocks design will be retrospectively executed upon completion of SHINE to later compare the designs based on their use of patient resources, time, and conclusions in a real world setting.

Trial registration

ClinicalTrials.gov NCT01369069 June 2011.

A surrogate-primary replacement algorithm for response-adaptive randomization in stroke clinical trials

Response-adaptive randomization (RAR) offers clinical investigators benefit by modifying the treatment allocation probabilities to optimize the ethical, operational, or statistical performance of the trial. Delayed primary outcomes and their effect on RAR have been studied in the literature; however, the incorporation of surrogate outcomes has not been fully addressed. We explore the benefits and limitations of surrogate outcome utilization in RAR in the context of acute stroke clinical trials. We propose a novel surrogate-primary (S-P) replacement algorithm where a patient's surrogate outcome is used in the RAR algorithm only until their primary outcome becomes available to replace it. Computer simulations investigate the effect of both the delay in obtaining the primary outcome and the underlying surrogate and primary outcome distributional discrepancies on complete randomization, standard RAR and the S-P replacement algorithm methods. Results show that when the primary outcome is delayed, the S-P replacement algorithm reduces the variability of the treatment allocation probabilities and achieves stabilization sooner. Additionally, the S-P replacement algorithm benefit proved to be robust in that it preserved power and reduced the expected number of failures across a variety of scenarios.

Bayesian methodology for the design and interpretation of clinical trials in critical care medicine: a primer for clinicians

OBJECTIVES

To review Bayesian methodology and its utility to clinical decision making and research in the critical care field.

DATA SOURCE AND STUDY SELECTION

Clinical, epidemiological, and biostatistical studies on Bayesian methods in PubMed and Embase from their inception to December 2013.

DATA SYNTHESIS

Bayesian methods have been extensively used by a wide range of scientific fields, including astronomy, engineering, chemistry, genetics, physics, geology, paleontology, climatology, cryptography, linguistics, ecology, and computational sciences. The application of medical knowledge in clinical research is analogous to the application of medical knowledge in clinical practice. Bedside physicians have to make most diagnostic and treatment decisions on critically ill patients every day without clear-cut evidence-based medicine (more subjective than objective evidence). Similarly, clinical researchers have to make most decisions about trial design with limited available data. Bayesian methodology allows both subjective and objective aspects of knowledge to be formally measured and transparently incorporated into the design, execution, and interpretation of clinical trials. In addition, various degrees of knowledge and several hypotheses can be tested at the same time in a single clinical trial without the risk of multiplicity. Notably, the Bayesian technology is naturally suited for the interpretation of clinical trial findings for the individualized care of critically ill patients and for the optimization of public health policies.

CONCLUSIONS

We propose that the application of the versatile Bayesian methodology in conjunction with the conventional statistical methods is not only ripe for actual use in critical care clinical research but it is also a necessary step to maximize the performance of clinical trials and its translation to the practice of critical care medicine.

Managing competing demands in the implementation of response-adaptive randomization in a large multicenter phase III acute stroke trial

It is well known that competing demands exist between the control of important covariate imbalance and protection of treatment allocation randomness in confirmative clinical trials. When implementing a response-adaptive randomization algorithm in confirmative clinical trials designed under a frequentist framework, additional competing demands emerge between the shift of the treatment allocation ratio and the preservation of the power. Based on a large multicenter phase III stroke trial, we present a patient randomization scheme that manages these competing demands by applying a newly developed minimal sufficient balancing design for baseline covariates and a cap on the treatment allocation ratio shift in order to protect the allocation randomness and the power. Statistical properties of this randomization plan are studied by computer simulation. Trial operation characteristics, such as patient enrollment rate and primary outcome response delay, are also incorporated into the randomization plan.

Achieving integration in mixed methods designs-principles and practices

Mixed methods research offers powerful tools for investigating complex processes and systems in health and health care. This article describes integration principles and practices at three levels in mixed methods research and provides illustrative examples. Integration at the study design level occurs through three basic mixed method designs-exploratory sequential, explanatory sequential, and convergent-and through four advanced frameworks-multistage, intervention, case study, and participatory. Integration at the methods level occurs through four approaches. In connecting, one database links to the other through sampling. With building, one database informs the data collection approach of the other. When merging, the two databases are brought together for analysis. With embedding, data collection and analysis link at multiple points. Integration at the interpretation and reporting level occurs through narrative, data transformation, and joint display. The fit of integration describes the extent the qualitative and quantitative findings cohere. Understanding these principles and practices of integration can help health services researchers leverage the strengths of mixed methods.

Bayesian adaptive trials offer advantages in comparative effectiveness trials: an example in status epilepticus

OBJECTIVE

We present a novel Bayesian adaptive comparative effectiveness trial comparing three treatments for status epilepticus that uses adaptive randomization with potential early stopping.

STUDY DESIGN AND SETTING

The trial will enroll 720 unique patients in emergency departments and uses a Bayesian adaptive design.

RESULTS

The trial design is compared to a trial without adaptive randomization and produces an efficient trial in which a higher proportion of patients are likely to be randomized to the most effective treatment arm while generally using fewer total patients and offers higher power than an analogous trial with fixed randomization when identifying a superior treatment.

CONCLUSION

When one treatment is superior to the other two, the trial design provides better patient care, higher power, and a lower expected sample size.

Biomarker-based adaptive trials for patients with glioblastoma--lessons from I-SPY 2

The traditional clinical trials infrastructure may not be ideally suited to evaluate the numerous therapeutic hypotheses that result from the increasing number of available targeted agents combined with the various methodologies to molecularly subclassify patients with glioblastoma. Additionally, results from smaller screening studies are rarely translated to successful larger confirmatory studies, potentially related to a lack of efficient control arms or the use of unvalidated surrogate endpoints. Streamlining clinical trials and providing a flexible infrastructure for biomarker development is clearly needed for patients with glioblastoma. The experience developing and implementing the I-SPY studies in breast cancer may serve as a guide to developing such trials in neuro-oncology.

Spinal cord injury neuroprotection and the promise of flexible adaptive clinical trials

Effective treatments for acute neurologic illness and injury are lacking, particularly for spinal cord injury (SCI). The very structure of clinical trials may be contributing to this because assumptions made during trial planning preclude additional learning within residual important areas of uncertainty, such as dose, timing, and duration of treatment. Adaptive clinical trials offer potential solutions to some of the factors that may be slowing the pace of discovery. Broadly defined, one can consider an adaptive clinical trial as any sort of clinical trial that makes use of information from within the trial to make decisions about how the trial is conducted going forward; however, it is important to emphasize that regardless of the degree of flexibility or complexity of an adaptive clinical trial design, the types of designs being described are only those in which all potential changes to the conduct of the trial are prospectively defined before the first patient is enrolled. Within this review, we describe the structure of flexible adaptive clinical trial designs, the process by which they are developed and conducted, and potential opportunities and drawbacks of these approaches. We must accept that there are some uncertainties that remain when both exploratory and confirmatory trials are designed. The process by which teams carefully consider which uncertainties are most important and most likely to potentially compromise the ability to detect an effective treatment can lead to trial designs that are more likely to find the right treatment for the right population of patients.

Overview, hurdles, and future work in adaptive designs: perspectives from a National Institutes of Health-funded workshop

BACKGROUND

The clinical trials community has a never-ending search for dependable and reliable ways to improve clinical research. This exploration has led to considerable interest in adaptive clinical trial designs, which provide the flexibility to adjust trial characteristics on the basis of data reviewed at interim stages. Statisticians and clinical investigators have proposed or implemented a wide variety of adaptations in clinical trials, but specific approaches have met with differing levels of support. Within industry, investigators are actively exploring the benefits and pitfalls associated with adaptive designs (ADs). For example, a Drug Information Association (DIA) working group on ADs has engaged regulatory agencies in discussions. Many researchers working on publicly funded clinical trials, however, are not yet fully engaged in this discussion. We organized the Scientific Advances in Adaptive Clinical Trial Designs Workshop to begin a conversation about using ADs in publicly funded research. Held in November of 2009, the 1½-day workshop brought together representatives from the National Institutes of Health (NIH), the Food and Drug Administration (FDA), the European Medicines Agency (EMA), the pharmaceutical industry, nonprofit foundations, the patient advocacy community, and academia. The workshop offered a forum for participants to address issues of ADs that arise at the planning, designing, and execution stages of clinical trials, and to hear the perspectives of influential members of the clinical trials community. The participants also set forth recommendations for guiding action to promote the appropriate use of ADs. These recommendations have since been presented, discussed, and vetted in a number of venues including the University of Pennsylvania Conference on Statistical Issues in Clinical Trials and the Society for Clinical Trials annual meeting.

PURPOSE

To provide a brief overview of ADs, describe the rationale behind conducting the workshop, and summarize the main recommendations that were produced as a result of this workshop.

CONCLUSIONS

There is a growing interest in the use of adaptive clinical trial designs. However, a number of logistical barriers need to be addressed in order to obtain the potential advantages of an AD. Currently, the pharmaceutical industry is well ahead of academic trialists with respect to addressing these barriers. Academic trialists will need to address important issues such as education, infrastructure, modifications to existing funding models, and the impact on Data and Safety Monitoring Boards (DSMB) in order to achieve the possible benefits of adaptive clinical trial designs.