Using short-term endpoints to improve interim decision making and trial duration in two-stage phase II trials with nested binary endpoints

In oncology, phase II clinical trials are often planned as single-arm two-stage designs with a binary endpoint, for example, progression-free survival after 12 months, and the option to stop for futility after the first stage. Simon's two-stage design is a very popular approach but depending on the follow-up time required to measure the patients' outcomes the trial may have to be paused undesirably long. To shorten this forced interruption, it was proposed to use a short-term endpoint for the interim decision, such as progression-free survival after 3 months. We show that if the assumptions for the short-term endpoint are misspecified, the decision-making in the interim can be misleading, resulting in a great loss of statistical power. For the setting of a binary endpoint with nested measurements, such as progression-free survival, we propose two approaches that utilize all available short-term and long-term assessments of the endpoint to guide the interim decision. One approach is based on conditional power and the other is based on Bayesian posterior predictive probability of success. In extensive simulations, we show that both methods perform similarly, when appropriately calibrated, and can greatly improve power compared to the existing approach in settings with slow patient recruitment. Software code to implement the methods is made publicly available.

Practical guidance for planning resources required to support publicly-funded adaptive clinical trials

Adaptive designs are a class of methods for improving efficiency and patient benefit of clinical trials. Although their use has increased in recent years, research suggests they are not used in many situations where they have potential to bring benefit. One barrier to their more widespread use is a lack of understanding about how the choice to use an adaptive design, rather than a traditional design, affects resources (staff and non-staff) required to set-up, conduct and report a trial. The Costing Adaptive Trials project investigated this issue using quantitative and qualitative research amongst UK Clinical Trials Units. Here, we present guidance that is informed by our research, on considering the appropriate resourcing of adaptive trials. We outline a five-step process to estimate the resources required and provide an accompanying costing tool. The process involves understanding the tasks required to undertake a trial, and how the adaptive design affects them. We identify barriers in the publicly funded landscape and provide recommendations to trial funders that would address them. Although our guidance and recommendations are most relevant to UK non-commercial trials, many aspects are relevant more widely.

Costs and staffing resource requirements for adaptive clinical trials: quantitative and qualitative results from the Costing Adaptive Trials project

BACKGROUND

Adaptive designs offer great promise in improving the efficiency and patient-benefit of clinical trials. An important barrier to further increased use is a lack of understanding about which additional resources are required to conduct a high-quality adaptive clinical trial, compared to a traditional fixed design. The Costing Adaptive Trials (CAT) project investigated which additional resources may be required to support adaptive trials.

METHODS

We conducted a mock costing exercise amongst seven Clinical Trials Units (CTUs) in the UK. Five scenarios were developed, derived from funded clinical trials, where a non-adaptive version and an adaptive version were described. Each scenario represented a different type of adaptive design. CTU staff were asked to provide the costs and staff time they estimated would be needed to support the trial, categorised into specified areas (e.g. statistics, data management, trial management). This was calculated separately for the non-adaptive and adaptive version of the trial, allowing paired comparisons. Interviews with 10 CTU staff who had completed the costing exercise were conducted by qualitative researchers to explore reasons for similarities and differences.

RESULTS

Estimated resources associated with conducting an adaptive trial were always (moderately) higher than for the non-adaptive equivalent. The median increase was between 2 and 4% for all scenarios, except for sample size re-estimation which was 26.5% (as the adaptive design could lead to a lengthened study period). The highest increase was for statistical staff, with lower increases for data management and trial management staff. The percentage increase in resources varied across different CTUs. The interviews identified possible explanations for differences, including (1) experience in adaptive trials, (2) the complexity of the non-adaptive and adaptive design, and (3) the extent of non-trial specific core infrastructure funding the CTU had.

CONCLUSIONS

This work sheds light on additional resources required to adequately support a high-quality adaptive trial. The percentage increase in costs for supporting an adaptive trial was generally modest and should not be a barrier to adaptive designs being cost-effective to use in practice. Informed by the results of this research, guidance for investigators and funders will be developed on appropriately resourcing adaptive trials.

Stakeholder perspectives on adaptive clinical trials: a scoping review

Background

Adaptive clinical trials (ACTs) represent an emerging approach to trial design where accumulating data are used to make decisions about future conduct. Adaptations can include comparisons of multiple dose tiers, response-adaptive randomization, sample size re-estimation, and efficacy/futility stopping rules. The objective of this scoping review is to assess stakeholder attitudes, perspectives, and understanding of adaptive trials.

Methods

We conducted a review of articles examining stakeholders encompassing the broad medical trial community’s perspectives of adaptive designs (ADs). A computerized search was conducted of four electronic databases with relevant search terms. Following screening of articles, the primary findings of each included article were coded for study design, population studied, purpose, and primary implications.

Results

Our team retrieved 167 peer-reviewed titles in total from the database search and 5 additional titles through searching web-based search engines for gray literature. Of those 172 titles, 152 were non-duplicate citations. Of these, 119 were not given full-text reviews, as their titles and abstracts indicated that they did not meet the inclusion criteria. Thirty-three articles were carefully examined for relevance, and of those, 18 were chosen to be part of the analysis; the other 15 were excluded, as they were not relevant upon closer inspection.

Perceived advantages to ADs included limiting ineffective treatments and efficiency in answering the research question; −perceived barriers included insufficient sample size for secondary outcomes, challenges of consent, potential for bias, risk of type 1 error, cost and time to adaptively design trials, unclear rationales for using Ads, and, most importantly, a lack of education regarding ADs among stakeholders within the clinical trial community. Perceptions among different types of stakeholders varied from sector to sector, with patient perspectives being noticeably absent from the literature.

Conclusion

There are diverse perceptions regarding ADs among stakeholders. Further training, guidelines, and toolkits on the proper use of ADs are needed at all levels to overcome many of these perceived barriers. While education for principal investigators is important, it is also crucial to educate other groups in the community, such as patients, as well as clinicians and staff involved in their daily implementation.

Maintenance therapy and need for cessation studies in multiple myeloma: Focus on the future

With ten years of follow-up since the advent of the modern paradigm of combination induction therapy, consolidative autologous stem-cell transplant, and lenalidomide maintenance, median survival for multiple myeloma has increased to almost 50% at 10 years. Given this outlook, the overarching goal of maintenance therapy is to spare patients from the toxicities of aggressive or otherwise intrusive therapies while ideally extending survival or, at the least, extending progression-free survival or time until next treatment. This review will focus on the current landscape of maintenance therapies for multiple myeloma. The historical context and evidence for choice of agent, duration of treatment, and current strategies and ongoing trials will be discussed - as well as a focus on unmet needs. The case for studies investigating cessation of therapy and risk and response-adapted approaches will be underscored given that the current paradigm likely results in overtreatment for some patients.

Seven unconfirmed ideas to improve future ICU practice

With imprecise definitions, inexact measurement tools, and flawed study execution, our clinical science often lags behind bedside experience and simply documents what appear to be the apparent faults or validity of ongoing practices. These impressions are later confirmed, modified, or overturned by the results of the next trial. On the other hand, insights that stem from the intuitions of experienced clinicians, scientists and educators-while often neglected-help place current thinking into proper perspective and occasionally point the way toward formulating novel hypotheses that direct future research. Both streams of information and opinion contribute to progress. In this paper we present a wide-ranging set of unproven 'out of the mainstream' ideas of our FCCM faculty, each with a defensible rationale and holding clear implications for altering bedside management. Each proposition was designed deliberately to be provocative so as to raise awareness, stimulate new thinking and initiate lively dialog.

Futility for subgroup analyses in the adaptive signature design

Subgroup analyses in clinical trials are becoming increasingly important. In cancer research, more and more targeted therapies are explored from which probably only a portion of the whole population will benefit. An adaptive design for subgroup selection with identification of a subgroup, the adaptive signature design, was proposed in the literature. Unfortunately, measuring and validating the variables defining the subgroup (i.e. biomarkers) can be extremely expensive. For this reason, we propose an extension of this design where subgroup analysis is not performed when the overall results suggest that it is unlikely to achieve statistical significance in the subgroup. Avoiding measuring and validating expensive biomarkers in this case can save resources that could be used on more promising research.

The life cycles of six multi-center adaptive clinical trials focused on neurological emergencies developed for the Advancing Regulatory Science initiative of the National Institutes of Health and US Food and Drug Administration: Case studies from the Adaptive Designs Accelerating Promising Treatments Into Trials Project

OBJECTIVES

Clinical trials are complicated, expensive, time-consuming, and frequently do not lead to discoveries that improve the health of patients with disease. Adaptive clinical trials have emerged as a methodology to provide more flexibility in design elements to better answer scientific questions regarding whether new treatments are efficacious. Limited observational data exist that describe the complex process of designing adaptive clinical trials. To address these issues, the Adaptive Designs Accelerating Promising Treatments Into Trials project developed six, tailored, flexible, adaptive, phase-III clinical trials for neurological emergencies, and investigators prospectively monitored and observed the processes. The objective of this work is to describe the adaptive design development process, the final design, and the current status of the adaptive trial designs that were developed.

METHODS

To observe and reflect upon the trial development process, we employed a rich, mixed methods evaluation that combined quantitative data from visual analog scale to assess attitudes about adaptive trials, along with in-depth qualitative data about the development process gathered from observations.

RESULTS

The Adaptive Designs Accelerating Promising Treatments Into Trials team developed six adaptive clinical trial designs. Across the six designs, 53 attitude surveys were completed at baseline and after the trial planning process completed. Compared to baseline, the participants believed significantly more strongly that the adaptive designs would be accepted by National Institutes of Health review panels and non-researcher clinicians. In addition, after the trial planning process, the participants more strongly believed that the adaptive design would meet the scientific and medical goals of the studies.

CONCLUSION

Introducing the adaptive design at early conceptualization proved critical to successful adoption and implementation of that trial. Involving key stakeholders from several scientific domains early in the process appears to be associated with improved attitudes towards adaptive designs over the life cycle of clinical trial development.

A conceptual model for the development process of confirmatory adaptive clinical trials within an emergency research network

BACKGROUND

Adaptive clinical trials use accumulating data from enrolled subjects to alter trial conduct in pre-specified ways based on quantitative decision rules. In this research, we sought to characterize the perspectives of key stakeholders during the development process of confirmatory-phase adaptive clinical trials within an emergency clinical trials network and to build a model to guide future development of adaptive clinical trials.

METHODS

We used an ethnographic, qualitative approach to evaluate key stakeholders' views about the adaptive clinical trial development process. Stakeholders participated in a series of multidisciplinary meetings during the development of five adaptive clinical trials and completed a Strengths-Weaknesses-Opportunities-Threats questionnaire. In the analysis, we elucidated overarching themes across the stakeholders' responses to develop a conceptual model.

RESULTS

Four major overarching themes emerged during the analysis of stakeholders' responses to questioning: the perceived statistical complexity of adaptive clinical trials and the roles of collaboration, communication, and time during the development process. Frequent and open communication and collaboration were viewed by stakeholders as critical during the development process, as were the careful management of time and logistical issues related to the complexity of planning adaptive clinical trials.

CONCLUSION

The Adaptive Design Development Model illustrates how statistical complexity, time, communication, and collaboration are moderating factors in the adaptive design development process. The intensity and iterative nature of this process underscores the need for funding mechanisms for the development of novel trial proposals in academic settings.

Attitudes and opinions regarding confirmatory adaptive clinical trials: a mixed methods analysis from the Adaptive Designs Accelerating Promising Trials into Treatments (ADAPT-IT) project

Background

Adaptive designs have been increasingly used in the pharmaceutical and device industries, but adoption within the academic setting has been less widespread — particularly for confirmatory phase trials. We sought to understand perceptions about understanding, acceptability, and scientific validity of adaptive clinical trials (ACTs).

Methods

We used a convergent mixed methods design using survey and mini-focus group data collection procedures to elucidate attitudes and opinions among “trial community” stakeholders regarding understanding, acceptability, efficiency, scientific validity, and speed of discovery with adaptive designs. Data were collected about various aspects of ACTs using self-administered surveys (paper or Web-based) with visual analog scales (VASs) with free text responses and with mini-focus groups of key stakeholders. Participants were recruited as part of an ongoing NIH/FDA-funded research project exploring the incorporation of ACTs into an existing NIH network that focuses on confirmatory phase clinical trials in neurological emergencies. “Trial community” representatives, namely, clinical investigators, biostatisticians, NIH officials, and FDA scientists involved in the planning of four clinical trials, were eligible to participate. In addition, recent and current members of a clinical trial-oriented NIH study section were also eligible.

Results

A total of 76 stakeholders completed the survey (out of 91 who were offered it, response rate 84 %). While the VAS attitudinal data showed substantial variability across respondents about acceptability and understanding of ACTs by various constituencies, respondents perceived clinicians to be less likely to understand ACTs and that ACTs probably would increase the efficiency of discovery. Textual and focus group responses emerged into several themes that enhanced understanding of VAS attitudinal data including the following: acceptability of adaptive designs depends on constituency and situation; there is variable understanding of ACTs (limited among clinicians, perceived to be higher at FDA); views about the potential for efficiency depend on the situation and implementation. Participants also frequently mentioned a need for greater education within the academic community. Finally, the empiric, non-quantitative selection of treatments for phase III trials based on limited phase II trials was highlighted as an opportunity for improvement and a potential explanation for the high number of neutral confirmatory trials.

Conclusions

These data show considerable variations in attitudes and beliefs about ACTs among trial community representatives. For adaptive trials to be fully considered when appropriate and for the research enterprise to realize the full potential of adaptive designs will likely require extensive experience and trust building within the trial community.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1493-z) contains supplementary material, which is available to authorized users.

Precision Bariatrics: Toward a New Paradigm of Personalized Devices in Obesity Therapeutics

The prevalence and complexity of obesity and its associated metabolic complications highlight the importance of building a rigorous investigative framework for the development of novel weight loss therapies. Device-based interventions in particular constitute a market poised for rapid expansion in the coming years. Optimizing outcomes for this new class of therapies requires attention to an evolving taxonomy of subdivisions within the broader obesity phenotype and a means for stratifying patients toward maximally effective interventions. Extant bariatric devices implicitly prioritize anatomic variables as surrogates for physiology, a somewhat arbitrary assumption that merits empiric validation. Utilizing the governing principles of systems biology and recent innovations in clinical trial design, a robust and precise research infrastructure can and should be developed to more effectively mitigate this contemporary epidemic.

Simulation of various randomization strategies for a clinical trial in sickle cell disease

OBJECTIVE

To use numerical simulation to evaluate various randomization strategies for a clinical trial in sickle cell disease (SCD).

METHODS

The Inhaled Mometasone to Promote Reduction in Vaso-occlusive Events trial* is a randomized, controlled, feasibility study of inhaled mometasone for individuals with SCD who do not have asthma. The target sample size is 45 patients and one goal is to limit imbalance with respect to two important covariates (1) hydroxyurea use and (2) historical emergency department (ED) utilization. We compared three methods of patient allocation (simple randomization, block randomization, and biased-coin adaptive randomization) using numerical simulation (10 000 trials). The primary outcome measure was the proportion of simulated trials with numerically apparent differences in the two covariates: hydroxyurea use (binary) and ED utilization (three-level ordinal).

RESULTS

Overall, only 1.6% of simulated trials had any covariate comparison with P < 0.3 across groups for simple randomization, and 0% for both the block and adaptive randomization. In trials where the total sample size was 45 patients, the block randomization strategy achieved the greatest balance because participants were deterministically assigned to the treatment arm that balanced covariates. The adaptive strategy achieved similar results without deterministic treatment assignments even when trials included only 45 patients.

DISCUSSION

Adaptive clinical trial designs have potential to mitigate some of the challenges that have hampered SCD trials. In small exploratory trials, even non-statistically significant differences in important covariates can threaten interpretability and external validity.

CONCLUSION

Adaptive randomization performed similarly to block randomization and offers advantages including better allocation concealment and less ability for investigators to predict the next assignment.

Spinal cord injury neuroprotection and the promise of flexible adaptive clinical trials

Effective treatments for acute neurologic illness and injury are lacking, particularly for spinal cord injury (SCI). The very structure of clinical trials may be contributing to this because assumptions made during trial planning preclude additional learning within residual important areas of uncertainty, such as dose, timing, and duration of treatment. Adaptive clinical trials offer potential solutions to some of the factors that may be slowing the pace of discovery. Broadly defined, one can consider an adaptive clinical trial as any sort of clinical trial that makes use of information from within the trial to make decisions about how the trial is conducted going forward; however, it is important to emphasize that regardless of the degree of flexibility or complexity of an adaptive clinical trial design, the types of designs being described are only those in which all potential changes to the conduct of the trial are prospectively defined before the first patient is enrolled. Within this review, we describe the structure of flexible adaptive clinical trial designs, the process by which they are developed and conducted, and potential opportunities and drawbacks of these approaches. We must accept that there are some uncertainties that remain when both exploratory and confirmatory trials are designed. The process by which teams carefully consider which uncertainties are most important and most likely to potentially compromise the ability to detect an effective treatment can lead to trial designs that are more likely to find the right treatment for the right population of patients.

Adaptive enrichment designs for clinical trials

Modern medicine has graduated from broad spectrum treatments to targeted therapeutics. New drugs recognize the recently discovered heterogeneity of many diseases previously considered to be fairly homogeneous. These treatments attack specific genetic pathways which are only dysregulated in some smaller subset of patients with the disease. Often this subset is only rudimentarily understood until well into large-scale clinical trials. As such, standard practice has been to enroll a broad range of patients and run post hoc subset analysis to determine those who may particularly benefit. This unnecessarily exposes many patients to hazardous side effects, and may vastly decrease the efficiency of the trial (especially if only a small subset of patients benefit). In this manuscript, we propose a class of adaptive enrichment designs that allow the eligibility criteria of a trial to be adaptively updated during the trial, restricting entry to patients likely to benefit from the new treatment. We show that our designs both preserve the type 1 error, and in a variety of cases provide a substantial increase in power.