A Review of Clinical Trials With an Adaptive Design and Health Economic Analysis

A landscape of methodology and implementation of adaptive designs in cancer clinical trials

BACKGROUND

The use of adaptive designs in cancer trials has considerably increased worldwide in recent years, along with the release of various guidelines for their application. This systematic review aims to comprehensively summarize the key methodological and executive features of adaptive designs in cancer clinical trials.

METHODS

A comprehensive search from PubMed, EMBASE, and the Cochrane Central Register of Controlled Trials was conducted to screen eligible clinical trials that employed adaptive designs and were conducted in cancer patients. The methodological and executive characteristics of adaptive designs were the main measurements extracted. Descriptive analyses, primarily consisting of frequency and percentage, were employed to analyzed and reported the data.

RESULTS

A total of 180 cancer clinical trials with adaptive designs were identified. The first three most common type of adaptive design was the group sequential design (n=114, 63.3 %), adaptive dose-finding design (n=22, 12.2 %), and adaptive platform design (n=16, 8.9 %). The results showed that 4.4 % (n=8) of trials conducted post hoc modifications, and around 29.4 % (n=53) did not provide the methods for controlling type I errors. Among phase II or above trials, 79.9 % (112/140) applied the surrogate endpoint as the primary outcome in these trials. Importantly, 27.2 % (49/180) of trials did not report clear information on the independent data monitoring committee (iDMC), and 13.3 % (n=24) without clear information on interim analyses. Interim analyses suggested 34.4 % (62/180) of trials being stopped for futility, 10.6 % (n=19) for efficacy, and 2.2 % (n=4) for safety concerns in the early stage.

CONCLUSIONS

This study emphasizes adaptive designs in cancer trials face significant challenges in their design or strict implementation according to protocol, which might significantly compromise the validity and integrity of trials. It is thus important for researchers, sponsors, and policymakers to actively oversee and guide their application.

Adaptive design trials in eating disorder research: A scoping review

OBJECTIVE

This scoping review sought to map the breadth of literature on the use of adaptive design trials in eating disorder research.

METHOD

A systematic literature search was conducted in Medline, Scopus, PsycInfo, Emcare, Econlit, CINAHL and ProQuest Dissertations and Theses. Articles were included if they reported on an intervention targeting any type of eating disorder (including anorexia nervosa, bulimia nervosa, binge-eating disorder, and other specified feeding or eating disorders), and employed the use of an adaptive design trial to evaluate the intervention. Two independent reviewers screened citations for inclusion, and data abstraction was performed by one reviewer and verified by a second.

RESULTS

We identified five adaptive design trials targeting anorexia nervosa, bulimia nervosa and binge-eating disorder conducted in the USA and Australia. All employed adaptive treatment arm switching based on early response to treatment and identified a priori stopping rules. None of the studies included value of information analysis to guide adaptive design decisions and none included lived experience perspectives.

DISCUSSION

The limited use of adaptive designs in eating disorder trials represents a missed opportunity to improve enrolment targets, attrition rates, treatment outcomes and trial efficiency. We outline the range of adaptive methodologies, how they could be applied to eating disorder research, and the specific operational and statistical considerations relevant to adaptive design trials.

PUBLIC SIGNIFICANCE

Adaptive design trials are increasingly employed as flexible, efficient alternatives to fixed trial designs, but they are not often used in eating disorder research. This first scoping review identified five adaptive design trials targeting anorexia nervosa, bulimia nervosa and binge-eating disorder that employed treatment arm switching adaptive methodology. We make recommendations on the use of adaptive design trials for future eating disorder trials.

Adaptive designs in critical care trials: a simulation study

BACKGROUND

Adaptive clinical trials are growing in popularity as they are more flexible, efficient and ethical than traditional fixed designs. However, notwithstanding their increased use in assessing treatments for COVID-19, their use in critical care trials remains limited. A better understanding of the relative benefits of various adaptive designs may increase their use and interpretation.

METHODS

Using two large critical care trials (ADRENAL.

CLINICALTRIALS

gov number, NCT01448109. Updated 12-12-2017; NICE-SUGAR.

CLINICALTRIALS

gov number, NCT00220987. Updated 01-29-2009), we assessed the performance of three frequentist and two bayesian adaptive approaches. We retrospectively re-analysed the trials with one, two, four, and nine equally spaced interims. Using the original hypotheses, we conducted 10,000 simulations to derive error rates, probabilities of making an early correct and incorrect decision, expected sample size and treatment effect estimates under the null scenario (no treatment effect) and alternative scenario (a positive treatment effect). We used a logistic regression model with 90-day mortality as the outcome and the treatment arm as the covariate. The null hypothesis was tested using a two-sided significance level (α) at 0.05.

RESULTS

Across all approaches, increasing the number of interims led to a decreased expected sample size. Under the null scenario, group sequential approaches provided good control of the type-I error rate; however, the type I error rate inflation was an issue for the Bayesian approaches. The Bayesian Predictive Probability and O'Brien-Fleming approaches showed the highest probability of correctly stopping the trials (around 95%). Under the alternative scenario, the Bayesian approaches showed the highest overall probability of correctly stopping the ADRENAL trial for efficacy (around 91%), whereas the Haybittle-Peto approach achieved the greatest power for the NICE-SUGAR trial. Treatment effect estimates became increasingly underestimated as the number of interims increased.

CONCLUSIONS

This study confirms the right adaptive design can reach the same conclusion as a fixed design with a much-reduced sample size. The efficiency gain associated with an increased number of interims is highly relevant to late-phase critical care trials with large sample sizes and short follow-up times. Systematically exploring adaptive methods at the trial design stage will aid the choice of the most appropriate method.

Point estimation for adaptive trial designs II: Practical considerations and guidance

In adaptive clinical trials, the conventional end-of-trial point estimate of a treatment effect is prone to bias, that is, a systematic tendency to deviate from its true value. As stated in recent FDA guidance on adaptive designs, it is desirable to report estimates of treatment effects that reduce or remove this bias. However, it may be unclear which of the available estimators are preferable, and their use remains rare in practice. This article is the second in a two-part series that studies the issue of bias in point estimation for adaptive trials. Part I provided a methodological review of approaches to remove or reduce the potential bias in point estimation for adaptive designs. In part II, we discuss how bias can affect standard estimators and assess the negative impact this can have. We review current practice for reporting point estimates and illustrate the computation of different estimators using a real adaptive trial example (including code), which we use as a basis for a simulation study. We show that while on average the values of these estimators can be similar, for a particular trial realization they can give noticeably different values for the estimated treatment effect. Finally, we propose guidelines for researchers around the choice of estimators and the reporting of estimates following an adaptive design. The issue of bias should be considered throughout the whole lifecycle of an adaptive design, with the estimation strategy prespecified in the statistical analysis plan. When available, unbiased or bias-reduced estimates are to be preferred.

Practical guidance for planning resources required to support publicly-funded adaptive clinical trials

Adaptive designs are a class of methods for improving efficiency and patient benefit of clinical trials. Although their use has increased in recent years, research suggests they are not used in many situations where they have potential to bring benefit. One barrier to their more widespread use is a lack of understanding about how the choice to use an adaptive design, rather than a traditional design, affects resources (staff and non-staff) required to set-up, conduct and report a trial. The Costing Adaptive Trials project investigated this issue using quantitative and qualitative research amongst UK Clinical Trials Units. Here, we present guidance that is informed by our research, on considering the appropriate resourcing of adaptive trials. We outline a five-step process to estimate the resources required and provide an accompanying costing tool. The process involves understanding the tasks required to undertake a trial, and how the adaptive design affects them. We identify barriers in the publicly funded landscape and provide recommendations to trial funders that would address them. Although our guidance and recommendations are most relevant to UK non-commercial trials, many aspects are relevant more widely.

An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial

BACKGROUND

Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial. In this article, we develop new methods for computing the EVSI of extending an existing trial's follow-up, first for an assumed survival model and then extending to capture uncertainty about the true survival model.

METHODS

We developed a nested Markov Chain Monte Carlo procedure and a nonparametric regression-based method. We compared the methods by computing single-model and model-averaged EVSI for collecting additional follow-up data in 2 synthetic case studies.

RESULTS

There was good agreement between the 2 methods. The regression-based method was fast and straightforward to implement, and scales easily to include any number of candidate survival models in the model uncertainty case. The nested Monte Carlo procedure, on the other hand, was extremely computationally demanding when we included model uncertainty.

CONCLUSIONS

We present a straightforward regression-based method for computing the EVSI of extending an existing trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. EVSI for ongoing trials can help decision makers determine whether early patient access to a new technology can be justified on the basis of the current evidence or whether more mature evidence is needed.

HIGHLIGHTS

Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life-expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial.In this article, we have developed new methods for computing the EVSI of extending a trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. We extend a previously described nonparametric regression-based method for computing EVSI, which we demonstrate in synthetic case studies is fast, straightforward to implement, and scales easily to include any number of candidate survival models in the EVSI calculations.The EVSI methods that we present in this article can quantify the need for collecting additional follow-up data before making an adoption decision given any decision-making context.

Designing and conducting adaptive trials to evaluate interventions in health services and implementation research: practical considerations

Randomized controlled clinical trials are widely considered the gold standard for evaluating the efficacy or effectiveness of interventions in health care. Adaptive trials incorporate changes as the study proceeds, such as modifying allocation probabilities or eliminating treatment arms that are likely to be ineffective. These designs have been widely used in drug discovery studies but can also be useful in health services and implementation research and have been minimally used. As motivating examples, we use an ongoing adaptive trial and two completed parallel group studies and highlight potential advantages, disadvantages, and important considerations when using adaptive trial designs in health services and implementation research. In addition, we investigate the impact on power and the study duration if the two completed parallel-group trials had instead been conducted using adaptive principles. Compared with traditional trial designs, adaptive designs can often allow one to evaluate more interventions, adjust participant allocation probabilities (e.g., to achieve covariate balance), and identify participants who are likely to agree to enroll. These features could reduce resources needed to conduct a trial. However, adaptive trials have potential disadvantages and practical aspects that need to be considered, most notably outcomes that can be rapidly measured and extracted (e.g., long-term outcomes that take significant time to measure from data sources can be challenging), minimal missing data, and time trends. In conclusion, adaptive designs are a promising approach to help identify how best to implement evidence-based interventions into real-world practice in health services and implementation research.

Expected Value of Sample Information to Guide the Design of Group Sequential Clinical Trials

Introduction

Adaptive designs allow changes to an ongoing trial based on prespecified early examinations of accrued data. Opportunities are potentially being missed to incorporate health economic considerations into the design of these studies.

Methods

We describe how to estimate the expected value of sample information for group sequential design adaptive trials. We operationalize this approach in a hypothetical case study using data from a pilot trial. We report the expected value of sample information and expected net benefit of sampling results for 5 design options for the future full-scale trial including the fixed-sample-size design and the group sequential design using either the Pocock stopping rule or the O’Brien-Fleming stopping rule with 2 or 5 analyses. We considered 2 scenarios relating to 1) using the cost-effectiveness model with a traditional approach to the health economic analysis and 2) adjusting the cost-effectiveness analysis to incorporate the bias-adjusted maximum likelihood estimates of trial outcomes to account for the bias that can be generated in adaptive trials.

Results

The case study demonstrated that the methods developed could be successfully applied in practice. The results showed that the O’Brien-Fleming stopping rule with 2 analyses was the most efficient design with the highest expected net benefit of sampling in the case study.

Conclusions

Cost-effectiveness considerations are unavoidable in budget-constrained, publicly funded health care systems, and adaptive designs can provide an alternative to costly fixed-sample-size designs. We recommend that when planning a clinical trial, expected value of sample information methods be used to compare possible adaptive and nonadaptive trial designs, with appropriate adjustment, to help justify the choice of design characteristics and ensure the cost-effective use of research funding.

Highlights

Urgently seeking efficiency and sustainability of clinical trials in global health

This paper shows the scale of global health research and the context in which we frame the subsequent papers in the Series. In this Series paper, we provide a historical perspective on clinical trial research by revisiting the 1948 streptomycin trial for pulmonary tuberculosis, which was the first documented randomised clinical trial in the English language, and we discuss its close connection with global health. We describe the current state of clinical trial research globally by providing an overview of clinical trials that have been registered in the WHO International Clinical Trial Registry since 2010. We discuss challenges with current trial planning and designs that are often used in clinical trial research undertaken in low-income and middle-income countries, as an overview of the global health trials landscape. Finally, we discuss the importance of collaborative work in global health research towards generating sustainable and culturally appropriate research environments.

Lifecycle evidence requirements for high-risk implantable medical devices: a European perspective

INTRODUCTION

The new European Union (EU) Regulations on medical devices and on in vitro diagnostics provide manufacturers and Notified Bodies with new tools to improve pre-market and post-market clinical evidence generation especially for high-risk products but fail to indicate what type of clinical evidence is appropriate at each stage of the whole lifecycle of medical devices. In this paper we address: i) the appropriate level and timing of clinical evidence throughout the lifecycle of high-risk implantable medical devices; and ii) how the clinical evidence generation ecosystem could be adapted to optimize patient access.

AREAS COVERED

The European regulatory and health technology assessment (HTA) contexts are reviewed, in relation to the lifecycle of high-risk medical devices and clinical evidence generation recommended by international network or endorsed by regulatory and HTA agencies in different jurisdictions.

EXPERT OPINION

Four stages are relevant for clinical evidence generation: i) pre-clinical, pre-market; ii) clinical, pre-market; iii) diffusion, post-market; and iv) obsolescence & replacement, post-market. Each stage has its own evaluation needs and specific studies are recommended to generate the appropriate evidence. Effective lifecycle planning requires anticipation of what evidence will be needed at each stage.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

The Inclusion of Economic Endpoints as Outcomes in Clinical Trials Reported to ClinicalTrials.gov

BACKGROUND

As medication expenditures rise, payers are increasingly demanding evidence of economic value for new medications. The 2015 Professional Society for Health Economics and Outcomes Research (ISPOR) Task Force on Cost-Effectiveness Analysis Alongside Clinical Trials noted that clinical trials are increasingly including health care utilization endpoints to address this rising interest in economic information.

OBJECTIVES

To (a) describe the prevalence of economic endpoints in clinical trials submitted to ClinicalTrials.gov and (b) examine associations between trial characteristics and the inclusion of economic endpoints.

METHODS

This retrospective review of ClinicalTrials.gov data extracted the characteristics of clinical trials that were submitted to ClinicalTrials.gov from January 2004 to December 2018; studied a drug and/or biological; and had a recruitment status of not yet recruiting, recruiting, active but not recruiting, or completed. Studies were classified as containing an economic endpoint based on 2 independent evaluations of the inclusion of endpoints relevant to costs, resource utilization, cost-effectiveness, productivity, absenteeism, presenteeism, or unemployment. Descriptive statistics were used to summarize trial characteristics, and chi-square analyses were used to evaluate differences in characteristics between trials with and without economic endpoints.

RESULTS

Of the 104,885 trials included in the study, 1,437 (1.37%) included an economic endpoint; among later phase (phase 2/3, 3, 4) trials, 939 (2.54%) included economic endpoints. Compared with studies that did not include economic endpoints, those that did were less often industry funded (48.0% vs. 52.0%, P < 0.001) and were for a high-spend specialty condition (24.1% vs. 27.4%, P < 0.001). The proportion of trials that included economic endpoints increased by a small but significant amount over the time period studied, from 1.2% (2004-2008) to 1.6% (2014-2018; P < 0.001).

CONCLUSIONS

A small but growing number of clinical trials are including economic endpoints. This finding may reflect continued industry concerns surrounding the cost and logistical challenges of piggybacking economic data collection alongside clinical trials and/or manufacturers' preferences for modeling for value demonstration. Future research is needed to better understand barriers to the inclusion of economic endpoints as well as the degree to which incorporating health care resource utilization collected during clinical trials into early economic modeling may reduce payer concerns about model transparency and bias.

DISCLOSURES

No outside funding supported this study. Patterson reports past employment by Indivior, unrelated to this study. Mitchell has nothing to disclose. The research included in this study was presented as a nonreviewed student pharmacist poster at AMCP Nexus 2019; October 30-November 1, 2019; National Harbor, MD.

How can health economics be used in the design and analysis of adaptive clinical trials? A qualitative analysis

INTRODUCTION

Adaptive designs offer a flexible approach, allowing changes to a trial based on examinations of the data as it progresses. Adaptive clinical trials are becoming a popular choice, as the prudent use of finite research budgets and accurate decision-making are priorities for healthcare providers around the world. The methods of health economics, which aim to maximise the health gained for money spent, could be incorporated into the design and analysis of adaptive clinical trials to make them more efficient. We aimed to understand the perspectives of stakeholders in health technology assessments to inform recommendations for the use of health economics in adaptive clinical trials.

METHODS

A qualitative study explored the attitudes of key stakeholders-including researchers, decision-makers and members of the public-towards the use of health economics in the design and analysis of adaptive clinical trials. Data were collected using interviews and focus groups (29 participants). A framework analysis was used to identify themes in the transcripts.

RESULTS

It was considered that answering the clinical research question should be the priority in a clinical trial, notwithstanding the importance of cost-effectiveness for decision-making. Concerns raised by participants included handling the volatile nature of cost data at interim analyses; implementing this approach in global trials; resourcing adaptive trials which are designed and adapted based on health economic outcomes; and training stakeholders in these methods so that they can be implemented and appropriately interpreted.

CONCLUSION

The use of health economics in the design and analysis of adaptive clinical trials has the potential to increase the efficiency of health technology assessments worldwide. Recommendations are made concerning the development of methods allowing the use of health economics in adaptive clinical trials, and suggestions are given to facilitate their implementation in practice.