Interim Analyses During Group Sequential Clinical Trials

Bayesian Interim Analysis and Efficiency of Phase III Randomized Trials

IMPORTANCE

Improving the efficiency of interim assessments in phase III trials should reduce trial costs, hasten the approval of efficacious therapies, and mitigate patient exposure to disadvantageous randomizations.

OBJECTIVE

We hypothesized that in silico Bayesian early stopping rules improve the efficiency of phase III trials compared with the original frequentist analysis without compromising overall interpretation.

DESIGN

Cross-sectional analysis.

SETTING

230 randomized phase III oncology trials enrolling 184,752 participants.

PARTICIPANTS

Individual patient-level data were manually reconstructed from primary endpoint Kaplan-Meier curves.

INTERVENTIONS

Trial accruals were simulated 100 times per trial and leveraged published patient outcomes such that only the accrual dynamics, and not the patient outcomes, were randomly varied.

MAIN OUTCOMES AND MEASURES

Early stopping was triggered per simulation if interim analysis demonstrated ≥ 85% probability of minimum clinically important difference/3 for efficacy or futility. Trial-level early closure was defined by stopping frequencies ≥ 0.75.

RESULTS

A total of 12,451 simulations (54%) met early stopping criteria. Trial-level early stopping frequency was highly predictive of the published outcome (OR, 7.24; posterior probability of association, >99.99%; AUC, 0.91; P < 0.0001). Trial-level early closure was recommended for 82 trials (36%), including 62 trials (76%) which had performed frequentist interim analysis. Bayesian early stopping rules were 96% sensitive (95% CI, 91% to 98%) for detecting trials with a primary endpoint difference, and there was a high level of agreement in overall trial interpretation (Bayesian Cohen's κ, 0.95; 95% CrI, 0.92 to 0.99). However, Bayesian interim analysis was associated with >99.99% posterior probability of reducing patient enrollment requirements ( P < 0.0001), with an estimated cumulative enrollment reduction of 20,543 patients (11%; 89 patients averaged equally over all studied trials) and an estimated cumulative cost savings of 851 million USD (3.7 million USD averaged equally over all studied trials).

CONCLUSIONS AND RELEVANCE

Bayesian interim analyses may improve randomized trial efficiency by reducing enrollment requirements without compromising trial interpretation. Increased utilization of Bayesian interim analysis has the potential to reduce costs of late-phase trials, reduce patient exposures to ineffective therapies, and accelerate approvals of effective therapies.

KEY POINTS

Question: What are the effects of Bayesian early stopping rules on the efficiency of phase III randomized oncology trials?Findings: Individual-patient level outcomes were reconstructed for 184,752 patients from 230 trials. Compared with the original interim analysis strategy, in silico Bayesian interim analysis reduced patient enrollment requirements and preserved the original trial interpretation. Meaning: Bayesian interim analysis may improve the efficiency of conducting randomized trials, leading to reduced costs, reduced exposure of patients to disadvantageous treatments, and accelerated approval of efficacious therapies.

Focused deterrence: A protocol for a realist multisite randomised controlled trial for evaluating a violence prevention intervention in the UK

INTRODUCTION

Focused deterrence (FD) is a frequently cited intervention for preventing violence, particularly against violent urban gangs. The Youth Endowment Fund (YEF) believes it could be effective in the UK, based primarily on research conducted in the US. However, we contend that these studies have inadequate methodological designs, lack of rigorous testing, and small sample sizes. Therefore, the evidence supporting focused deterrence as an effective method, particularly outside the US, is inconclusive. The aim of the protocol is to better understand the potential effects of FD in the context of the UK, using a multisite evaluation experimental design to more closely investigate the evidence of its likely impact.

METHODS

We planned a realist randomised controlled trial. The design is focused on a multisite trial consisting of two-arm randomised experiments in five locations. Each trial location will test their implementation of a core programme specified by the funder. The multisite nature will allow us to understand differential impacts between locations, improving the external validity of the results. Participants will be randomly selected from a wider pool of eligible individuals for the intervention. We estimate a sample size of approximately N = 1,700 individuals is required. Based on this pooled sample size, a relative reduction of 26% would be detectable in 80% of trials. The trial is coupled with a formative process evaluation of delivery and fidelity. The formative evaluation will use a mixed methods design. The qualitative aspect will include semi-structured cross-sectional and longitudinal interviews with programme leads, programme delivery team, and programme participants, as well as observations of the meetings between the programme delivery team (i.e., community navigators/mentors) and programme participants. The quantitative data for the formative evaluation will be gathered by the sites themselves and consist of routine outcome performance monitoring using administrative data. Sampling for interviews and observations will vary, with the researchers aiming for a higher number of individuals included in the first round of cross-sectional interviews and retaining as many as possible for repeat interviews and observations.

DISCUSSION

This protocol outlines the process and impact evaluation methodology for the most extensive multisite evaluation of focused deterrence to date in the UK. Spanning five distinct sites with seven trials, the evaluation includes a cohort of 2,000 individuals, marking it as the only multisite trial of focused deterrence. Employing an integrated realist evaluation framework, the study uses qualitative and quantitative research methods. The anticipated findings will offer pivotal insights for formulating future violence prevention policies in the UK. They are also expected to contribute significantly to the corpus of literature on violence prevention and intervention evaluation.

TRIAL REGISTRATION

Protocol registration: ISRCTN: 11650008 4th June 2023.

Incorporating Efficacy Data from Initial Trials Into Subsequent Evaluations: Application to Vaccines Against Respiratory Syncytial Virus

BACKGROUND

When a randomized controlled trial fails to demonstrate statistically significant efficacy against the primary endpoint, a potentially costly new trial would need to be conducted to receive licensure. Incorporating data from previous trials might allow for more efficient follow-up trials to demonstrate efficacy, speeding the availability of effective vaccines.

METHODS

Based on the outcomes from a failed trial of a maternal vaccine against respiratory syncytial virus (RSV), we simulated data for a new Bayesian group-sequential trial. We analyzed the data either ignoring data from the previous trial (i.e., weakly informative prior distributions) or using prior distributions incorporating the historical data into the analysis. We evaluated scenarios where efficacy in the new trial was the same, greater than, or less than that in the original trial. For each scenario, we evaluated the statistical power and type I error rate for estimating the vaccine effect following interim analyses.

RESULTS

When we used a stringent threshold to control the type I error rate, analyses incorporating historical data had a small advantage over trials that did not. If control of type I error is less important (e.g., in a postlicensure evaluation), the incorporation of historical data can provide a substantial boost in efficiency.

CONCLUSIONS

Due to the need to control the type I error rate in trials used to license a vaccine, incorporating historical data provides little additional benefit in terms of stopping the trial early. However, these statistical approaches could be promising in evaluations that use real-world evidence following licensure.

An explainable machine learning-based phenomapping strategy for adaptive predictive enrichment in randomized clinical trials

Randomized clinical trials (RCT) represent the cornerstone of evidence-based medicine but are resource-intensive. We propose and evaluate a machine learning (ML) strategy of adaptive predictive enrichment through computational trial phenomaps to optimize RCT enrollment. In simulated group sequential analyses of two large cardiovascular outcomes RCTs of (1) a therapeutic drug (pioglitazone versus placebo; Insulin Resistance Intervention after Stroke (IRIS) trial), and (2) a disease management strategy (intensive versus standard systolic blood pressure reduction in the Systolic Blood Pressure Intervention Trial (SPRINT)), we constructed dynamic phenotypic representations to infer response profiles during interim analyses and examined their association with study outcomes. Across three interim timepoints, our strategy learned dynamic phenotypic signatures predictive of individualized cardiovascular benefit. By conditioning a prospective candidate's probability of enrollment on their predicted benefit, we estimate that our approach would have enabled a reduction in the final trial size across ten simulations (IRIS: -14.8% ± 3.1%, pone-sample t-test = 0.001; SPRINT: -17.6% ± 3.6%, pone-sample t-test < 0.001), while preserving the original average treatment effect (IRIS: hazard ratio of 0.73 ± 0.01 for pioglitazone vs placebo, vs 0.76 in the original trial; SPRINT: hazard ratio of 0.72 ± 0.01 for intensive vs standard systolic blood pressure, vs 0.75 in the original trial; all simulations with Cox regression-derived p value of  < 0.01 for the effect of the intervention on the respective primary outcome). This adaptive framework has the potential to maximize RCT enrollment efficiency.

An explainable machine learning-based phenomapping strategy for adaptive predictive enrichment in randomized controlled trials

Randomized controlled trials (RCT) represent the cornerstone of evidence-based medicine but are resource-intensive. We propose and evaluate a machine learning (ML) strategy of adaptive predictive enrichment through computational trial phenomaps to optimize RCT enrollment. In simulated group sequential analyses of two large cardiovascular outcomes RCTs of (1) a therapeutic drug (pioglitazone versus placebo; Insulin Resistance Intervention after Stroke (IRIS) trial), and (2) a disease management strategy (intensive versus standard systolic blood pressure reduction in the Systolic Blood Pressure Intervention Trial (SPRINT)), we constructed dynamic phenotypic representations to infer response profiles during interim analyses and examined their association with study outcomes. Across three interim timepoints, our strategy learned dynamic phenotypic signatures predictive of individualized cardiovascular benefit. By conditioning a prospective candidate's probability of enrollment on their predicted benefit, we estimate that our approach would have enabled a reduction in the final trial size across ten simulations (IRIS: -14.8% ± 3.1%, pone-sample t-test=0.001; SPRINT: -17.6% ± 3.6%, pone-sample t-test<0.001), while preserving the original average treatment effect (IRIS: hazard ratio of 0.73 ± 0.01 for pioglitazone vs placebo, vs 0.76 in the original trial; SPRINT: hazard ratio of 0.72 ± 0.01 for intensive vs standard systolic blood pressure, vs 0.75 in the original trial; all with pone-sample t-test<0.01). This adaptive framework has the potential to maximize RCT enrollment efficiency.

Effect of propranolol and clonidine after severe traumatic brain injury: a pilot randomized clinical trial

OBJECTIVE

To evaluate the safety, feasibility, and efficacy of combined adrenergic blockade with propranolol and clonidine in patients with severe traumatic brain injury (TBI).

BACKGROUND

Administration of adrenergic blockade after severe TBI is common. To date, no prospective trial has rigorously evaluated this common therapy for benefit.

METHODS

This phase II, single-center, double-blinded, pilot randomized placebo-controlled trial included patients aged 16-64 years with severe TBI (intracranial hemorrhage and Glasgow Coma Scale score ≤ 8) within 24 h of ICU admission. Patients received propranolol and clonidine or double placebo for 7 days. The primary outcome was ventilator-free days (VFDs) at 28 days. Secondary outcomes included catecholamine levels, hospital length of stay, mortality, and long-term functional status. A planned futility assessment was performed mid-study.

RESULTS

Dose compliance was 99%, blinding was intact, and no open-label agents were used. No treatment patient experienced dysrhythmia, myocardial infarction, or cardiac arrest. The study was stopped for futility after enrolling 47 patients (26 placebo, 21 treatment), per a priori stopping rules. There was no significant difference in VFDs between treatment and control groups [0.3 days, 95% CI (- 5.4, 5.8), p = 1.0]. Other than improvement of features related to sympathetic hyperactivity (mean difference in Clinical Features Scale (CFS) 1.7 points, CI (0.4, 2.9), p = 0.012), there were no between-group differences in the secondary outcomes.

CONCLUSION

Despite the safety and feasibility of adrenergic blockade with propranolol and clonidine after severe TBI, the intervention did not alter the VFD outcome. Given the widespread use of these agents in TBI care, a multi-center investigation is warranted to determine whether adrenergic blockade is of therapeutic benefit in patients with severe TBI. Trial Registration Number NCT01322048.

Guidance on interim analysis methods in clinical trials

Interim analyses in clinical trials can take on a multitude of forms. They are often used to guide Data and Safety Monitoring Board (DSMB) recommendations to study teams regarding recruitment targets for large, later-phase clinical trials. As collaborative biostatisticians working and teaching in multiple fields of research and across a broad array of trial phases, we note the large heterogeneity and confusion surrounding interim analyses in clinical trials. Thus, in this paper, we aim to provide a general overview and guidance on interim analyses for a nonstatistical audience. We explain each of the following types of interim analyses: efficacy, futility, safety, and sample size re-estimation, and we provide the reader with reasoning, examples, and implications for each. We emphasize that while the types of interim analyses employed may differ depending on the nature of the study, we would always recommend prespecification of the interim analytic plan to the extent possible with risk mitigation and trial integrity remaining a priority. Finally, we posit that interim analyses should be used as tools to help the DSMB make informed decisions in the context of the overarching study. They should generally not be deemed binding, and they should not be reviewed in isolation.

The West Africa Lassa fever Consortium pre-positioned protocol for a Phase II/III adaptive, randomised, controlled, platform trial to evaluate multiple Lassa fever therapeutics

Background: This is a standardized, pre-positioned protocol for the coordinated evaluation of Lassa fever therapeutics. The protocol is the product of discussions that took place in 2021 and 2022 among international investigators from a wide range of scientific and medical disciplines working together within the West Africa Lassa fever Consortium (WALC). Methods: This is a clinical Phase II/III multicentre randomised controlled platform trial using a superiority framework with an equal allocation ratio and a composite primary endpoint of all-cause mortality OR new onset of i) acute kidney failure (AKF), OR ii) acute respiratory failure (ARF), OR iii) shock assessed from enrolment (D0) to D28. Discussion: This pre-positioned protocol was developed by the WALC and made available for adaptation and implementation by the wider Lassa fever research community in order to generate efficient, reliable, and comparable evidence for Lassa fever therapeutics.