An overview of methodological considerations regarding adaptive stopping, arm dropping, and randomization in clinical trials

Empirical meropenem versus piperacillin/tazobactam for adult patients with sepsis (EMPRESS) trial: Protocol

BACKGROUND

Piperacillin/tazobactam may be associated with less favourable outcomes than carbapenems in patients with severe bacterial infections, but the certainty of evidence is low.

METHODS

The Empirical Meropenem versus Piperacillin/Tazobactam for Adult Patients with Sepsis (EMPRESS) trial is an investigator-initiated, international, parallel-group, randomised, open-label, adaptive clinical trial with an integrated feasibility phase. We will randomise adult, critically ill patients with sepsis to empirical treatment with meropenem or piperacillin/tazobactam for up to 30 days. The primary outcome is 30-day all-cause mortality. The secondary outcomes are serious adverse reactions within 30 days; isolation precautions due to resistant bacteria within 30 days; days alive without life support and days alive and out of hospital within 30 and 90 days; 90- and 180-day all-cause mortality and 180-day health-related quality of life. EMPRESS will use Bayesian statistical models with weak to somewhat sceptical neutral priors. Adaptive analyses will be conducted after follow-up of the primary outcome for the first 400 participants concludes and after every 300 subsequent participants, with adaptive stopping for superiority/inferiority and practical equivalence (absolute risk difference <2.5%-points) and response-adaptive randomisation. The expected sample sizes in scenarios with no, small or large differences are 5189, 5859 and 2570 participants, with maximum 14,000 participants and ≥99% probability of conclusiveness across all scenarios.

CONCLUSIONS

EMPRESS will compare the effects of empirical meropenem against piperacillin/tazobactam in adult, critically ill patients with sepsis. Due to the pragmatic, adaptive design with high probability of conclusiveness, the trial results are expected to directly inform clinical practice.

Effects of sceptical priors on the performance of adaptive clinical trials with binary outcomes

It is unclear how sceptical priors impact adaptive trials. We assessed the influence of priors expressing a spectrum of scepticism on the performance of several Bayesian, multi-stage, adaptive clinical trial designs using binary outcomes under different clinical scenarios. Simulations were conducted using fixed stopping rules and stopping rules calibrated to keep type 1 error rates at approximately 5%. We assessed total sample sizes, event rates, event counts, probabilities of conclusiveness and selecting the best arm, root mean squared errors (RMSEs) of the estimated treatment effect in the selected arms, and ideal design percentages (IDPs; which combines arm selection probabilities, power, and consequences of selecting inferior arms), with RMSEs and IDPs estimated in conclusive trials only and after selecting the control arm in inconclusive trials. Using fixed stopping rules, increasingly sceptical priors led to larger sample sizes, more events, higher IDPs in simulations ending in superiority, and lower RMSEs, lower probabilities of conclusiveness/selecting the best arm, and lower IDPs when selecting controls in inconclusive simulations. With calibrated stopping rules, the effects of increased scepticism on sample sizes and event counts were attenuated, and increased scepticism increased the probabilities of conclusiveness/selecting the best arm and IDPs when selecting controls in inconclusive simulations without substantially increasing sample sizes. Results from trial designs with gentle adaptation and non-informative priors resembled those from designs with more aggressive adaptation using weakly-to-moderately sceptical priors. In conclusion, the use of somewhat sceptical priors in adaptive trial designs with binary outcomes seems reasonable when considering multiple performance metrics simultaneously.

The Rise of Adaptive Platform Trials in Critical Care

As durable learning research systems, adaptive platform trials represent a transformative new approach to accelerating clinical evaluation and discovery in critical care. This Perspective provides a brief introduction to the concept of adaptive platform trials, describes several established and emerging platforms in critical care, and surveys some opportunities and challenges for their implementation and impact.

Effects of duration of follow-up and lag in data collection on the performance of adaptive clinical trials

Different combined outcome-data lags (follow-up durations plus data-collection lags) may affect the performance of adaptive clinical trial designs. We assessed the influence of different outcome-data lags (0-105 days) on the performance of various multi-stage, adaptive trial designs (2/4 arms, with/without a common control, fixed/response-adaptive randomisation) with undesirable binary outcomes according to different inclusion rates (3.33/6.67/10 patients/day) under scenarios with no, small, and large differences. Simulations were conducted under a Bayesian framework, with constant stopping thresholds for superiority/inferiority calibrated to keep type-1 error rates at approximately 5%. We assessed multiple performance metrics, including mean sample sizes, event counts/probabilities, probabilities of conclusiveness, root mean squared errors (RMSEs) of the estimated effect in the selected arms, and RMSEs between the analyses at the time of stopping and the final analyses including data from all randomised patients. Performance metrics generally deteriorated when the proportions of randomised patients with available data were smaller due to longer outcome-data lags or faster inclusion, that is, mean sample sizes, event counts/probabilities, and RMSEs were larger, while the probabilities of conclusiveness were lower. Performance metric impairments with outcome-data lags ≤45 days were relatively smaller compared to those occurring with ≥60 days of lag. For most metrics, the effects of different outcome-data lags and lower proportions of randomised patients with available data were larger than those of different design choices, for example, the use of fixed versus response-adaptive randomisation. Increased outcome-data lag substantially affected the performance of adaptive trial designs. Trialists should consider the effects of outcome-data lags when planning adaptive trials.

Pragmatic trial design to compare real-world effectiveness of different treatments for inflammatory bowel diseases: the PRACTICE-IBD European consensus

BACKGROUND AND AIMS

Pragmatic studies designed to test interventions in everyday clinical settings can successfully complement the evidence from registration and explanatory clinical trials. The European consensus project PRACTICE-IBD was developed to identify essential criteria and address key methodological issues needed to design valid comparative pragmatic studies in inflammatory bowel diseases (IBDs).

METHODS

Statements were issued by a panel of 11 European experts in IBD management and trial methodology on four main topics: (I) study design; (II) eligibility, recruitment and organization, flexibility; (III) outcomes; (IV) analysis. The consensus process followed a modified Delphi approach, involving two rounds of assessment and rating of the level of agreement (1 to 9; cut-off ≥7 for approval) with the statements by 18 additional European experts in IBD.

RESULTS

At the first voting round, 25 out of the 26 statements reached a mean score ≥7. Following the discussion that preceded the second round of voting, it was decided to eliminate two statements and to split one into two. At the second voting round, 25 final statements were approved: 7 for study design, 6 for eligibility, recruitment and organization, flexibility, 8 for outcomes, and 4 for analysis.

CONCLUSIONS

Pragmatic randomized clinical trials can address important questions in IBD clinical practice, and may provide complementary high-level evidence, as long as they follow a methodologically rigorous approach. These 25 statements intend to offer practical guidance in the design of high-quality pragmatic clinical trials that can aid decision making in choosing a management strategy for IBDs.

Real-world causal evidence for planned predictive enrichment in critical care trials: A scoping review

BACKGROUND

Randomised clinical trials in critical care are prone to inconclusiveness due, in part, to undue optimism about effect sizes and suboptimal accounting for heterogeneous treatment effects. Although causal evidence from rich real-world critical care can help overcome these challenges by informing predictive enrichment, no overview exists.

METHODS

We conducted a scoping review, systematically searching 10 general and speciality journals for reports published on or after 1 January 2018, of randomised clinical trials enrolling adult critically ill patients. We collected trial metadata on 22 variables including recruitment period, intervention type and early stopping (including reasons) as well as data on the use of causal evidence from secondary data for planned predictive enrichment.

RESULTS

We screened 9020 records and included 316 unique RCTs with a total of 268,563 randomised participants. One hundred seventy-three (55%) trials tested drug interventions, 101 (32%) management strategies and 42 (13%) devices. The median duration of enrolment was 2.2 (IQR: 1.3-3.4) years, and 83% of trials randomised less than 1000 participants. Thirty-six trials (11%) were restricted to COVID-19 patients. Of the 55 (17%) trials that stopped early, 23 (42%) used predefined rules; futility, slow enrolment and safety concerns were the commonest stopping reasons. None of the included RCTs had used causal evidence from secondary data for planned predictive enrichment.

CONCLUSION

Work is needed to harness the rich multiverse of critical care data and establish its utility in critical care RCTs. Such work will likely need to leverage methodology from interventional and analytical epidemiology as well as data science.

Intravenous Vitamin C for Patients Hospitalized With COVID-19: Two Harmonized Randomized Clinical Trials

Importance

The efficacy of vitamin C for hospitalized patients with COVID-19 is uncertain.

Objective

To determine whether vitamin C improves outcomes for patients with COVID-19.

Design, Setting, and Participants

Two prospectively harmonized randomized clinical trials enrolled critically ill patients receiving organ support in intensive care units (90 sites) and patients who were not critically ill (40 sites) between July 23, 2020, and July 15, 2022, on 4 continents.

Interventions

Patients were randomized to receive vitamin C administered intravenously or control (placebo or no vitamin C) every 6 hours for 96 hours (maximum of 16 doses).

Main Outcomes and Measures

The primary outcome was a composite of organ support-free days defined as days alive and free of respiratory and cardiovascular organ support in the intensive care unit up to day 21 and survival to hospital discharge. Values ranged from -1 organ support-free days for patients experiencing in-hospital death to 22 organ support-free days for those who survived without needing organ support. The primary analysis used a bayesian cumulative logistic model. An odds ratio (OR) greater than 1 represented efficacy (improved survival, more organ support-free days, or both), an OR less than 1 represented harm, and an OR less than 1.2 represented futility.

Results

Enrollment was terminated after statistical triggers for harm and futility were met. The trials had primary outcome data for 1568 critically ill patients (1037 in the vitamin C group and 531 in the control group; median age, 60 years [IQR, 50-70 years]; 35.9% were female) and 1022 patients who were not critically ill (456 in the vitamin C group and 566 in the control group; median age, 62 years [IQR, 51-72 years]; 39.6% were female). Among critically ill patients, the median number of organ support-free days was 7 (IQR, -1 to 17 days) for the vitamin C group vs 10 (IQR, -1 to 17 days) for the control group (adjusted proportional OR, 0.88 [95% credible interval {CrI}, 0.73 to 1.06]) and the posterior probabilities were 8.6% (efficacy), 91.4% (harm), and 99.9% (futility). Among patients who were not critically ill, the median number of organ support-free days was 22 (IQR, 18 to 22 days) for the vitamin C group vs 22 (IQR, 21 to 22 days) for the control group (adjusted proportional OR, 0.80 [95% CrI, 0.60 to 1.01]) and the posterior probabilities were 2.9% (efficacy), 97.1% (harm), and greater than 99.9% (futility). Among critically ill patients, survival to hospital discharge was 61.9% (642/1037) for the vitamin C group vs 64.6% (343/531) for the control group (adjusted OR, 0.92 [95% CrI, 0.73 to 1.17]) and the posterior probability was 24.0% for efficacy. Among patients who were not critically ill, survival to hospital discharge was 85.1% (388/456) for the vitamin C group vs 86.6% (490/566) for the control group (adjusted OR, 0.86 [95% CrI, 0.61 to 1.17]) and the posterior probability was 17.8% for efficacy.

Conclusions and Relevance

In hospitalized patients with COVID-19, vitamin C had low probability of improving the primary composite outcome of organ support-free days and hospital survival.

Trial Registration

ClinicalTrials.gov Identifiers: NCT04401150 (LOVIT-COVID) and NCT02735707 (REMAP-CAP).

A Bayesian-adaptive decision-theoretic approach can reduce the sample sizes for multiarm exercise oncology trials

OBJECTIVES

Adaptive designs may reduce trial sample sizes and costs. This study illustrates a Bayesian-adaptive decision-theoretic design applied to a multiarm exercise oncology trial.

STUDY DESIGN AND SETTING

In the Physical exercise during Adjuvant Chemotherapy Effectiveness Study (PACES) trial, 230 breast cancer patients receiving chemotherapy were randomized to supervised resistance and aerobic exercise (OnTrack), home-based physical activity (OncoMove) or usual care (UC). Data were reanalyzed as an adaptive trial using both Bayesian decision-theoretic and a frequentist group-sequential approach incorporating interim analyses after every 36 patients. Endpoint was chemotherapy treatment modifications (any vs. none). Bayesian analyses were performed for different continuation thresholds and settings with and without arm dropping and both in a 'pick-the-winner' and a 'pick-all-treatments-superior-to-control' setting.

RESULTS

Treatment modifications occurred in 34% of patients in UC and OncoMove vs. 12% in OnTrack (P = 0.002). Using a Bayesian-adaptive decision-theoretic design, OnTrack was identified as most effective after 72 patients in the 'pick-the-winner' setting and after 72-180 patients in the 'pick-all-treatments-superior-to-control' setting. In a frequentist setting, the trial would have been stopped after 180 patients, and the proportion of patients with treatment modifications was significantly lower for OnTrack than UC.

CONCLUSION

A Bayesian-adaptive decision-theoretic approach substantially reduced the sample size required for this three-arm exercise trial, especially in the 'pick-the-winner' setting.