A systematic survey of randomised trials that stopped early for reasons of futility

Epistemic uncertainty in Bayesian predictive probabilities

Bayesian predictive probabilities have become a ubiquitous tool for design and monitoring of clinical trials. The typical procedure is to average predictive probabilities over the prior or posterior distributions. In this paper, we highlight the limitations of relying solely on averaging, and propose the reporting of intervals or quantiles for the predictive probabilities. These intervals formalize the intuition that uncertainty decreases with more information. We present four different applications (Phase 1 dose escalation, early stopping for futility, sample size re-estimation, and assurance/probability of success) to demonstrate the practicality and generality of the proposed approach.

Beta spending function based on conditional power in group sequential design

Conditional power (CP) serves as a widely utilized approach for futility monitoring in group sequential designs. However, adopting the CP methods may lead to inadequate control of the type II error rate at the desired level. In this study, we introduce a flexible beta spending function tailored to regulate the type II error rate while employing CP based on a predetermined standardized effect size for futility monitoring (a so-called CP-beta spending function). This function delineates the expenditure of type II error rate across the entirety of the trial. Unlike other existing beta spending functions, the CP-beta spending function seamlessly incorporates beta spending concept into the CP framework, facilitating precise stagewise control of the type II error rate during futility monitoring. In addition, the stopping boundaries derived from the CP-beta spending function can be calculated via integration akin to other traditional beta spending function methods. Furthermore, the proposed CP-beta spending function accommodates various thresholds on the CP-scale at different stages of the trial, ensuring its adaptability across different information time scenarios. These attributes render the CP-beta spending function competitive among other forms of beta spending functions, making it applicable to any trials in group sequential designs with straightforward implementation. Both simulation study and example from an acute ischemic stroke trial demonstrate that the proposed method accurately captures expected power, even when the initially determined sample size does not consider futility stopping, and exhibits a good performance in maintaining overall type I error rates for evident futility.

Design and monitoring of clinical trials with an interim analysis and a negative binomial endpoint

There are very rich publications devoted to group sequential design, adaptive design and trial monitoring for continuous, binary and time to event endpoints. Many authors also discuss fixed design, blinded sample size re-estimation design and group sequential design for studies with a negative binomial outcome. Nonetheless, literature is sparse in adaptive design for a trial with a negative binomial endpoint. The features of such an endpoint in a flexible trial design setting remains inadequately understood. In this research, we seek to bridge this knowledge gap by offering a thorough examination of utilizing data components from a two-stage adaptive design for unblinded conditional power calculation and corresponding sample size re-estimation. We also provide expression for calculating the probability of meeting the futility criterion to determine the appropriate timing for the interim analysis. To evaluate the performance of the design, we conduct simulations to assess its operation characteristics. Finally, we provide a helpful and illustrative example to demonstrate the practical applications of the methods.

Estimation of conditional power in the presence of auxiliary data

Conditional power (CP) is a commonly used tool to inform interim decision-making in clinical trials, but the conventional approach using only primary endpoint data to calculate CP may not perform well when the primary endpoint requires a long follow-up period, or the treatment effect size changes during the trial. Several methods have been proposed to use additional short term auxiliary data observed at the interim analysis to improve the CP estimation in these situations, however, they may rely on strong assumptions, have limited applications, or use ad hoc choices of information fraction. In this paper we propose a general framework where the true CP formula is first derived in the presence of auxiliary data, and CP estimation is obtained by substituting the unknown parameters with consistent estimators. We conducted extensive simulations to examine the performance of both proposed and conventional approaches using the true CP as the benchmark. As the proposed approach is based on the true underlying CP, the simulations confirmed its superiority over the conventional approach in terms of efficiency and accuracy, especially if observed auxiliary data reflect the change of treatment effect size. The simulations also indicate that the magnitude of improvement in CP estimation is associated with the correlation between auxiliary and primary endpoints and/or the magnitude of the effect size change during the trial.

Outcomes of IncobotulinumtoxinA Injection on Myalgia and Arthralgia in Patients Undergoing Temporomandibular Joint Arthroscopy: A Randomized Controlled Trial

BACKGROUND

Several studies have considered Botulinum Neurotoxin Type A injections effective in treating temporomandibular joint disorder (TMD) symptoms. A double-blind, randomized, controlled clinical trial investigated the benefit of complementary incobotulinumtoxinA (inco-BoNT/A) injections in the masticatory muscles of patients submitted to bilateral temporomandibular joint (TMJ) arthroscopy.

METHODS

Fifteen patients with TMD and an indication for bilateral TMJ arthroscopy were randomized into inco-BoNT/A (Xeomin, 100 U) or placebo groups (saline solution). Injections were carried out five days before TMJ arthroscopy. The primary outcome variable was a Visual Analogue Scale for TMJ arthralgia, and secondary outcomes were the myalgia degree, maximum mouth opening, and joint clicks. All outcome variables were assessed preoperatively (T0) and postoperatively (T1-week 5; T2-6-month follow-up).

RESULTS

At T1, the outcomes in the inco-BoNT/A group were improved, but not significantly more than in the placebo group. At T2, significant improvements in the TMJ arthralgia and myalgia scores were observed in the inco-BoNT/A group compared to the placebo. A higher number of postoperative reinterventions with further TMJ treatments were observed in the placebo group compared to inco-BoNT/A (63% vs. 14%).

CONCLUSIONS

In patients submitted to TMJ arthroscopy, statistically significant long-term differences were observed between the placebo and inco-BoNT/A groups.

Avoiding future controversies in the Alzheimer's disease space through understanding the aducanumab data and FDA review

Key points of disagreement between the aducanumab FDA statistical review, which had primarily negative conclusions, and the clinical review, which had primarily positive conclusions, were investigated. Results from secondary endpoints in positive Study 302 were significant and these endpoints provided meaningful additional information. Findings indicate the statistical review of the aducanumab data was incorrect in a number of key areas. Greater placebo decline was not responsible for the significant results in Study 302. Correlations did exist between reduction in β-amyloid and clinical outcomes. Missing data and functional unblinding did not likely bias results. In contrast, the clinical review went too far in saying the negative results in Study 301 did not detract from the positive results in Study 302, as all clinical data should be considered in the evaluation, and the clinical review accepted the company's explanation for divergence of the results between the studies although much of the divergence remained unexplained. Interestingly, both the statistical review and the clinical review considered the available efficacy evidence despite both studies being terminated early. Implications of these findings include that the divergence in results seen in the two phase 3 aducanumab studies can be expected in other studies with similar design and analysis. Therefore, further research is needed to determine if analysis methods other than MMRM and/or optimized outcomes will provide more consistent results across studies.

Taking a chance: How likely am I to receive my preferred treatment in a clinical trial?

Researchers should ideally conduct clinical trials under a presumption of clinical equipoise, but in fact trial patients will often prefer one or other of the treatments being compared. Receiving an unblinded preferred treatment may affect the study outcome, possibly beneficially, but receiving a non-preferred treatment may induce 'reluctant acquiescence', and poorer outcomes. Even in blinded trials, patients' primary motivation to enrol may be the chance of potentially receiving a desirable experimental treatment, which is otherwise unavailable. Study designs with a higher probability of receiving a preferred treatment (denoted as 'concordance') will be attractive to potential participants, and investigators, because they may improve recruitment and hence enhance study efficiency. Therefore, it is useful to consider the concordance rates associated with various study designs. We consider this question with a focus on comparing the standard, randomised, two-arm, parallel group design with the two-stage randomised patient preference design and Zelen designs; we also mention the fully randomised and partially randomised patient preference designs. For each of these designs, we evaluate the concordance rate as a function of the proportions randomised to the alternative treatments, the distribution of preferences over treatments, and (for the Zelen designs) the proportion of patients who consent to receive their assigned treatment. We also examine the equity of each design, which we define as the similarity between the concordance rates for participants with different treatment preferences. Finally, we contrast each of the alternative designs with the standard design in terms of gain in concordance and change in equity.

Premature termination of clinical trials in Spain: reasons, characteristics, and opportunities to improve

PURPOSE

The aim of this study is to determine the rate of prematurely terminated clinical trials (CTs) and describe primary reasons and characteristics, and suggest strategies to improve.

METHODS

We performed a retrospective, observational, cross-sectional study including all CTs registered in the Spanish Registry of Clinical Studies (REec) from January 1, 2013 to November 31, 2021. A descriptive analysis of reasons for premature termination was made. To assess characteristics associated with a premature termination, the relative risks (RR) with a 95% confidence interval were calculated.

RESULTS

In total, 21% (718) of CT were prematurely terminated. Reasons for premature termination included patient recruitment issues in 25% (179) of cases, efficacy or futility problems in 18% (132), and commercial or strategic decisions from the sponsor in 12% (87). Characteristics significantly associated with an increased risk of premature termination included the following: presence of placebo (RR 2.08); multiple study sites (RR 1.32); pediatric and geriatric populations (RR 1.29 children; RR 1.47 preschoolers; RR 1.92 newborns; RR 1.27 > 64 years of age). In addition, circumstances such as investigations in phase II (RR 1.21), of cancer (RR 1.37), and of digestive pathology (RR 1.65) were also associated with increased risk of premature termination.

CONCLUSION

Recruitment of the study subjects in a CT must be meticulous and account for age of participants. In addition, CT study sites should be evaluated to ensure they have appropriate resources and the desired patient population. Based on intermediate analyses, CT protocols should describe the criteria to terminate a study due to futility. These approaches are essential to avoid harm to participants, ensure internal validity of studies, and improve the use of resources in CT development.

Ensuring the Scientific Value and Feasibility of Clinical Trials: A Qualitative Interview Study

BACKGROUND

Ethical and scientific principles require that clinical trials address an important question and have the resources needed to complete the study. However, there are no clear standards for review that would ensure that these principles are upheld.

METHODS

We conducted semi-structured interviews with a convenience sample of nineteen experts in clinical trial design, conduct, and/or oversight to elucidate current practice and identify areas of need with respect to ensuring the scientific value and feasibility of clinical trials prior to initiation and while ongoing. We used a priori and grounded theory to analyze the data and constant comparative method to induce higher order themes.

RESULTS

Interviewees perceived determination of scientific value as the responsibility of the investigator and, secondarily, other parties who review or oversee research. Interviewees reported that ongoing trials are rarely reevaluated due to emerging evidence from external sources, evaluation is complex, and there would be value in the development of standards for monitoring and evaluating evidence systematically. Investigators, IRBs, and/or data monitoring committees (DMCs) could undertake these responsibilities. Feasibility assessments are performed but are typically inadequate; potential solutions are unclear.

CONCLUSIONS

There are three domains where current approaches are suboptimal and in which further guidance is needed. First, who has the responsibility for conducting scientific review, whether it be the investigator, IRB, and/or DMC is often unclear. Second, the standards for scientific review (e.g., appropriate search terms, data sources, and analytic plan) should be defined. Third, guidance is needed on the evaluation of ongoing studies in light of potentially new and evolving evidence, with particular reference to evidence from outside the trial itself.

Impact of immediate and continuous heart rate feedback by dry electrode ECG on time to initiation of ventilation after birth: protocol for a randomised controlled trial

INTRODUCTION

3%-8% of newborns need positive pressure ventilation (PPV) after birth. Heart rate (HR) is considered the most sensitive indicator of the newborns' condition and response to resuscitative interventions. According to guidelines, HR should be assessed and PPV initiated within 60 s after birth in non-breathing newborns. Dry electrode ECG can provide accurate feedback on HR immediately after birth and continuously during resuscitation. The impact of early and continuous HR feedback is unknown.

METHOD AND ANALYSIS

This single-centre randomised controlled trial seeks to determine if HR feedback by dry electrode ECG immediately after birth and continuously during newborn resuscitation results in more timely initiation of PPV, improved ventilation and short-term outcomes compared with standard HR assessment.In all newborns≥34 gestational weeks, the dry electrode ECG sensor is placed on the upper abdomen immediately after birth as an additional modality of HR assessment. The device records and stores HR signals. In intervention subjects, the HR display is visible to guide decision-making and further management, in control subjects the display is masked. Standard HR assessment is by stethoscope, gel-electrode ECG and/or pulse oximetry (PO).Time of birth is registered in the Liveborn app. Time of initiation and duration of PPV is calculated from video recordings. Ventilation parameters are retrieved from the ventilation monitor, oxygen saturation and HR from the PO and gel-electrode ECG monitors.The primary endpoint is proportion of resuscitated newborns who receive PPV within 60 s after birth. To detect a 50% increase with power of 90% using an overall significance level of 0.05 and 1 interim analysis, 169 newborns are needed in each group.

ETHICS AND DISSEMINATION

Approval by the Norwegian National Research Ethics Committee West (2018/338). Parental consent is sought at routine screening early in pregnancy. The results will be published in peer-reviewed journal and presented at conferences.

TRIAL REGISTRATION NUMBER

NCT03849781.

Early Termination of Acute Stroke Randomized Controlled Trials Published Between 2013 and 2020: A Systematic Review

BACKGROUND

Termination of a clinical trial before the maximum planned sample size is accrued can occur for multiple valid reasons but has implications for the interpretation of results. We undertook a systematic review of contemporary acute stroke trials to document the prevalence of and reasons for early termination.

METHODS

We searched MEDLINE for randomized controlled trials of acute stroke therapies published between 2013 and 2020 in 9 major clinical journals. Manuscripts describing the primary results of phase 2 and phase 3 trials of acute stroke care were included. Data on study characteristics and adherence to CONSORT reporting guidelines were abstracted and summarized using descriptive statistics. Where feasible, we compared treatment effect sizes between trials terminated early and those not terminated early.

RESULTS

Of 96 randomized controlled trials, 39 (41%) were terminated early, 84 (88%) had a data and safety monitoring board, and 57 (59%) reported a prespecified statistical stopping rule. Among the 39 trials terminated early, 10 were discontinued for benefit, 10 due to logistical issues, 8 for futility, 6 because of newly available evidence, 1 for harm, and 4 for other or a combination of reasons. The median percentage of the maximum planned sample size accrued among trials terminated early was 63% (range, 8%-89%). Only 55% of trials (53 of 96) reported whether interim efficacy analyses were conducted, as recommended by the CONSORT guidelines. When 10 endovascular therapy trials were compared according to early termination status, the effect sizes of trials terminated early for benefit were only modestly larger than those not terminated early.

CONCLUSIONS

The high prevalence of early termination in combination with the wide variety of reasons underscores the necessity of meticulous trial planning and adherence to methodological and reporting guidelines for early termination. Registration: URL: https://www.crd.york.ac.uk/prospero/; Unique identifier: CRD42019128727.

Prospective phase 2 trial of PSMA-targeted molecular RadiothErapy with 177Lu-PSMA-617 for metastatic castration-reSISTant Prostate Cancer (RESIST-PC): efficacy results of the UCLA cohort

The objective of this study was to determine prospectively the efficacy profile of 2 activity regimens of ¹⁷⁷Lu-PSMA therapy in patients with progressive metastatic castrate-resistant prostate cancer (mCRPC): 6.0 vs. 7.4 GBq. Methods: RESIST-PC (NCT03042312) was a prospective multicenter phase 2 trial. Patients with progressive mCRPC after ≥ 1 novel androgen-axis drug, either chemotherapy naïve or postchemotherapy, with sufficient bone marrow reserve, normal kidney function, and sufficient PSMA expression by PSMA PET were eligible. Patients were randomized (1:1) into 2 activity groups (6.0 or 7.4 GBq) and received up to 4 cycles every 8 wk. The primary endpoint was the efficacy of ¹⁷⁷Lu-PSMA measured by the prostate-specific antigen (PSA) response rate (RR) after 2 cycles (≥50% decline from baseline). Secondary endpoints included the PSA RR (≥50% decline) at any time (best response), and overall survival (OS). Results: The study was closed at enrollment of 71/200 planned patients because of sponsorship transfer. We report here the efficacy of the University of California Los Angeles cohort results only (n = 43). The PSA RRs after 2 cycles and at any time were 11/40 (28%, 95% CI 15-44), 6/13 (46%, 95% CI 19-75), and 5/27 (19%, 95% CI 6-38), and 16/43 (37%, 95% CI 23-53), 7/14 (50%, 95% CI 23-77), and 9/29 (31%, 95% CI 15-51) in the whole cohort, the 6.0-GBq group, and the 7.4-GBq group, respectively (P = 0.12 and P = 0.31). The median OS was 14.0 mo (95% CI 10.1-17.9), 15.8 (95% CI 11.8-19.4), and 13.5 (95% CI 10.0-17.0) in the whole cohort, the 6.0-GBq group, and the 7.4 GBq group, respectively (P = 0.87). OS was longer in patients who experienced a PSA decline ≥ 50% at any time than in those who did not: median, 20.8 versus 10.8 mo (P = 0.005). Conclusion: In this prospective phase 2 trial of ¹⁷⁷Lu-PSMA for mCRPC, the median OS was 14 mo. Despite the heterogeneous study population and the premature study termination, the efficacy profile of ¹⁷⁷Lu-PSMA appeared to be favorable and comparable with both activity regimens (6.0 vs. 7.4 GBq). Results justify confirmation with real-world data matched-pair analysis and further clinical trials to refine and optimize the ¹⁷⁷Lu-PSMA therapy administration scheme to improve tumor radiation dose delivery and efficacy.

A multi-center phase II randomized clinical trial of losartan on symptomatic outpatients with COVID-19

BACKGROUND

The SARS-CoV-2 virus enters cells via Angiotensin-converting enzyme 2 (ACE2), disrupting the renin-angiotensin-aldosterone axis, potentially contributing to lung injury. Treatment with angiotensin receptor blockers (ARBs), such as losartan, may mitigate these effects, though induction of ACE2 could increase viral entry, replication, and worsen disease.

METHODS

This study represents a placebo-controlled blinded randomized clinical trial (RCT) to test the efficacy of losartan on outpatients with COVID-19 across three hospital systems with numerous community sites in Minnesota, U.S. Participants included symptomatic outpatients with COVID-19 not already taking ACE-inhibitors or ARBs, enrolled within 7 days of symptom onset. Patients were randomized to 1:1 losartan (25 mg orally twice daily unless estimated glomerular filtration rate, eGFR, was reduced, when dosing was reduced to once daily) versus placebo for 10 days, and all patients and outcome assesors were blinded. The primary outcome was all-cause hospitalization within 15 days. Secondary outcomes included functional status, dyspnea, temperature, and viral load. (clinicatrials.gov, NCT04311177, closed to new participants).

FINDINGS

From April to November 2020, 117 participants were randomized 58 to losartan and 59 to placebo, and all were analyzed under intent to treat principles. The primary outcome did not differ significantly between the two arms based on Barnard's test [losartan arm: 3 events (5.2% 95% CI 1.1, 14.4%) versus placebo arm: 1 event (1.7%; 95% CI 0.0, 9.1%)]; proportion difference -3.5% (95% CI -13.2, 4.8%); p = 0.32]. Viral loads were not statistically different between treatment groups at any time point. Adverse events per 10 patient days did not differ signifcantly [0.33 (95% CI 0.22-0.49) for losartan vs. 0.37 (95% CI 0.25-0.55) for placebo]. Due to a lower than expected hospitalization rate and low likelihood of a clinically important treatment effect, the trial was terminated early.

INTERPRETATION

In this multicenter blinded RCT for outpatients with mild symptomatic COVID-19 disease, losartan did not reduce hospitalizations, though assessment was limited by low event rate. Importantly, viral load was not statistically affected by treatment. This study does not support initiation of losartan for low-risk outpatients.

Conditional power of antidepressant network meta-analysis

BACKGROUND

Conditional power of network meta-analysis (NMA) can support the planning of randomized controlled trials (RCTs) assessing medical interventions. Conditional power is the probability that updating existing inconclusive evidence in NMA with additional trial(s) will result in conclusive evidence, given assumptions regarding trial design, anticipated effect sizes, or event probabilities.

METHODS

The present work aimed to estimate conditional power for potential future trials on antidepressant treatments. Existing evidence was based on a published network of 502 RCTs conducted between 1979-2018 assessing acute antidepressant treatment in major depressive disorder (MDD). Primary outcomes were efficacy in terms of the symptom change on the Hamilton Depression Scale (HAMD) and tolerability in terms of the dropout rate due to adverse events. The network compares 21 antidepressants consisting of 231 relative treatment comparisons, 164 (efficacy) and 127 (tolerability) of which are currently assumed to have inconclusive evidence.

RESULTS

Required sample sizes to achieve new conclusive evidence with at least 80% conditional power were estimated to range between N = 894 - 4190 (efficacy) and N = 521 - 1246 (tolerability). Otherwise, sample sizes ranging between N = 49 - 485 (efficacy) and N = 40 - 320 (tolerability) may require stopping for futility based on a boundary at 20% conditional power. Optimizing trial designs by considering multiple trials that contribute both direct and indirect evidence, anticipating alternative effect sizes or alternative event probabilities, may increase conditional power but required sample sizes remain high. Antidepressants having the greatest conditional power associated with smallest required sample sizes were identified as those on which current evidence is low, i.e., clomipramine, levomilnacipran, milnacipran, nefazodone, and vilazodone, with respect to both outcomes.

CONCLUSIONS

The present results suggest that conditional power to achieve new conclusive evidence in ongoing or future trials on antidepressant treatments is low. Limiting the use of the presented conditional power analysis are primarily due to the estimated large sample sizes which would be required in future trials as well as due to the well-known small effect sizes in antidepressant treatments. These findings may inform researchers and decision-makers regarding the clinical relevance and justification of research in ongoing or future antidepressant RCTs in MDD.

Trial monitoring via a futility criterion for interim results on a count data endpoint and a continuous endpoint

Assumptions made at design stage regarding the true treatment effect, background event rate and other factors may not always hold. Thus, long-term and large-scale studies may be designed with an interim analysis in order that the trials may be stopped early due to futility to save resource. There are many considerations of trial conducts for this type of trials. In this paper, we use a mock study to illustrate systematically the thinking and procedures for trial monitoring with a futility criterion for the interim results on a count data endpoint and a continuous endpoint. We focus on the discussions of blinded trial monitoring, the probability of meeting the futility criterion, conditional power/probability of success, Bayesian inference, potential delayed treatment effect and subgroup analysis. The experience should be applicable to future studies with similar features.

The efficacy and safety of hydroxychloroquine for COVID-19 prophylaxis: A systematic review and meta-analysis of randomized trials

Background

Populations such as healthcare workers (HCW) that are unable to practice physical distancing are at high risk of acquiring Coronavirus disease-2019 (COVID-19). In these cases pharmacological prophylaxis would be a solution to reduce severe acute respiratory syndrome coronavirus-2 (SARS CoV-2) transmission. Hydroxychloroquine has in vitro antiviral properties against SARS CoV-2. We therefore sought to determine the efficacy and safety of hydroxychloroquine as prophylaxis for COVID-19.

Methods and findings

We electronically searched EMBASE, MEDLINE, the Cochrane COVID-19 Register of Controlled Trials, Epistemonikos COVID-19, clinicaltrials.gov, and the World Health Organization International Clinical Trials Registry Platform up to September 28^th, 2020 for randomized controlled trials (RCTs). We calculated pooled relative risks (RRs) for dichotomous outcomes with the corresponding 95% confidence intervals (CIs) using a random-effect model. We identified four RCTs (n = 4921) that met our eligibility criteria. The use of hydroxychloroquine, compared to placebo, did not reduce the risks of developing COVID-19 (RR 0.82, 95% CI 0.65 to 1.04, moderate certainty), hospitalization (RR 0.72, 95% CI 0.34 to 1.50, moderate certainty), or mortality (RR 3.26, 95% CI 0.13 to 79.74, low certainty), however, hydroxychloroquine use increased the risk of adverse events (RR 2.76, 95% CI 1.38 to 5.55, moderate certainty).

Conclusion

Although pharmacologic prophylaxis is an attractive preventive strategy against COVID-19, the current body of evidence failed to show clinical benefit for prophylactic hydroxychloroquine and showed a higher risk of adverse events when compared to placebo or no prophylaxis.