The Fragility of Statistically Significant Results in Randomized Clinical Trials for COVID-19

Nebulized Furosemide for Pulmonary Inflammation in Intubated Patients With COVID-19: A Phase 2 Randomized Controlled Double-Blind Study

OBJECTIVES

Respiratory failure secondary to COVID-19 is associated with morbidity and mortality. Current anti-inflammatory therapies are effective but are given systemically and have significant side effects. Furosemide has anti-inflammatory properties, can be administered by inhalation, and is inexpensive. We investigated the efficacy of nebulized furosemide as an adjunctive therapy for COVID-19 respiratory failure.

DESIGN

A double-blind, randomized, placebo-controlled trial.

SETTING

Multicenter ICU study.

PATIENTS

Adults requiring invasive mechanical ventilation secondary to COVID-19.

INTERVENTION

Patients were randomized within 48 hours of intubation to receive inhaled furosemide or placebo until day 28, death, or liberation from mechanical ventilation.

MEASUREMENTS AND MAIN RESULTS

The study was stopped early due to waning incidence of COVID-19; 39 patients were available for analysis with mean ± sd age of 70.5 (10.8) years, Acute Physiology and Chronic Health Evaluation II 26.1 (7.8) and Fio2 60.0% (21.9). Baseline characteristics were similar between the groups. For the primary outcome of change in Pao2/Fio2 ratio between day 1 and day 6, it was +31.4 (83.5) in the furosemide arm versus +20.1 (92.8) in the control (p = 0.58). For secondary outcomes, furosemide versus control: 60-day mortality was 48% versus 71% (p = 0.20), hospital stay was 25.6 (21.9) versus 27.4 (25.0) days, p = 0.94 and VFD was 6.0 (9.1) versus 3.1 (7.1), p value of equals to 0.28. A post hoc analysis of the hierarchical composite outcome, alive and ventilator-free favored furosemide. There were no adverse events.

CONCLUSIONS

In this trial of inhaled furosemide for COVID-19 respiratory failure, differences in Pao2/Fio2 ratio to day 6 and other clinical outcomes were not significantly different, although the trial was underpowered due to early termination. Given the favorable profile of inhaled furosemide, further study is warranted in disease states where acute pulmonary inflammation contributes to the underlying pathophysiology.

Region of Attainable Redaction, an extension of Ellipse of Insignificance analysis for gauging impacts of data redaction in dichotomous outcome trials

In biomedical science, it is a reality that many published results do not withstand deeper investigation, and there is growing concern over a replicability crisis in science. Recently, Ellipse of Insignificance (EOI) analysis was introduced as a tool to allow researchers to gauge the robustness of reported results in dichotomous outcome design trials, giving precise deterministic values for the degree of miscoding between events and non-events tolerable simultaneously in both control and experimental arms (Grimes, 2022). While this is useful for situations where potential miscoding might transpire, it does not account for situations where apparently significant findings might result from accidental or deliberate data redaction in either the control or experimental arms of an experiment, or from missing data or systematic redaction. To address these scenarios, we introduce Region of Attainable Redaction (ROAR), a tool that extends EOI analysis to account for situations of potential data redaction. This produces a bounded cubic curve rather than an ellipse, and we outline how this can be used to identify potential redaction through an approach analogous to EOI. Applications are illustrated, and source code, including a web-based implementation that performs EOI and ROAR analysis in tandem for dichotomous outcome trials is provided.

Anakinra authorized to treat severe coronavirus disease 2019; Sepsis breakthrough or time to reflect?

Introduction

The European Medicines Agency (EMA) and the United States Food and Drug Administration (FDA) announced conditions for using recombinant human interleukin-1 receptor antagonist (rhIL-1ra) to treat hospitalized patients with Coronavirus disease 2019 (COVID-19) and risk for progression. These decisions followed publication of the suPAR-guided Anakinra treatment for Validation of the risk and early Management OF seveRE respiratory failure by COVID-19 (SAVE- MORE) phase 3 clinical trial that yielded positive results.

Methods

We conducted a literature review and theoretical analysis of IL-1 blockade as a therapy to treat COVID-19. Using a stepwise analysis, we assessed clinical applicability of the SAVE-MORE results and evaluated conceptual support for interleukin-1 suppression as a suitable approach to COVID-19 treatment. This therapeutic approach was then examined as an example of inflammation-suppressing measures used to treat sepsis.

Results

Anakinra use as a COVID-19 therapy seems to rely on a view of pathogenesis that incorrectly reflects human disease. Since COVID-19 is an example of sepsis, COVID-19 benefit due to anti-inflammatory therapy contradicts an extensive history of unsuccessful clinical study. Repurposing rhIL-1ra to treat COVID-19 appears to exemplify a cycle followed by inflammation-suppressing sepsis treatments. A landscape of treatment failures is interrupted by a successful clinical trial. However, subsequent confirmatory study fails to replicate the positive data.

Discussion

We suggest further experimentation is not a promising pathway to discover game-changing sepsis therapies. A different kind of approach may be necessary.

Statistical Significance Versus Clinical Relevance: A Head-to-Head Comparison of the Fragility Index and Relative Risk Index

Background In biostatistics, assessing the fragility of research findings is crucial for understanding their clinical significance. This study focuses on the fragility index, unit fragility index, and relative risk index as measures to evaluate statistical fragility. The fragility indices assess the susceptibility of p-values to change significance with minor alterations in outcomes within a 2x2 contingency table. In contrast, the relative risk index quantifies the deviation of observed findings from therapeutic equivalence, the point at which the relative risk equals 1. While the fragility indices have intuitive appeal and have been widely applied, their behavior across a wide range of contingency tables has not been rigorously evaluated. Methods Using a Python software program, a simulation approach was employed to generate random 2x2 contingency tables. All tables under consideration exhibited p-values < 0.05 according to Fisher's exact test. Subsequently, the fragility indices and the relative risk index were calculated. To account for sample size variations, the indices were divided by the sample size to give fragility and risk quotients. A correlation matrix assessed the collinearity between each metric and the p-value. Results The analysis included 2,000 contingency tables with cell counts ranging from 20 to 480. Notably, the formulas for calculating the fragility indices encountered limitations when cell counts approached zero or duplicate cell counts hindered standardized application. The correlation coefficients with p-values were as follows: unit fragility index (-0.806), fragility index (-0.802), fragility quotient (-0.715), unit fragility quotient (-0.695), relative risk index (-0.403), and risk quotient (-0.261). Conclusion The fragility indices and fragility quotients demonstrated a strong correlation with p-values below 0.05, while the relative risk index and relative risk quotient exhibited a weak association with p-values below this threshold. This implies that the fragility indices offer limited additional information beyond the p-value alone. In contrast, the relative risk index and risk quotient exhibit independence from the p-value, indicating that they may provide important additional information about statistical fragility by evaluating the divergence of observed results from therapeutic equivalence, irrespective of the p-value-based statistical significance.

Assessing the robustness of COVID-19 vaccine efficacy trials: systematic review and meta-analysis, January 2023

BackgroundVaccines play a crucial role in the response to COVID-19 and their efficacy is thus of great importance.AimTo assess the robustness of COVID-19 vaccine efficacy (VE) trial results using the fragility index (FI) and fragility quotient (FQ) methodology.MethodsWe conducted a Cochrane and PRISMA-compliant systematic review and meta-analysis of COVID-19 VE trials published worldwide until 22 January 2023. We calculated the FI and FQ for all included studies and assessed their associations with selected trial characteristics using Wilcoxon rank sum tests and Kruskal-Wallis H tests. Spearman correlation coefficients and scatter plots were used to quantify the strength of correlation of FIs and FQs with trial characteristics.ResultsOf 6,032 screened records, we included 40 trials with 54 primary outcomes, comprising 909,404 participants with a median sample size per outcome of 13,993 (interquartile range (IQR): 8,534-25,519). The median FI and FQ was 62 (IQR: 22-123) and 0.50% (IQR: 0.24-0.92), respectively. FIs were positively associated with sample size (p < 0.001), and FQs were positively associated with type of blinding (p = 0.023). The Spearman correlation coefficient for FI with sample size was moderately strong (0.607), and weakly positive for FI and FQ with VE (0.138 and 0.161, respectively).ConclusionsThis was the largest study on trial robustness to date. Robustness of COVID-19 VE trials increased with sample size and varied considerably across several other important trial characteristics. The FI and FQ are valuable complementary parameters for the interpretation of trial results and should be reported alongside established trial outcome measures.

Treating COVID-19: Targeting the Host Response, Not the Virus

In low- and middle-income countries (LMICs), inexpensive generic drugs like statins, ACE inhibitors, and ARBs, especially if used in combination, might be the only practical way to save the lives of patients with severe COVID-19. These drugs will already be available in all countries on the first pandemic day. Because they target the host response to infection instead of the virus, they could be used to save lives during any pandemic. Observational studies show that inpatient statin treatment reduces 28–30-day mortality but randomized controlled trials have failed to show this benefit. Combination treatment has been tested for antivirals and dexamethasone but, with the exception of one observational study in Belgium, not for inexpensive generic drugs. Future pandemic research must include testing combination generic drug treatments that could be used in LMICs.

GM-CSF targeting in COVID-19: an approach based on fragile foundations

Coronavirus disease 2019 (COVID-19) arises as a result of a pathological inflammatory response following infection with the coronavirus SARS-CoV-2. Although the majority of people infected with this virus will experience minimal or mild symptoms, a proportion will go on to develop more severe disease requiring hospitalisation and oxygen therapy. The most severe forms produce acute respiratory failure, necessitating mechanical ventilation or extracorporeal membrane oxygenation (ECMO). The advent of SARS-CoV-2 vaccination has substantially altered the risk profile of COVID-19, with marked reductions in the severity of illness and hospitalisation. However, for unvaccinated patients and those who do not mount an effective immune response to vaccination, it remains a potentially lethal infection.

Trials of anti-GM-CSF therapies in COVID-19 show divergent results; this may be explained by underlying biology and the fragility of the study findings. Further investigation of the pathophysiology of COVID-19 is required to better target therapies.http://bit.ly/3O1AuIo

Computer clinical decision support that automates personalized clinical care: a challenging but needed healthcare delivery strategy

How to deliver best care in various clinical settings remains a vexing problem. All pertinent healthcare-related questions have not, cannot, and will not be addressable with costly time- and resource-consuming controlled clinical trials. At present, evidence-based guidelines can address only a small fraction of the types of care that clinicians deliver. Furthermore, underserved areas rarely can access state-of-the-art evidence-based guidelines in real-time, and often lack the wherewithal to implement advanced guidelines. Care providers in such settings frequently do not have sufficient training to undertake advanced guideline implementation. Nevertheless, in advanced modern healthcare delivery environments, use of eActions (validated clinical decision support systems) could help overcome the cognitive limitations of overburdened clinicians. Widespread use of eActions will require surmounting current healthcare technical and cultural barriers and installing clinical evidence/data curation systems. The authors expect that increased numbers of evidence-based guidelines will result from future comparative effectiveness clinical research carried out during routine healthcare delivery within learning healthcare systems.

Management of Severe and Critical COVID-19 Infection with Immunotherapies

Following the reduction in mortality demonstrated by dexamethasone treatment in severe COVID-19, many targeted immunotherapies have been investigated. Thus far, inhibition of IL-6 and JAK pathways have the most robust data and have been granted Emergency Use Authorization for treatment of severe disease. However, it must be noted that critically ill patients comprised a relatively small proportion of most of the trials of COVID-19 therapeutics, despite bearing a disproportionate burden of morbidity and mortality. Furthermore, the rapidity and fluidity with which clinical trials have been conducted in the pandemic setting have contributed to difficulty in extrapolating available trial data to critically ill patients. The exclusion of many patients requiring invasive mechanical ventilation, preponderance of ordinal scale based endpoints, and frequent lack of blinding are particular challenges. More data is needed to identify beneficial treatments in the complex milieu of critical illness from COVID-19 infection.

Efficacy and safety of baricitinib and tocilizumab in hospitalized patients with COVID-19: A comparison using systematic review and meta-analysis

Objective: This review was performed to compare the efficacy and safety among hospitalized patients with COVID-19 who received baricitinib and those who received tocilizumab independently with placebo or the standard of care (SOC). Methods: Relevant databases were searched for randomized controlled trials which evaluated the effect of baricitinib or tocilizumab as compared to placebo or the SOC in hospitalized patients with COVID-19. The primary endpoint was the comparison of the 28-day mortality. Risk ratios (RR) and mean differences were compared and pooled for dichotomous and continuous variables, respectively. A two-staged exploratory network meta-analysis using a multivariate meta-analysis was also performed. All analyses were performed in Stata version 16.0. The GRADE approach was used to assess the quality of the generated evidence (PROSPERO ID: CRD42022323363). Results: Treatment with baricitinib [RR, 0.69 (95% CI, 0.50-0.94), p = 0.02, i2 = 64.86%] but not with tocilizumab [RR, 0.87 (95% CI, 0.71-1.07), p = 0.19, i2 = 24.41%] led to a significant improvement in the 28-day mortality as compared to that with the SOC. Treatment with baricitinib or tocilizumab, both independently led to a significant reduction in the duration of hospitalization [baricitinib: mean difference, -1.13 days (95% CI, -1.51 to -0.76), p < 0.001, i2 = 0.00%; tocilizumab: mean difference, -2.80 days (95% CI, -4.17 to -1.43), p < 0.001, i2 = 55.47%] and a significant improvement in the proportion of patients recovering clinically by day 28 [baricitinib: RR, 1.24 (95% CI, 1.03-1.48), p = 0.02, i2 = 27.20%; tocilizumab: RR, 1.41 (95% CI, 1.12-1.78), p < 0.001, i2 = 34.59%] as compared to those with the SOC. From the safety point of view, both these drugs showed similar results. There were fewer patients who experienced any serious adverse event following treatment with barictinib and tocilizumab as compared to those following treatment with the SOC [baricitinib: RR, 0.76 (95% CI, 0.62-0.92), p = 0.01, i2 = 12.63%; tocilizumab: RR, 0.85 (95% CI, 0.72-1.01), p = 0.07, i2 = 0.00%]. Conclusion: As baricitinib and tocilizumab are recommended interchangeably by various guidelines for the management of COVID-19, considering the better 28-day mortality data and other comparable efficacy and safety outcomes, baricitinib may be favored over tocilizumab considering its ease of administration, shorter half-life, and lower cost of treatment.

Failed clinical trials on COVID-19 acute respiratory distress syndrome in hospitalized patients: common oversights and streamlining the development of clinically effective therapeutics

INTRODUCTION

The coronavirus disease 2019 (COVID-19) pandemic has put a strain on global healthcare systems. Despite admirable efforts to develop rapidly new pharmacotherapies, supportive treatments remain the standard of care. Multiple clinical trials have failed due to design issues, biased patient enrollment, small sample sizes, inadequate control groups, and lack of long-term outcomes monitoring.

AREAS COVERED

This narrative review depicts the current situation around failed and success COVID-19 clinical trials and recommendations in hospitalized patients with COVID-19, oversights and streamlining of clinically effective therapeutics. PubMed, EBSCO, Cochrane Library, and WHO and NIH guidelines were searched for relevant literature up to 5 August 2022.

EXPERT OPINION

The WHO, NIH, and IDSA have issued recommendations to better clarify which drugs should be used during the different phases of the disease. Given the biases and high heterogeneity of published studies, interpretation of the current literature is difficult. Future clinical trials should be designed to standardize clinical approaches, with appropriate organization, patient selection, addition of control groups, and careful identification of disease phase to reduce heterogeneity and bias and should rely on the integration of scientific societies to promote a consensus on interpretation of the data and recommendations for optimal COVID-19 therapies.