1
|
Liu CC, Wu P, Yu RX. Delta Inflation, Optimism Bias, and Uncertainty in Clinical Trials. Ther Innov Regul Sci 2024:10.1007/s43441-024-00697-4. [PMID: 39242461 DOI: 10.1007/s43441-024-00697-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Accepted: 08/23/2024] [Indexed: 09/09/2024]
Abstract
The phenomenon of delta inflation, in which design treatment effects tend to exceed observed treatment effects, has been documented in several therapeutic areas. Delta inflation has often been attributed to investigators' optimism bias, or an unwarranted belief in the efficacy of new treatments. In contrast, we argue that delta inflation may be a natural consequence of clinical equipoise, that is, genuine uncertainty about the relative benefits of treatments before a trial is initiated. We review alternative methodologies that can offer more direct evidence about investigators' beliefs, including Bayesian priors and forecasting analysis. The available evidence for optimism bias appears to be mixed, and can be assessed only where uncertainty is expressed explicitly at the trial design stage.
Collapse
Affiliation(s)
- Charles C Liu
- Department of Biostatistics, Gilead Sciences, 333 Lakeside Drive, Foster City, CA, 94404, USA.
| | - Peiwen Wu
- Department of Biostatistics, Gilead Sciences, 333 Lakeside Drive, Foster City, CA, 94404, USA
| | - Ron Xiaolong Yu
- Department of Biostatistics, Gilead Sciences, 333 Lakeside Drive, Foster City, CA, 94404, USA
| |
Collapse
|
2
|
Olivier CB, Struß L, Sünnen N, Kaier K, Heger LA, Westermann D, Meerpohl JJ, Mahaffey KW. Accuracy of Event Rate and Effect Size Estimation in Major Cardiovascular Trials: A Systematic Review. JAMA Netw Open 2024; 7:e248818. [PMID: 38687478 PMCID: PMC11061773 DOI: 10.1001/jamanetworkopen.2024.8818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Accepted: 02/29/2024] [Indexed: 05/02/2024] Open
Abstract
Importance For the design of a randomized clinical trial (RCT), estimation of the expected event rate and effect size of an intervention is needed to calculate the sample size. Overestimation may lead to an underpowered trial. Objective To evaluate the accuracy of published estimates of event rate and effect size in contemporary cardiovascular RCTs. Evidence Review A systematic search was conducted in MEDLINE for multicenter cardiovascular RCTs associated with MeSH (Medical Subject Headings) terms for cardiovascular diseases published in the New England Journal of Medicine, JAMA, or the Lancet between January 1, 2010, and December 31, 2019. Identified trials underwent abstract review; eligible trials then underwent full review, and those with insufficiently reported data were excluded. Data were extracted from the original publication or the study protocol, and a random-effects model was used for data pooling. This review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline. The primary outcome was the accuracy of event rate and effect size estimation. Accuracy was determined by comparing the observed event rate in the control group and the effect size with their hypothesized values. Linear regression was used to determine the association between estimation accuracy and trial characteristics. Findings Of the 873 RCTs identified, 374 underwent full review and 30 were subsequently excluded, resulting in 344 trials for analysis. The median observed event rate was 9.0% (IQR, 4.3% to 21.4%), which was significantly lower than the estimated event rate of 11.0% (IQR, 6.0% to 25.0%) with a median deviation of -12.3% (95% CI, -16.4% to -5.6%; P < .001). More than half of the trials (196 [61.1%]) overestimated the expected event rate. Accuracy of event rate estimation was associated with a higher likelihood of refuting the null hypothesis (0.13 [95% CI, 0.01 to 0.25]; P = .03). The median observed effect size in superiority trials was 0.91 (IQR, 0.74 to 0.99), which was significantly lower than the estimated effect size of 0.72 (IQR, 0.60 to 0.80), indicating a median overestimation of 23.1% (95% CI, 17.9% to 28.3%). A total of 216 trials (82.1%) overestimated the effect size. Conclusions and Relevance In this systematic review of contemporary cardiovascular RCTs, event rates of the primary end point and effect sizes of an intervention were frequently overestimated. This overestimation may have contributed to the inability to adequately test the trial hypothesis.
Collapse
Affiliation(s)
- Christoph B. Olivier
- Department of Cardiology and Angiology, Cardiovascular Clinical Research Center, University Heart Center Freiburg–Bad Krozingen, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Lasse Struß
- Department of Cardiology and Angiology, Cardiovascular Clinical Research Center, University Heart Center Freiburg–Bad Krozingen, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nathalie Sünnen
- Department of Cardiology and Angiology, Cardiovascular Clinical Research Center, University Heart Center Freiburg–Bad Krozingen, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Klaus Kaier
- Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Lukas A. Heger
- Department of Cardiology and Angiology, Cardiovascular Clinical Research Center, University Heart Center Freiburg–Bad Krozingen, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Dirk Westermann
- Department of Cardiology and Angiology, Cardiovascular Clinical Research Center, University Heart Center Freiburg–Bad Krozingen, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Joerg J. Meerpohl
- Institute for Evidence in Medicine, University Medical Center Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Cochrane Germany, Cochrane Germany Foundation, University of Freiburg, Freiburg, Germany
| | - Kenneth W. Mahaffey
- Department of Medicine, Stanford Center for Clinical Research, Stanford University School of Medicine, Stanford, California
| |
Collapse
|
3
|
Kaas-Hansen BS, Granholm A, Sivapalan P, Anthon CT, Schjørring OL, Maagaard M, Kjaer MBN, Mølgaard J, Ellekjaer KL, Fagerberg SK, Lange T, Møller MH, Perner A. Real-world causal evidence for planned predictive enrichment in critical care trials: A scoping review. Acta Anaesthesiol Scand 2024; 68:16-25. [PMID: 37649412 DOI: 10.1111/aas.14321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Revised: 08/01/2023] [Accepted: 08/12/2023] [Indexed: 09/01/2023]
Abstract
BACKGROUND Randomised clinical trials in critical care are prone to inconclusiveness due, in part, to undue optimism about effect sizes and suboptimal accounting for heterogeneous treatment effects. Although causal evidence from rich real-world critical care can help overcome these challenges by informing predictive enrichment, no overview exists. METHODS We conducted a scoping review, systematically searching 10 general and speciality journals for reports published on or after 1 January 2018, of randomised clinical trials enrolling adult critically ill patients. We collected trial metadata on 22 variables including recruitment period, intervention type and early stopping (including reasons) as well as data on the use of causal evidence from secondary data for planned predictive enrichment. RESULTS We screened 9020 records and included 316 unique RCTs with a total of 268,563 randomised participants. One hundred seventy-three (55%) trials tested drug interventions, 101 (32%) management strategies and 42 (13%) devices. The median duration of enrolment was 2.2 (IQR: 1.3-3.4) years, and 83% of trials randomised less than 1000 participants. Thirty-six trials (11%) were restricted to COVID-19 patients. Of the 55 (17%) trials that stopped early, 23 (42%) used predefined rules; futility, slow enrolment and safety concerns were the commonest stopping reasons. None of the included RCTs had used causal evidence from secondary data for planned predictive enrichment. CONCLUSION Work is needed to harness the rich multiverse of critical care data and establish its utility in critical care RCTs. Such work will likely need to leverage methodology from interventional and analytical epidemiology as well as data science.
Collapse
Affiliation(s)
- Benjamin Skov Kaas-Hansen
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Anders Granholm
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Praleene Sivapalan
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Carl Thomas Anthon
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Olav Lilleholt Schjørring
- Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark
- Department of Clinical Medicine, Aalborg University, Aalborg, Denmark
| | - Mathias Maagaard
- Centre for Anaesthesiological Research, Department of Anaesthesiology, Zealand University Hospital, Køge, Denmark
| | | | - Jesper Mølgaard
- Department of Anesthesiology, Centre for Cancer and Organ Dysfunction, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Karen Louise Ellekjaer
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Steen Kåre Fagerberg
- Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark
| | - Theis Lange
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Morten Hylander Møller
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Anders Perner
- Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| |
Collapse
|
4
|
Deng C, Sidebotham D. Using the Delphi process to determine the minimum clinically important effect size for the Balanced-2 randomised controlled trial. Clin Trials 2023; 20:473-478. [PMID: 37144615 DOI: 10.1177/17407745231173058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
BACKGROUND The sample size calculation is an important step in designing randomised controlled trials. For a trial comparing a control and an intervention group, where the outcome is binary, the sample size calculation requires choosing values for the anticipated event rates in both the control and intervention groups (the effect size), and the error rates. The Difference ELicitation in TriAls guidance recommends that the effect size should be both realistic, and clinically important to stakeholder groups. Overestimating the effect size leads to sample sizes that are too small to reliably detect the true population effect size, which in turn results in low achieved power. In this study, we use the Delphi approach to gain consensus on what the minimum clinically important effect size is for Balanced-2, a randomised controlled trial comparing processed electroencephalogram-guided 'light' to 'deep' general anaesthesia on the incidence of postoperative delirium in older adults undergoing major surgery. METHODS Delphi rounds were conducted using electronic surveys. Surveys were administered to two stakeholder groups: specialist anaesthetists from a general adult department in Auckland City Hospital, New Zealand (Group 1), and specialist anaesthetists with expertise in clinical research, identified from the Australian and New Zealand College of Anaesthetist's Clinical Trials Network (Group 2). A total of 187 anaesthetists were invited to participate (81 from Group 1 and 106 from Group 2). Results from each Delphi round were summarised and presented in subsequent rounds until consensus was reached (>70% agreement). RESULTS The overall response rate for the first Delphi survey was 47% (88/187). The median minimum clinically important effect size was 5.0% (interquartile range: 5.0-10.0) for both stakeholder groups. The overall response rate for the second Delphi survey was 51% (95/187). Consensus was reached after the second round, as 74% of respondents in Group 1 and 82% of respondents in Group 2 agreed with the median effect size. The combined minimum clinically important effect size across both groups was 5.0% (interquartile range: 3.0-6.5). CONCLUSIONS This study demonstrates that surveying stakeholder groups using a Delphi process is a simple way of defining a minimum clinically important effect size, which aids the sample size calculation and determines whether a randomised study is feasible.
Collapse
Affiliation(s)
- Carolyn Deng
- Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand
- University of Auckland, Auckland, New Zealand
| | - David Sidebotham
- University of Auckland, Auckland, New Zealand
- Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand
| |
Collapse
|
5
|
Sidebotham D, Barlow CJ, Martin J, Jones PM. Interpreting frequentist hypothesis tests: insights from Bayesian inference. Can J Anaesth 2023; 70:1560-1575. [PMID: 37794259 PMCID: PMC10600289 DOI: 10.1007/s12630-023-02557-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Revised: 03/25/2023] [Accepted: 03/27/2023] [Indexed: 10/06/2023] Open
Abstract
Randomized controlled trials are one of the best ways of quantifying the effectiveness of medical interventions. Therefore, when the authors of a randomized superiority trial report that differences in the primary outcome between the intervention group and the control group are "significant" (i.e., P ≤ 0.05), we might assume that the intervention has an effect on the outcome. Similarly, when differences between the groups are "not significant," we might assume that the intervention does not have an effect on the outcome. Nevertheless, both assumptions are frequently incorrect.In this article, we explore the relationship that exists between real treatment effects and declarations of statistical significance based on P values and confidence intervals. We explain why, in some circumstances, the chance an intervention is ineffective when P ≤ 0.05 exceeds 25% and the chance an intervention is effective when P > 0.05 exceeds 50%.Over the last decade, there has been increasing interest in Bayesian methods as an alternative to frequentist hypothesis testing. We provide a robust but nontechnical introduction to Bayesian inference and explain why a Bayesian posterior distribution overcomes many of the problems associated with frequentist hypothesis testing.Notwithstanding the current interest in Bayesian methods, frequentist hypothesis testing remains the default method for statistical inference in medical research. Therefore, we propose an interim solution to the "significance problem" based on simplified Bayesian metrics (e.g., Bayes factor, false positive risk) that can be reported along with traditional P values and confidence intervals. We calculate these metrics for four well-known multicentre trials. We provide links to online calculators so readers can easily estimate these metrics for published trials. In this way, we hope decisions on incorporating the results of randomized trials into clinical practice can be enhanced, minimizing the chance that useful treatments are discarded or that ineffective treatments are adopted.
Collapse
Affiliation(s)
- David Sidebotham
- Department of Anaesthesia and the Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand.
- Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand.
- Cardiothoracic and Vascular Intensive Care Unit (Ward 48), Building 32, Auckland City Hospital, 2 Park Road, Grafton, Auckland, 1023, New Zealand.
| | - C Jake Barlow
- Department of Anaesthesia and the Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand
| | - Janet Martin
- Department of Anesthesia & Perioperative Medicine, University of Western Ontario, London, ON, Canada
- Department of Epidemiology & Biostatistics, University of Western Ontario, London, ON, Canada
| | - Philip M Jones
- Department of Anesthesia & Perioperative Medicine, University of Western Ontario, London, ON, Canada
- Department of Epidemiology & Biostatistics, University of Western Ontario, London, ON, Canada
| |
Collapse
|
6
|
Kampman JM, Sperna Weiland NH, Hermanides J, Hollmann MW, Repping S, Horn J. Randomized Controlled Trials in ICU in the Four Highest-Impact General Medicine Journals. Crit Care Med 2023; 51:e179-e183. [PMID: 37199541 PMCID: PMC10426774 DOI: 10.1097/ccm.0000000000005937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/19/2023]
Abstract
OBJECTIVE To study ICU trials published in the four highest-impact general medicine journals by comparing them with concurrently published non-ICU trials in the same journals. DATA SOURCES PubMed was searched for randomized controlled trials (RCTs) published between January 2014 and October 2021 in the New England Journal of Medicine , The Lancet , the Journal of the American Medical Association , and the British Medical Journal. STUDY SELECTION Original RCT publications investigating any type of intervention in any patient population. DATA EXTRACTION ICU RCTs were defined as RCTs exclusively including patients admitted to the ICU. Year and journal of publication, sample size, study design, funding source, study outcome, type of intervention, Fragility Index (FI), and Fragility Quotient were collected. DATA SYNTHESIS A total of 2,770 publications were screened. Of 2,431 original RCTs, 132 (5.4%) were ICU RCTs, gradually rising from 4% in 2014 to 7.5% in 2021. ICU RCTs and non-ICU RCTs included a comparable number of patients (634 vs 584, p = 0.528). Notable differences for ICU RCTs were the low occurrence of commercial funding (5% vs 36%, p < 0.001), the low number of RCTs that reached statistical significance (29% vs 65%, p < 0.001), and the low FI when they did reach significance (3 vs 12, p = 0.008). CONCLUSIONS In the last 8 years, RCTs in ICU medicine made up a meaningful, and growing, portion of RCTs published in high-impact general medicine journals. In comparison with concurrently published RCTs in non-ICU disciplines, statistical significance was rare and often hinged on the outcome events of just a few patients. Increased attention should be paid to realistic expectations of treatment effects when designing ICU RCTs to detect differences in treatment effects that are reliable and clinically relevant.
Collapse
Affiliation(s)
- Jasper M Kampman
- Department of Anesthesiology, Amsterdam UMC, Amsterdam, The Netherlands
- Department of Intensive Care Medicine, Amsterdam UMC, Amsterdam, The Netherlands
| | | | - Jeroen Hermanides
- Department of Anesthesiology, Amsterdam UMC, Amsterdam, The Netherlands
| | - Markus W Hollmann
- Department of Anesthesiology, Amsterdam UMC, Amsterdam, The Netherlands
| | - Sjoerd Repping
- Department of Health Evaluation and Appropriate Use, Amsterdam UMC, Amsterdam, The Netherlands
- National Healthcare Institute, Diemen, The Netherlands
| | - Janneke Horn
- Department of Intensive Care Medicine, Amsterdam UMC, Amsterdam, The Netherlands
| |
Collapse
|
7
|
Granholm A, Schjørring OL, Jensen AKG, Kaas-Hansen BS, Munch MW, Klitgaard TL, Crescioli E, Kjaer MBN, Strøm T, Lange T, Perner A, Rasmussen BS, Møller MH. Association between days alive without life support/out of hospital and health-related quality of life. Acta Anaesthesiol Scand 2023; 67:762-771. [PMID: 36915265 DOI: 10.1111/aas.14231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 02/28/2023] [Accepted: 03/07/2023] [Indexed: 03/16/2023]
Abstract
BACKGROUND Trials in critically ill patients increasingly focus on days alive without life support (DAWOLS) or days alive out of hospital (DAOOH) and health-related quality of life (HRQoL). DAWOLS and DAOOH convey more information than mortality and are simpler and faster to collect than HRQoL. However, whether these outcomes are associated with HRQoL is uncertain. We thus aimed to assess the associations between DAWOLS and DAOOH and long-term HRQoL. METHODS Secondary analysis of the COVID STEROID 2 trial including adults with COVID-19 and severe hypoxaemia and the Handling Oxygenation Targets in the Intensive Care Unit (HOT-ICU) trial including adult intensive care unit patients with acute hypoxaemic respiratory failure. Associations between DAWOLS and DAOOH at day 28 and 90 and long-term HRQoL (after 6 or 12 months) using the EuroQol 5-dimension 5-level survey (EQ VAS and EQ-5D-5L index values) were assessed using flexible models and evaluated using measures of fit and prediction adequacy in both datasets (comprising internal performance and external validation), non-parametric correlation coefficients and graphical presentations. RESULTS We found no strong associations between DAWOLS or DAOOH and HRQoL in survivors at HRQoL-follow-up (615 and 1476 patients, respectively). There was substantial variability in outcomes, and predictions from the best fitted models were poor both internally and externally in the other trial dataset, which also showed inadequate calibration. Moderate associations were found when including non-survivors, although predictions remained uncertain and calibration inadequate. CONCLUSION DAWOLS and DAOOH were poorly associated with HRQoL in adult survivors of severe or critical illness included in the COVID STEROID 2 and HOT-ICU trials.
Collapse
Affiliation(s)
- Anders Granholm
- Department of Intensive Care, Copenhagen University Hospital-Rigshospitalet, Copenhagen, Denmark
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
| | - Olav Lilleholt Schjørring
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
- Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark
- Department of Clinical Medicine, Aalborg University, Aalborg, Denmark
| | - Aksel Karl Georg Jensen
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Benjamin Skov Kaas-Hansen
- Department of Intensive Care, Copenhagen University Hospital-Rigshospitalet, Copenhagen, Denmark
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Marie Warrer Munch
- Department of Intensive Care, Copenhagen University Hospital-Rigshospitalet, Copenhagen, Denmark
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
| | - Thomas Lass Klitgaard
- Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark
| | - Elena Crescioli
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
- Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark
- Department of Clinical Medicine, Aalborg University, Aalborg, Denmark
| | - Maj-Brit Nørregaard Kjaer
- Department of Intensive Care, Copenhagen University Hospital-Rigshospitalet, Copenhagen, Denmark
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
| | - Thomas Strøm
- Department of Anaesthesia and Critical Care Medicine, Odense University Hospital, Odense, Denmark
- Department of Anaesthesia and Critical Care Medicine, Hospital Sønderjylland, University Hospital of Southern Denmark, Odense, Denmark
| | - Theis Lange
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Anders Perner
- Department of Intensive Care, Copenhagen University Hospital-Rigshospitalet, Copenhagen, Denmark
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
| | - Bodil Steen Rasmussen
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
- Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark
- Department of Clinical Medicine, Aalborg University, Aalborg, Denmark
| | - Morten Hylander Møller
- Department of Intensive Care, Copenhagen University Hospital-Rigshospitalet, Copenhagen, Denmark
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
| |
Collapse
|
8
|
Nielsen N, Friberg H. Changes in Practice of Controlled Hypothermia after Cardiac Arrest in the Past 20 Years: A Critical Care Perspective. Am J Respir Crit Care Med 2023; 207:1558-1564. [PMID: 37104654 DOI: 10.1164/rccm.202211-2142cp] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Accepted: 04/26/2023] [Indexed: 04/29/2023] Open
Abstract
For 20 years, induced hypothermia and targeted temperature management have been recommended to mitigate brain injury and increase survival after cardiac arrest. On the basis of animal research and small clinical trials, the International Liaison Committee on Resuscitation strongly advocated hypothermia at 32-34 °C for 12-24 hours for comatose patients with out-of-hospital cardiac arrest with initial rhythm of ventricular fibrillation or nonperfusing ventricular tachycardia. The intervention was implemented worldwide. In the past decade, hypothermia and targeted temperature management have been investigated in larger clinical randomized trials focusing on target temperature depth, target temperature duration, prehospital versus in-hospital initiation, nonshockable rhythms, and in-hospital cardiac arrest. Systematic reviews suggest little or no effect of delivering the intervention on the basis of the summary of evidence, and the International Liaison Committee on Resuscitation today recommends only to treat fever and keep body temperature below 37.5 °C (weak recommendation, low-certainty evidence). Here we describe the evolution of temperature management for patients with cardiac arrest during the past 20 years and how the accrued evidence has influenced not only the recommendations but also the guideline process. We also discuss possible paths forward in this field, bringing up both whether fever management is at all beneficial for patients with cardiac arrest and which knowledge gaps future clinical trials in temperature management should address.
Collapse
Affiliation(s)
- Niklas Nielsen
- Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Anesthesiology and Intensive Care, Helsingborg Hospital, Helsingborg, Sweden; and
| | - Hans Friberg
- Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Anesthesiology and Intensive Care, Skåne University Hospital, Malmö, Sweden
| |
Collapse
|
9
|
Hansen ML, Jørgensen CK, Thabane L, Rulli E, Biagioli E, Chiaruttini M, Mbuagbaw L, Mathiesen O, Gluud C, Jakobsen JC. Observed intervention effects for mortality in randomised clinical trials: a methodological study protocol. BMJ Open 2023; 13:e072550. [PMID: 37316319 DOI: 10.1136/bmjopen-2023-072550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/16/2023] Open
Abstract
INTRODUCTION It is essential to choose a realistic anticipated intervention effect when calculating a sample size for a randomised clinical trial. Unfortunately, anticipated intervention effects are often inflated, when compared with the 'true' intervention effects. This is documented for mortality in critical care trials. A similar pattern might exist across different medical specialties. This study aims to estimate the range of observed intervention effects for all-cause mortality in trials included in Cochrane Reviews, within each Cochrane Review Group. METHODS AND ANALYSIS We will include randomised clinical trials assessing all-cause mortality as an outcome. Trials will be identified from Cochrane Reviews published in the Cochrane Database of Systematic Reviews. Cochrane Reviews will be clustered according to the registered Cochrane Review Group (eg, Anaesthesia, Emergency and Critical Care) and the statistical analyses will be conducted for each Cochrane Review Group and overall. The median relative risk and IQR for all-cause mortality and the proportion of trials with a relative all-cause mortality risk within seven different ranges will be reported (relative risk below 0.70, 0.70-0.79, 0.80-0.89, 0.90-1.09, 1.10-1.19, 1.20-1.30 and above 1.30). Subgroup analyses will explore the effects of original design, sample size, risk of bias, disease, intervention type, follow-up length, participating centres, funding type, information size and outcome hierarchy. ETHICS AND DISSEMINATION Since we will use summary data from trials already approved by relevant ethical committees, this study does not require ethical approval. Regardless of our findings, the results will be published in an international peer-reviewed journal.
Collapse
Affiliation(s)
- Mathias Lühr Hansen
- Department of Neonatology, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Caroline Kamp Jørgensen
- Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Department of Regional Health Research, The Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and Impact, McMaster University, and St Joseph's Healthcare, Hamilton, Ontario, Canada
- Faculty of Health Sciences, University of Johannesburg, Johannesburg, South Africa
| | - Eliana Rulli
- Methodology for Clinical Research Laboratory, Oncology Department, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy
| | - Elena Biagioli
- Methodology for Clinical Research Laboratory, Oncology Department, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy
| | - Maria Chiaruttini
- Methodology for Clinical Research Laboratory, Oncology Department, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy
| | - Lawrence Mbuagbaw
- Department of Health Research Methods, Evidence, and Impact, McMaster University, and St Joseph's Healthcare, Hamilton, Ontario, Canada
- Centre for Development of Best Practices in Health (CDBPH), Yaoundé Central Hospital, Yaoundé, Cameroon
| | - Ole Mathiesen
- Department of Clinical Medicine, Copenhagen University, Copenhagen, Denmark
- Centre for Anaesthesiological Research, Department of Anaesthesiology, Zealand University Hospital, Køge, Denmark
| | - Christian Gluud
- Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Department of Regional Health Research, The Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark
| | - Janus Christian Jakobsen
- Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Department of Regional Health Research, The Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
10
|
Granholm A, Kaas-Hansen BS, Lange T, Munch MW, Harhay MO, Zampieri FG, Perner A, Møller MH, Jensen AKG. Use of days alive without life support and similar count outcomes in randomised clinical trials - an overview and comparison of methodological choices and analysis methods. BMC Med Res Methodol 2023; 23:139. [PMID: 37316785 DOI: 10.1186/s12874-023-01963-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 06/03/2023] [Indexed: 06/16/2023] Open
Abstract
BACKGROUND Days alive without life support (DAWOLS) and similar outcomes that seek to summarise mortality and non-mortality experiences are increasingly used in critical care research. The use of these outcomes is challenged by different definitions and non-normal outcome distributions that complicate statistical analysis decisions. METHODS We scrutinized the central methodological considerations when using DAWOLS and similar outcomes and provide a description and overview of the pros and cons of various statistical methods for analysis supplemented with a comparison of these methods using data from the COVID STEROID 2 randomised clinical trial. We focused on readily available regression models of increasing complexity (linear, hurdle-negative binomial, zero-one-inflated beta, and cumulative logistic regression models) that allow comparison of multiple treatment arms, adjustment for covariates and interaction terms to assess treatment effect heterogeneity. RESULTS In general, the simpler models adequately estimated group means despite not fitting the data well enough to mimic the input data. The more complex models better fitted and thus better replicated the input data, although this came with increased complexity and uncertainty of estimates. While the more complex models can model separate components of the outcome distributions (i.e., the probability of having zero DAWOLS), this complexity means that the specification of interpretable priors in a Bayesian setting is difficult. Finally, we present multiple examples of how these outcomes may be visualised to aid assessment and interpretation. CONCLUSIONS This summary of central methodological considerations when using, defining, and analysing DAWOLS and similar outcomes may help researchers choose the definition and analysis method that best fits their planned studies. TRIAL REGISTRATION COVID STEROID 2 trial, ClinicalTrials.gov: NCT04509973, ctri.nic.in: CTRI/2020/10/028731.
Collapse
Affiliation(s)
- Anders Granholm
- Department of Intensive Care 4131, Copenhagen University Hospital - Rigshospitalet, DK-2100, Copenhagen, Denmark.
| | - Benjamin Skov Kaas-Hansen
- Department of Intensive Care 4131, Copenhagen University Hospital - Rigshospitalet, DK-2100, Copenhagen, Denmark
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Theis Lange
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - Marie Warrer Munch
- Department of Intensive Care 4131, Copenhagen University Hospital - Rigshospitalet, DK-2100, Copenhagen, Denmark
| | - Michael O Harhay
- Clinical Trials Methods and Outcomes Lab, Palliative and Advanced Illness Research Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, USA
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, USA
| | - Fernando G Zampieri
- HCor Research Institute, São Paulo, Brazil
- Department of Critical Care Medicine, Faculty of Medicine and Dentistry, University of Alberta, Alberta, Canada
| | - Anders Perner
- Department of Intensive Care 4131, Copenhagen University Hospital - Rigshospitalet, DK-2100, Copenhagen, Denmark
| | - Morten Hylander Møller
- Department of Intensive Care 4131, Copenhagen University Hospital - Rigshospitalet, DK-2100, Copenhagen, Denmark
| | - Aksel Karl Georg Jensen
- Section of Biostatistics, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
11
|
Ghossein J, Fernando SM, Rochwerg B, Inaba K, Lampron J, Tran A. A systematic review and meta-analysis of sample size methodology for traumatic hemorrhage trials. J Trauma Acute Care Surg 2023; 94:870-876. [PMID: 36879398 DOI: 10.1097/ta.0000000000003944] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/08/2023]
Abstract
BACKGROUND Trauma hemorrhage remains the most common cause of preventable mortality in trauma. To guide clinical practice, RCTs provide high-quality evidence to inform clinical decision making. The clinical relevance and inferences made by RCTs are dependent on assumptions made during sample size calculation. METHODS To describe the quality of methodology for sample size determination, we conducted a systemic review RCTs evaluating interventions that aim to improve survival in adults with trauma-related hemorrhage. Estimated and actual outcome data are compared, including components of sample size determination. RESULTS A total of 13 RCTs were included. We noted a high rate of negative trial results (11 of 13 studies). Most studies were multi-center and conducted in North America, evaluating patients with blunt and penetrating injuries. The criteria for hemorrhagic shock varied across studies. All studies did not accurately estimate the mortality rate during sample size calculation. All but one study overestimated the mortality reduction during sample size calculation; the median absolute mortality reduction was 3%, compared with a target of 10%. Only the CRASH-2 study used a minimal clinically important different for treatment effect target. No RCTs employed prognostic enrichment. Most studies were terminated (8 of 13), mainly for futility. CONCLUSION Taken together, this review highlights that current clinical trial methodology is limited by imprecise control group risk estimates, overly optimistic treatment effect estimates, and lack of transparent justification for such targets. These limitations result in studies at high risk for futility and potentially premature abandonment of promising therapies. Given the high morbidity and mortality of trauma-related hemorrhage, we recommend that future conduct of trauma RCTs incorporate (1) prognostic enrichment to inform baseline risk, (2) justify target treatment differences based on clinical importance and realistic estimates of feasibility, and (3) be transparent and provide justification for the assumptions made. LEVEL OF EVIDENCE Systematic Review/Meta-Analysis; Level III.
Collapse
Affiliation(s)
- Jamie Ghossein
- From the Faculty of Medicine (J.G.), University of Ottawa, Ottawa, Ontario, Canada; Department of Medicine (J.G.), The Ottawa Hospital, Ottawa, Ontario, Canada; Department of Critical Care (S.M.F.), Lakeridge Health Corporation, Oshawa, Canada; Division of Critical Care, Department of Medicine (B.R.), McMaster University, Hamilton, Canada; Department of Health Research Methods (B.R.), Evidence, and Impact, McMaster University, Hamilton, Canada; Division of Acute Care Surgery, Department of Surgery (K.I.), University of Southern California, Los Angeles, California; Division of General Surgery, Department of Surgery (J.L., A.T.), University of Ottawa, Ontario, Canada; and Division of Critical Care, Department of Medicine (A.T.), University of Ottawa, Ontario, Canada
| | | | | | | | | | | |
Collapse
|
12
|
Bahnam P, Hanzel J, Ma C, Zou L, Narula N, Singh S, Kahan B, Jairath V. Most Placebo-Controlled Trials in Inflammatory Bowel Disease were Underpowered Because of Overestimated Drug Efficacy Rates: Results from a Systematic Review of Induction Studies. J Crohns Colitis 2023; 17:404-417. [PMID: 36219564 DOI: 10.1093/ecco-jcc/jjac150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Indexed: 12/13/2022]
Abstract
BACKGROUND AND AIMS Most pharmaceutical clinical trials for inflammatory bowel disease [IBD] are placebo-controlled and require effect size estimation for a drug relative to placebo. We compared expected effect sizes in sample size calculations [SSCs] to actual effect sizes in IBD clinical trials. METHODS MEDLINE, EMBASE, CENTRAL and the Cochrane library were searched from inception to March 26, 2021, to identify placebo-controlled induction studies for luminal Crohn's disease [CD] and ulcerative colitis [UC] that reported an SSC and a primary endpoint of clinical remission/response. Expected effects were subtracted from actual effects, and interquartile ranges [IQRs] for each corresponding median difference were calculated. Linear regression was used to assess whether placebo or drug event rate misspecifications were responsible for these differences. RESULTS Of eligible studies, 36.9% [55/149] were excluded because of incomplete SSC reporting, yielding 94 studies [46 CD, 48 UC]. Treatment effects were overestimated in CD for remission (-12.6% [IQR: -16.3 to -1.6%]), in UC for remission (-10.2% [IQR: -16.5 to -5.6%]) and in CD for response (-15.3% [IQR: -27.1 to -5.8%]). Differences observed were due to overestimated drug event rates, whereas expected and actual placebo event rates were similar. A meta-regression demonstrated associations between overestimated treatment effect sizes and several trial characteristics: isolated ileal disease, longer CD duration, extensive colitis [UC], single-centre, phase 2 and no endoscopic endpoint component [UC]. CONCLUSION Overestimation of IBD therapy efficacy rates resulted in smaller-than-expected treatment effects. These results should be used to inform SSCs and trial design for IBD drug development.
Collapse
Affiliation(s)
- Paul Bahnam
- Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Jurij Hanzel
- Department of Gastroenterology, Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia
- Alimentiv Inc, London, Ontario, Canada
| | - Christopher Ma
- Alimentiv Inc, London, Ontario, Canada
- Division of Gastroenterology & Hepatology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Lily Zou
- Department of Statistics and Actuarial Sciences, University of Waterloo, Waterloo, Ontario, Canada
| | - Neeraj Narula
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Siddharth Singh
- Division of Gastroenterology, University of California San Diego, La Jolla, California, USA
| | | | - Vipul Jairath
- Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
- Alimentiv Inc, London, Ontario, Canada
- Department of Epidemiology and Biostatistics, Western University, London, Ontario, Canada
| |
Collapse
|
13
|
Seretny M, Barlow J, Sidebotham D. Multicentre randomised trials in anaesthesia: an analysis using Bayesian metrics. Anaesthesia 2023; 78:73-80. [PMID: 36128627 DOI: 10.1111/anae.15867] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/01/2022] [Indexed: 12/13/2022]
Abstract
Are the results of randomised trials reliable and are p values and confidence intervals the best way of quantifying efficacy? Low power is common in medical research, which reduces the probability of obtaining a 'significant result' and declaring the intervention had an effect. Metrics derived from Bayesian methods may provide an insight into trial data unavailable from p values and confidence intervals. We did a structured review of multicentre trials in anaesthesia that were published in the New England Journal of Medicine, The Lancet, Journal of the American Medical Association, British Journal of Anaesthesia and Anesthesiology between February 2011 and November 2021. We documented whether trials declared a non-zero effect by an intervention on the primary outcome. We documented the expected and observed effect sizes. We calculated a Bayes factor from the published trial data indicating the probability of the data under the null hypothesis of zero effect relative to the alternative hypothesis of a non-zero effect. We used the Bayes factor to calculate the post-test probability of zero effect for the intervention (having assumed 50% belief in zero effect before the trial). We contacted all authors to estimate the costs of running the trials. The median (IQR [range]) hypothesised and observed absolute effect sizes were 7% (3-13% [0-25%]) vs. 2% (1-7% [0-24%]), respectively. Non-zero effects were declared for 12/56 outcomes (21%). The Bayes factor favouring a zero effect relative to a non-zero effect for these 12 trials was 0.000001-1.9, with post-test zero effect probabilities for the intervention of 0.0001-65%. The other 44 trials did not declare non-zero effects, with Bayes factors favouring zero effect of 1-688, and post-test probabilities of zero effect of 53-99%. The median (IQR [range]) study costs reported by 20 corresponding authors in US$ were $1,425,669 ($514,766-$2,526,807 [$120,758-$24,763,921]). We think that inadequate power and mortality as an outcome are why few trials declared non-zero effects. Bayes factors and post-test probabilities provide a useful insight into trial results, particularly when p values approximate the significance threshold.
Collapse
Affiliation(s)
- M Seretny
- Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand.,Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand
| | - J Barlow
- University of Auckland, Auckland, New Zealand
| | - D Sidebotham
- Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand.,Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand
| |
Collapse
|
14
|
Patel B, Yver H, Woods-Hill CZ, Harhay MO, Yehya N. Elements of Statistical Power in Pediatric Critical Care Trials. Ann Am Thorac Soc 2023; 20:152-155. [PMID: 36044710 PMCID: PMC9819260 DOI: 10.1513/annalsats.202202-154rl] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023] Open
Affiliation(s)
- Bhavesh Patel
- Children’s Hospital of PhiladelphiaPhiladelphia, Pennsylvania
| | - Hugues Yver
- Children’s Hospital of PhiladelphiaPhiladelphia, Pennsylvania
| | - Charlotte Z. Woods-Hill
- Children’s Hospital of PhiladelphiaPhiladelphia, Pennsylvania
- University of PennsylvaniaPhiladelphia, Pennsylvania
| | | | - Nadir Yehya
- Children’s Hospital of PhiladelphiaPhiladelphia, Pennsylvania
- University of PennsylvaniaPhiladelphia, Pennsylvania
| |
Collapse
|
15
|
Fooled by Significance Testing: An Analysis of the LOVIT Vitamin C Trial. THE JOURNAL OF EXTRA-CORPOREAL TECHNOLOGY 2022; 54:324-329. [PMID: 36742025 PMCID: PMC9891468 DOI: 10.1182/ject-2200030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
In this article, I discuss the potential pitfalls of interpreting p values, confidence intervals, and declarations of statistical significance. To illustrate the issues, I discuss the LOVIT trial, which compared high-dose vitamin C with placebo in mechanically ventilated patients with sepsis. The primary outcome - the proportion of patients who died or had persisting organ dysfunction at day 28 - was significantly higher in patients who received vitamin C (p = .01). The authors had hypothesized that vitamin C would have a beneficial effect, although the prior evidence for benefit was weak. There was no prior evidence for a harmful effect of high-dose vitamin C. Consequently, the pretest probability for harm was low. The sample size was calculated assuming a 10% absolute risk difference, which was optimistic. Overestimating the effect size when calculating the sample size leads to low power. For these reasons, we should be skeptical that vitamin C causes harm in septic patients, despite the significant result. p-values and confidence intervals are probabilities concerning the chance of obtaining the observed data. However, we are more interested in the chance the intervention has a real effect on the outcome. That is to say, we are more interested in whether the hypothesis is true. A Bayesian approach allows us to estimate the false positive risk, which is the post-test probability there is no effect of the intervention. The false positive risk for the LOVIT trial (calculated from the published summary data using uniform priors for the parameter values) is 70%. Most likely, high-dose vitamin C does not cause harm in septic patients. Most likely it has no effect at all. If there is an effect, it is probably small and most likely beneficial.
Collapse
|
16
|
Kaas‐Hansen BS, Granholm A, Anthon CT, Kjær MN, Sivapalan P, Maagaard M, Schjørring OL, Fagerberg SK, Ellekjær KL, Mølgaard J, Ekstrøm CT, Møller MH, Perner A. Causal inference for planning randomised critical care trials: Protocol for a scoping review. Acta Anaesthesiol Scand 2022; 66:1274-1278. [PMID: 36054374 PMCID: PMC9826202 DOI: 10.1111/aas.14142] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 08/17/2022] [Indexed: 01/11/2023]
Abstract
BACKGROUND Randomised clinical trials in critical care are prone to inconclusiveness owing, in part, to undue optimism about effect sizes and suboptimal accounting for heterogeneous treatment effects. Planned predictive enrichment based on secondary critical care data (often very rich with respect to both data types and temporal granularity) and causal inference methods may help overcome these challenges, but no overview exists about their use to this end. METHODS We will conduct a scoping review to assess the extent and nature of the use of causal inference from secondary data for planned predictive enrichment of randomised clinical trials in critical care. We will systematically search 10 general and specialty journals for reports published on or after 1 January 2018, of randomised clinical trials enrolling adult critically ill patients. We will collect trial metadata (e.g., recruitment period and phase) and, when available, information pertaining to the focus of the review (predictive enrichment based on causal inference estimates from secondary data): causal inference methods, estimation techniques and software used; types of patient populations; data provenance, types and models; and the availability of the data (public or not). The results will be reported in a descriptive manner. DISCUSSION The outlined scoping review aims to assess the use of causal inference methods and secondary data for planned predictive enrichment in randomised critical care trials. This will help guide methodological improvements to increase the utility, and facilitate the use, of causal inference estimates when planning such trials in the future.
Collapse
Affiliation(s)
- Benjamin Skov Kaas‐Hansen
- Department of Intensive CareCopenhagen University HospitalCopenhagenDenmark
- Section for Biostatistics, Department of Public HealthUniversity of CopenhagenCopenhagenDenmark
| | - Anders Granholm
- Department of Intensive CareCopenhagen University HospitalCopenhagenDenmark
| | - Carl Thomas Anthon
- Department of Intensive CareCopenhagen University HospitalCopenhagenDenmark
| | | | - Praleene Sivapalan
- Department of Intensive CareCopenhagen University HospitalCopenhagenDenmark
| | - Mathias Maagaard
- Department of Anaesthesiology, Centre for Anaesthesiological Research, Zealand University Hospital KøgeKøgeDenmark
| | - Olav Lilleholt Schjørring
- Department of Anaesthesia and Intensive CareAalborg University HospitalAalborgDenmark
- Department of Clinical MedicineAalborg UniversityAalborgDenmark
| | - Steen Kåre Fagerberg
- Department of Anaesthesia and Intensive CareAalborg University HospitalAalborgDenmark
| | | | - Jesper Mølgaard
- Department of Anesthesiology, Centre for Cancer and Organ DysfunctionCopenhagen University HospitalCopenhagenDenmark
| | - Claus Thorn Ekstrøm
- Section for Biostatistics, Department of Public HealthUniversity of CopenhagenCopenhagenDenmark
| | | | - Anders Perner
- Department of Intensive CareCopenhagen University HospitalCopenhagenDenmark
| |
Collapse
|
17
|
The Effect of Intensive Care Unit Diaries on Posttraumatic Stress Disorder, Anxiety, and Depression: A Systematic Review and Meta-analysis of Randomized Controlled Trials. Dimens Crit Care Nurs 2022; 41:256-263. [PMID: 35905428 DOI: 10.1097/dcc.0000000000000539] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
Abstract
INTRODUCTION Patients discharged from the intensive care unit (ICU) suffer from long-term symptoms affecting the physical, psychological, and cognitive well-being and cannot understand memories and dreams. Intensive care unit diaries describe daily events about the patient and may allow them to reconstruct their experience. OBJECTIVE We conducted a systematic review and meta-analysis of randomized controlled trials (RCTs) to investigate the effects of ICU diaries on posttraumatic stress disorder (PTSD), depression, and anxiety. METHODS Five electronic databases were searched up to May 6, 2022. We included RCTs comparing patients admitted to the ICU who received a diary to those who did not receive a diary. The primary outcome was the rate of PTSD. Secondary outcomes were rates of depression and anxiety. RESULTS We included 7 RCTs. Patients who received a diary during the ICU admission had reduced rate of PTSD (78/432 [18%] vs 106/422 [25%]; risk ratio [RR], 0.73; 95% confidence interval [CI], 0.57-0.94; P = .02; I2 = 0%; trial sequential analysis-adjusted CI, 0.55-0.97) when compared with patients who did not receive a diary. We found a non-statistically significant difference toward a reduction in the rate of depression (38/232 [16%] vs 54/224 [24%]; RR, 0.70; 95% CI, 0.49-1.01; P = .06; I2 = 0%) and anxiety (63/232 [27%] vs 70/224 [31%]; RR, 0.64; 95% CI, 0.29-1.40; P = .26; I2 = 67%). CONCLUSIONS Providing an ICU diary to patients admitted to the ICU reduced the rate of PTSD symptoms compared with usual care.
Collapse
|
18
|
Vail EA, Avidan MS. Trials with 'non-significant' results are not insignificant trials: a common significance threshold distorts reporting and interpretation of trial results. Br J Anaesth 2022; 129:643-646. [PMID: 35871898 DOI: 10.1016/j.bja.2022.06.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Revised: 06/09/2022] [Accepted: 06/23/2022] [Indexed: 11/02/2022] Open
Abstract
We discuss a newly published study examining how phrases are used in clinical trials to describe results when the estimated P-value is close to (slightly above or slightly below) 0.05, which has been arbitrarily designated by convention as the boundary for 'statistical significance'. Terms such as 'marginally significant', 'trending towards significant', and 'nominally significant' are well represented in biomedical literature, but are not actually scientifically meaningful. Acknowledging that 'statistical significance' remains a major determinant of publication, we propose that scientific journals de-emphasise the use of P-values for null hypothesis significance testing, a purpose for which they were never intended, and avoid the use of these ambiguous and confusing terms in scientific articles. Instead, investigators could simply report their findings: effect sizes, P-values, and confidence intervals (or their Bayesian equivalents), and leave it to the discerning reader to infer the clinical applicability and importance. Our goal should be to move away from describing studies (or trials) as positive or negative based on an arbitrary P-value threshold, and rather to judge whether the scientific evidence provided is informative or uninformative.
Collapse
Affiliation(s)
- Emily A Vail
- Department of Anesthesiology and Critical Care, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA.
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine in St. Louis, St. Louis, MO, USA
| |
Collapse
|
19
|
Saha R, Assouline B, Mason G, Douiri A, Summers C, Shankar-Har M. The Impact of Sample Size Misestimations on the Interpretation of ARDS Trials: Systematic Review and Meta-analysis. Chest 2022; 162:1048-1062. [PMID: 35643115 DOI: 10.1016/j.chest.2022.05.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 04/06/2022] [Accepted: 05/04/2022] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND Indeterminate randomized controlled trials (RCTs) in ARDS may arise from sample size misspecification, leading to abandonment of efficacious therapies. RESEARCH QUESTIONS If evidence exists for sample size misspecification in ARDS RCTs, has this led to rejection of potentially beneficial therapies? Does evidence exist for prognostic enrichment in RCTs using mortality as a primary outcome? STUDY DESIGN AND METHODS We identified 150 ARDS RCTs commencing recruitment after the 1994 American European Consensus Conference ARDS definition and published before October 31, 2020. We examined predicted-observed sample size, predicted-observed control event rate (CER), predicted-observed average treatment effect (ATE), and the relationship between observed CER and observed ATE for RCTs with mortality and nonmortality primary outcome measures. To quantify the strength of evidence, we used Bayesian-averaged meta-analysis, trial sequential analysis, and Bayes factors. RESULTS Only 84 of 150 RCTs (56.0%) reported sample size estimations. In RCTs with mortality as the primary outcome, CER was overestimated in 16 of 28 RCTs (57.1%). To achieve predicted ATE, interventions needed to prevent 40.8% of all deaths, compared with the original prediction of 29.3%. Absolute reduction in mortality ≥ 10% was observed in 5 of 28 RCTs (17.9%), but predicted in 21 of 28 RCTs (75%). For RCTs with mortality as the primary outcome, no association was found between observed CER and observed ATE (pooled OR: β = -0.04; 95% credible interval, -0.18 to 0.09). We identified three interventions that are not currently standard of care with a Bayesian-averaged effect size of > 0.20 and moderate strength of existing evidence: corticosteroids, airway pressure release ventilation, and noninvasive ventilation. INTERPRETATION Reporting of sample size estimations was inconsistent in ARDS RCTs, and misspecification of CER and ATE was common. Prognostic enrichment strategies in ARDS RCTs based on all-cause mortality are unlikely to be successful. Bayesian methods can be used to prioritize interventions for future effectiveness RCTs.
Collapse
Affiliation(s)
- Rohit Saha
- Critical Care Centre, King's College London, London, United Kingdom; School of Immunology & Microbial Sciences, King's College London, London, United Kingdom
| | - Benjamin Assouline
- Service de Médecine Intensive Réanimation, Faculté de Médecine Sorbonne Université, Hôpital Pitié Salpêtrière, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Georgina Mason
- Critical Care Centre, King's College London, London, United Kingdom
| | - Abdel Douiri
- School of Population Health & Environmental Sciences, King's College London, London, United Kingdom; National Institute for Health Research Comprehensive Biomedical Research Centre, Guy's and St. Thomas' NHS Foundation Trust, London, United Kingdom
| | - Charlotte Summers
- Department of Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Manu Shankar-Har
- Centre for Inflammation Research, University of Edinburgh, Edinburgh, United Kingdom.
| |
Collapse
|
20
|
Nostedt S, Joffe AR. Critical Care Randomized Trials Demonstrate Power Failure: A Low Positive Predictive Value of Findings in the Critical Care Research Field. J Intensive Care Med 2022; 37:1082-1093. [PMID: 35179408 DOI: 10.1177/08850666221077203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND We aimed to determine the post-hoc power of randomized controlled trials (RCTs) in critical care, and describe the implications for long-term positive (PPV) and negative predictive value (NPV) of statistically significant and non-significant findings respectively in the research field. METHODS We reviewed three cohorts of RCTs. "Adult-RCTs" were 216 multicenter RCTs with a mortality outcome from a published systematic review. "Pediatric-RCTs" were 120 RCTs with a mortality outcome, obtained by search of picutrials.net. "Consecutive-RCTs" were 90 recent RCTs obtained by screening publications in 6 journals. Post-hoc power for each study was calculated at α 0.05 and 0.005, for measures of small, medium, and large effect-size, using G*Power software. Long-run expected PPV and NPV of critical care research field findings were then calculated. RESULTS With α 0.05, post-hoc power for small effect-size was very low in all RCT-cohorts (eg, median 24% in Adult-RCTs). For medium effect-size, post-hoc power was low, except for Adult-RCTs (eg, median 9% in Pediatric-RCTs). For large effect-size, post-hoc power for non-human-animal Consecutive-RCTs was low (median 32%). With α 0.005, post-hoc power was even lower. The corollary was that both PPV and NPV were poor for small effect-size, unless α 0.005 was used. Even with α 0.005, with realistic (vs. optimistic) prior probability of the alternative hypothesis, the PPV was low (eg, in Adult-RCTs 57.1% vs. 92.3%). Adding mild bias (0.1) reduced the PPV even further. For medium effect-size both PPV and NPV were better; nevertheless, with α 0.05 and realistic prior probability of the alternative hypothesis the PPV was poor, and with α 0.005 and mild bias (0.1) the PPV was very low (eg, Adult-RCTs median 44.1%). CONCLUSIONS To improve the predictive value of findings in the critical care research field, RCTs should be designed to have 80% power for realistic effect-size at α 0.005.
Collapse
Affiliation(s)
- Sarah Nostedt
- Department of Pediatrics, Division of Critical Care Medicine, University of Alberta, Edmonton, Alberta, Canada.,Stollery Children's Hospital, Edmonton, Alberta, Canada
| | - Ari R Joffe
- Department of Pediatrics, Division of Critical Care Medicine, University of Alberta, Edmonton, Alberta, Canada.,Stollery Children's Hospital, Edmonton, Alberta, Canada
| |
Collapse
|
21
|
Granholm A, Schjørring OL, Jensen AKG, Kaas‐Hansen BS, Munch MW, Klitgaard TL, Crescioli E, Kjær MN, Strøm T, Perner A, Rasmussen BS, Møller MH. Health-related quality of life and days alive without life support or out of hospital: Protocol. Acta Anaesthesiol Scand 2022; 66:295-301. [PMID: 34811741 DOI: 10.1111/aas.14001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 11/07/2021] [Indexed: 11/26/2022]
Abstract
BACKGROUND Mortality is often the primary outcome in randomised clinical trials (RCTs) conducted in critically ill patients. Due to increased awareness on survivors after critical illness and outcomes other than mortality, health-related quality of life (HRQoL) and days alive without life support (DAWOLS) or days alive and out of hospital (DAAOOH) are increasingly being used. DAWOLS and DAAOOH convey more information than mortality, are easier to collect than HRQoL, and are usually assessed at earlier time points, which may be preferable in some situations. However, the associations between DAWOLS-DAAOOH and HRQoL are uncertain. METHODS We will assess associations between DAWOLS-DAAOOH at day 28 and 90 (independent variables/predictors) and HRQoL assessed using the EuroQol EQ-5D-5L questionnaire (EQ-VAS and EQ-5D-5L index values) at 6 or 12 months (dependent variables) in two RCTs: the COVID STEROID 2 RCT conducted in adult patients with COVID-19 and severe hypoxaemia and the Handling Oxygenation Targets in the Intensive Care Unit (HOT-ICU) RCT conducted in adult intensive care patients with acute hypoxaemic respiratory failure. We will describe associations using best-fitting fractional polynomial transformations separately in each dataset, with the resulting models presented and assessed in both datasets graphically and using measures of fit and prediction adequacy (i.e., internal performance and external validation). We will use multiple imputation if missingness exceeds 5%. DISCUSSION The outlined study will provide important knowledge on the associations between DAWOLS-DAAOOH and HRQoL in adult critically ill patients, which may help researchers and clinical trialists prioritise and select outcomes in future RCTs conducted in this population.
Collapse
Affiliation(s)
- Anders Granholm
- Department of Intensive Care Copenhagen University Hospital – Rigshospitalet Copenhagen Denmark
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
| | - Olav Lilleholt Schjørring
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
- Department of Anaesthesia and Intensive Care Aalborg University Hospital Aalborg Denmark
- Department of Clinical Medicine Aalborg University Aalborg Denmark
| | - Aksel Karl Georg Jensen
- Section of Biostatistics, Department of Public Health University of Copenhagen Copenhagen Denmark
| | - Benjamin Skov Kaas‐Hansen
- Section of Biostatistics, Department of Public Health University of Copenhagen Copenhagen Denmark
- Clinical Pharmacology Unit Zealand University Hospital Roskilde Denmark
| | - Marie Warrer Munch
- Department of Intensive Care Copenhagen University Hospital – Rigshospitalet Copenhagen Denmark
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
| | - Thomas Lass Klitgaard
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
- Department of Anaesthesia and Intensive Care Aalborg University Hospital Aalborg Denmark
- Department of Clinical Medicine Aalborg University Aalborg Denmark
| | - Elena Crescioli
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
- Department of Anaesthesia and Intensive Care Aalborg University Hospital Aalborg Denmark
- Department of Clinical Medicine Aalborg University Aalborg Denmark
| | - Maj‐Brit Nørregaard Kjær
- Department of Intensive Care Copenhagen University Hospital – Rigshospitalet Copenhagen Denmark
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
| | - Thomas Strøm
- Department of Anaesthesia and Critical Care Medicine Odense University Hospital Odense Denmark
- Department of Anaesthesia and Critical Care Medicine Hospital Sønderjylland, University Hospital of Southern Denmark Odense Denmark
| | - Anders Perner
- Department of Intensive Care Copenhagen University Hospital – Rigshospitalet Copenhagen Denmark
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
| | - Bodil Steen Rasmussen
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
- Department of Anaesthesia and Intensive Care Aalborg University Hospital Aalborg Denmark
- Department of Clinical Medicine Aalborg University Aalborg Denmark
| | - Morten Hylander Møller
- Department of Intensive Care Copenhagen University Hospital – Rigshospitalet Copenhagen Denmark
- Collaboration for Research in Intensive Care (CRIC) Copenhagen Denmark
| |
Collapse
|
22
|
Sidebotham D. In Defense of Science. THE JOURNAL OF EXTRA-CORPOREAL TECHNOLOGY 2021; 53:239-244. [PMID: 34992313 PMCID: PMC8717720 DOI: 10.1182/ject-2100052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Affiliation(s)
- David Sidebotham
- Anesthesiologist and Intensivist, Department of Anaesthesia and the Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand. E-mail:
| |
Collapse
|
23
|
Affiliation(s)
- D Sidebotham
- Department of Anaesthesia and the Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand
| |
Collapse
|
24
|
Sidebotham D, Popovich I, Lumley T. A Bayesian analysis of mortality outcomes in multicentre clinical trials in critical care. Br J Anaesth 2021; 127:487-494. [PMID: 34275603 DOI: 10.1016/j.bja.2021.06.026] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2021] [Revised: 06/15/2021] [Accepted: 06/20/2021] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND Multicentre RCTs are widely used by critical care researchers to answer important clinical questions. However, few trials evaluating mortality outcomes report statistically significant results. We hypothesised that the low proportion of trials reporting statistically significant differences for mortality outcomes is plausibly explained by lower-than-expected effect sizes combined with a low proportion of participants who could realistically benefit from studied interventions. METHODS We reviewed multicentre trials in critical care published over a 10-yr period in the New England Journal of Medicine, the Journal of the American Medical Association, and the Lancet. To test our hypothesis, we analysed the results using a Bayesian model to investigate the relationship between the proportion of effective interventions and the proportion of statistically significant results for prior distributions of effect size and trial participant susceptibility. RESULTS Five of 54 trials (9.3%) reported a significant difference in mortality between the control and the intervention groups. The median expected and observed differences in absolute mortality were 8.0% and 2.0%, respectively. Our modelling shows that, across trials, a lower-than-expected effect size combined with a low proportion of potentially susceptible participants is consistent with the observed proportion of trials reporting significant differences even when most interventions are effective. CONCLUSIONS When designing clinical trials, researchers most likely overestimate true population effect sizes for critical care interventions. Bayesian modelling demonstrates that that it is not necessarily the case that most studied interventions lack efficacy. In fact, it is plausible that many studied interventions have clinically important effects that are missed.
Collapse
Affiliation(s)
- David Sidebotham
- Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand.
| | - Ivor Popovich
- Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand
| | - Thomas Lumley
- Department of Statistics, University of Auckland, Auckland, New Zealand
| |
Collapse
|
25
|
Powering Bias and Clinically Important Treatment Effects in Randomized Trials of Critical Illness. Crit Care Med 2021; 48:1710-1719. [PMID: 33031148 DOI: 10.1097/ccm.0000000000004568] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
OBJECTIVES Recurring issues in clinical trial design may bias results toward the null, yielding findings inconclusive for treatment effects. This study evaluated for powering bias among high-impact critical care trials and the associated risk of masking clinically important treatment effects. DESIGN, SETTING, AND PATIENTS Secondary analysis of multicenter randomized trials of critically ill adults in which mortality was the main endpoint. Trials were eligible for inclusion if published between 2008 and 2018 in leading journals. Analyses evaluated for accuracy of estimated control group mortality, adaptive sample size strategy, plausibility of predicted treatment effect, and results relative to the minimal clinically important difference. The main outcome was the mortality risk difference at the study-specific follow-up interval. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS Of 101 included trials, 12 met statistical significance for their main endpoint, five for increased intervention-associated mortality. Most trials (77.3%) overestimated control group mortality in power calculations (observed minus predicted difference, -6.7% ± 9.8%; p < 0.01). Due to this misestimation of control group mortality, in 14 trials, the intervention would have had to prevent at least half of all deaths to achieve the hypothesized treatment effect. Seven trials prespecified adaptive sample size strategies that might have mitigated this issue. The observed risk difference for mortality fell within 5% of predicted in 20 trials, of which 16 did not reach statistical significance. Half of trials (47.0%) were powered for an absolute risk reduction greater than or equal to 10%, but this effect size was observed in only three trials with a statistically significant treatment benefit. Most trials (67.3%) could not exclude clinically important treatment benefit or harm. CONCLUSIONS The design of most high-impact critical care trials biased results toward the null by overestimating control group mortality and powering for unrealistic treatment effects. Clinically important treatment effects often cannot be excluded.
Collapse
|
26
|
"Paying the Piper": The Downstream Implications of Manipulating Sample Size Assumptions for Critical Care Randomized Control Trials. Crit Care Med 2021; 48:1885-1886. [PMID: 33255102 DOI: 10.1097/ccm.0000000000004664] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
27
|
Seehra J, Stonehouse-Smith D, Cobourne MT, Tsagris M, Pandis N. Are treatment effect assumptions in orthodontic studies overoptimistic? Eur J Orthod 2021; 43:583-587. [PMID: 33991101 PMCID: PMC8488969 DOI: 10.1093/ejo/cjab018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Background At the clinical trial design stage, assumptions regarding the treatment effects to be detected should be appropriate so that the required sample size can be calculated. There is evidence in the medical literature that sample size assumption can be overoptimistic. The aim of this study was to compare the distribution of the assumed effects versus that of the observed effects as a proxy for overoptimistic treatment effect assumptions at the study design stage. Materials and method Systematic reviews (SRs) published between 1 January 2010 and 31 December 2019 containing at least one meta-analysis on continuous outcomes were identified electronically. SR and primary study level characteristics were extracted from the SRs and the individual trials. Details on the sample size calculation process and assumptions and the observed treatment effects were extracted. Results Eighty-five SRs with meta-analysis containing 347 primary trials were included. The median number of SR authors was 5 (interquartile range: 4–7). At the primary study level, the majority were single centre (78.1%), utilized a parallel design (52%), and rated as an unclear/moderate level of risk of bias (34.3%). A sample size was described in only 31.7% (110/347) of studies. From this cohort of 110 studies, in only 37 studies was the assumed clinical difference that the study was designed to detect reported (37/110). The assumed treatment effect was recalculated for the remaining 73 studies (73/110). The one-sided exact signed rank test showed a significant difference between the assumed and observed treatment effects (P < 0.001) suggesting greater values for the assumed effect sizes. Conclusions Careful consideration of the assumptions at the design stage of orthodontic studies are necessary in order to reduce the unreliability of clinical study results and research waste.
Collapse
Affiliation(s)
- Jadbinder Seehra
- Department of Orthodontics, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, Floor 25, Guy's Hospital, Guy's and St Thomas NHS Foundation Trust, London, UK
| | - Daniel Stonehouse-Smith
- Department of Orthodontics, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, Floor 25, Guy's Hospital, Guy's and St Thomas NHS Foundation Trust, London, UK
| | - Martyn T Cobourne
- Department of Orthodontics, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, Floor 25, Guy's Hospital, Guy's and St Thomas NHS Foundation Trust, London, UK
| | - Michail Tsagris
- Department of Economics, University of Crete, Rethimnon, Greece
| | - Nikolaos Pandis
- Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
28
|
Juschten J, Tuinman PR, Guo T, Juffermans NP, Schultz MJ, Loer SA, Girbes ARJ, de Grooth HJ. Between-trial heterogeneity in ARDS research. Intensive Care Med 2021; 47:422-434. [PMID: 33713156 PMCID: PMC7955690 DOI: 10.1007/s00134-021-06370-w] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Accepted: 02/15/2021] [Indexed: 02/07/2023]
Abstract
Purpose Most randomized controlled trials (RCTs) in patients with acute respiratory distress syndrome (ARDS) revealed indeterminate or conflicting study results. We aimed to systematically evaluate between-trial heterogeneity in reporting standards and trial outcome. Methods A systematic review of RCTs published between 2000 and 2019 was performed including adult ARDS patients receiving lung-protective ventilation. A random-effects meta-regression model was applied to quantify heterogeneity (non-random variability) and to evaluate trial and patient characteristics as sources of heterogeneity. Results In total, 67 RCTs were included. The 28-day control-group mortality rate ranged from 10 to 67% with large non-random heterogeneity (I2 = 88%, p < 0.0001). Reported baseline patient characteristics explained some of the outcome heterogeneity, but only six trials (9%) reported all four independently predictive variables (mean age, mean lung injury score, mean plateau pressure and mean arterial pH). The 28-day control group mortality adjusted for patient characteristics (i.e. the residual heterogeneity) ranged from 18 to 45%. Trials with significant benefit in the primary outcome reported a higher control group mortality than trials with an indeterminate outcome or harm (mean 28-day control group mortality: 44% vs. 28%; p = 0.001). Conclusion Among ARDS RCTs in the lung-protective ventilation era, there was large variability in the description of baseline characteristics and significant unexplainable heterogeneity in 28-day control group mortality. These findings signify problems with the generalizability of ARDS research and underline the urgent need for standardized reporting of trial and baseline characteristics. Supplementary Information The online version of this article (10.1007/s00134-021-06370-w) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- J Juschten
- Department of Intensive Care, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, mail stop ZH 7D-172, 1081HV, Amsterdam, The Netherlands. .,Research VUmc Intensive Care (REVIVE), Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands. .,Department of Anesthesiology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.
| | - P R Tuinman
- Department of Intensive Care, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, mail stop ZH 7D-172, 1081HV, Amsterdam, The Netherlands.,Research VUmc Intensive Care (REVIVE), Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - T Guo
- Department of Intensive Care, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, mail stop ZH 7D-172, 1081HV, Amsterdam, The Netherlands.,Research VUmc Intensive Care (REVIVE), Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.,Division of System Biomedicine and Pharmacology, Leiden Academic Centre for Drug Research (LACDR), Leiden University, Leiden, The Netherlands
| | - N P Juffermans
- Laboratory of Experimental Intensive Care and Anesthesiology (LEICA), Amsterdam UMC, Universiteit Van Amsterdam, Amsterdam, The Netherlands.,Department of Intensive Care, OLVG Hospital, Amsterdam, The Netherlands
| | - M J Schultz
- Department of Intensive Care, Amsterdam UMC, Universiteit Van Amsterdam, Amsterdam, The Netherlands.,Mahidol-Oxford Tropical Medicine Research Unit (MORU), Mahidol University, Bangkok, Thailand.,Nuffield Department of Medicine, University of Oxford, Oxford, UK
| | - S A Loer
- Department of Anesthesiology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - A R J Girbes
- Department of Intensive Care, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, mail stop ZH 7D-172, 1081HV, Amsterdam, The Netherlands.,Research VUmc Intensive Care (REVIVE), Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - H J de Grooth
- Department of Anesthesiology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
29
|
Zangrillo A, Bignami E, Noè B, Nardelli P, Licheri M, Gerli C, Crivellari M, Oriani A, Di Prima AL, Fominskiy E, Di Tomasso N, Lembo R, Landoni G, Crescenzi G, Monaco F. Esmolol in Cardiac Surgery: A Randomized Controlled Trial. J Cardiothorac Vasc Anesth 2021; 35:1106-1114. [PMID: 33451954 DOI: 10.1053/j.jvca.2020.12.029] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Revised: 12/11/2020] [Accepted: 12/16/2020] [Indexed: 12/19/2022]
Abstract
OBJECTIVE To assess whether the administration of the ultra-short-acting β-blocker esmolol in cardiac surgery could have a cardioprotective effect that translates into improved postoperative outcomes. DESIGN Single-center, double-blinded, parallel-group randomized controlled trial. SETTING A tertiary care referral center. PARTICIPANTS Patients undergoing elective cardiac surgery with preoperative evidence of left ventricular end-diastolic diameter >60 mm and/or left ventricular ejection fraction <50%. INTERVENTIONS Patients were assigned randomly to receive either esmolol (1 mg/kg as a bolus before aortic cross-clamping and 2 mg/kg mixed in the cardioplegia solution) or placebo in a 1:1 allocation ratio. MEASUREMENTS AND MAIN RESULTS The primary composite endpoint of prolonged intensive care unit stay and/or in-hospital mortality occurred in 36/98 patients (36%) in the placebo group versus 27/102 patients (27%) in the esmolol group (p = 0.13). In the esmolol group, a reduction in the maximum inotropic score during the first 24 postoperative hours was observed (10 [interquartile range 5-15] v 7 [interquartile range 5-10.5]; p = 0.04), as well as a trend toward a reduction in postoperative low-cardiac-output syndrome (13/98 v 6/102; p = 0.08) and the rate of hospital admission at one year (26/95 v 16/96; p = 0.08). A trend toward an increase in the number of patients with ejection fraction ≥60% at hospital discharge also was observed (4/95 v 11/92; p = 0.06). CONCLUSIONS In the present trial, esmolol as a cardioplegia adjuvant enhanced postoperative cardiac performance but did not reduce a composite endpoint of prolonged intensive care unit stay and/or mortality.
Collapse
Affiliation(s)
- Alberto Zangrillo
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
| | - Elena Bignami
- Anesthesiology, Critical Care and Pain Medicine Division, Department of Medicine and Surgery, University of Parma, Parma, Italy
| | - Beatrice Noè
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Pasquale Nardelli
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Margherita Licheri
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Chiara Gerli
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Martina Crivellari
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Alessandro Oriani
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Ambra Licia Di Prima
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Evgeny Fominskiy
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Nora Di Tomasso
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Rosalba Lembo
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Giovanni Landoni
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy.
| | - Giuseppe Crescenzi
- Department of Anaesthesia and Intensive Care Medicine, Humanitas Clinical and Research Center - IRCCS, Rozzano, Milan, Italy
| | - Fabrizio Monaco
- Department of Anesthesia and Intensive Care, IRCCS San Raffaele Scientific Institute, Milan, Italy
| |
Collapse
|
30
|
Clinical trials in critical care: can a Bayesian approach enhance clinical and scientific decision making? THE LANCET RESPIRATORY MEDICINE 2020; 9:207-216. [PMID: 33227237 DOI: 10.1016/s2213-2600(20)30471-9] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 09/28/2020] [Accepted: 10/01/2020] [Indexed: 02/07/2023]
Abstract
Recent Bayesian reanalyses of prominent trials in critical illness have generated controversy by contradicting the initial conclusions based on conventional frequentist analyses. Many clinicians might be sceptical that Bayesian analysis, a philosophical and statistical approach that combines prior beliefs with data to generate probabilities, provides more useful information about clinical trials than the frequentist approach. In this Personal View, we introduce clinicians to the rationale, process, and interpretation of Bayesian analysis through a systematic review and reanalysis of interventional trials in critical illness. In the majority of cases, Bayesian and frequentist analyses agreed. In the remainder, Bayesian analysis identified interventions where benefit was probable despite the absence of statistical significance, where interpretation depended substantially on choice of prior distribution, and where benefit was improbable despite statistical significance. Bayesian analysis in critical care medicine can help to distinguish harm from uncertainty and establish the probability of clinically important benefit for clinicians, policy makers, and patients.
Collapse
|
31
|
Amrein K, Lasky-Su JA, Dobnig H, Christopher KB. Metabolomic basis for response to high dose vitamin D in critical illness. Clin Nutr 2020; 40:2053-2060. [PMID: 33087250 DOI: 10.1016/j.clnu.2020.09.028] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Revised: 08/04/2020] [Accepted: 09/21/2020] [Indexed: 12/12/2022]
Abstract
BACKGROUND & AIMS It is unclear if intervention can mitigate the dramatic alterations of metabolic homeostasis present in critical illness. Our objective was to determine the associations between increased 25-hydroxyvitamin D levels following high dose vitamin D3 and more favorable metabolomic profiles in critical illness. METHODS We performed a post-hoc metabolomics study of the VITdAL-ICU randomized double-blind, placebo-controlled trial. Trial patients from Medical and Surgical Intensive Care Units at a tertiary university hospital with 25-hydroxyvitamin D level ≤20 ng/mL received either high dose oral vitamin D3 (540,000 IU) or placebo. We performed an analysis of 578 metabolites from 1215 plasma samples from 428 subjects at randomization (day 0), day 3 and 7. Using mixed-effects modeling, we studied changes in metabolite profiles in subjects receiving intervention or placebo relative to absolute increases in 25-hydroxyvitamin D levels from day 0 to day 3. RESULTS 55.2% of subjects randomized to high dose vitamin D3 demonstrated an absolute increase in 25-hydroxyvitamin D ≥ 15 ng/ml from day 0 to day 3. With an absolute increase in 25-hydroxyvitamin D ≥ 15 ng/ml, multiple members of the sphingomyelin, plasmalogen, lysoplasmalogen and lysophospholipid metabolite classes had significantly positive Bonferroni corrected associations over time. Further, multiple representatives of the acylcarnitine and phosphatidylethanolamine metabolite classes had significantly negative Bonferroni corrected associations over time with an absolute increase in 25-hydroxyvitamin D ≥ 15 ng/ml. Changes in these highlighted metabolite classes were associated with decreased 28-day mortality. CONCLUSIONS Increases in 25-hydroxyvitamin D following vitamin D3 intervention are associated with favorable changes in metabolites involved in endothelial protection, enhanced innate immunity and improved mitochondrial function.
Collapse
Affiliation(s)
- Karin Amrein
- Division of Endocrinology and Diabetology, Medical University of Graz, Graz, Austria
| | - Jessica A Lasky-Su
- Channing Division of Network Medicine, Brigham and Women's Hospital, USA
| | - Harald Dobnig
- Thyroid Endocrinology Osteoporosis Institute Dobnig, Graz, Austria
| | - Kenneth B Christopher
- Division of Renal Medicine, Channing Division of Network Medicine, Brigham and Women's Hospital, USA.
| |
Collapse
|
32
|
Lactated Ringer's Versus 4% Albumin on Lactated Ringer's in Early Sepsis Therapy in Cancer Patients: A Pilot Single-Center Randomized Trial. Crit Care Med 2020; 47:e798-e805. [PMID: 31356475 DOI: 10.1097/ccm.0000000000003900] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Abstract
OBJECTIVE To investigate the effects of the administration of 4% albumin on lactated Ringer's, when compared with lactated Ringer's alone, in the early phase of sepsis in cancer patients. DESIGN Single-center, randomized, double-blind, controlled-parallel trial. SETTING A tertiary care university cancer hospital. PATIENTS Cancer patients with severe sepsis or septic shock. INTERVENTIONS Between October 2014 and December 2016, patients were randomly assigned to receive either bolus of albumin in a lactated Ringer's solution or lactated Ringer's solution alone during the first 6 hours of fluid resuscitation after intensive care medicine (ICU) admission. Primary outcome was defined as death from any cause at 7 days. Secondary outcomes were defined as death from any cause within 28 days, change in Sequence Organ Failure Assessment scores from baseline to day 7, days alive and free of mechanical ventilation, days alive and free of vasopressor, renal replacement therapy during ICU stay, and length of ICU and hospital stay. MEASUREMENTS AND MAIN RESULTS A total of 360 patients were enrolled in the trial. At 7 days, 46 of 180 patients (26%) died in the albumin group and 40 of 180 (22%) died in the lactated Ringer's group (p = 0.5). At 28 days, 96 of 180 patients (53%) died in the albumin group and 83 of 180 (46%) died in the lactated Ringer's group (p = 0.2). No significant differences in secondary outcomes were observed. CONCLUSIONS Adding albumin to early standard resuscitation with lactated Ringer's in cancer patients with sepsis did not improve 7-day survival.
Collapse
|
33
|
Typology of Published Randomized Controlled Trials Investigating Initial Ventilation Strategy in Critically Ill Patients With Acute Respiratory Failure: A Methodologic Review. Chest 2020; 158:986-998. [PMID: 32387523 DOI: 10.1016/j.chest.2020.03.082] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2019] [Revised: 03/02/2020] [Accepted: 03/22/2020] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND Randomized controlled trials (RCTs) in patients with hypoxemic acute respiratory failure (ARF) often failed to show survival benefits and resulted in varying clinical end points. RESEARCH QUESTION This methodologic review was conducted of published RCTs on ARF, with a careful attention to whether the study results were positive or negative. STUDY DESIGN AND METHODS MEDLINE, Cochrane Central Register of Controlled Trials, and Web of Science were searched for published RCTs in adult patients with ARF between January 1995 and December 2019. The objective was to investigate sources of heterogeneity and factors associated with a positive RCT (ie, with a significant difference on the primary end point). To determine the importance of the primary end point choice, end points were classified as follows: (1) respiratory event end points (corresponding to modification of the ventilation support); (2) physiologic/clinical end points (corresponding to oxygenation or clinical parameters); and (3) mortality. The Cochrane risk-of-bias tool was used to assess study quality. RESULTS Seventy-four RCTs were included (57% were single-center RCTs) comparing mainly oxygenation/ventilation strategies (95%) in patients with various ARF causes (62%); studies were stopped prematurely in 20% of the trials. A standardized management of ARF was observed in 24 (32%) trials. Twenty-two distinct primary end points have been used, the first of which were those based on respiratory events (44 RCTs [49%]), namely intubation in 76% of the cases. Physiologic/clinical end points have been used in 21 trials (29%) and mortality in nine (12%). Overall, 42 (57%) RCTs were positive, 52% in studies with respiratory event end points, 76% in studies with physiologic/clinical end points, and 33% in studies with mortality end points. Adjusted for study quality (Cochrane risk-of-bias tool), factors associated with a positive RCT included clinically based primary end points (OR, 8.40; 95% CI, 1.35-65.79), the use of standardized ARF management (OR, 4.55; 95% CI, 1.02-22.88), and single-center trials (OR, 3.85; 95% CI, 1.25-13.11). INTERPRETATION The typology of published RCTs in patients with ARF could be used to frame future trial designs in this field and guide clinicians and researchers toward optimal research transfer to the bedside.
Collapse
|
34
|
Contemporary strategies to improve clinical trial design for critical care research: insights from the First Critical Care Clinical Trialists Workshop. Intensive Care Med 2020; 46:930-942. [PMID: 32072303 PMCID: PMC7224097 DOI: 10.1007/s00134-020-05934-6] [Citation(s) in RCA: 48] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2019] [Accepted: 01/11/2020] [Indexed: 02/06/2023]
Abstract
BACKGROUND Conducting research in critically-ill patient populations is challenging, and most randomized trials of critically-ill patients have not achieved pre-specified statistical thresholds to conclude that the intervention being investigated was beneficial. METHODS In 2019, a diverse group of patient representatives, regulators from the USA and European Union, federal grant managers, industry representatives, clinical trialists, epidemiologists, and clinicians convened the First Critical Care Clinical Trialists (3CT) Workshop to discuss challenges and opportunities in conducting and assessing critical care trials. Herein, we present the advantages and disadvantages of available methodologies for clinical trial design, conduct, and analysis, and a series of recommendations to potentially improve future trials in critical care. CONCLUSION The 3CT Workshop participants identified opportunities to improve critical care trials using strategies to optimize sample size calculations, account for patient and disease heterogeneity, increase the efficiency of trial conduct, maximize the use of trial data, and to refine and standardize the collection of patient-centered and patient-informed outcome measures beyond mortality.
Collapse
|
35
|
Harhay MO, Ratcliffe SJ, Small DS, Suttner LH, Crowther MJ, Halpern SD. Measuring and Analyzing Length of Stay in Critical Care Trials. Med Care 2019; 57:e53-e59. [PMID: 30664613 PMCID: PMC6635104 DOI: 10.1097/mlr.0000000000001059] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
BACKGROUND In randomized clinical trials among critically ill patients, it is uncertain how choices regarding the measurement and analysis of nonmortal outcomes measured in terms of duration, such as intensive care unit (ICU) length of stay (LOS), affect studies' conclusions. OBJECTIVES Assess the definitions and analytic methods used for ICU LOS analyses in published randomized clinical trials. RESEARCH DESIGN This is a systematic review and statistical simulation study. RESULTS Among the 80 of 150 trials providing sufficient information regarding the chosen definition of ICU LOS, 3 different start times (ICU admission, trial enrollment/randomization, receipt of intervention) and 2 end times (discharge readiness, actual discharge) were used. In roughly three quarters of these studies, ICU LOS was compared using approaches that did not explicitly account for death, either by ignoring it entirely or stratifying the analyses by survival status. The remaining studies used time-to-event (discharge) models censoring at death or applied a fixed LOS value to patients who died. In statistical simulations, we showed that each analytic approach tested a different question regarding ICU LOS, and that approaches that do not explicitly account for death often produce misleading or ambiguous conclusions when treatments produce small effects on mortality, even if those are not detected as significant in the trial. CONCLUSIONS There is considerable variability in how ICU LOS is measured and analyzed which impairs the ability to compare results across trials and can produce spurious conclusions. Analyses of duration-based outcomes such as LOS should jointly assess the impact of the intervention on mortality to yield correct interpretations.
Collapse
Affiliation(s)
- Michael O Harhay
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine
- Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania, Philadelphia, PA
| | - Sarah J Ratcliffe
- Department of Public Health Sciences, University of Virginia, Division of Biostatistics, Charlottesville, VA
| | - Dylan S Small
- Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania, Philadelphia, PA
- Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Leah H Suttner
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine
| | - Michael J Crowther
- Biostatistics Research Group, Department of Health Sciences, Centre for Medicine, University of Leicester, Leicester, UK
| | - Scott D Halpern
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of Medicine
- Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania, Philadelphia, PA
- Department of Medicine, Division of Pulmonary, Allergy, and Critical Care Medicine
- Department of Medical Ethics and Health Policy, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA
| |
Collapse
|
36
|
Yeh DD, Martin M, Sakran JV, Meier K, Mendoza A, Grant AA, Parks J, Byerly S, Lee EE, McKinley WI, McClave SA, Miller K, Mazuski J, Taylor B, Luckhurst C, Fagenholz P. Advances in nutrition for the surgical patient. Curr Probl Surg 2019; 56:343-398. [DOI: 10.1067/j.cpsurg.2019.04.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
37
|
|
38
|
|
39
|
Kaufmann T, Granholm A, Keus F, Myburgh J, Perner A, Møller MH, van der Horst ICC. Foresight over hindsight: Mandatory publication of clinical research protocols prior to conduct. Acta Anaesthesiol Scand 2019; 63:267-269. [PMID: 30311209 DOI: 10.1111/aas.13267] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2018] [Accepted: 09/18/2018] [Indexed: 01/19/2023]
Affiliation(s)
- Thomas Kaufmann
- Department of Anesthesiology, University Medical Center Groningen; University of Groningen; Groningen The Netherlands
| | - Anders Granholm
- Department of Intensive Care, Rigshospitalet; University of Copenhagen; Copenhagen Denmark
| | - Frederik Keus
- Department of Critical Care, University Medical Center Groningen; University of Groningen; Groningen The Netherlands
| | - John Myburgh
- Critical Care Division; George Institute for Global Health, University of New South Wales; Sydney Australia
| | - Anders Perner
- Department of Intensive Care, Rigshospitalet; University of Copenhagen; Copenhagen Denmark
- Centre for Research in Intensive Care (CRIC); Copenhagen Denmark
| | - Morten Hylander Møller
- Department of Intensive Care, Rigshospitalet; University of Copenhagen; Copenhagen Denmark
- Centre for Research in Intensive Care (CRIC); Copenhagen Denmark
| | - Iwan C. C. van der Horst
- Department of Critical Care, University Medical Center Groningen; University of Groningen; Groningen The Netherlands
| |
Collapse
|
40
|
Miskimins R, Pati S, Schreiber M. Barriers to clinical research in trauma. Transfusion 2018; 59:846-853. [PMID: 30585332 DOI: 10.1111/trf.15097] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2018] [Accepted: 11/06/2018] [Indexed: 12/13/2022]
Abstract
As with all areas of medicine, high-quality clinical research is essential to improving the care of trauma patients. This research is crucial in developing evidence-based treatments that decrease cost, decrease morbidity, and improve mortality. Trauma continues to extract a significant toll on society and is the single largest cause of years of life lost in the United States. The need to conduct high-quality clinical research in trauma is not disputed. However, significant challenges and barriers unique to the field of trauma make performing this research more difficult. It is critical to be aware of these challenges and barriers to performing clinical research involving trauma patients so these challenges can be accounted for and solutions implemented to minimize their impact on research. This review will focus on the barriers and challenges that are encountered while performing clinical research in trauma.
Collapse
Affiliation(s)
- Richard Miskimins
- Department of Surgery, University of New Mexico School of Medicine, Albuquerque, New Mexico
| | - Shibani Pati
- Department of Pathology and Laboratory Medicine, University of California San Francisco, San Francisco, California, USA
| | - Martin Schreiber
- Department of Surgery, Oregon Health & Science University, Portland, Oregon, USA
| |
Collapse
|
41
|
Gasparini M, Chevret S. Intensive care medicine in 2050: clinical trials designs. Intensive Care Med 2018; 45:668-670. [PMID: 30430206 DOI: 10.1007/s00134-018-5364-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2018] [Accepted: 08/30/2018] [Indexed: 01/15/2023]
Affiliation(s)
- M Gasparini
- Department of Mathematical Sciences, Politecnico di Torino, Turin, Italy
| | - S Chevret
- AP-HP, Saint-Louis Hospital, Biostatistics Team, Paris, France.
- Paris Diderot University, Inserm, ECSTRRA Team, Paris, France.
| |
Collapse
|
42
|
Ciarleglio MM, Arendt CD. Sample size re-estimation in a superiority clinical trial using a hybrid classical and Bayesian procedure. Stat Methods Med Res 2018; 28:1852-1878. [PMID: 29869564 DOI: 10.1177/0962280218776991] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
When designing studies involving a continuous endpoint, the hypothesized difference in means ( θA ) and the assumed variability of the endpoint ( σ2 ) play an important role in sample size and power calculations. Traditional methods of sample size re-estimation often update one or both of these parameters using statistics observed from an internal pilot study. However, the uncertainty in these estimates is rarely addressed. We propose a hybrid classical and Bayesian method to formally integrate prior beliefs about the study parameters and the results observed from an internal pilot study into the sample size re-estimation of a two-stage study design. The proposed method is based on a measure of power called conditional expected power (CEP), which averages the traditional power curve using the prior distributions of θ and σ2 as the averaging weight, conditional on the presence of a positive treatment effect. The proposed sample size re-estimation procedure finds the second stage per-group sample size necessary to achieve the desired level of conditional expected interim power, an updated CEP calculation that conditions on the observed first-stage results. The CEP re-estimation method retains the assumption that the parameters are not known with certainty at an interim point in the trial. Notional scenarios are evaluated to compare the behavior of the proposed method of sample size re-estimation to three traditional methods.
Collapse
Affiliation(s)
- Maria M Ciarleglio
- 1 Yale University School of Public Health, Department of Biostatistics, New Haven, CT, USA.,2 Clinical Epidemiology Research Center, VA Cooperative Studies Program, West Haven, CT, USA
| | | |
Collapse
|
43
|
Chow JTY, Turkstra TP, Yim E, Jones PM. Sample size calculations for randomized clinical trials published in anesthesiology journals: a comparison of 2010 versus 2016. Can J Anaesth 2018; 65:611-618. [DOI: 10.1007/s12630-018-1109-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2017] [Revised: 02/14/2018] [Accepted: 02/26/2018] [Indexed: 12/01/2022] Open
|
44
|
Niven DJ, McCormick TJ, Straus SE, Hemmelgarn BR, Jeffs L, Barnes TRM, Stelfox HT. Reproducibility of clinical research in critical care: a scoping review. BMC Med 2018; 16:26. [PMID: 29463308 PMCID: PMC5820784 DOI: 10.1186/s12916-018-1018-6] [Citation(s) in RCA: 49] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/19/2017] [Accepted: 01/31/2018] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND The ability to reproduce experiments is a defining principle of science. Reproducibility of clinical research has received relatively little scientific attention. However, it is important as it may inform clinical practice, research agendas, and the design of future studies. METHODS We used scoping review methods to examine reproducibility within a cohort of randomized trials examining clinical critical care research and published in the top general medical and critical care journals. To identify relevant clinical practices, we searched the New England Journal of Medicine, The Lancet, and JAMA for randomized trials published up to April 2016. To identify a comprehensive set of studies for these practices, included articles informed secondary searches within other high-impact medical and specialty journals. We included late-phase randomized controlled trials examining therapeutic clinical practices in adults admitted to general medical-surgical or specialty intensive care units (ICUs). Included articles were classified using a reproducibility framework. An original study was the first to evaluate a clinical practice. A reproduction attempt re-evaluated that practice in a new set of participants. RESULTS Overall, 158 practices were examined in 275 included articles. A reproduction attempt was identified for 66 practices (42%, 95% CI 33-50%). Original studies reported larger effects than reproduction attempts (primary endpoint, risk difference 16.0%, 95% CI 11.6-20.5% vs. 8.4%, 95% CI 6.0-10.8%, P = 0.003). More than half of clinical practices with a reproduction attempt demonstrated effects that were inconsistent with the original study (56%, 95% CI 42-68%), among which a large number were reported to be efficacious in the original study and to lack efficacy in the reproduction attempt (34%, 95% CI 19-52%). Two practices reported to be efficacious in the original study were found to be harmful in the reproduction attempt. CONCLUSIONS A minority of critical care practices with research published in high-profile journals were evaluated for reproducibility; less than half had reproducible effects.
Collapse
Affiliation(s)
- Daniel J Niven
- Department of Critical Care Medicine, University of Calgary, 3134 Hospital Drive NW, Calgary, AB, T2N 2T9, Canada.
| | - T Jared McCormick
- Department of Anesthesiology and Pain Medicine, University of Ottawa, 1053 Carling Avenue, B302, Ottawa, ON, K1Y 4E9, Canada
| | - Sharon E Straus
- Li Ka Shing Knowledge Institute of St. Michael's Hospital, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
| | - Brenda R Hemmelgarn
- Department of Community Health Sciences, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada
| | - Lianne Jeffs
- St. Michael's Hospital Volunteer Association Chair in Nursing and Scientist with the Keenan Research Center, Li Ka Shing Knowledge Institute of St. Michael's Hospital, Institute of Health Policy Management and Faculty of Nursing, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
| | - Tavish R M Barnes
- Department of Critical Care Medicine, University of Calgary, 3134 Hospital Drive NW, Calgary, AB, T2N 2T9, Canada
| | - Henry T Stelfox
- Department of Critical Care Medicine, University of Calgary, 3134 Hospital Drive NW, Calgary, AB, T2N 2T9, Canada
| |
Collapse
|
45
|
Empirical Consequences of Current Recommendations for the Design and Interpretation of Noninferiority Trials. J Gen Intern Med 2018; 33:88-96. [PMID: 28875400 PMCID: PMC5756156 DOI: 10.1007/s11606-017-4161-4] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/19/2017] [Revised: 07/28/2017] [Accepted: 08/09/2017] [Indexed: 12/19/2022]
Abstract
BACKGROUND Noninferiority trials are increasingly common, though they have less standardized designs and their interpretation is less familiar to clinicians than superiority trials. OBJECTIVE To empirically evaluate a cohort of noninferiority trials to determine 1) their interpretation as recommended by CONSORT, 2) choice of alpha threshold and its sidedness, and 3) differences between methods of analysis such as intention-to-treat and per-protocol. DESIGN We searched MEDLINE for parallel-group randomized controlled noninferiority trials published in the five highest-impact general medical journals between 2011 and 2016. MAIN MEASURES Data abstracted included trial design parameters, results, and interpretation of results based on CONSORT recommendations. KEY RESULTS One hundred sixty-three trials and 182 noninferiority comparisons were included in our analysis. Based on CONSORT-recommended interpretation, 79% of experimental therapies met criteria for noninferiority, 13% met criteria for superiority, 20% were declared inconclusive, and 2% met criteria for inferiority. However, for 12% of trials, the experimental therapy was statistically significantly worse than the active control, but CONSORT recommended an interpretation of inconclusive or noninferior. A two-sided alpha equivalent of greater than 0.05 was used in 34% of the trials, and in five of these trials, the use of a standard two-sided alpha of 0.05 led to changes in the interpretation of results that disfavored the experimental therapy. In four of the five comparisons where different methods of analysis (e.g., intention-to-treat and per-protocol) yielded different results, the intention-to-treat analysis was the more conservative. In 11% of trials, a secondary advantage of the new therapy was neither reported nor could it be inferred by reviewers. CONCLUSIONS In this cohort, the design and interpretation of noninferiority trials led to significant and systematic bias in favor of the experimental therapy. Clinicians should exercise caution when interpreting these trials. Future trials may be more reliable if design parameters are standardized.
Collapse
|
46
|
Harhay MO, Ratcliffe SJ, Halpern SD. Measurement Error Due to Patient Flow in Estimates of Intensive Care Unit Length of Stay. Am J Epidemiol 2017; 186:1389-1395. [PMID: 28605399 DOI: 10.1093/aje/kwx222] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2016] [Accepted: 04/17/2017] [Indexed: 12/17/2022] Open
Abstract
Clinical endpoints measured in terms of duration, such as intensive care unit (ICU) length of stay (LOS), are widely used in randomized clinical trials (RCTs) and observational research. In analyses of patient-level data from a recent RCT, in which ICU LOS was the primary endpoint, and in administrative data, we showed that additional ICU time is often accrued by patients after they are deemed ready for discharge. This "immutable" time (which cannot plausibly be altered by interventions under study) varies by day, week, and year, adding on average one-third of a day to total LOS. We then used statistical simulations, informed by the administrative data and RCT, to assess the impact of immutable time on the measurement and statistical comparison of patients' ICU LOS. These simulations demonstrated that immutable time combines with clinically necessary ICU time (neither of which is likely to be normally distributed) to produce overall LOS distributions that might either mask true treatment effects or suggest false treatment effects relative to analyses of time to discharge readiness. The extent and direction of bias were complex functions of the statistical method used, mortality rates and distributions, and the magnitude of immutable time relative to intervention-associated reductions in LOS.
Collapse
Affiliation(s)
- Michael O Harhay
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Sarah J Ratcliffe
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Scott D Halpern
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
47
|
Scales DC, Cheskes S, Verbeek PR, Pinto R, Austin D, Brooks SC, Dainty KN, Goncharenko K, Mamdani M, Thorpe KE, Morrison LJ. Prehospital cooling to improve successful targeted temperature management after cardiac arrest: A randomized controlled trial. Resuscitation 2017; 121:187-194. [PMID: 28988962 DOI: 10.1016/j.resuscitation.2017.10.002] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2017] [Revised: 09/25/2017] [Accepted: 10/02/2017] [Indexed: 12/22/2022]
Abstract
RATIONALE Targeted temperature management (TTM) improves survival with good neurological outcome after out-of-hospital cardiac arrest (OHCA), but is delivered inconsistently and often with delay. OBJECTIVE To determine if prehospital cooling by paramedics leads to higher rates of 'successful TTM', defined as achieving a target temperature of 32-34°C within 6h of hospital arrival. METHODS Pragmatic RCT comparing prehospital cooling (surface ice packs, cold saline infusion, wristband reminders) initiated 5min after return of spontaneous circulation (ROSC) versus usual resuscitation and transport. The primary outcome was rate of 'successful TTM'; secondary outcomes were rates of applying TTM in hospital, survival with good neurological outcome, pulmonary edema in emergency department, and re-arrest during transport. RESULTS 585 patients were randomized to receive prehospital cooling (n=279) or control (n=306). Prehospital cooling did not increase rates of 'successful TTM' (30% vs 25%; RR, 1.17; 95% confidence interval [CI] 0.91-1.52; p=0.22), but increased rates of applying TTM in hospital (68% vs 56%; RR, 1.21; 95%CI 1.07-1.37; p=0.003). Survival with good neurological outcome (29% vs 26%; RR, 1.13, 95%CI 0.87-1.47; p=0.37) was similar. Prehospital cooling was not associated with re-arrest during transport (7.5% vs 8.2%; RR, 0.94; 95%CI 0.54-1.63; p=0.83) but was associated with decreased incidence of pulmonary edema in emergency department (12% vs 18%; RR, 0.66; 95%CI 0.44-0.99; p=0.04). CONCLUSIONS Prehospital cooling initiated 5min after ROSC did not increase rates of achieving a target temperature of 32-34°C within 6h of hospital arrival but was safe and increased application of TTM in hospital.
Collapse
Affiliation(s)
- D C Scales
- Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada; Interdepartmental Division of Critical Care, Department of Medicine, University of Toronto, Toronto, Ontario, Canada; Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada; Institute of Clinical and Evaluative Sciences, Toronto, Ontario, Canada.
| | - S Cheskes
- Division of Emergency Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada; Sunnybrook Centre for Prehospital Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | - P R Verbeek
- Division of Emergency Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada; Sunnybrook Centre for Prehospital Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | - R Pinto
- Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | - D Austin
- Department of Emergency Medicine, Markham Stouffville Hospital, Markham, Ontario, Canada
| | - S C Brooks
- Department of Emergency Medicine, Faculty of Health Sciences Queen's University, Kingston, Ontario, Canada; Rescu, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
| | - K N Dainty
- Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada; Rescu, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
| | - K Goncharenko
- Rescu, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
| | - M Mamdani
- Applied Health Research Centre, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
| | - K E Thorpe
- Applied Health Research Centre, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada; Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - L J Morrison
- Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada; Division of Emergency Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada; Rescu, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
| | | |
Collapse
|
48
|
Latour-Pérez J. Clinical research in critical care. Difficulties and perspectives. Med Intensiva 2017; 42:184-195. [PMID: 28943024 DOI: 10.1016/j.medin.2017.07.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2017] [Revised: 07/10/2017] [Accepted: 07/27/2017] [Indexed: 12/30/2022]
Abstract
In the field of Intensive Care Medicine, improved survival has resulted from better patient care, the early detection of clinical deterioration, and the prevention of iatrogenic complications, while research on new treatments has been followed by an overwhelming number of disappointments. The origins of these fiascos must be sought in the conjunction of methodological problems - common to other disciplines - and the particularities of critically ill patients. The present article discusses both aspects and suggests some options for progress.
Collapse
Affiliation(s)
- J Latour-Pérez
- Servicio de Medicina Intensiva, Hospital General Universitario de Elche, Elche, España; Departamento de Medicina Clínica, Universidad Miguel Hernández, Sant Joan d'Alacant, España.
| |
Collapse
|
49
|
Ridgeon EE, Bellomo R, Aberegg SK, Sweeney RM, Varughese RS, Landoni G, Young PJ. Effect sizes in ongoing randomized controlled critical care trials. CRITICAL CARE : THE OFFICIAL JOURNAL OF THE CRITICAL CARE FORUM 2017; 21:132. [PMID: 28583149 PMCID: PMC5460326 DOI: 10.1186/s13054-017-1726-x] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/09/2017] [Accepted: 05/22/2017] [Indexed: 02/06/2023]
Abstract
Background An important limitation of many critical care trial designs is that they hypothesize large, and potentially implausible, reductions in mortality. Interpretation of trial results could be improved by systematic assessment of the plausibility of trial hypotheses; however, such assessment has not been attempted in the field of critical care medicine. The purpose of this study was to determine clinicians’ views about prior probabilities and plausible effect sizes for ongoing critical care trials where the primary endpoint is landmark mortality. Methods We conducted a systematic review of clinical trial registries in September 2015 to identify ongoing critical care medicine trials where landmark mortality was the primary outcome, followed by a clinician survey to obtain opinions about ten large trials. Clinicians were asked to estimate the probability that each trial would demonstrate a mortality effect equal to or larger than that used in its sample size calculations. Results Estimates provided by individual clinicians varied from 0% to 100% for most trials, with a median estimate of 15% (IQR 10–20%). The median largest absolute mortality reduction considered plausible was 4.5% (IQR 3.5–5%), compared with a median absolute mortality reduction used in sample size calculations of 5% (IQR 3.6–10%) (P = 0.27). Conclusions For some of the largest ongoing critical care trials, many clinicians regard prior probabilities as low and consider that plausible effects on absolute mortality are less than 5%. Further work is needed to determine whether pooled estimates obtained by surveying clinicians are replicable and accurate or whether other methods of estimating prior probability are preferred. Electronic supplementary material The online version of this article (doi:10.1186/s13054-017-1726-x) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
| | - Rinaldo Bellomo
- Australian and New Zealand Intensive Care Research Centre, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia.,Intensive Care Unit, Austin Hospital, Melbourne, Australia
| | - Scott K Aberegg
- Division of Pulmonary and Critical Care Medicine, University of Utah School of Medicine, Salt Lake City, UT, USA
| | | | | | - Giovanni Landoni
- Department of Anesthesia and Intensive Care, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), San Raffaele Scientific Institute, Milan, Italy.,Vita-Salute San Raffaele University, Milan, Italy
| | - Paul J Young
- Intensive Care Unit, Wellington Regional Hospital, Wellington, New Zealand.
| |
Collapse
|
50
|
Interventions affecting mortality in critically ill and perioperative patients: A systematic review of contemporary trials. J Crit Care 2017; 41:107-111. [PMID: 28505485 DOI: 10.1016/j.jcrc.2017.05.005] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2017] [Accepted: 05/03/2017] [Indexed: 11/22/2022]
Abstract
PURPOSE Confounders in randomized controlled trials (RCTs) reporting significant effects on mortality in critically ill patients using non-surgical techniques have not been systematically explored. We aimed to identify factors unrelated to the reported intervention that might have affected the findings and robustness of such trials. METHODS We searched Pubmed/MEDLINE for all RCTs on any non-surgical interventions reporting an effect on unadjusted mortality in critically ill patients between 1/1/2000 and 1/12/2015. We assessed: the number needed to treat/harm (NNT or NNH), sample size, trial design (blinded/unblinded, single or multinational, single or multicenter (sRCT or mRCT)), intention to treat (ITT) analysis, and countries of origin. RESULTS Almost half of RCTs were sRCTs. Median sample size was small, and 1/3 were not analyzed according to ITT principle. Lack of ITT analysis was associated with greater effect size (p=0.0028). Harm was more likely in mRCTs (p=0.002) and/or in blinded RCTs (p=0.003). Blinded RCTs had double sample size (p=0.007) and an increased NNT/NNH (p=0.002). Finally, mRCTs had higher NNT (p=0.005) and NNH (p=0.02), and harm was only detected in studies from Western countries (p=0.007). CONCLUSIONS These observations imply that major systematic biases exist and affect trial findings irrespective of the intervention being studied.
Collapse
|