1
|
Naji L, Dennis B, Rodrigues M, Bawor M, Hillmer A, Chawar C, Deck E, Worster A, Paul J, Thabane L, Samaan Z. Assessing fragility of statistically significant findings from randomized controlled trials assessing pharmacological therapies for opioid use disorders: a systematic review. Trials 2024; 25:286. [PMID: 38678289 PMCID: PMC11055220 DOI: 10.1186/s13063-024-08104-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2022] [Accepted: 04/10/2024] [Indexed: 04/29/2024] Open
Abstract
BACKGROUND The fragility index is a statistical measure of the robustness or "stability" of a statistically significant result. It has been adapted to assess the robustness of statistically significant outcomes from randomized controlled trials. By hypothetically switching some non-responders to responders, for instance, this metric measures how many individuals would need to have responded for a statistically significant finding to become non-statistically significant. The purpose of this study is to assess the fragility index of randomized controlled trials evaluating opioid substitution and antagonist therapies for opioid use disorder. This will provide an indication as to the robustness of trials in the field and the confidence that should be placed in the trials' outcomes, potentially identifying ways to improve clinical research in the field. This is especially important as opioid use disorder has become a global epidemic, and the incidence of opioid related fatalities have climbed 500% in the past two decades. METHODS Six databases were searched from inception to September 25, 2021, for randomized controlled trials evaluating opioid substitution and antagonist therapies for opioid use disorder, and meeting the necessary requirements for fragility index calculation. Specifically, we included all parallel arm or two-by-two factorial design RCTs that assessed the effectiveness of any opioid substitution and antagonist therapies using a binary primary outcome and reported a statistically significant result. The fragility index of each study was calculated using methods described by Walsh and colleagues. The risk of bias of included studies was assessed using the Revised Cochrane Risk of Bias tool for randomized trials. RESULTS Ten studies with a median sample size of 82.5 (interquartile range (IQR) 58, 179, range 52-226) were eligible for inclusion. Overall risk of bias was deemed to be low in seven studies, have some concerns in two studies, and be high in one study. The median fragility index was 7.5 (IQR 4, 12, range 1-26). CONCLUSIONS Our results suggest that approximately eight participants are needed to overturn the conclusions of the majority of trials in opioid use disorder. Future work should focus on maximizing transparency in reporting of study results, by reporting confidence intervals, fragility indexes, and emphasizing the clinical relevance of findings. TRIAL REGISTRATION PROSPERO CRD42013006507. Registered on November 25, 2013.
Collapse
Affiliation(s)
- Leen Naji
- Department of Family Medicine, David Braley Health Sciences Centre, McMaster University, 100 Main St W, 3rdFloor, Hamilton, ON, L8P 1H6, Canada.
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
- Department of Medicine, Montefiore Medical Center, New York, NY, USA.
| | - Brittany Dennis
- Department of Medicine, McMaster University, Hamilton, ON, Canada
- Department of Medicine, University of British Columbia, Vancouver, Canada
| | - Myanca Rodrigues
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Monica Bawor
- Department of Medicine, Imperial College Healthcare NHS Trust, London, UK
| | - Alannah Hillmer
- Department of Psychiatry and Behavaioral Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Caroul Chawar
- Physician Assistant Program, University of Toronto, Toronto, ON, Canada
| | - Eve Deck
- Department of Family Medicine, Western University, London, ON, Canada
| | - Andrew Worster
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - James Paul
- Department of Anesthesia, McMaster University, Hamilton, ON, Canada
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Biostatistics Unit, Research Institute at St Joseph's Healthcare, Hamilton, ON, Canada
| | - Zainab Samaan
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Psychiatry and Behavioral Neurosciences, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
2
|
Li KD, Venishetty N, Fernandez AM, Hakam N, Ghaffar U, Gupta S, Patel HV, Breyer BN. Fragility of overactive bladder medication clinical trials: A systematic review. Neurourol Urodyn 2024. [PMID: 38594889 DOI: 10.1002/nau.25468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Accepted: 04/01/2024] [Indexed: 04/11/2024]
Abstract
PURPOSE Overactive bladder (OAB) syndrome significantly impairs quality of life, often necessitating pharmacological interventions with associated risks. The fragility of OAB trial outcomes, as measured by the fragility index (FI: smallest number of event changes to reverse statistical significance) and quotient (FQ: FI divided by total sample size expressed as a percentage), is critical yet unstudied. MATERIALS AND METHODS We conducted a systematic search for randomized controlled trials on OAB medications published between January 2000 and August 2023. Inclusion criteria were trials with two parallel arms reporting binary outcomes related to OAB medications. We extracted trial details, outcomes, and statistical tests employed. We calculated FI and FQ, analyzing associations with trial characteristics through linear regression. RESULTS We included 57 trials with a median sample size of 211 participants and a 12% median lost to follow-up. Most studies investigated anticholinergics (37/57, 65%). The median FI/FQ was 5/3.5%. Larger trials were less fragile (median FI 8; FQ 1.0%) compared to medium (FI: 4; FQ 2.5%) and small trials (FI: 4; FQ 8.3%). Double-blinded studies exhibited higher FQs (median 2.9%) than unblinded trials (6.7%). Primary and secondary outcomes had higher FIs (median 5 and 6, respectively) than adverse events (FI: 4). Each increase in 10 participants was associated with a +0.19 increase in FI (p < 0.001). CONCLUSIONS A change in outcome for a median of five participants, or 3.5% of the total sample size, could reverse the direction of statistical significance in OAB trials. Studies with larger sample sizes and efficacy outcomes from blinded trials were less fragile.
Collapse
Affiliation(s)
- Kevin D Li
- Department of Urology, University of California San Francisco, San Francisco, California, USA
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA
| | - Nikit Venishetty
- Paul L. Foster School of Medicine, Texas Tech University Health Sciences Center, El Paso, Texas, USA
| | - Adrian M Fernandez
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Nizar Hakam
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Umar Ghaffar
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Shiv Gupta
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Hiren V Patel
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Benjamin N Breyer
- Department of Urology, University of California San Francisco, San Francisco, California, USA
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
3
|
Strippoli GFM, Green SC. Actioning the findings of hard endpoint clinical trials as they emerge in the realm of chronic kidney disease care: a review and a call to action. Clin Kidney J 2024; 17:sfae035. [PMID: 38425707 PMCID: PMC10903297 DOI: 10.1093/ckj/sfae035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Indexed: 03/02/2024] Open
Abstract
Fewer than half of patients treated with hemodialysis survive 5 years. Multiple therapeutics are used to address the complications of advanced chronic kidney disease but most have not been found to improve clinical outcomes. Clinical trials of treatment innovations for chronic kidney diseases and dialysis care have been suboptimal in number and quality. Recent trials are changing this trend. Practice and policy change when new evidence emerges remains frequently impeded by resource and organizational constraints and accordingly, clinical practice guidelines are updated years or decades after definitive evidence is produced. Ultimately, practice change in health systems is slow, leading to impaired uptake of effective medical interventions and lower value healthcare, although innovations in rapid guideline production are emerging. What can be done to ensure that conclusive evidence is taken up in practice, policy and healthcare funding? We use the example of the recently published hard endpoint study "Comparison of high-dose HDF with high-flux HD" (CONVINCE) (hemodiafiltration versus hemodialysis), to explain how a new trial can impact on medical knowledge and change in practices. We (i) assess how the trial can be placed in the context of the totality of the evidence, (ii) define whether or not further trials of convective dialysis therapies are still needed and (iii) examine whether the evidence for convective therapies is now ready to inform practice, policy and funding change. When looking at CONVINCE in the context of the totality of evidence, we show that it addresses dialysis quality improvement priorities and is consistent with other trials evaluating convective dialysis therapies, and that the evidence for convective dialysis therapies is now definitive. Once updated evidence for cost-effectiveness in specific healthcare settings and patient-reported outcomes become available, we should therefore determine whether or not clinical practice guidelines should recommend uptake of convective dialysis therapies routinely, and move on to evaluating other treatments.
Collapse
Affiliation(s)
- Giovanni F M Strippoli
- Sydney School of Public Health, The University of Sydney, NSW Australia
- Department of Precision and Regenerative Medicine and Ionian Area (DIMEPRE-J) University of Bari Aldo Moro, Bari, Italy
| | - Suetonia C Green
- Department of Medicine, University of Otago, Christchurch, Christchurch, New Zealand
| |
Collapse
|
4
|
Hayes J, Zuercher M, Gai N, Chowdhury AR, Aoyama K. The Fragility Index of randomized controlled trials in pediatric anesthesiology. Can J Anaesth 2023; 70:1449-1460. [PMID: 37286747 DOI: 10.1007/s12630-023-02513-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 01/16/2023] [Accepted: 01/23/2023] [Indexed: 06/09/2023] Open
Abstract
PURPOSE The P value is a widely used measure of statistical importance but has many drawbacks and limitations, one being that it does not reflect the robustness of the results of a clinical trial. The Fragility Index (FI) was developed as a measure of how many outcome events would need to change to nonevents to render a significant P value nonsignificant (P ≥ 0.05). The FI of trials from other medical specialties is typically < 5. We aimed to determine the FI of pediatric anesthesiology randomized controlled trials (RCT) and to test for association with various characteristics of the included trials. METHODS We conducted a comprehensive systematic search of high-impact anesthesia, surgical, and medical journals from the last 25 years for trials comparing an intervention between two groups with a statistically significant P value (< 0.05) for a dichotomous outcome. We also compared FI values for variables that reflect the quality and importance of a trial. RESULTS The median [interquartile range] FI was 3 [1-7] and correlated positively with the number of participants (rS = 0.41; P < 0.001) and events (rS = 0.42; P < 0.001), and negatively with the P value (rPB = -0.36; P < 0.001). Other measures of trial quality and impact or importance were not strongly associated with the FI. CONCLUSIONS The FI of published trials in pediatric anesthesiology is similarly low as in other medical specialties. Larger trials with more events and P values ≤ 0.01 were associated with a higher FI.
Collapse
Affiliation(s)
- Jason Hayes
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada.
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada.
| | - Mael Zuercher
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
| | - Nan Gai
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
| | - Apala Roy Chowdhury
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
| | - Kazuyoshi Aoyama
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids), 555 University Avenue, Toronto, ON, M5G 1X8, Canada
- Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, ON, Canada
- Program in Child Health Evaluative Sciences, SickKids Research Institute, Toronto, ON, Canada
| |
Collapse
|
5
|
Demarquette A, Perrault T, Alapetite T, Bouizegarene M, Bronnert R, Fouré G, Masson C, Nicolas V, Lasocki S, Léger M. Spin and fragility in randomised controlled trials in the anaesthesia literature: a systematic review. Br J Anaesth 2023; 130:528-535. [PMID: 36759291 DOI: 10.1016/j.bja.2023.01.001] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 12/22/2022] [Accepted: 01/02/2023] [Indexed: 02/10/2023] Open
Abstract
BACKGROUND Given variable frequency of misleading reports and the potential for spin (a way of describing results that can mislead readers) to influence interpretation of randomised controlled trials (RCTs), we have undertaken a spin reassessment. We evaluated the quality of recent literature in anaesthesia journals by assessing the presence of spin and calculating the fragility index. METHODS This systematic review of randomised trials was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. We searched via PubMed® from January 1, 2019 to January 1, 2021 to identify all RCTs published in one of the 20 anaesthesia journals with the highest journal impact factors during this time. Four pairs of reviewers assessed articles independently for eligibility using a piloted electronic data extraction form. They assessed the presence of spin in statistically negative RCTs and calculated the fragility index for statistically positive RCTs. RESULTS Of the 802 screened records, 162 (20%) articles were analysed for spin, and 65 (8%) trials were analysed for fragility index. For the statistically negative studies, 66 articles (40%) presented spin; 89% of these occurrences of spin were described in the conclusion of the abstract. The primary type of spin was the highlight of secondary outcomes (67%). For statistically positive trials, the median fragility index was 4 [1-8]. CONCLUSIONS This systematic review showed that 40% of statistically negative trials in high-impact anaesthesia journals could mislead readers. For statistically positive RCTs, the results relied on few subjects, with a median fragility index of 4 [1-8]. Efforts must be continued to reduce spin and fragility in the medical literature.
Collapse
Affiliation(s)
- Achille Demarquette
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France.
| | - Tristan Perrault
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Thomas Alapetite
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Madjid Bouizegarene
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Romain Bronnert
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Gaël Fouré
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Charline Masson
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Vivian Nicolas
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Sigismond Lasocki
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France
| | - Maxime Léger
- Anaesthesiology and Critical Care Department, Angers University Hospital, Angers, France; INSERM UMR 1246, SPHERE, Nantes University, Tours University, Nantes, France
| |
Collapse
|
6
|
Schröder A, Muensterer OJ, Oetzmann von Sochaczewski C. Paediatric surgical trials, their fragility index, and why to avoid using it to evaluate results. Pediatr Surg Int 2022; 38:1057-1066. [PMID: 35524787 PMCID: PMC9162995 DOI: 10.1007/s00383-022-05133-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/24/2022] [Indexed: 12/30/2022]
Abstract
BACKGROUND The fragility index has been gaining ground in the evaluation of comparative clinical studies. Many scientists evaluated trials in their fields and deemed them to be fragile, although there is no consensus on the definition of fragility. We aimed to calculate the fragility index and its permutations for paediatric surgical trials. METHODS We searched pubmed for prospectively conducted paediatric surgical trials with intervention and control group without limitations and calculated their (reverse) fragility indices and respective quotients along with posthoc-power. Relationships between variables were evaluated using Spearman's ρ. We also calculated S values by negative log transformation base-2 of P values. RESULTS Of 516 retrieved records, we included 87. The median fragility index was 1.5 (interquartile range: 0-4) and the median reverse fragility index was 3 (interquartile range: 2-4), although they were statistically not different (Mood's test: χ2 = 0.557, df = 1, P = 0.4556). P values and fragility indices were strongly inversely correlated (ρ = - 0.71, 95% confidence interval: - 0.53 to - 0.85, P < 0.0001), while reverse fragility indices were moderately correlated to P values (ρ = 0.5, 95% confidence interval: 0.37-0.62, P < 0.0001). A fragility index of 1 resulted from P values between 0.039 and 0.003, which resulted in S values between 4 and 8. CONCLUSIONS Fragility indices, reverse fragility indices, and their respective fragility quotients of paediatric surgical trials are low. The fragility index can be viewed as no more than a transformed P value with even more substantial limitations. Its inherent penalisation of small studies irrespective of their clinical relevance is particularly harmful for paediatric surgery. Consequently, the fragility index should be avoided.
Collapse
Affiliation(s)
- Arne Schröder
- Klinik für Kinder- und Jugendmedizin, Klinikum Dortmund, Dortmund, Germany
| | - Oliver J Muensterer
- Kinderchirurgische Klinik und Poliklinik im Dr. von Haunerschen Kinderspital, Ludwig-Maximilians-Universität München, München, Germany
- Klinik und Poliklinik für Kinderchirurgie, Universitätsmedizin der Johannes-Gutenberg-Universität Mainz, Mainz, Germany
| | - Christina Oetzmann von Sochaczewski
- Klinik und Poliklinik für Kinderchirurgie, Universitätsmedizin der Johannes-Gutenberg-Universität Mainz, Mainz, Germany.
- Sektion Kinderchirurgie der Klinik und Poliklinik für Allgemein, Viszeral, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Venusberg-Campus 1, 53127, Bonn, Germany.
| |
Collapse
|
7
|
Ho AK. The Fragility Index for Assessing the Robustness of the Statistically Significant Results of Experimental Clinical Studies. J Gen Intern Med 2022; 37:206-211. [PMID: 34357573 PMCID: PMC8739402 DOI: 10.1007/s11606-021-06999-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/13/2021] [Accepted: 06/23/2021] [Indexed: 01/03/2023]
Affiliation(s)
- Adrienne K Ho
- Department of Thoracic Oncology, Wythenshawe Hospital, Manchester, UK.
- Present address: Department of Public Health Sciences (Epidemiology), Queen's University, Kingston, Ontario, Canada.
| |
Collapse
|
8
|
Fragility indices for only sufficiently likely modifications. Proc Natl Acad Sci U S A 2021; 118:2105254118. [PMID: 34848537 DOI: 10.1073/pnas.2105254118] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/13/2021] [Indexed: 11/18/2022] Open
Abstract
The fragility index is a clinically meaningful metric based on modifying patient outcomes that is increasingly used to interpret the robustness of clinical trial results. The fragility index relies on a concept that explores alternative realizations of the same clinical trial by modifying patient measurements. In this article, we propose to generalize the fragility index to a family of fragility indices called the incidence fragility indices that permit only outcome modifications that are sufficiently likely and provide an exact algorithm to calculate the incidence fragility indices. Additionally, we introduce a far-reaching generalization of the fragility index to any data type and explain how to permit only sufficiently likely modifications for nondichotomous outcomes. All of the proposed methodologies follow the fragility index concept.
Collapse
|
9
|
Baer BR, Fremes SE, Gaudino M, Charlson M, Wells MT. On clinical trial fragility due to patients lost to follow up. BMC Med Res Methodol 2021; 21:254. [PMID: 34800976 PMCID: PMC8606097 DOI: 10.1186/s12874-021-01446-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 10/19/2021] [Indexed: 11/21/2022] Open
Abstract
BACKGROUND Clinical trials routinely have patients lost to follow up. We propose a methodology to understand their possible effect on the results of statistical tests by altering the concept of the fragility index to treat the outcomes of observed patients as fixed but incorporate the potential outcomes of patients lost to follow up as random and subject to modification. METHODS We reanalyse the statistical results of three clinical trials on coronary artery bypass grafting (CABG) to study the possible effect of patients lost to follow up on the treatment effect statistical significance. To do so, we introduce the LTFU-aware fragility indices as a measure of the robustness of a clinical trial's statistical results with respect to patients lost to follow up. RESULTS The analyses illustrate that clinical trials can either be completely robust to the outcomes of patients lost to follow up, extremely sensitive to the outcomes of patients lost to follow up, or in an intermediate state. When a clinical trial is in an intermediate state, the LTFU-aware fragility indices provide an interpretable measure to quantify the degree of fragility or robustness. CONCLUSIONS The LTFU-aware fragility indices allow researchers to rigorously explore the outcomes of patients who are lost to follow up, when their data is the appropriate kind. The LTFU-aware fragility indices are sensitivity measures in a way that the original fragility index is not.
Collapse
Affiliation(s)
- Benjamin R. Baer
- Department of Statistics and Data Science, Cornell University, Ithaca, NY US
| | - Stephen E. Fremes
- Schulich Heart Centre, Sunnybrook Health Science Centre, University of Toronto, Toronto, ON Canada
| | - Mario Gaudino
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY US
| | - Mary Charlson
- Department of Medicine, Weill Cornell Medicine, Weill Cornell Medicine, New York, NY US
| | - Martin T. Wells
- Department of Statistics and Data Science, Cornell University, Ithaca, NY US
- Department of Medicine, Weill Cornell Medicine, Weill Cornell Medicine, New York, NY US
| |
Collapse
|
10
|
Baer BR, Gaudino M, Fremes SE, Charlson M, Wells MT. The fragility index can be used for sample size calculations in clinical trials. J Clin Epidemiol 2021; 139:199-209. [PMID: 34403756 DOI: 10.1016/j.jclinepi.2021.08.010] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Revised: 08/09/2021] [Accepted: 08/11/2021] [Indexed: 12/14/2022]
Abstract
OBJECTIVE The fragility index is a clinically interpretable metric increasingly used to interpret the robustness of clinical trials results that is generally not incorporated in sample size calculation and applied post-hoc. In this manuscript, we propose to base the sample size calculation on the fragility index in a way that supplements the classical prefixed alpha and power cutoffs and we provide a dedicated R software package for the design and analysis tools. STUDY DESIGN AND SETTING This approach follows from a novel hypothesis testing framework that is based on the fragility index and builds on the classical testing approach. As case studies, we re-analyse the design of two important trials in cardiovascular medicine, the FAME and FAMOUS-NSTEMI trials. RESULTS The analyses show that approach returns sample sizes which results in a higher power for the P value based test and most importantly a lower and context dependent Type I error rate for the fragility index based test compared to standard tests. CONCLUSION Our method allows clinicians to control for the fragility index during clinical trial design.
Collapse
Affiliation(s)
- Benjamin R Baer
- Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA.
| | - Mario Gaudino
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Stephen E Fremes
- Schulich Heart Centre, Sunnybrook Health Science, University of Toronto, Toronto, Ontario, Canada
| | - Mary Charlson
- Department of Medicine, Weill Cornell Medicine, New York, NY, USA
| | - Martin T Wells
- Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA; Department of Medicine, Weill Cornell Medicine, New York, NY, USA
| |
Collapse
|