1
|
Bai X, Wan Z, Li Y, Jiang Q, Wu X, Xu B, Li X, Zhou R, Mi J, Sun Y, Ruan G, Han W, Li G, Yang H. Fragility index analysis for randomized controlled trials of approved biologicals and small molecule drugs in inflammatory bowel diseases. Int Immunopharmacol 2024; 130:111752. [PMID: 38422772 DOI: 10.1016/j.intimp.2024.111752] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 01/30/2024] [Accepted: 02/21/2024] [Indexed: 03/02/2024]
Abstract
INTRODUCTION Biologics and small molecules have been increasingly applied in Crohn's disease (CD) and ulcerative colitis (UC). But the robustness of their trials has not been evaluated. METHODS We initially collected all the approved biologics or small molecules for CD or UC up to December 1, 2022. Databases were then queried by keywords in chemical name and CD or UC. Randomized controlled trials (RCTs) in the two-arm, 1:1 design were included. Fragility index (FI) and fragility quotient (FQ) were subsequently calculated. RESULTS We included twenty-eight RCTs, including nine pivotal trials listed in approval labels, nineteen non-pivotal trials not included in the labels. The median sample size was 99 [IQR, 60-262] and the median number of loss-of-follow-up (LFU) was 14 [IQR, 8-43]. Pivotal trials in the labels had the median FI of 8 [IQR, 4-14, n = 6] that was marginally higher than non-pivotal trials (3 [IQR, 2-4], p = 0.08). The median FQ was 0.0330 [IQR, 0.1220-0.0466] and 0.0310 [IQR, 0.0129-0.0540] for pivotal and non-pivotal trials, respectively (p = 1.0). The sample size and FI were significantly correlated (Spearman correlation coefficient [r] = 0.56, 95 %CI 0.21-0.78, p = 0.003). The number of total events was also significantly correlated with FI (r = 0.53, 95 %CI 0.17-0.77, p = 0.006). Study p-values were significantly associated with FI (p = 0.01): trials with p-values < 0.001 had the highest median FI of 10 [IQR, 6-17]. No factor was found strongly correlated with FQ. CONCLUSION Results from trials assessing administration-approved biologics or small molecules for treating CD or UC were vulnerable to small changes by measuring FI or FQ. Pivotal studies contributing to regulatory approvals exhibited a relatively higher degree of resilience compared to non-pivotal trials.
Collapse
Affiliation(s)
- Xiaoyin Bai
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Ziqi Wan
- Eight-year Program, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing, China
| | - Yi Li
- Tsinghua Clinical Research Institute, School of Medicine, Tsinghua University, Beijing, China
| | - Qingwei Jiang
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Xia Wu
- Department of Medicine, Tufts Medical Center, Boston, MA 02111, USA
| | | | | | - Runing Zhou
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Jiarui Mi
- Department of Cell and Molecular Biology, Karolinska Institutet, Solna, Sweden
| | - Yinghao Sun
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Gechong Ruan
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Wei Han
- Institute of Basic Medical Sciences, School of Basic Medicine, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
| | | | - Hong Yang
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
| |
Collapse
|
2
|
Placer-Galán C, Arocena EB. Fragility Index, Reverse Fragility Index, Fragility Quotients, and Susceptibility Index. Novel tools for assessing of randomized clinical trials. Cir Esp 2024; 102:41-43. [PMID: 37949365 DOI: 10.1016/j.cireng.2023.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 05/29/2023] [Indexed: 11/12/2023]
Affiliation(s)
- Carlos Placer-Galán
- Servicio de Cirugía General y Digestiva, Hospital Universitario Donostia, San Sebastián, Spain.
| | - Elisabet Bollo Arocena
- Servicio de Cirugía General y Digestiva, Hospital Universitario Donostia, San Sebastián, Spain
| |
Collapse
|
3
|
Anderson RM, Peña A, Johnson BS, Nowlin R, Hudson T, Vassar M. What's in a P-Value? A Fragility Analysis of RCTs in the AUA Guidelines for Benign Prostatic Hyperplasia. Urology 2023; 176:127-136. [PMID: 36894031 DOI: 10.1016/j.urology.2022.12.063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 12/28/2022] [Indexed: 03/09/2023]
Abstract
OBJECTIVE To use the fragility index (FI) and fragility quotient (FQ) to assess the strength of statistically significant findings for randomized controlled trials (RCTs) cited in the American Urological Association (AUA) guidelines for benign prostatic hyperplasia. METHODS Two investigators independently screened the AUA guidelines for management of benign prostatic hyperplasia for RCTs cited as evidence for recommendations. Investigators extracted data related to event rate per group and loss to follow-up which was compared against the FI. Stata 17.0 was used to calculate the FI and FQ which was then summarized and reported according to primary or secondary endpoints. RESULTS Among the 373 citations in the AUA guidelines, 24 RCTs met inclusion criteria with 29 distinct outcomes analyzed. The median fragility index was 12 (IQR = 4-38), indicating that twelve alternative events to either study arm would nullify statistical significance. Six studies had a FI of ≤2, indicating that only 1-2 outcomes would need to be changed in order to render nonsignificance of results. In 10/24 RCTs, the number of patients lost to follow-up was greater than the FI. CONCLUSION The AUA Clinical Practice Guidelines for management of benign prostatic hyperplasia cite RCTs with more robust findings when compared to previous studies assessing fragility in the field of Urology. While several included studies had high fragility, the median FI in our analysis was approximately 4-5 times higher than comparable studies of urologic RCTs. However, there are areas where improvement is necessary to support the highest quality of evidence-based medicine.
Collapse
Affiliation(s)
- Reece M Anderson
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK
| | - Andriana Peña
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK
| | - Bradley S Johnson
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK; Department of Urology, Sparrow Hospital, Lansing, MI; Michigan State Urology, Michigan State University, East Lansing, MI
| | - Ross Nowlin
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK
| | - Tillman Hudson
- Department of Urology, St. Francis Health System, Tulsa, OK
| | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK; Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK
| |
Collapse
|
4
|
Ho AK. The Fragility Index for Assessing the Robustness of the Statistically Significant Results of Experimental Clinical Studies. J Gen Intern Med 2022; 37:206-211. [PMID: 34357573 PMCID: PMC8739402 DOI: 10.1007/s11606-021-06999-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/13/2021] [Accepted: 06/23/2021] [Indexed: 01/03/2023]
Affiliation(s)
- Adrienne K Ho
- Department of Thoracic Oncology, Wythenshawe Hospital, Manchester, UK.
- Present address: Department of Public Health Sciences (Epidemiology), Queen's University, Kingston, Ontario, Canada.
| |
Collapse
|
5
|
Williams MO, Sedarous M, Dennis B, Dlamini V, Nwaiwu O, Nguyen L, Okafor PN. The fragility of randomized placebo-controlled trials for irritable bowel syndrome. Neurogastroenterol Motil 2021; 33:e14166. [PMID: 34399023 DOI: 10.1111/nmo.14166] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/26/2020] [Revised: 02/21/2021] [Indexed: 12/20/2022]
Abstract
BACKGROUND The fragility index (FI) represents the number of participants whose status in a trial would have to change from a non-event (not experiencing the primary endpoint) to an event (experiencing the primary endpoint) in order to turn a statistically significant result into a non-significant result. We sought to evaluate the fragility indices of irritable bowel syndrome [IBS-mixed (IBS-M), IBS-constipation (IBS-C), & IBS-diarrhea (IBS-D)] trials. METHODS Irritable bowel syndrome trials published in high-impact journals were identified from Medline. Trials had to be in adults, randomized, parallel-armed, with at least one statistically significant binary outcome, and an achieved primary endpoint of therapeutic efficacy. FI and correlation coefficients were calculated, and regression modeling used to identify predictors of a high FI. KEY RESULTS Twelve trials were analyzed with a median FI of 6 (range: 0-38). Median sample size in all trials was 366 (range: 44-856). Trial publication year (p = 0.71), journal impact factor (p = 0.52), duration of study (p = 0.12), and number need to treat [NNT] (p = 0.29) were not predictive of a high FI. While a lower p-value correlated with a higher FI (p = 0.039), no correlation was noted between FI and impact factor (R = -0.20, p = 0.52), trial publication year (R = 0.12, p = 0.71), duration of trial (R = -0.46, p = 0.13), NNT (R = -0.34, p = 0.29), and sample size (R = 0.23, p = 0.5). The highest FI was in a Ramosetron trial (FI = 30) for IBS-D. CONCLUSION & INFERENCES A median of six participants is needed to nullify results in the included IBS trials suggesting how easily statistical significance based on a threshold p-value may be overturned.
Collapse
Affiliation(s)
- MaryJane O Williams
- Division of Gastroenterology, Hepatology and Nutrition, Vidant Medical Center, East Carolina University, Greenville, North Carolina, USA
| | - Mary Sedarous
- Department of Medicine, McMaster University, Hamilton, Ontorio, Canada
| | - Brittany Dennis
- Department of Medicine, McMaster University, Hamilton, Ontorio, Canada
| | - Vanessa Dlamini
- Meharry Medical College School of Medicine, Nashville, Tennessee, USA
| | - Obioma Nwaiwu
- Department of Medicine, University of Arkansas School of Medical Sciences, Fayetteville, Arkansas, USA
| | - Linda Nguyen
- Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California, USA
| | - Philip N Okafor
- Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California, USA
| |
Collapse
|
6
|
Tzelves L, Chatzikrachtis N, Lazarou L, Mourmouris P, Pinitas A, Tsirkas K, Petropoulos O, Berdempes M, Feretzakis G, Glykas I, Fragkoulis C, Varkarakis I, Skolarikos A. Fragility index of urological literature regarding medical expulsive treatment. World J Urol 2021; 39:3741-3746. [PMID: 33978811 DOI: 10.1007/s00345-021-03725-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Accepted: 05/03/2021] [Indexed: 12/23/2022] Open
Abstract
INTRODUCTION The role of medical expulsive treatment (MET) is controversial. Fragility index is an additional metric to assess randomized controlled trials (RCTs) outcome validity and indicates how many patients would be required to convert a trial from being statistically significant, to not significant. The larger is the FI, the better the trial's data. The aim of this study is to assess FI of RCTs regarding MET for ureteral stones. MATERIALS AND METHODS A systematic literature search was performed. RCTs, reporting stone expulsion as a dichotomous outcome, showing statistical significance were eligible. FI (the number of patients needed to change from a non-event to event group, to lose statistical significance) and Fragility quotient (FI divided by total sample size), were calculated while Pearson's correlation and Mann-Whitney U test were used as appropriate. RESULTS Thirty-six RCTs were eligible, with median FI = 3.5 and fragility quotient = 0.042, median sample size = 81, median journal impact factor = 1.73 and median reported p value = 0.008. In 33.3% of the studies, number of patients lost during follow-up was larger than FI, while in 13.89% of the studies, FI was 0, indicating use of inappropriate statistical method. Pearson's correlation showed significant positive association between FI and sample size (r = 0.981), number of events (r = 0.982) and impact factor (r = 0.731), while no association was found with p value or publication year. CONCLUSIONS In this analysis, a calculated FI of 3.5 indicates that findings from RCTs on MET for ureteral stones are fragile and should be interpreted in combination with clinical thinking and expertise.
Collapse
Affiliation(s)
- Lazaros Tzelves
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Nikolaos Chatzikrachtis
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Lazaros Lazarou
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Panagiotis Mourmouris
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Alexandros Pinitas
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Kimon Tsirkas
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Orestis Petropoulos
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Marinos Berdempes
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Georgios Feretzakis
- Department of Quality Control, Research and Continuing Education, Sismanogleio General Hospital, 15126, Marousi, Greece
| | - Ioannis Glykas
- Department of Urology, General Hospital of Athens "G. Gennimatas", Athens, Greece.
| | | | - Ioannis Varkarakis
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| | - Andreas Skolarikos
- Second Department of Urology, Sismanogleio General Hospital, National and Kapodistrian University of Athens, Athens, Greece
| |
Collapse
|
7
|
Zhang H, Li J, Zeng W. Frequent fragility of randomized controlled trials for HCC treatment. BMC Cancer 2021; 21:389. [PMID: 33836710 PMCID: PMC8034173 DOI: 10.1186/s12885-021-08133-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Accepted: 12/16/2020] [Indexed: 11/18/2022] Open
Abstract
Background The fragility index (FI) of trial results can provide a measure of confidence in the positive effects reported in randomized controlled trials (RCTs). The aim of this study was to calculate the FI of RCTs supporting HCC treatments. Methods A methodological systematic review of RCTs in HCC treatments was conducted. Two-arm studies with randomized and positive results for a time-to-event outcome were eligible for the FI calculation. Results A total of 6 trails were included in this analysis. The median FI was 0.5 (IQR 0–10). FI was ≤7 in 4 (66.7%) of 6 trials; in those trials the fragility quotient was ≤1%. Conclusion Many phase 3 RCTs supporting HCC treatments have a low FI, which challenges the confidence in concluding the superiority of these drugs over control treatments. Supplementary Information The online version contains supplementary material available at 10.1186/s12885-021-08133-8.
Collapse
Affiliation(s)
- Hao Zhang
- Department of Infectious Diseases, The Key Discipline of Gguangdong Province, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou Medical University, #151 Yanjiang Road, Guangzhou, 510120, Guangdong Province, China
| | - Jingtao Li
- Department of liver diseases (I), The Hospital Affiliated to Shaanxi University of Chinese Medicine, Xianyang, 712000, Shaanxi Province, China.
| | - Wenting Zeng
- Department of Infectious Diseases, The Key Discipline of Gguangdong Province, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou Medical University, #151 Yanjiang Road, Guangzhou, 510120, Guangdong Province, China.
| |
Collapse
|
8
|
Grammatikopoulou MG, Nigdelis MP, Theodoridis X, Gkiouras K, Tranidou A, Papamitsou T, Bogdanos DP, Goulis DG. How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials. BMJ Nutr Prev Health 2021; 4:115-131. [PMID: 34308119 PMCID: PMC8258081 DOI: 10.1136/bmjnph-2020-000188] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 02/04/2021] [Accepted: 02/13/2021] [Indexed: 11/25/2022] Open
Abstract
INTRODUCTION The Mediterranean diet (MD) is a traditional regional dietary pattern and a healthy diet recommended for the primary and secondary prevention of various diseases and health conditions. Results from the higher level of primary evidence, namely randomised controlled trials (RCTs), are often used to produce dietary recommendations; however, the robustness of RCTs with MD interventions is unknown. METHODS A systematic search was conducted and all MD RCTs with dichotomous primary outcomes were extracted from PubMed. The fragility (FI) and the reverse fragility index (RFI) were calculated for the trials with significant and non-significant comparisons, respectively. RESULTS Out of 27 RCTs of parallel design, the majority failed to present a significant primary outcome, exhibiting an FI equal to 0. The median FI of the significant comparisons was 5, ranging between 1 and 39. More than half of the comparisons had an FI <5, indicating that the addition of 1-4 events to the treatment arm eliminated the statistical significance. For the comparisons with an FI=0, the RFI ranged between 1 and 29 (Median RFI: 7). When the included RCTs were stratified according to masking, the use of a composite primary endpoint, sample size, outcome category, or dietary adherence assessment method, no differences were exhibited in the FI and RFI between groups, except for the RFI among different compliance assessment methods. CONCLUSIONS In essence, the present study shows that even in the top tiers of evidence hierarchy, research on the MD may lack robustness, setting concerns for the formulation of nutrition recommendations.
Collapse
Affiliation(s)
- Maria G Grammatikopoulou
- Unit of Reproductive Endocrinology, 1st Department of Obstetrics and Gynecology, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
- Rheumatology and Clinical Immunology, Faculty of Health Sciences, University of Thessaly, Larissa, Greece
- Nutritional Sciences & Dietetics, Faculty of Health Sciences, International Hellenic University, Thessaloniki, Greece
| | - Meletios P Nigdelis
- Unit of Reproductive Endocrinology, 1st Department of Obstetrics and Gynecology, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
| | - Xenophon Theodoridis
- Rheumatology and Clinical Immunology, Faculty of Health Sciences, University of Thessaly, Larissa, Greece
- Medical School, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
| | - Konstantinos Gkiouras
- Rheumatology and Clinical Immunology, Faculty of Health Sciences, University of Thessaly, Larissa, Greece
- Medical School, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
| | - Antigoni Tranidou
- Medical School, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
- Department of Endocrinology, Diabetes and Metabolism, Hippokration General Hospital of Thessaloniki, Thessaloniki, Central Macedonia, Greece
| | - Theodora Papamitsou
- Laboratory of Histology and Embryology, Medical School, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
| | - Dimitrios P Bogdanos
- Rheumatology and Clinical Immunology, Faculty of Health Sciences, University of Thessaly, Larissa, Greece
- Division of Transplantation, Immunology and Mucosal Biology, MRC Centre for Transplantation, School of Medical Education, King's College London, London, UK
| | - Dimitrios G Goulis
- Unit of Reproductive Endocrinology, 1st Department of Obstetrics and Gynecology, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Central Macedonia, Greece
| |
Collapse
|
9
|
Choupoo NS, Das SK, Saikia P, Dey S, Ray S. How Robust are the Evidences that Formulate Surviving Sepsis Guidelines? An Analysis of Fragility and Reverse Fragility of Randomized Controlled Trials that were Referred in these Guidelines. Indian J Crit Care Med 2021; 25:773-779. [PMID: 34316171 PMCID: PMC8286372 DOI: 10.5005/jp-journals-10071-23895] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022] Open
Abstract
Objectives “Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016” provides guidelines in regard to prompt management and resuscitation of sepsis or septic shock. The study is aimed to assess the robustness of randomized controlled trials (RCTs) that formulate these guidelines in terms of fragility index and reverse fragility index. Method RCTs that contributed to these guidelines having parallel two-group design, 1:1 allocation ratio, and at least one dichotomous outcome were included in the study. The median fragility index was calculated for RCTs with significant statistical outcomes, whereas the median reverse fragility index was calculated for RCTs with nonsignificant statistical results. Results Hundred RCTs that met the inclusion criteria were analyzed. The median fragility index was 5.5 [95% confidence interval (CI) 1–30] and median reverse fragility index was 13 (95% CI 12.07–16.8) at a p value of 0.05. The median reverse fragility index was 16 (95% CI 10–26) at a p value of 0.01. Most of the RCTs included in this analysis were of good quality, having a median Jadad score of 6. Conclusion This analysis found that the surviving sepsis guidelines were based on highly robust RCTs with statistically insignificant results and on some moderately robust RCTs with statistically significant results. RCTs with statistically insignificant results were more robust than RCTs with statistically significant results in regard to these guidelines. Highlights The study assessed the robustness of randomized controlled trials (RCTs) that were used to formulate surviving sepsis guidelines. Most RCTs showed statistically nonsignificant results. RCTs with statistically significant results were moderately fragile whereas RCTs with nonsignificant results were more robust. How to cite this article Choupoo NS, Das SK, Saikia P, Dey S, Ray S. How Robust are the Evidences that Formulate Surviving Sepsis Guidelines? An Analysis of Fragility and Reverse Fragility of Randomized Controlled Trials that were Referred in these Guidelines. Indian J Crit Care Med 2021;25(7):773–779.
Collapse
Affiliation(s)
- Nang S Choupoo
- Department of Anesthesia, Atal Bihari Vajpayee Medical Institute and Dr RML Hospital, Delhi, India
| | - Saurabh K Das
- Department of Critical Care Medicine, Artemis Hospital, Gurugram, Haryana, India
| | - Priyam Saikia
- Department of Anaesthesiology and Critical Care, Gauhati Medical College and Hospital, Guwahati, Assam, India
| | - Samarjit Dey
- Department of Anaesthesia and Critical Care, AIIMS, Raipur, Chhattisgarh, India
| | - Sumit Ray
- Department of Critical Care, Holy Family Hospital, Delhi, India
| |
Collapse
|
10
|
Potter GE. Dismantling the Fragility Index: A demonstration of statistical reasoning. Stat Med 2020; 39:3720-3731. [PMID: 32781488 DOI: 10.1002/sim.8689] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 05/09/2020] [Accepted: 06/11/2020] [Indexed: 12/22/2022]
Abstract
The Fragility Index has been introduced as a complement to the P-value to summarize the statistical strength of evidence for a trial's result. The Fragility Index (FI) is defined in trials with two equal treatment group sizes, with a dichotomous or time-to-event outcome, and is calculated as the minimum number of conversions from nonevent to event in the treatment group needed to shift the P-value from Fisher's exact test over the .05 threshold. As the index lacks a well-defined probability motivation, its interpretation is challenging for consumers. We clarify what the FI may be capturing by separately considering two scenarios: (a) what the FI is capturing mathematically when the probability model is correct and (b) how well the FI captures violations of probability model assumptions. By calculating the posterior probability of a treatment effect, we show that when the probability model is correct, the FI inappropriately penalizes small trials for using fewer events than larger trials to achieve the same significance level. The analysis shows that for experiments conducted without bias, the FI promotes an incorrect intuition of probability, which has not been noted elsewhere and must be dispelled. We illustrate shortcomings of the FI's ability to quantify departures from model assumptions and contextualize the FI concept within current debate around the null hypothesis significance testing paradigm. Altogether, the FI creates more confusion than it resolves and does not promote statistical thinking. We recommend against its use. Instead, sensitivity analyses are recommended to quantify and communicate robustness of trial results.
Collapse
|
11
|
Understanding the fragility index in experimental clinical studies: An example using the meta-analysis of compression-only v. conventional CPR in out-of-hospital cardiac arrest. CAN J EMERG MED 2020; 22:633-636. [PMID: 32342833 DOI: 10.1017/cem.2020.35] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|