1
|
Williamson TK, Martinez VH, Aziz AW, Kotzur T, Verlinsky L, Buttacavoli FA. Aspirin in prevention of venous thromboembolism following hip fracture surgery: A systematic review and meta-analysis. J Orthop 2024; 58:75-81. [PMID: 39070114 PMCID: PMC11269802 DOI: 10.1016/j.jor.2024.06.039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/17/2024] [Revised: 06/24/2024] [Accepted: 06/26/2024] [Indexed: 07/30/2024] Open
Abstract
Background Many orthopaedic surgeons routinely prescribe aspirin (ASA) as prophylaxis for venous thromboembolism (VTE) following hip fracture surgery (HFS). The purpose of this study is to assess the effectiveness of aspirin to other agents in preventing VTE and mortality following hip fracture surgery. Methods Following PRISMA guidelines, we performed a search for HFS studies from 1998 to 2023 reporting comparisons between aspirin and other chemoprophylaxis methods for VTE (DVT - deep vein thrombosis; PE - pulmonary embolism). SPSS Meta-analysis function was used to calculate Mean Effect Size Estimate (MESE) and 95 % Confidence Intervals for each outcome. Reverse Fragility Index (RFI) and Fragility Quotient (FQ) were calculated for each study. Results Of the 847 articles screened, 4 studies with 5 comparisons met the search criteria to be included for analysis. A total of 1194 participants were included in these studies. There was a decreased risk of mortality seen with use of aspirin compared to other agents (MESE = 0.86, 95 % CI: [0.07-1.66]; p=.03). There was no increased risk of DVT or PE with use of aspirin (both p>.4). The overall RFI and FQ for all 19 outcomes were 12 (IQR: 6.5-15) and 0.080 (IQR: 0.027-0.110), respectively. Ten studies (52.6 %) reported a loss-to-follow-up (LTF) greater than the overall RFI. Conclusions Aspirin demonstrates similar protective effects on prevention of VTE compared to other agents and may have significant protective effects on overall mortality following surgical intervention for hip fractures. However, the current evidence concerning its use in this arena is less than robust, with more than half of the studied outcomes considered statistically fragile.
Collapse
Affiliation(s)
- Tyler K. Williamson
- Department of Orthopaedic Surgery, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Victor H. Martinez
- University of the Incarnate Word School of Osteopathic Medicine, San Antonio, TX, USA
| | - Adam W. Aziz
- University of the Incarnate Word School of Osteopathic Medicine, San Antonio, TX, USA
| | - Travis Kotzur
- Long School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
| | - Luke Verlinsky
- Department of Orthopaedic Surgery, University of Texas Health San Antonio, San Antonio, TX, USA
| | - Frank A. Buttacavoli
- Department of Orthopaedic Surgery, University of Texas Health San Antonio, San Antonio, TX, USA
| |
Collapse
|
2
|
Nanji K, Xie J, Hatamnejad A, Pur DR, Phillips M, Zeraatkar D, Wong TY, Guymer RH, Kaiser PK, Sivaprasad S, Bhandari M, Steel DH, Wykoff CC, Chaudhary V. Exploring the fragility of meta-analyses in ophthalmology: a systematic review. Eye (Lond) 2024:10.1038/s41433-024-03255-2. [PMID: 39033242 DOI: 10.1038/s41433-024-03255-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Revised: 06/13/2024] [Accepted: 07/15/2024] [Indexed: 07/23/2024] Open
Abstract
OBJECTIVE The fragility index (FI) of a meta-analysis evaluates the extent that the statistical significance can be changed by modifying the event status of individuals from included trials. Understanding the FI improves the interpretation of the results of meta-analyses and can help to inform changes to clinical practice. This review determined the fragility of ophthalmology-related meta-analyses. METHODS Meta-analyses of randomized controlled trials with binary outcomes published in a journal classified as 'Ophthalmology' according to the Journal Citation Report or an Ophthalmology-related Cochrane Review were included. An iterative process determined the FI of each meta-analysis. Multivariable linear regression modeling evaluated the relationship between the FI and potential predictive factors in statistically significant and non-significant meta-analyses. RESULTS 175 meta-analyses were included. The median FI was 6 (Q1-Q3: 3-12). This meant that moving 6 outcomes from one group to another would reverse the study's findings. The FI was 1 for 18 (10.2%) of the included meta-analyses and was ≤5 for 75 (42.4%) of the included meta-analyses. The number of events (p < 0.001) and the p-value (p < 0.001) were the best predictors of the FI in both significant and non-significant meta-analyses. CONCLUSION The statistical significance of meta-analyses in ophthalmology often hinges on the outcome of a few patients. The number of events and the p-value are the most important factors in determining the fragility of the evidence. The FI is an easily interpretable measure that can supplement the reader's understanding of the strength of the evidence being presented. PROSPERO REGISTRATION CRD42022377589.
Collapse
Affiliation(s)
- Keean Nanji
- Department of Surgery, Division of Ophthalmology, McMaster University, Hamilton, ON, Canada
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Jim Xie
- Department of Surgery, Division of Ophthalmology, McMaster University, Hamilton, ON, Canada
| | - Amin Hatamnejad
- Department of Surgery, Division of Ophthalmology, McMaster University, Hamilton, ON, Canada
| | - Daiana R Pur
- Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Mark Phillips
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Dena Zeraatkar
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
- Department of Anesthesia, McMaster University, Hamilton, ON, Canada
| | - Tien Yin Wong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Tsinghua Medicine, Tsinghua University, Beijing, China
| | - Robyn H Guymer
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Peter K Kaiser
- Cole Eye Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Sobha Sivaprasad
- NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, UK
| | - Mohit Bhandari
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
- Department of Surgery, Division of Orthopedic Surgery, McMaster University, Hamilton, ON, Canada
| | - David H Steel
- Bioscience Institute, Newcastle University, Newcastle Upon Tyne, UK
- Sunderland Eye Infirmary, Sunderland, UK
| | - Charles C Wykoff
- Retina Consultants of Texas, Houston, TX, USA
- Blanton Eye Institute, Houston Methodist Hospital, Houston, TX, USA
| | - Varun Chaudhary
- Department of Surgery, Division of Ophthalmology, McMaster University, Hamilton, ON, Canada.
| |
Collapse
|
3
|
Murphy E, Gillies K, Shiely F. Retention strategies are routinely communicated to potential trial participants but often differ from what was planned in the trial protocol: an analysis of adult participant information leaflets and their corresponding protocols. Trials 2024; 25:372. [PMID: 38858790 PMCID: PMC11163762 DOI: 10.1186/s13063-024-08194-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Accepted: 05/23/2024] [Indexed: 06/12/2024] Open
Abstract
BACKGROUND Retaining participants in randomised controlled trials (RCTs) is challenging and trial teams are often required to use strategies to ensure retention or improve it. Other than monetary incentives, there is no requirement to disclose the use of retention strategies to the participant. Additionally, not all retention strategies are developed at the planning stage, i.e. post-funding during protocol development, but some protocols include strategies for participant retention as retention is considered and planned for early in the trial planning stage. It is yet unknown if these plans are communicated in the corresponding participant information leaflets (PILs). The purpose of our study was to determine if PILs communicate plans to promote participant retention and, if so, are these outlined in the corresponding trial protocol. METHODS Ninety-two adult PILs and their 90 corresponding protocols from Clinical Trial Units (CTUs) in the UK were analysed. Directed (deductive) content analysis was used to analyse the participant retention text from the PILs. Data were presented using a narrative summary and frequencies where appropriate. RESULTS Plans to promote participant retention were communicated in 81.5% (n = 75/92) of PILs. Fifty-seven percent (n = 43/75) of PILs communicated plans to use "combined strategies" to promote participant retention. The most common individual retention strategy was telling the participants that data collection for the trial would be scheduled during routine care visits (16%; n = 12/75 PILs). The importance of retention and the impact that missing or deleted data (deleting data collected prior to withdrawal) has on the ability to answer the research question were explained in 6.5% (n = 6/92) and 5.4% (n = 5/92) of PILs respectively. Out of the 59 PILs and 58 matching protocols that both communicated plans to use strategies to promote participant retention, 18.6% (n = 11/59) communicated the same information, the remaining 81.4% (n = 48/59) of PILs either only partially communicated (45.8%; n = 27/59) the same information or did not communicate the same information (35.6%; n = 21/59) as the protocol with regard to the retention strategy(ies). CONCLUSION Retention strategies are frequently communicated to potential trial participants in PILs; however, the information provided often differs from the content in the corresponding protocol. Participant retention considerations are best done at the planning stage of the trial and we encourage trial teams to be consistent in the communication of these strategies in both the protocol and PIL.
Collapse
Affiliation(s)
- Ellen Murphy
- Health Research Board-Trials Methodology Research Network (HRB-TMRN), Galway, Ireland.
- Trials Research and Methodologies Unit (TRAMS), Health Research Board Clinical Research Facility University College Cork, Cork, Ireland.
| | - Katie Gillies
- Health Services Research Unit, University of Aberdeen, Aberdeen, UK
| | - Frances Shiely
- Health Research Board-Trials Methodology Research Network (HRB-TMRN), Galway, Ireland
- Trials Research and Methodologies Unit (TRAMS), Health Research Board Clinical Research Facility University College Cork, Cork, Ireland
- School of Public Health, University College Cork, Cork, Ireland
| |
Collapse
|
4
|
Ortiz-Babilonia CD, Gupta A, Cartagena-Reyes MA, Xu AL, Raad M, Durand WM, Skolasky RL, Jain A. The Statistical Fragility of Trials Comparing Cervical Disc Arthroplasty and Anterior Cervical Discectomy and Fusion: A Meta Analysis. Spine (Phila Pa 1976) 2024; 49:708-714. [PMID: 37368958 DOI: 10.1097/brs.0000000000004756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 06/11/2023] [Indexed: 06/29/2023]
Abstract
STUDY DESIGN Meta-analysis. OBJECTIVE To assess the robustness of randomized controlled trials (RCTs) that compared cervical disc arthroplasty (CDA) and anterior cervical discectomy and fusion (ACDF) for the treatment of symptomatic degenerative cervical pathology by using fragility indices. SUMMARY OF BACKGROUND DATA RCTs comparing these surgical approaches have shown that CDA may be equivalent or even superior to ACDF due to better preservation of normal spinal kinematics. MATERIALS AND METHODS RCTs reporting clinical outcomes after CDA versus ACDF for degenerative cervical disc disease were evaluated. Data for outcome measures were classified as continuous or dichotomous. Continuous outcomes included: Neck Disability Index, overall pain, neck pain, radicular arm pain, and modified Japanese Orthopedic Association scores. Dichotomous outcomes included: any adjacent segment disease (ASD), superior-level ASD, and inferior-level ASD. The fragility index (FI) and continuous FI (CFI) were determined for dichotomous and continuous outcomes, respectively. The corresponding fragility quotient (FQ) and continuous FQ were calculated by dividing FI/CFI by sample size. RESULTS Twenty-five studies (78 outcome events) were included. Thirteen dichotomous events had a median FI of 7 [interquartile range (IQR): 3-10], and the median FQ was 0.043 (IQR: 0.035-0.066). Sixty-five continuous events had a median CFI of 14 (IQR: 9-22) and a median continuous FQ of 0.145 (IQR: 0.074-0.188). This indicates that, on average, altering the outcome of 4.3 patients out of 100 for the dichotomous outcomes and 14.5 out of 100 for continuous outcomes would reverse trial significance. Of the 13 dichotomous events that included a loss to follow-up data, 8 (61.5%) represented ≥7 patients lost. Of the 65 continuous events reporting the loss to follow-up data, 22 (33.8%) represented ≥14 patients lost. CONCLUSION RCTs comparing ACDF and CDA have fair to moderate statistical robustness and do not suffer from statistical fragility.
Collapse
Affiliation(s)
- Carlos D Ortiz-Babilonia
- Department of Orthopedic Surgery, Johns Hopkins University, Baltimore, MD
- Department of Orthopedic Surgery, University of Puerto Rico, PR
| | - Arjun Gupta
- Department of Orthopedic Surgery, Rutgers University, New Jersey, NJ
| | | | - Amy L Xu
- Department of Orthopedic Surgery, Johns Hopkins University, Baltimore, MD
| | - Micheal Raad
- Department of Orthopedic Surgery, Johns Hopkins University, Baltimore, MD
| | - Wesley M Durand
- Department of Orthopedic Surgery, Johns Hopkins University, Baltimore, MD
| | - Richard L Skolasky
- Department of Orthopedic Surgery, Johns Hopkins University, Baltimore, MD
| | - Amit Jain
- Department of Orthopedic Surgery, Johns Hopkins University, Baltimore, MD
| |
Collapse
|
5
|
McKechnie T, Brennan K, Eskicioglu C, Farooq A, Patel SV. Applying the fragility index to randomized controlled trials evaluating total neoadjuvant therapy for rectal cancer: A methodological survey. Radiother Oncol 2024; 194:110148. [PMID: 38341094 DOI: 10.1016/j.radonc.2024.110148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Revised: 01/08/2024] [Accepted: 02/05/2024] [Indexed: 02/12/2024]
Abstract
BACKGROUND Recently, there has been significant interest in, and adoption of, total neoadjuvant therapy (TNT) for locally advanced rectal cancer (LARC). We designed the present study to assess the robustness of the randomized controlled trials (RCTs) evaluating contemporary TNTs for LARC using the fragility index (FI). MATERIALS AND METHODS Relevant articles were identified through a review article by Johnson et al. in the Canadian Journal of Surgery. Dichotomous outcomes within these RCTs were eligible for inclusion if the reported effect size had a p-value < 0.05. The main outcome was FI for each included outcome. Walsh et al.'s method of calculating FI was utilized. Correlations between FI and research characteristics were assessed using the Spearman's rank correlation coefficients. Risk of bias was assessed using Cochrane recommended tools. RESULTS Ten RCTs were identified with 25 outcomes having statistically significant differences between groups. Eleven outcomes were time-to-event outcomes, while the remainder were dichotomous outcomes. Approximately half (n = 13) were oncologic outcomes. The median FI was 2 (interquartile range [IQR] 1-16). The number of patients lost to follow-up exceeded the FI in 17 outcomes (68.0 %) and thus these results were considered "fragile". Lower FI was associated with high risk of bias (rho = -0.5594) and greater loss to follow-up (rho = -0.4394), while higher FI was associated with large study size (rho = 0.5120). CONCLUSIONS The robustness of outcomes from trials assessing TNT for LARC was found to be questionable. Most outcomes were fragile, as determined by the FI. This survey is limited by the number of included studies.
Collapse
Affiliation(s)
- Tyler McKechnie
- Division of General Surgery, Department of Surgery, McMaster University, Hamilton, ON, Canada
| | - Kelly Brennan
- Division of General Surgery, Department of Surgery, Queen's University, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Cagla Eskicioglu
- Division of General Surgery, Department of Surgery, McMaster University, Hamilton, ON, Canada
| | - Ameer Farooq
- Division of General Surgery, Department of Surgery, Queen's University, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Sunil V Patel
- Division of General Surgery, Department of Surgery, Queen's University, Kingston Health Sciences Centre, Kingston, ON, Canada.
| |
Collapse
|
6
|
Li KD, Venishetty N, Fernandez AM, Hakam N, Ghaffar U, Gupta S, Patel HV, Breyer BN. Fragility of overactive bladder medication clinical trials: A systematic review. Neurourol Urodyn 2024. [PMID: 38594889 DOI: 10.1002/nau.25468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Accepted: 04/01/2024] [Indexed: 04/11/2024]
Abstract
PURPOSE Overactive bladder (OAB) syndrome significantly impairs quality of life, often necessitating pharmacological interventions with associated risks. The fragility of OAB trial outcomes, as measured by the fragility index (FI: smallest number of event changes to reverse statistical significance) and quotient (FQ: FI divided by total sample size expressed as a percentage), is critical yet unstudied. MATERIALS AND METHODS We conducted a systematic search for randomized controlled trials on OAB medications published between January 2000 and August 2023. Inclusion criteria were trials with two parallel arms reporting binary outcomes related to OAB medications. We extracted trial details, outcomes, and statistical tests employed. We calculated FI and FQ, analyzing associations with trial characteristics through linear regression. RESULTS We included 57 trials with a median sample size of 211 participants and a 12% median lost to follow-up. Most studies investigated anticholinergics (37/57, 65%). The median FI/FQ was 5/3.5%. Larger trials were less fragile (median FI 8; FQ 1.0%) compared to medium (FI: 4; FQ 2.5%) and small trials (FI: 4; FQ 8.3%). Double-blinded studies exhibited higher FQs (median 2.9%) than unblinded trials (6.7%). Primary and secondary outcomes had higher FIs (median 5 and 6, respectively) than adverse events (FI: 4). Each increase in 10 participants was associated with a +0.19 increase in FI (p < 0.001). CONCLUSIONS A change in outcome for a median of five participants, or 3.5% of the total sample size, could reverse the direction of statistical significance in OAB trials. Studies with larger sample sizes and efficacy outcomes from blinded trials were less fragile.
Collapse
Affiliation(s)
- Kevin D Li
- Department of Urology, University of California San Francisco, San Francisco, California, USA
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA
| | - Nikit Venishetty
- Paul L. Foster School of Medicine, Texas Tech University Health Sciences Center, El Paso, Texas, USA
| | - Adrian M Fernandez
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Nizar Hakam
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Umar Ghaffar
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Shiv Gupta
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Hiren V Patel
- Department of Urology, University of California San Francisco, San Francisco, California, USA
| | - Benjamin N Breyer
- Department of Urology, University of California San Francisco, San Francisco, California, USA
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
7
|
Ahn BJ, Quinn M, Zhao L, He EW, Dworkin M, Naphade O, Byrne RA, Molino J, Blankenhorn B. Statistical Fragility Analysis of Open Reduction Internal Fixation vs Primary Arthrodesis to Treat Lisfranc Injuries: A Systematic Review. Foot Ankle Int 2024; 45:298-308. [PMID: 38327213 DOI: 10.1177/10711007231224797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
BACKGROUND There is a lack of consensus in the use of open reduction internal fixation (ORIF) vs primary arthrodesis (PA) in the management of Lisfranc injuries. Statistical fragility represents the number of events needed to flip statistical significance and provides context to interpret P values of outcomes from conflicting studies. The current study evaluates the statistical fragility of existing research with an outcome-specific approach to provide statistical clarity to the ORIF vs PA discussion. We hypothesized that statistical fragility analysis would offer clinically relevant insight when interpreting conflicting outcomes regarding ORIF vs PA management of Lisfranc injuries. METHODS All comparative studies, RCTs, and case-series investigating ORIF vs PA management of Lisfranc injuries published through October 5, 2023, were identified. Descriptive characteristics, dichotomous outcomes, and continuous outcomes were extracted. Fragility index and continuous fragility index were calculated by the number of event reversals needed to alter significance. Outcomes were categorized by clinical relevance, and median FI and CFI were reported. RESULTS A total of 244 studies were screened. Ten studies and 67 outcomes (44 dichotomous, 23 continuous) were included in the fragility analysis. Of the 10 studies, 4 studies claimed PA to correlate with superior outcomes compared to ORIF with regard to functional scores and return to function outcomes. Of these 4 studies, 3 were statistically robust. Six studies claimed PA and ORIF to have no differences in outcomes, in which only 2 studies were statistically robust. CONCLUSION The overall research regarding ORIF vs PA is relatively robust compared with other orthopaedic areas of controversy. Although the full statistical context of each article must be considered, studies supporting PA superiority with regard to functional scores and return to function metrics were found to be statistically robust. Outcome-specific analysis revealed moderate fragility in several clinically relevant outcomes such as functional score, return to function, and wound complications.
Collapse
Affiliation(s)
- Benjamin J Ahn
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Matthew Quinn
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Leon Zhao
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Elaine W He
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Myles Dworkin
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Om Naphade
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Rory A Byrne
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Janine Molino
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Brad Blankenhorn
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| |
Collapse
|
8
|
Parsons N, Whitehouse MR, Costa ML. What is a fragility index? Bone Joint J 2024; 106-B:319-322. [PMID: 38555942 DOI: 10.1302/0301-620x.106b4.bjj-2023-1043.r1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/02/2024]
Affiliation(s)
- Nick Parsons
- Warwick Clinical Trials Unit, Warwick Medical School, University of Warwick, Coventry, UK
| | | | - Matthew L Costa
- Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
9
|
Wang A, Kwon D, Kim E, Oleru O, Seyidova N, Taub PJ. Statistical fragility of outcomes in acellular dermal matrix literature: A systematic review of randomized controlled trials. J Plast Reconstr Aesthet Surg 2024; 91:284-292. [PMID: 38432086 PMCID: PMC10984759 DOI: 10.1016/j.bjps.2024.02.047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Accepted: 02/04/2024] [Indexed: 03/05/2024]
Abstract
BACKGROUND Acellular dermal matrix (ADM) is commonly used in plastic and reconstructive surgery. With the abundance of randomized controlled trials (RCTs) reporting P-values for ADM outcomes, this study used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate the statistical stability of the outcomes in ADM RCTs. METHODS PubMed, Embase, SCOPUS, Medline, and Cochrane databases were reviewed for ADM RCTs (2003-present) reporting a dichotomous, categorical outcome. FI and rFI (event reversals influencing outcome significance) and FQ (standardized fragility) were calculated and reported as median. Subgroup analysis was performed based on intervention types. RESULTS Among the 127 studies screened, 56 RCTs with 579 outcomes were included. The median FI stood at 4 (3-5) and FQ was 0.04 (0.03-0.07). Only 101 outcomes were statistically significant with a median FI of 3 (1-6) and FQ of 0.04 (0.02-0.08). The nonsignificant outcomes had a median FI of 4 (3-5) and FQ of 0.04 (0.03-0.07). Notably, 26% of the outcomes had several patients lost to follow up equal to or surpassing the FI. Based on the intervention type, the median FIs showed minor fluctuations but remained low. CONCLUSIONS Outcomes from ADM-related RCTs were statistically fragile. Slight outcome reversals or maintenance of patient follow-up can alter the significance of results. Therefore, future researchers are recommended to jointly report FI, FQ, and P-values to offer a comprehensive view of the robustness in ADM literature.
Collapse
Affiliation(s)
- Anya Wang
- Icahn School of Medicine at Mount Sinai, Division of Plastic and Reconstructive Surgery, New York, NY 10029, USA
| | - Daniel Kwon
- Icahn School of Medicine at Mount Sinai, Division of Plastic and Reconstructive Surgery, New York, NY 10029, USA
| | - Esther Kim
- Icahn School of Medicine at Mount Sinai, Division of Plastic and Reconstructive Surgery, New York, NY 10029, USA
| | - Olachi Oleru
- Icahn School of Medicine at Mount Sinai, Division of Plastic and Reconstructive Surgery, New York, NY 10029, USA
| | - Nargiz Seyidova
- Icahn School of Medicine at Mount Sinai, Division of Plastic and Reconstructive Surgery, New York, NY 10029, USA
| | - Peter J Taub
- Icahn School of Medicine at Mount Sinai, Division of Plastic and Reconstructive Surgery, New York, NY 10029, USA.
| |
Collapse
|
10
|
Megafu M, Megafu E, Mian H, Singhal S, Lee A, Gladstone JN, Parisien RL. Fragile Statistical Findings in Randomized Controlled Trials Evaluating Autograft Versus Allograft Use in Anterior Cruciate Ligament Reconstruction: A Systematic Review. Arthroscopy 2024; 40:1009-1018. [PMID: 37579956 DOI: 10.1016/j.arthro.2023.07.055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 07/25/2023] [Accepted: 07/28/2023] [Indexed: 08/16/2023]
Abstract
PURPOSE To analyze the statistical stability of randomized controlled trials (RCTs) evaluating the surgical management of autografts versus allografts in the anterior cruciate ligament reconstruction (ACLR) literature and calculate the fragility index (FI) and fragility quotient and explore a subgroup analysis by calculating the proportion of outcome events where the FI was less than the number of patients lost to follow-up. METHODS Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic search in the PubMed and Cochrane databases to identify RCTs published between 2000 and 2022 that investigated the use of autografts versus allografts in ACLR literature and reported dichotomous data. The fragility index of each dichotomous variable was calculated through the reversal of a single outcome event until significance was reversed. The fragility quotient was calculated by dividing each fragility index by the study sample size. The interquartile range also was calculated. RESULTS Of the 4407 articles screened, 23 met the search criteria, with 11 RCTs evaluating ALCR using autografts and allografts included for analysis. Two hundred and 18 outcome events with 32 significant (P < .05) outcomes and 186 nonsignificant (P ≥ .05) outcomes were identified. The overall fragility index and fragility quotient for all 218 outcomes were 6 subjects (interquartile range 5-8) and 0.058 (interquartile range 0.039-0.077). Fragility analysis of statistically significant outcomes and nonsignificant outcomes had a fragility index of 3.5 (interquartile range 1-5.5) and 6 (interquartile range 5-8), respectively. All of the studies reported a loss to follow-up where 45.5% (5) reported a loss to follow-up greater or equal to 6. CONCLUSIONS The RCTs in the ACLR peer-reviewed literature evaluating autograft versus allograft use are vulnerable to a small number of outcome event reversals and exemplify significant statistical fragility in statistically significant findings. LEVEL OF EVIDENCE Level I, systematic review of Level I studies.
Collapse
Affiliation(s)
- Michael Megafu
- A.T. Still University, Kirksville College of Osteopathic Medicine, Kirksville, Missouri, U.S.A..
| | - Emmanuel Megafu
- Geisinger Commonwealth School of Medicine, Scranton, Pennsylvania, U.S.A
| | - Hassan Mian
- University of Minnesota Medical School, Twin Cities Campus, Minneapolis, Minnesota, U.S.A
| | - Sulabh Singhal
- Drexel University College of Medicine, Philadelphia, Pennsylvania, U.S.A
| | - Alexander Lee
- University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, U.S.A
| | - James N Gladstone
- Mount Sinai Hospital, Department of Orthopedic Surgery and Sports Medicine, New York, New York, U.S.A
| | - Robert L Parisien
- Mount Sinai Hospital, Department of Orthopedic Surgery and Sports Medicine, New York, New York, U.S.A
| |
Collapse
|
11
|
Yendluri A, Alexanian A, Chari RR, Corvi JJ, Namiri NK, Song J, Alaia MJ, Li X, Parisien RL. The Statistical Fragility of Marrow Stimulation for Cartilage Defects of the Knee: A Systematic Review of Randomized Controlled Trials. Cartilage 2024:19476035241233441. [PMID: 38403983 DOI: 10.1177/19476035241233441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/27/2024] Open
Abstract
OBJECTIVE Marrow stimulation is used to address knee cartilage defects. In this study, we used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate statistical fragility of outcomes reported in randomized controlled trials (RCTs) evaluating marrow stimulation. DESIGN PubMed, Embase, and MEDLINE were queried for recent RCTs (January 1, 2010-September 5, 2023) assessing marrow stimulation for cartilage defects of the knee. The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI by the study sample size. RESULTS Across 155 total outcomes from 21 RCTs, the median FI was 3 (interquartile range [IQR], 2-5), with an associated median FQ of 0.067 (IQR, 0.033-0.010). Thirty-two outcomes were statistically significant, with a median FI of 2 (IQR, 1-3.25) and FQ of 0.050 (IQR, 0.025-0.069). Ten of the 32 (31.3%) outcomes reported as statistically significant had an FI of 1. In total, 123 outcomes were nonsignificant, with a median rFI of 3 (IQR, 2-5). Studies assessing stem cell augments were the most fragile, with a median FI of 2. In 55.5% of outcomes, the number of patients lost to follow-up was greater than or equal to the FI. CONCLUSION Statistical findings in RCTs evaluating marrow stimulation for cartilage defects of the knee are statistically fragile. We recommend combined reporting of P-values with FI and FQ metrics to aid in the interpretation of clinical findings in comparative trials assessing cartilage restoration.
Collapse
Affiliation(s)
- Avanish Yendluri
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Rohit R Chari
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - John J Corvi
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Nikan K Namiri
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Junho Song
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Michael J Alaia
- Department of Orthopaedic Surgery, New York University Langone Health, New York, NY, USA
| | - Xinning Li
- Department of Orthopaedic Surgery, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
| | - Robert L Parisien
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
12
|
Figueroa-Parra G, Putman MS, Crowson CS, Duarte-García A. Fragility of randomised controlled trials for systemic lupus erythematosus and lupus nephritis therapies. Lupus Sci Med 2024; 11:e001068. [PMID: 38199863 PMCID: PMC10806519 DOI: 10.1136/lupus-2023-001068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 12/20/2023] [Indexed: 01/12/2024]
Abstract
OBJECTIVE We aimed to evaluate the robustness of phase III randomised controlled trials (RCTs) for SLE and lupus nephritis (LN) using the fragility index (FI), the reverse FI (RFI) and the fragility quotient (FQ). METHODS We searched for phase III RCTs that included patients with active SLE or LN. Data on primary endpoints, total participants and the number of events for each arm were obtained. We calculated the FI score for RCTs with statistically significant results (number of patients required to change from event to non-event to make the study lose statistical significance), the RFI for RCTs without statistically significant results (number of patients required to change from non-event to event to make study gain statistical significance) and the FQ score for both (FI or RFI score divided by the sample size). RESULTS We evaluated 20 RCTs (16 SLE, four LN). The mean FI/RFI score was 13.6 (SD 6.6). There were nine RCTs with statistically significant results (seven SLE, two LN), and the mean FI score was 10.2 (SD 6.2). The lowest FI was for the ILLUMINATE-2 trial (FI=2), and the highest FI was for the BLISS-52 trial (FI=17).Twelve studies had non-statistically significant results (10 SLE, two LN) with a mean RFI score of 15.6 (SD 6.1). The lowest RFI was for the ILLUMINATE-1 trial (RFI=4), and the highest RFI was for the TULIP-1 trial (RFI=27). The lowest FQ scores were found in the ILLUMINATE trials and the highest in the Rituximab trials (EXPLORER and LUNAR), meaning that the last ones were the most robust results after accounting for sample size. CONCLUSIONS The evidence of therapies for patients with SLE and LN is derived mostly from fragile RCTs. Clinicians and trialists must be aware of the fragility of these RCTs for clinical decision-making and designing trials for novel therapeutics.
Collapse
Affiliation(s)
| | - Michael S Putman
- Division of Rheumatology, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Cynthia S Crowson
- Division of Rheumatology, Mayo Clinic, Rochester, Minnesota, USA
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, Minnesota, USA
| | - Alí Duarte-García
- Division of Rheumatology, Mayo Clinic, Rochester, Minnesota, USA
- Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, USA
| |
Collapse
|
13
|
Shah S, Chow R, Simone CB. Bacterial Decolonization for Prevention of Radiation Dermatitis-Fragility Index Analysis. JAMA Oncol 2024; 10:142. [PMID: 37943535 DOI: 10.1001/jamaoncol.2023.5007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2023]
Affiliation(s)
- Shubhangi Shah
- VCU School of Medicine, Virginia Commonwealth University, Richmond
| | | | | |
Collapse
|
14
|
Javidan A, Benipal H, Vi L, Li A, Lee Y, Feridooni T, Alaichi J, Naji F. Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index. J Vasc Surg 2024; 79:148-158.e3. [PMID: 37315910 DOI: 10.1016/j.jvs.2023.05.051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2021] [Revised: 05/15/2023] [Accepted: 05/17/2023] [Indexed: 06/16/2023]
Abstract
OBJECTIVE The fragility index (FI) measures the robustness of statistically significant findings in randomized controlled trials (RCTs) by quantifying the minimum number of event conversions required to reverse a dichotomous outcome's statistical significance. In vascular surgery, many clinical guidelines and critical decision-making points are informed by a handful of key RCTs, especially regarding open surgical versus endovascular treatment. The objective of this study is to evaluate the FI of RCTs with statistically significant primary outcomes that compared open vs endovascular surgery in vascular surgery. METHODS In this meta-epidemiological study and systematic review, MEDLINE, Embase, and CENTRAL were searched for RCTs comparing open versus endovascular treatments for abdominal aortic aneurysms, carotid artery stenosis, and peripheral arterial disease to December 2022. RCTs with statistically significant primary outcomes were included. Data screening and extraction were conducted in duplicate. The FI was calculated by adding an event to the group with the smaller number of events while subtracting a nonevent to the same group until Fisher's exact test produced a nonstatistically significant result. The primary outcome was the FI and proportion of outcomes where the loss to follow-up was greater than the FI. The secondary outcomes assessed the relationship of the FI to disease state, presence of commercial funding, and study design. RESULTS Overall, 5133 articles were captured in the initial search with 21 RCTs reporting 23 different primary outcomes being included in the final analysis. The median FI (first quartile, third quartile) was 3 (3, 20) with 16 outcomes (70%) reporting a loss to follow-up greater than its FI. Mann-Whitney U test revealed that commercially funded RCTs and composite outcomes had greater FIs (median, 20.0 [5.5, 24.5] vs median, 3.0 [2.0, 5.5], P = .035; median, 21 [8, 38] vs median, 3.0 [2.0, 8.5], P = .01, respectively). The FI did not vary between disease states (P = .285) or between index and follow-up trials (P = .147). There were significant correlations between the FI and P values (Pearson r = 0.90; 95% confidence interval, 0.77-0.96), and the number of events (r = 0.82; 95% confidence interval, 0.48-0.97). CONCLUSIONS A small number of event conversions (median, 3) are needed to alter the statistical significance of primary outcomes in vascular surgery RCTs evaluating open surgical and endovascular treatments. Most studies had loss to follow-up greater than its FI, which can call into question trial results, and commercially funded studies had a greater FI. The FI and these findings should be considered in future trial design in vascular surgery.
Collapse
Affiliation(s)
- Arshia Javidan
- Division of Vascular Surgery, University of Toronto, Toronto, Ontario, Canada.
| | - Harsukh Benipal
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Lisa Vi
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Allen Li
- Faculty of Medicine/The Ottawa Hospital Research Institute, University of Ottawa, Ottawa, Ontario, Canada
| | - Yung Lee
- Division of General Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA
| | - Tiam Feridooni
- Division of Vascular Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Jacob Alaichi
- Division of Vascular Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada
| | - Faysal Naji
- Division of Vascular Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
15
|
Ormseth BH, ElHawary H, Janis JE. The Fragility of Landmark Randomized Controlled Trials in the Plastic Surgery Literature. PLASTIC AND RECONSTRUCTIVE SURGERY-GLOBAL OPEN 2024; 12:e5352. [PMID: 38235350 PMCID: PMC10793969 DOI: 10.1097/gox.0000000000005352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 08/24/2023] [Indexed: 01/19/2024]
Abstract
Background Randomized controlled trials (RCTs) are integral to the progress of evidenced-based medicine and help guide changes in the standards of care. Although results are traditionally evaluated according to their corresponding P value, the universal utility of this statistical metric has been called into question. The fragility index (FI) has been developed as an adjunct method to provide additional statistical perspective. In this study, we aimed to determine the fragility of 25 highly cited RCTs in the plastic surgery literature. Methods A PubMed search was used to identify the 25 highest cited RCTs with statistically significant dichotomous outcomes across 24 plastic surgery journals. Article characteristics were extracted, and the FI of each article was calculated. Additionally, Altmetric scores were determined for each study to determine article attention across internet platforms. Results The median FI score across included studies was 4 (2-7.5, interquartile range). The two highest FI scores were 208 and 58, respectively. Four studies (16%) had scores of 0 or 1. Three studies (12%) had scores of 2. All other studies (72%) had FI scores of 3 or higher. The median Altmetric score was 0 (0-3). Conclusion The FI can provide additional perspective on the robustness of study results, but like the P value, it should be interpreted in the greater context of other study elements.
Collapse
Affiliation(s)
- Benjamin H. Ormseth
- From the Department of Plastic and Reconstructive Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | - Hassan ElHawary
- Division of Plastic and Reconstructive Surgery, McGill University Health Center, Montreal, Canada
| | - Jeffrey E. Janis
- From the Department of Plastic and Reconstructive Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio
| |
Collapse
|
16
|
Ayuso SA, Holland AM, Lorenz WR, Scarola GT, Fischer JP, Smart NJ, Heniford BT. Fragility of Randomized Clinical Trials Using Mesh in Abdominal Wall Reconstruction. JAMA Netw Open 2023; 6:e2347534. [PMID: 38091044 PMCID: PMC10719754 DOI: 10.1001/jamanetworkopen.2023.47534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 10/31/2023] [Indexed: 12/17/2023] Open
Abstract
This systematic review evaluates the fragility of randomized clinical trials that used mesh in abdominal wall reconstruction.
Collapse
Affiliation(s)
- Sullivan A. Ayuso
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - Alexis M. Holland
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - William R. Lorenz
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - Gregory T. Scarola
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| | - John P. Fischer
- Division of Plastic Surgery, Department of Surgery, Perelman School of Medicine, Philadelphia, Pennsylvania
| | - Neil J. Smart
- Department of Colorectal Surgery, Royal Devon and Exeter NHS Foundation Trust, Royal Devon and Exeter Hospital, Exeter, Devon, England
| | - B. Todd Heniford
- Division of Gastrointestinal and Minimally Invasive Surgery, Department of Surgery, Carolinas Medical Center, Charlotte, North Carolina
| |
Collapse
|
17
|
Minhas A, Berkay F, Ehlers CB, Froehle AW, Krishnamurthy AB. The statistical fragility of intramedullary reaming in tibial nail fixation: a systematic review. Eur J Trauma Emerg Surg 2023; 49:2347-2354. [PMID: 36879152 DOI: 10.1007/s00068-023-02254-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 02/19/2023] [Indexed: 03/08/2023]
Abstract
PURPOSE To report the statistical stability of prospective clinical trials evaluating the effect of intramedullary reaming on rates of non-union in tibial fractures through calculation of the fragility metrics for non-union rates and all other dichotomous outcomes. METHODS Literature search was conducted for prospective clinical trials evaluating the effect of intramedullary reaming on non-union rates in tibial nailing. All dichotomous outcomes were extracted from the manuscripts. The fragility index (FI) and reverse fragility index (RFI) were calculated by determining the number of event reversals required for a statistically significant outcome to lose significance and vice-versa. The fragility quotient (FQ) and reverse fragility quotient (RFQ) were calculated by dividing the FI or RFI by the sample size, respectively. Outcomes were defined as "fragile" if the FI or RFI was found to be less than or equal to the number of patients lost to follow-up. RESULTS Literature search identified 579 results which produced ten studies meeting the criteria for review. There were 111 outcomes identified for analysis, of which 89 (80%) exhibited statistical fragility. For reported outcomes across the studies the median and mean FI was 2, the median FQ was 0.019, the mean FQ was 0.030, the median RFI was 4, the mean RFI was 3.95, the median RFQ was 0.045, and the mean RFQ was 0.030. Four studies reported outcomes which were found to have an FI of 0. CONCLUSIONS The studies evaluating the effect of intramedullary reaming on tibial nail fixation demonstrate considerable fragility. On average, two event reversals for significant findings, and four event reversals for insignificant findings are sufficient to alter statistical significance. LEVEL OF EVIDENCE Level II, systematic review of Level I and Level II studies.
Collapse
Affiliation(s)
- Arjun Minhas
- Department of Orthopaedic Surgery, Wright State University Boonshoft School of Medicine, Dayton, OH, 45409, USA.
- Department of Orthopaedic Surgery, Miami Valley Hospital, 30 E. Apple Street, Suite 2200, Dayton, OH, 45409, USA.
| | - Fehmi Berkay
- Department of Orthopaedic Surgery, Wright State University Boonshoft School of Medicine, Dayton, OH, 45409, USA
| | - Cooper B Ehlers
- Department of Orthopaedic Surgery, University of California San Diego School of Medicine, San Diego, CA, 92103, USA
| | - Andrew W Froehle
- Department of Orthopaedic Surgery, Wright State University Boonshoft School of Medicine, Dayton, OH, 45409, USA
| | - Anil B Krishnamurthy
- Department of Orthopaedic Surgery, Wright State University Boonshoft School of Medicine, Dayton, OH, 45409, USA
| |
Collapse
|
18
|
Bleakley CM, Wagemans J, Schurz AP, Smoliga JM. How robust are clinical trials in primary and secondary ankle sprain prevention? Phys Ther Sport 2023; 64:85-90. [PMID: 37801794 DOI: 10.1016/j.ptsp.2023.08.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 08/24/2023] [Accepted: 08/25/2023] [Indexed: 10/08/2023]
Abstract
OBJECTIVES Determine the statistical stability of RCTs examining primary and secondary prevention of ankle sprains. METHODS Databases were searched to August 2023. We included parallel design RCTs, using conservative interventions for preventing ankle sprain, reporting dichotomous injury event outcomes. Statistical stability was quantified using Fragility Index (FI) and Fragility Quotient (FQ). Subgroup analyses were undertaken to test if FI varied based on by study objective, original approach to analysis (frequency vs time to event), follow-up duration, and pre-registration. RESULTS 3559 studies were screened with 45 RCTs included. The median number of events required to change the statistical significance (FI) was 4 (IQR 1-6). FI was similar regardless of study objective, original analysis, follow-up duration, and pre-registration status. Median (IQR) FQ was 0.015 (0.005-0.046), therefore reversing events <2 patients/100 would alter significance. In 80% of studies the number of patients lost to follow-up was greater than the FI. CONCLUSION RCTs informing primary and secondary prevention of ankle sprain are fragile. Only a small percentage of outcome event reversals would reverse study significance, and this is often exceeded by the number of drop outs. Robust reporting of dichotomous outcomes requires the use P values and key metrics such as FI or FQ.
Collapse
Affiliation(s)
- C M Bleakley
- Faculty of Life and Health Sciences, Ulster University, Belfast, United Kingdom.
| | - J Wagemans
- Faculty of Medicine and Health Sciences, University of Antwerp, Belgium
| | - A P Schurz
- Department of Health Professions, Bern University of Applied Sciences, Switzerland; Faculty of Physical Education and Physiotherapy, Vrije Universiteit Brussels, Belgium
| | - J M Smoliga
- Department of Physical Therapy, High Point University, United States; School of Medicine, Tufts University, United States
| |
Collapse
|
19
|
Shapiro L, Scherger S, Franco-Paredes C, Gharamti A, Henao-Martinez AF. Anakinra authorized to treat severe coronavirus disease 2019; Sepsis breakthrough or time to reflect? Front Microbiol 2023; 14:1250483. [PMID: 37928695 PMCID: PMC10620707 DOI: 10.3389/fmicb.2023.1250483] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Accepted: 09/26/2023] [Indexed: 11/07/2023] Open
Abstract
Introduction The European Medicines Agency (EMA) and the United States Food and Drug Administration (FDA) announced conditions for using recombinant human interleukin-1 receptor antagonist (rhIL-1ra) to treat hospitalized patients with Coronavirus disease 2019 (COVID-19) and risk for progression. These decisions followed publication of the suPAR-guided Anakinra treatment for Validation of the risk and early Management OF seveRE respiratory failure by COVID-19 (SAVE- MORE) phase 3 clinical trial that yielded positive results. Methods We conducted a literature review and theoretical analysis of IL-1 blockade as a therapy to treat COVID-19. Using a stepwise analysis, we assessed clinical applicability of the SAVE-MORE results and evaluated conceptual support for interleukin-1 suppression as a suitable approach to COVID-19 treatment. This therapeutic approach was then examined as an example of inflammation-suppressing measures used to treat sepsis. Results Anakinra use as a COVID-19 therapy seems to rely on a view of pathogenesis that incorrectly reflects human disease. Since COVID-19 is an example of sepsis, COVID-19 benefit due to anti-inflammatory therapy contradicts an extensive history of unsuccessful clinical study. Repurposing rhIL-1ra to treat COVID-19 appears to exemplify a cycle followed by inflammation-suppressing sepsis treatments. A landscape of treatment failures is interrupted by a successful clinical trial. However, subsequent confirmatory study fails to replicate the positive data. Discussion We suggest further experimentation is not a promising pathway to discover game-changing sepsis therapies. A different kind of approach may be necessary.
Collapse
Affiliation(s)
- Leland Shapiro
- Division of Infectious Diseases, Department of Medicine, Rocky Mountain Regional Veterans Affairs Medical Center, Aurora, CO, United States
- Division of Infectious Diseases, Department of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Sias Scherger
- Division of Infectious Diseases, Department of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Carlos Franco-Paredes
- Hospital Infantil de México, Federico Gómez, México City, México
- Department of Microbiology, Immunology, and Pathology, Colorado State University, Collins, CO, United States
| | - Amal Gharamti
- Department of Internal Medicine, Yale University, Waterbury, CT, United States
| | - Andrés F. Henao-Martinez
- Division of Infectious Diseases, Department of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| |
Collapse
|
20
|
Heston TF. Statistical Significance Versus Clinical Relevance: A Head-to-Head Comparison of the Fragility Index and Relative Risk Index. Cureus 2023; 15:e47741. [PMID: 37899890 PMCID: PMC10602368 DOI: 10.7759/cureus.47741] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 10/25/2023] [Indexed: 10/31/2023] Open
Abstract
Background In biostatistics, assessing the fragility of research findings is crucial for understanding their clinical significance. This study focuses on the fragility index, unit fragility index, and relative risk index as measures to evaluate statistical fragility. The fragility indices assess the susceptibility of p-values to change significance with minor alterations in outcomes within a 2x2 contingency table. In contrast, the relative risk index quantifies the deviation of observed findings from therapeutic equivalence, the point at which the relative risk equals 1. While the fragility indices have intuitive appeal and have been widely applied, their behavior across a wide range of contingency tables has not been rigorously evaluated. Methods Using a Python software program, a simulation approach was employed to generate random 2x2 contingency tables. All tables under consideration exhibited p-values < 0.05 according to Fisher's exact test. Subsequently, the fragility indices and the relative risk index were calculated. To account for sample size variations, the indices were divided by the sample size to give fragility and risk quotients. A correlation matrix assessed the collinearity between each metric and the p-value. Results The analysis included 2,000 contingency tables with cell counts ranging from 20 to 480. Notably, the formulas for calculating the fragility indices encountered limitations when cell counts approached zero or duplicate cell counts hindered standardized application. The correlation coefficients with p-values were as follows: unit fragility index (-0.806), fragility index (-0.802), fragility quotient (-0.715), unit fragility quotient (-0.695), relative risk index (-0.403), and risk quotient (-0.261). Conclusion The fragility indices and fragility quotients demonstrated a strong correlation with p-values below 0.05, while the relative risk index and relative risk quotient exhibited a weak association with p-values below this threshold. This implies that the fragility indices offer limited additional information beyond the p-value alone. In contrast, the relative risk index and risk quotient exhibit independence from the p-value, indicating that they may provide important additional information about statistical fragility by evaluating the divergence of observed results from therapeutic equivalence, irrespective of the p-value-based statistical significance.
Collapse
Affiliation(s)
- Thomas F Heston
- Medical Education and Clinical Sciences, Washington State University, Spokane, USA
- Family Medicine, University of Washington, Spokane, USA
| |
Collapse
|
21
|
Sudah SY, Moverman MA, Masood R, Mojica ES, Pagani NR, Puzzitiello RN, Menendez ME, Salzler MJ. The Majority of Sports Medicine and Arthroscopy-Related Randomized Controlled Trials Reporting Nonsignificant Results Are Statistically Fragile. Arthroscopy 2023; 39:2071-2083.e1. [PMID: 36868530 DOI: 10.1016/j.arthro.2023.02.022] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 02/14/2023] [Accepted: 02/16/2023] [Indexed: 03/05/2023]
Abstract
PURPOSE To evaluate the robustness of sports medicine and arthroscopy related randomized controlled trials (RCTs) reporting nonsignificant results by calculating the reverse fragility index (RFI) and reverse fragility quotient (RFQ). METHODS All sports medicine and arthroscopic-related RCTs from January 1, 2010, through August 3, 2021, were identified. Randomized-controlled trials comparing dichotomous variables with a reported P value ≥ .05 were included. Study characteristics, such as publication year and sample size, as well as loss to follow-up and number of outcome events were recorded. The RFI at a threshold of P < .05 and respective RFQ were calculated for each study. Coefficients of determination were calculated to determine the relationships between RFI and the number of outcome events, sample size, and number of patients lost to follow-up. The number of RCTs in which the loss to follow-up was greater than the RFI was determined. RESULTS Fifty-four studies and 4,638 patients were included in this analysis. The mean sample size and loss to follow-up were 85.9 patients and 12.5 patients, respectively. The mean RFI was 3.7, signifying that a change of 3.7 events in one arm was needed to flip the results of the study from non-significant to significant (P < .05). Of the 54 studies investigated, 33 (61%) had a loss to follow-up greater than their calculated RFI. The mean RFQ was 0.05. A significant correlation between RFI with sample size (R2 = 0.10, P = .02) and the total number of observed events (R2 = 0.13, P < .01) was found. No significant correlation existed between RFI and loss to follow-up in the lesser arm (R2 = 0.01, P = .41). CONCLUSIONS The RFI and RFQ are statistical tools that allow the fragility of studies reporting nonsignificant results to be appraised. Using this methodology, we found that the majority of sports medicine and arthroscopy-related RCTs reporting nonsignificant results are fragile. CLINICAL RELEVANCE RFI and RFQ serve as tools that can be used to assess the validity of RCT results and provide additional context for appropriate conclusions.
Collapse
Affiliation(s)
- Suleiman Y Sudah
- Department of Orthopedics, Monmouth Medical Center, Long Branch, New Jersey
| | - Michael A Moverman
- Department of Orthopaedic Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, Massachusetts
| | - Raisa Masood
- Department of Orthopaedic Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, Massachusetts
| | - Edward S Mojica
- Department of Orthopaedic Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, Massachusetts
| | - Nicholas R Pagani
- Department of Orthopaedic Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, Massachusetts
| | - Richard N Puzzitiello
- Department of Orthopaedic Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, Massachusetts
| | - Mariano E Menendez
- Oregon Shoulder Institute at Southern Oregon Orthopedics, Medford, OR; Midwest Orthopaedics at Rush, Rush University Medical Center, Chicago, IL, U.S.A
| | - Matthew J Salzler
- Department of Orthopaedic Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, Massachusetts.
| |
Collapse
|
22
|
Daldoul O, Bel Haj Ali K, Sekma A, Sassi S, Kouraichi C, Boubaker H, Zorgati A, Boukef R, Nouira S. Reply to Gowthaman and Vasoo. Clin Infect Dis 2023; 77:667. [PMID: 37216379 DOI: 10.1093/cid/ciad302] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Accepted: 05/17/2023] [Indexed: 05/24/2023] Open
Affiliation(s)
- Oussema Daldoul
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| | - Khaoula Bel Haj Ali
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| | - Adel Sekma
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| | - Sarra Sassi
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| | - Cyrine Kouraichi
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| | - Hamdi Boubaker
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| | - Asma Zorgati
- Emergency Department, Sahloul University Hospital, Sousse, Tunisia
| | - Riadh Boukef
- Emergency Department, Sahloul University Hospital, Sousse, Tunisia
| | - Semir Nouira
- Emergency Department, Fattouma Bourguiba University Hospital, Monastir, Tunisia
- Research Laboratory LR12SP18, University of Monastir, Monastir, Tunisia
| |
Collapse
|
23
|
Doyle TR, Hurley ET, Davey MS, Klifto C, Mullett H. The statistical fragility of the management options for reverse shoulder arthroplasty: a systematic review of randomized control trial with fragility analysis. JSES REVIEWS, REPORTS, AND TECHNIQUES 2023; 3:279-284. [PMID: 37588503 PMCID: PMC10426549 DOI: 10.1016/j.xrrt.2023.03.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 08/18/2023]
Abstract
Reverse shoulder arthroplasty (RSA) is used in the treatment of traumatic and arthritic pathologies, with expanding clinical indications and as a result there has been an increase in clinical research on the topic. The purpose of this study was to examine the statistical fragility of randomized control trials (RCTs) reporting outcomes from RSA. A systematic search was undertaken to find RCTs investigating RSA. The Fragility Index (FI) was calculated using Fisher's exact test, by sequentially altering the number of events until there was a reversal of significance. The Fragility Quotient (FQ) was calculated by dividing the FI by the trial population. Each trial was assigned an overall FI and FQ calculated as the median result of its reported findings. Overall, 19 RCTs warranted inclusion in the review, representing 1146 patients, of which 41.2% were male, with a mean age of 74.2 ± 4.3 years and mean follow-up of 22.1 ± 9.9 months. The median RCT population was 59, with a median of 9 patients lost to follow-up. The median FI was 4.5, and median FQ was 0.083, indicating more patients did not complete the trial than the number of outcomes which would have to change to reverse the finding of significance. This review found that the RCT evidence for RSA management may be vulnerable to statistical fragility, with a handful of events required to reverse a finding of significance.
Collapse
Affiliation(s)
- Tom R. Doyle
- Sports Surgery Clinic, Santry, Dublin, Ireland
- Galway University Department of Surgery, Galway, Ireland
| | | | | | | | | |
Collapse
|
24
|
Singh A, Jefferson J. Post-Cardiac Arrest Care. Emerg Med Clin North Am 2023; 41:617-632. [PMID: 37391254 DOI: 10.1016/j.emc.2023.03.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2023]
Abstract
A structured approach to postcardiac arrest care is needed. Although immediate goals include obtaining a blood pressure reading and ECG immediately after return of spontaneous circulation, other more advanced goals include minimizing CNS injury, managing cardiovascular dysfunction, reducing systemic ischemic/reperfusion injury, and identifying and treating the underlying cause to the arrest. This article summarizes the current understanding of the hemodynamic, neurologic, and metabolic abnormalities encountered in postarrest patients.
Collapse
Affiliation(s)
- Amandeep Singh
- Department of Emergency Medicine, Highland Hospital, 1411 East 31st Street, Oakland, CA 94602, USA.
| | - Jamal Jefferson
- Department of Emergency Medicine, Highland Hospital, 1411 East 31st Street, Oakland, CA 94602, USA
| |
Collapse
|
25
|
Placer-Galán C, Enriquez-Navascués JM, Lopetegui AE, Ansorena YS. An analysis of randomized controlled trials on anal fistula conducted between 2000 and 2020 based on the Fragility Index and Reverse Fragility Index. Colorectal Dis 2023; 25:1572-1577. [PMID: 37400967 DOI: 10.1111/codi.16645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 04/29/2023] [Accepted: 05/30/2023] [Indexed: 07/05/2023]
Abstract
AIM The aim of this work was to evaluate the robustness of randomized controlled trials (RCTs) on anal fistula management using the news tools of Fragility Index (FI), Reverse Fragility Index (RFI) and their corresponding fragility quotients. METHOD A systematic search was conducted based on the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines which utilized MEDLINE, EMBASE, Cochrane Library and Web of Science databases. Inclusion criteria included RCTs related to the management of anal fistula published from 2000 to 2022 with dichotomous outcomes measures and 1:1 allocation. Calculation of FI and RFI was performed by creating 2 × 2 contingency tables by successively changing one nonevent to an event for each outcome measure until the result was made nonsignificant or significant, respectively. The Fragility Quotients were calculated by dividing the FI or RFI by the total sample size. Fragile results were defined as those with a FI or RFI equal to or less than the number of patients lost to follow-up. Additionally, those with a FI or RFI less than 3 were also considered fragile. Studies were considered extremely fragile if FI was ≤1 or FQ was ≤0.01. RESULTS There were 36 RCTs that met our criteria, with 3223 patients. Among these, 19 (53%) were positive RCTs (p < 0.005) and 17 (47%) were negative RCTs (p > 0.05). The median FI was 2 (0-5). The analysis by categorical subgroup showed a strong correlation between FI and the p-value (p = 0.000) and the number of events (p = 0.011). The median RFI was 5 (3.5-9.5) and the subgroup analysis showed a strong correlation between RFI and the p-value (p = 0.000), sample size (0.021) and number needed to treat/number needed to harm (0.000). We considered 63.2% of positive RCTs to be fragile and 35.3% of negative RCTs. CONCLUSIONS In the present study we demonstrated the lack of robustness of study findings in published RCTs in the field of anal fistula.
Collapse
Affiliation(s)
- Carlos Placer-Galán
- Colorectal Unit, Department of Surgery, Hospital Universitario Donostia, San Sebastian, Spain
| | | | - Ane Etxart Lopetegui
- Colorectal Unit, Department of Surgery, Hospital Universitario Donostia, San Sebastian, Spain
| | | |
Collapse
|
26
|
Sidali S, Sritharan N, Campani C, Gregory J, Durand F, Ganne-Carrié N, Ronot M, Lévy V, Nault JC. Fragility index of positive phase II and III randomised clinical trials of treatments for hepatocellular carcinoma (2002-2022). JHEP Rep 2023; 5:100755. [PMID: 37425214 PMCID: PMC10326696 DOI: 10.1016/j.jhepr.2023.100755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 03/18/2023] [Accepted: 03/21/2023] [Indexed: 07/11/2023] Open
Abstract
Background & Aims The fragility index (FI), i.e., theminimum number of best survivors reassigned to the control group required to revert the statistically significant result of a clinical trial to non-significant, is a metric to evaluate the robustness of randomized controlled trials (RCTs). We aimed to assess the FI in the field of HCC. Methods This is a retrospective analysis of phase 2 and 3 RCTs for the treatment of HCC published between 2002 and 2022. We included two-arm studies with 1:1 randomization and significant positive results for a primary time-to-event endpoint for the FI calculation, which involves the iterative addition of a best survivor from the experimental group to the control group, until positive significance (p <0,05, Log-rank test) is lost. Results We identified 51 phase 2 and 3 positive RCTs, of which 29 (57%) were eligible for fragility index calculation. After reconstruction of the Kaplan-Meier curves, 25/29 studies remained significant, among which the analysis was performed. The median (interquartile range (IQR)) FI was 5 (2-10) and Fragility Quotient (FQ) was 3% (1%-6%). Ten trials (40%) had a FI of 2 or less. FI was positively correlated to the blind assessment of the primary endpoint (median FI 9 with blind assessment versus 2 without, p = 0.01), the number of reported events in the control arm (RS = 0.45, p = 0.02) and to impact factor (RS = 0.58, p = 0.003). Conclusions Several phases 2 and 3 RCTs in HCC have a low fragility index, underlying the limited robustness on the conclusion of their superiority over control treatments. The fragility index might provide an additional tool to assess the robustness of clinical trial data in HCC. Impact and implications The fragility index is a method to assess robustness of a clinical trial and is defined the minimum number of best survivors reassigned to the control group required to revert the statistically significant result of a clinical trial to non-significant. Among 25 randomised controlled trials in HCC, the median fragility index was 5, and 10 trials among 25 (40%) had a fragility index of 2 or less, indicating an important fragility.
Collapse
Affiliation(s)
- Sabrina Sidali
- Université de Paris, Service d’Hépatologie, DMU DIGEST, Hôpital Beaujon, APHP Nord, Clichy, France
- Centre de Recherche des Cordeliers, Sorbonne Université, Inserm, Université de Paris, Team ‘Functional Genomics of Solid Tumors’, Equipe Labellisée Ligue Nationale Contre le Cancer, Labex OncoImmunology, Paris, France
| | - Nanthara Sritharan
- Department of Clinical Research, Paris Seine Saint Denis Hospital, Sorbonne Paris University, APHP, Bobigny, France
| | - Claudia Campani
- Centre de Recherche des Cordeliers, Sorbonne Université, Inserm, Université de Paris, Team ‘Functional Genomics of Solid Tumors’, Equipe Labellisée Ligue Nationale Contre le Cancer, Labex OncoImmunology, Paris, France
| | - Jules Gregory
- Department of Radiology, FHU MOSAIC, Hôpital Beaujon APHP Nord, Clichy, France
- Université de Paris, INSERM, UMR1153, Epidemiology and Biostatistics Sorbonne Paris Cité Center (CRESS), METHODS Team, Paris, France
| | - François Durand
- Université de Paris, Service d’Hépatologie, DMU DIGEST, Hôpital Beaujon, APHP Nord, Clichy, France
| | - Nathalie Ganne-Carrié
- Centre de Recherche des Cordeliers, Sorbonne Université, Inserm, Université de Paris, Team ‘Functional Genomics of Solid Tumors’, Equipe Labellisée Ligue Nationale Contre le Cancer, Labex OncoImmunology, Paris, France
- Liver Unit, Hôpital Avicenne, Hôpitaux Universitaires Paris-Seine-Saint-Denis, Assistance-Publique Hôpitaux de Paris, Bobigny, France
- Unité de Formation et de Recherche Santé Médecine et Biologie Humaine, Université Sorbonne Paris Nord, Bobigny, France
| | - Maxime Ronot
- Department of Radiology, FHU MOSAIC, Hôpital Beaujon APHP Nord, Clichy, France
- Université de Paris, INSERM U1149 ‘Centre de Recherche sur L'inflammation’, CRI, Paris, France
| | - Vincent Lévy
- Department of Clinical Research, Paris Seine Saint Denis Hospital, Sorbonne Paris University, APHP, Bobigny, France
- ECSTRRA Team, CRESS UMR 1153, Hôpital Saint-Louis, APHP, Paris, France
| | - Jean-Charles Nault
- Centre de Recherche des Cordeliers, Sorbonne Université, Inserm, Université de Paris, Team ‘Functional Genomics of Solid Tumors’, Equipe Labellisée Ligue Nationale Contre le Cancer, Labex OncoImmunology, Paris, France
- Liver Unit, Hôpital Avicenne, Hôpitaux Universitaires Paris-Seine-Saint-Denis, Assistance-Publique Hôpitaux de Paris, Bobigny, France
- Unité de Formation et de Recherche Santé Médecine et Biologie Humaine, Université Sorbonne Paris Nord, Bobigny, France
| |
Collapse
|
27
|
Garcia MVF, Coz-Yataco A, Al-Jaghbeer MJ. Pulmonary arterial hypertension trials put to the test: Using the fragility index to assess trials robustness. Heart Lung 2023; 61:147-152. [PMID: 37271106 DOI: 10.1016/j.hrtlng.2023.05.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 05/29/2023] [Accepted: 05/30/2023] [Indexed: 06/06/2023]
Abstract
BACKGROUND Randomized clinical trials (RCTs) are considered the gold standard for evidence-based medicine. The Fragility Index (FI) is a tool to assess the robustness of RCT results. FI was validated for dichotomous outcomes and recent work expanded its use to continuous outcomes. Studying the robustness of RCTs in Pulmonary Arterial Hypertension (PAH) treatments is crucial due to the severity and mortality risks associated with this rare condition. OBJECTIVES Analyze FI and Fragility quotient (FQ) of significant primary outcomes in PAH RCTs and study FI correlation with sample size and journal impact factor. METHODS FI and FQ calculation followed by Spearman correlation to assess the correlation between FI and sample size, and FI and impact factor. RESULTS The median sample size of the 21 trials was 202 patients (IQR 106-267), with 6 trials reporting primary outcomes as dichotomous and 15 reporting continuous primary outcomes. The median FI was 10 (IQR 3-20), and the median FQ was 0.044 (0.026-0.097). A moderate correlation was found between FI and sample size, with r = 0.56; P = 0.008 and FI and journal impact factor (r=0.50; P=0.019). The FI for continuous outcomes was similar to that for dichotomous outcomes. CONCLUSIONS This study represents the first analysis of the FI and FQ of PAH treatment RCTs, and expands the use of FI to continuous outcomes in this context. The moderate correlation between FI and sample size suggests that increasing sample size alone is partially correlated to a higher FI. The similarity between FI for continuous and dichotomous outcomes supports the broader use of FI in PAH RCTs.
Collapse
Affiliation(s)
| | - Angel Coz-Yataco
- Cleveland Clinic Foundation, Fairview Hospital, 18101 Lorain Ave, Cleveland, OH 44111, United States; Respiratory Institute, Cleveland Clinic Foundation, Cleveland, OH, United States
| | - Mohammed J Al-Jaghbeer
- Cleveland Clinic Foundation, Fairview Hospital, 18101 Lorain Ave, Cleveland, OH 44111, United States; Respiratory Institute, Cleveland Clinic Foundation, Cleveland, OH, United States; Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, Cleveland, OH, United States
| |
Collapse
|
28
|
Hoang TNA, Quach HL, Hoang VN, Tran VT, Pham QT, Vogt F. Assessing the robustness of COVID-19 vaccine efficacy trials: systematic review and meta-analysis, January 2023. Euro Surveill 2023; 28:2200706. [PMID: 37261728 PMCID: PMC10236928 DOI: 10.2807/1560-7917.es.2023.28.22.2200706] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 04/19/2023] [Indexed: 06/02/2023] Open
Abstract
BackgroundVaccines play a crucial role in the response to COVID-19 and their efficacy is thus of great importance.AimTo assess the robustness of COVID-19 vaccine efficacy (VE) trial results using the fragility index (FI) and fragility quotient (FQ) methodology.MethodsWe conducted a Cochrane and PRISMA-compliant systematic review and meta-analysis of COVID-19 VE trials published worldwide until 22 January 2023. We calculated the FI and FQ for all included studies and assessed their associations with selected trial characteristics using Wilcoxon rank sum tests and Kruskal-Wallis H tests. Spearman correlation coefficients and scatter plots were used to quantify the strength of correlation of FIs and FQs with trial characteristics.ResultsOf 6,032 screened records, we included 40 trials with 54 primary outcomes, comprising 909,404 participants with a median sample size per outcome of 13,993 (interquartile range (IQR): 8,534-25,519). The median FI and FQ was 62 (IQR: 22-123) and 0.50% (IQR: 0.24-0.92), respectively. FIs were positively associated with sample size (p < 0.001), and FQs were positively associated with type of blinding (p = 0.023). The Spearman correlation coefficient for FI with sample size was moderately strong (0.607), and weakly positive for FI and FQ with VE (0.138 and 0.161, respectively).ConclusionsThis was the largest study on trial robustness to date. Robustness of COVID-19 VE trials increased with sample size and varied considerably across several other important trial characteristics. The FI and FQ are valuable complementary parameters for the interpretation of trial results and should be reported alongside established trial outcome measures.
Collapse
Affiliation(s)
- Thi Ngoc Anh Hoang
- Faculty of Medicine, PHENIKAA University, Yen Nghia, Ha Dong, Hanoi, Vietnam
| | - Ha-Linh Quach
- National Centre for Epidemiology and Population Health, Research School of Population Health, College of Health and Medicine, Australian National University, Canberra, ACT, Australia
- Department of Communicable Diseases Control and Prevention, National Institute of Hygiene and Epidemiology, Hanoi, Vietnam
- Centre for Ageing Research and Education (CARE), Duke-NUS Medical School, Singapore, Singapore
| | - Van Ngoc Hoang
- The General Department of Preventive Medicine, Ministry of Health, Hanoi, Vietnam
| | | | - Quang Thai Pham
- Department of Communicable Diseases Control and Prevention, National Institute of Hygiene and Epidemiology, Hanoi, Vietnam
- School of Preventive Medicine and Public Health, Hanoi Medical University, Hanoi, Vietnam
| | - Florian Vogt
- National Centre for Epidemiology and Population Health, Research School of Population Health, College of Health and Medicine, Australian National University, Canberra, ACT, Australia
- The Kirby Institute, University of New South Wales, Sydney, New South Wales, Australia
| |
Collapse
|
29
|
Aslan A, Stevens C, Aldine AS, Mamilly A, De Alba L, Arevalo O, Ahuja C, Cuellar HH. The reproducibility of interventional radiology randomized controlled trials and external validation of a classification system. Diagn Interv Radiol 2023; 29:529-534. [PMID: 37070845 PMCID: PMC10679611 DOI: 10.4274/dir.2023.222052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Accepted: 02/24/2023] [Indexed: 04/19/2023]
Abstract
PURPOSE The fragility index (FI) measures the robustness of randomized controlled trials (RCTs). It complements the P value by taking into account the number of outcome events. In this study, the authors measured the FI for major interventional radiology RCTs. METHODS Interventional radiology RCTs published between January 2010 and December 2022 relating to trans-jugular intrahepatic portosystemic shunt, trans-arterial chemoembolization, needle biopsy, angiography, angioplasty, thrombolysis, and nephrostomy tube insertion were analyzed to measure the FI and robustness of the studies. RESULTS A total of 34 RCTs were included. The median FI of those studies was 4.5 (range 1-68). Seven trials (20.6%) had a number of patients lost to follow-up that was higher than their FI, and 15 (44.1%) had a FI of 1-3. CONCLUSION The median FI, and hence the reproducibility of interventional radiology RCTs, is low compared to other medical fields, with some having a FI of 1, which should be interrupted cautiously.
Collapse
Affiliation(s)
- Assala Aslan
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Christopher Stevens
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Amro Saad Aldine
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Ahmed Mamilly
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Luis De Alba
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Octavio Arevalo
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Chaitanya Ahuja
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| | - Hugo H. Cuellar
- Department of Radiology and Interventional Radiology, Ochsner-Louisiana State University, Shreveport, United States
| |
Collapse
|
30
|
Lee Y, Samarasinghe Y, Chen LH, Hapugall A, Javidan A, McKechnie T, Doumouras A, Hong D. Fragility of statistically significant outcomes in randomized trials comparing bariatric surgeries. Int J Obes (Lond) 2023:10.1038/s41366-023-01298-1. [PMID: 37005473 DOI: 10.1038/s41366-023-01298-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Revised: 03/08/2023] [Accepted: 03/14/2023] [Indexed: 04/04/2023]
Abstract
BACKGROUND Randomized controlled trials (RCTs) are regarded as high-level evidence, but the strength of their P values can be difficult to ascertain. The Fragility Index (FI) is a novel metric that evaluates the frailty of trial findings. It is defined as the minimum number of patients required to change from a non-event to event for the findings to lose statistical significance. This study aims to characterize the robustness of bariatric surgery RCTs by examining their FIs. METHODS A search was conducted in MEDLINE, EMBASE, and CENTRAL from January 2000 to February 2022 for RCTs comparing two bariatric surgeries with statistically significant dichotomous outcomes. Bivariate correlation was conducted to identify associations between FI and trial characteristics. RESULTS A total of 35 RCTs were included with a median sample size of 80 patients (Interquartile range [IQR] 58-109). The median FI was 2 (IQR 0-5), indicating that altering the status of two patients in one treatment arm would overturn the statistical significance of results. Subgroup analyses of RCTs evaluating diabetes-related outcomes produced a FI of 4 (IQR 2-6.5), while RCTs comparing Roux-en-Y gastric bypass and sleeve gastrectomy had an FI of 2 (IQR 0.5-5). Increasing FI was found to be correlated with decreasing P value, increasing sample size, increasing number of events, and increasing journal impact factor. CONCLUSION Bariatric surgery RCTs are fragile, with only a few patients required to change from non-events to events to reverse the statistical significance of most trials. Future research should examine the use of FI in trial design.
Collapse
Affiliation(s)
- Yung Lee
- Division of General Surgery, McMaster University, Hamilton, ON, Canada
- Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA
| | | | - Lucy H Chen
- Division of General Surgery, McMaster University, Hamilton, ON, Canada
| | - Akithma Hapugall
- Division of General Surgery, McMaster University, Hamilton, ON, Canada
| | - Arshia Javidan
- Division of Vascular Surgery, University of Toronto, Toronto, ON, Canada
| | - Tyler McKechnie
- Division of General Surgery, McMaster University, Hamilton, ON, Canada
- Department of Health Research Methods and Evidence, McMaster University, Hamilton, ON, Canada
| | | | - Dennis Hong
- Division of General Surgery, McMaster University, Hamilton, ON, Canada.
| |
Collapse
|
31
|
Garcia MVF, Ferreira JC, Caruso P. Fragility index and fragility quotient in randomized clinical trials. J Bras Pneumol 2023; 49:e20230034. [PMID: 36946820 PMCID: PMC10171299 DOI: 10.36416/1806-3756/e20230034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/03/2023] Open
Affiliation(s)
| | - Juliana Carvalho Ferreira
- . Divisão de Pneumologia, Instituto do Coração, Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo (SP) Brasil
- . Unidade de Terapia Intensiva, AC Camargo Cancer Center, São Paulo (SP) Brasil
- . Methods in Epidemiologic, Clinical, and Operations Research-MECOR-program, American Thoracic Society/Asociación Latinoamericana del Tórax, Montevideo, Uruguay
| | - Pedro Caruso
- . Divisão de Pneumologia, Instituto do Coração, Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo (SP) Brasil
- . Unidade de Terapia Intensiva, AC Camargo Cancer Center, São Paulo (SP) Brasil
| |
Collapse
|
32
|
Megafu MN, Megafu EC, Nguyen JT, Mian HS, Singhal SS, Parisien RL. The Statistical Fragility of Orbital Fractures: A Systematic Review of Randomized Controlled Trials. J Oral Maxillofac Surg 2023:S0278-2391(23)00209-4. [PMID: 36931316 DOI: 10.1016/j.joms.2023.02.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 02/14/2023] [Indexed: 03/15/2023]
Abstract
BACKGROUND The P value has often been used as a tool to determine the statistical significance and evaluate the statistical robustness of study findings in orthopedic literature. The purpose of this study is to apply both the fragility index (FI) and the fragility quotient (FQ) to evaluate the degree of statistical fragility in orbital fracture literature. We hypothesized that the dichotomous outcomes within the orbital fracture literature will be vulnerable to a small number of outcome event reversals and will be statistically fragile. METHODS Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), the authors identified all dichotomous data for randomized controlled trials (RCTs) in orbital fracture literature and performed a PubMed search from 2000 to 2022. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each FI by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ. RESULTS Of the 3,329 studies screened, 28 met the criteria with 10 RCTs evaluating orbital fractures included for analysis. A total of 58 outcome events with 22 significant (P < .05) outcomes and 36 nonsignificant (P ≥ .05) outcomes were identified. The overall FI and FQ for all 58 outcomes was 5 (IQR: 4 to 5) and 0.140 (IQR: 0.075 to 0.250), respectively. Fragility analysis of statistical significant outcomes and nonsignificant outcomes had an FI of 3.5 with no IQR and 5 (IQR 4-5), respectively. All of the studies reported a loss to follow-up data, where 20% (2) was greater than the overall FI of 5. CONCLUSION The orbital fracture literature provides treatment guidance by relying on statistical significant results from RCTs. However, the RCTs in the orbital fracture peer-reviewed literature may not be statistically stable as previously thought. The sole reliance of the P value may depict misleading results. Thus, we recommend standardizing the reporting of the P value, FI, and FQ in the orbital fracture literature to aid readers in reliably drawing conclusions based on fragility outcome measures impacting clinical decision-making.
Collapse
Affiliation(s)
- Michael N Megafu
- A.T. Still University, Kirksville College of Osteopathic Medicine, Kirksville, MO.
| | | | | | - Hassan S Mian
- University of Minnesota Medical School, Twin Cities Campus, Minneapolis, MN
| | | | - Robert L Parisien
- Mount Sinai Hospital, Department of Orthopedic Surgery, New York, NY
| |
Collapse
|
33
|
Xing A, Lin L. Empirical assessment of fragility index based on a large database of clinical studies in the Cochrane Library. J Eval Clin Pract 2023; 29:359-370. [PMID: 36322140 PMCID: PMC9928801 DOI: 10.1111/jep.13787] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 10/10/2022] [Accepted: 10/16/2022] [Indexed: 11/05/2022]
Abstract
RATIONALE AIMS AND OBJECTIVES The fragility index (FI) and fragility quotient (FQ) are increasingly used measures for assessing the robustness of clinical studies with binary outcomes in terms of statistical significance. The FI is the minimum number of event status modifications that can alter a study result's statistical significance (or nonsignificance), and the FQ is calculated as the FI divided by the study's total sample size. The literature has no widely recognized criteria for interpreting the fragility measures' magnitudes. This article aims to provide an empirical assessment for the FI and FQ based on a large database of clinical studies in the Cochrane Library. METHODS We explored the overall empirical distributions of the FI and FQ based on five common methods (Fisher's exact test, χ2 test, risk difference, odds ratio, and relative risk) for determining statistical significance of binary outcomes in clinical research. We also considered three different scenarios for the FI calculation and evaluated the relationship between p values and FIs or FQs using Spearman'sρ $\rho $ . Finally, we summarized empirical thresholds based on the overall distributions of the FI and FQ to facilitate their interpretations in future research. RESULTS For about 20% of studies with significant results, the statistical significance was changed after modifying the event status of only one participant. Studies with significant results were considered slightly fragile if the significance hinged on the statuses of about five events. Studies were extremely fragile if FI≤ $\le $ 1 or FQ≤ $\le $ 0.01. The FIs were strongly correlated with p values for significant studies, while Spearman'sρ $\rho $ varied according to the total sample sizes of studies. CONCLUSIONS The statistical significance of clinical studies could be changed after modifying a few events' statuses. Many studies' findings are fairly fragile. The distributions of the FI and FQ provide insights for appraising the robustness of evidence in clinical decision-making.
Collapse
Affiliation(s)
- Aiwen Xing
- Biostatistics and Research Decision Sciences, Merck & Co., Inc., Rahway, NJ, USA
- Department of Statistics, Florida State University, Tallahassee, FL, USA
| | - Lifeng Lin
- Department of Epidemiology and Biostatistics, University of Arizona, Tucson, AZ, USA
| |
Collapse
|
34
|
Use of NEedle Versus suRFACE Recording Electrodes for Detection of Intraoperative Motor Warnings: A Non-Inferiority Trial. The NERFACE Study Part II. J Clin Med 2023; 12:jcm12051753. [PMID: 36902539 PMCID: PMC10003423 DOI: 10.3390/jcm12051753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 02/15/2023] [Accepted: 02/17/2023] [Indexed: 02/24/2023] Open
Abstract
In the NERFACE study part I, the characteristics of muscle transcranial electrical stimulation motor evoked potentials (mTc-MEPs) recorded from the tibialis anterior (TA) muscles with surface and subcutaneous needle electrodes were compared. The aim of this study (NERFACE part II) was to investigate whether the use of surface electrodes was non-inferior to the use of subcutaneous needle electrodes in detecting mTc-MEP warnings during spinal cord monitoring. mTc-MEPs were simultaneously recorded from TA muscles with surface and subcutaneous needle electrodes. Monitoring outcomes (no warning, reversible warning, irreversible warning, complete loss of mTc-MEP amplitude) and neurological outcomes (no, transient, or permanent new motor deficits) were collected. The non-inferiority margin was 5%. In total, 210 (86.8%) out of 242 consecutive patients were included. There was a perfect agreement between both recording electrode types for the detection of mTc-MEP warnings. For both electrode types, the proportion of patients with a warning was 0.12 (25/210) (difference, 0.0% (one-sided 95% CI, 0.014)), indicating non-inferiority of the surface electrode. Moreover, reversible warnings for both electrode types were never followed by permanent new motor deficits, whereas among the 10 patients with irreversible warnings or complete loss of amplitude, more than half developed transient or permanent new motor deficits. In conclusion, the use of surface electrodes was non-inferior to the use of subcutaneous needle electrodes for the detection of mTc-MEP warnings recorded over the TA muscles.
Collapse
|
35
|
Emile SH, Horesh N. Underestimating the benefits of screening colonoscopy for colorectal cancer. Int J Colorectal Dis 2023; 38:1. [PMID: 36598581 DOI: 10.1007/s00384-022-04298-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 12/14/2022] [Indexed: 01/05/2023]
Affiliation(s)
- Sameh Hany Emile
- Colorectal Surgery Department, Ellen Leifer Shulman and Steven Shulman Digestive Disease Center, Cleveland Clinic Florida, Weston, USA.
- General Surgery Department, Colorectal Surgery Unit, Mansoura University Hospitals, Mansoura, Egypt.
| | - Nir Horesh
- Colorectal Surgery Department, Ellen Leifer Shulman and Steven Shulman Digestive Disease Center, Cleveland Clinic Florida, Weston, USA
- Department of Surgery and Transplantation, Sheba Medical Center, Ramat-Gan, Israel
| |
Collapse
|
36
|
Liu Q, Chen H, Gao Y, Zhu C. Robustness of Significant Dichotomous Outcomes in Randomized Controlled Trials in the Treatment of Patients with COVID-19: A Systematic Analysis. INTENSIVE CARE RESEARCH 2023; 3:38-49. [PMID: 36687387 PMCID: PMC9836340 DOI: 10.1007/s44231-022-00027-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 12/23/2022] [Indexed: 01/13/2023]
Abstract
Purpose Significant results of randomized controlled trials (RCTs) should be properly weighed. This study adopted fragility index (FI) to evaluate the robustness of significant dichotomous outcomes from RCTs on coronavirus disease 2019 (COVID-19) treatment. Materials and methods ClinicalTrials.gov and PubMed were searched from inception to July 31, 2021. FIs were calculated and their distribution was depicted. FI's categorical influential factors were analyzed. Spearman correlation coefficient (r s) was reported for the relationship between FI and the continuous characteristics of RCTs. Results Fifty RCTs with 120 outcomes in 7869 patients were included. The FI distribution was abnormal with median 3 (interquartile range 1-7, P = 0.0001). The FIs and robustness were affected by the outcomes of interest, various patient populations, and interventions (T = 18.215,16.667, 23.107; P = 0.02,0.0001, 0.001, respectively). A cubic relationship between the FIs and absolute difference of events between groups with R square of 0.848 (T = 215.828, P = 0.0001, R square = 0.865) was observed. A strong negative logarithmic relationship existed between FI and the P value with R square = - 0.834. Conclusion The robustness of significant dichotomous outcomes of COVID-19 treatments was fragile and affected by the outcomes of interest, patients, interventions, P value, and absolute difference of events between the groups. FI was an useful quantitative metric for the binary significant outcomes on COVID-19 treatments. Registration PROSPERO (CRD42021272455). Supplementary Information The online version contains supplementary material available at 10.1007/s44231-022-00027-y.
Collapse
Affiliation(s)
- Qi Liu
- Emergency Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, No. 1st, Jianshe Eastern Road, Zhengzhou, Henan Province People's Republic of China.,Department of Translational Medicine Center, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, Zhengzhou, Henan Province People's Republic of China
| | - Hong Chen
- Emergency Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, No. 1st, Jianshe Eastern Road, Zhengzhou, Henan Province People's Republic of China.,Department of Translational Medicine Center, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, Zhengzhou, Henan Province People's Republic of China
| | - Yonghua Gao
- Department of Respiratory and Critical Care Medicine, Shanghai Pulmonary Hospital, Tongji University School of Medicine, Shanghai, People's Republic of China
| | - Changju Zhu
- Emergency Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, No. 1st, Jianshe Eastern Road, Zhengzhou, Henan Province People's Republic of China.,Henan Medical Key Laboratory of Emergency and Trauma Research, The First Affiliated Hospital of Zhengzhou University, Zhengzhou University, Zhengzhou, Henan Province People's Republic of China
| |
Collapse
|
37
|
Broekema AEH, Simões de Souza NF, Soer R, Koopmans J, van Santbrink H, Arts MP, Burhani B, Bartels RHMA, van der Gaag NA, Verhagen MHP, Tamási K, van Dijk JMC, Reneman MF, Groen RJM, Kuijlen JMA. Noninferiority of Posterior Cervical Foraminotomy vs Anterior Cervical Discectomy With Fusion for Procedural Success and Reduction in Arm Pain Among Patients With Cervical Radiculopathy at 1 Year: The FACET Randomized Clinical Trial. JAMA Neurol 2023; 80:40-48. [PMID: 36409485 PMCID: PMC9679957 DOI: 10.1001/jamaneurol.2022.4208] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Accepted: 09/18/2022] [Indexed: 11/23/2022]
Abstract
Importance The choice between posterior cervical foraminotomy (posterior surgery) and anterior cervical discectomy with fusion (anterior surgery) for cervical foraminal radiculopathy remains controversial. Objective To investigate the noninferiority of posterior vs anterior surgery in patients with cervical foraminal radiculopathy with regard to clinical outcomes after 1 year. Design, Setting, and Participants This multicenter investigator-blinded noninferiority randomized clinical trial was conducted from January 2016 to May 2020 with a total follow-up of 2 years. Patients were included from 9 hospitals in the Netherlands. Of 389 adult patients with 1-sided single-level cervical foraminal radiculopathy screened for eligibility, 124 declined to participate or did not meet eligibility criteria. Patients with pure axial neck pain without radicular pain were not eligible. Of 265 patients randomized (132 to posterior and 133 to anterior), 15 were lost to follow-up and 228 were included in the 1-year analysis (110 in posterior and 118 in anterior). Interventions Patients were randomly assigned 1:1 to posterior foraminotomy or anterior cervical discectomy with fusion. Main Outcomes and Measures Primary outcomes were proportion of success using Odom criteria and decrease in arm pain using a visual analogue scale from 0 to 100 with a noninferiority margin of 10% (assuming advantages with posterior surgery over anterior surgery that would justify a tolerable loss of efficacy of 10%). Secondary outcomes were neck pain, disability, quality of life, work status, treatment satisfaction, reoperations, and complications. Analyses were performed with 2-proportion z tests at 1-sided .05 significance levels with Bonferroni corrections. Results Among 265 included patients, the mean (SD) age was 51.2 (8.3) years; 133 patients (50%) were female and 132 (50%) were male. Patients were randomly assigned to posterior (132) or anterior (133) surgery. The proportion of success was 0.88 (86 of 98) in the posterior surgery group and 0.76 (81 of 106) in the anterior surgery group (difference, -0.11 percentage points; 1-sided 95% CI, -0.01) and the between-group difference in arm pain was -2.8 (1-sided 95% CI, -9.4) at 1-year follow-up, indicating noninferiority of posterior surgery. Decrease in arm pain had a between-group difference of 3.4 (1-sided 95% CI, 11.8), crossing the noninferiority margin with 1.8 points. All secondary outcomes had 2-sided 95% CIs clustered around 0 with small between-group differences. Conclusions and Relevance In this randomized clinical trial, posterior surgery was noninferior to anterior surgery for patients with cervical radiculopathy regarding success rate and arm pain at 1 year. Decrease in arm pain and secondary outcomes had small between-group differences. These results may be used to enhance shared decision-making. Trial Registration Netherlands Trial Register Identifier: NTR5536.
Collapse
Affiliation(s)
- Anne E. H. Broekema
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Nádia F. Simões de Souza
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Remko Soer
- Department of Anesthesiology, Groningen Pain Center, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
- Research Group Smart Health, Saxion University of Applied Sciences, Enschede, the Netherlands
| | - Jan Koopmans
- Department of Neurosurgery, Martini Hospital Groningen, Groningen, the Netherlands
| | - Henk van Santbrink
- Care and Public Health Research Institute School for Public Health and Primary Care, Maastricht University, Maastricht, the Netherlands
- Department of Neurosurgery, Zuyderland Medical Center, Heerlen, the Netherlands
- Department of Neurosurgery, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Mark P. Arts
- Department of Neurosurgery, Medical Center Haaglanden, the Hague, the Netherlands
| | - Bachtiar Burhani
- Department of Neurosurgery, Elisabeth Tweesteden Ziekenhuis, Tilburg, the Netherlands
| | - Ronald H. M. A. Bartels
- Department of Neurosurgery, Radboud University Medical Center Nijmegen and Canisius Wilhelmina Hospital Nijmegen, Nijmegen, the Netherlands
| | - Niels A. van der Gaag
- Department of Neurosurgery, Haaglanden Medical Center, the Hague, the Netherlands
- Department of Neurosurgery Haga Teaching Hospital, the Hague, the Netherlands
- Department of Neurosurgery, Leiden University Medical Center, Leiden, the Netherlands
| | | | - Katalin Tamási
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
- Department of Epidemiology, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - J. Marc C. van Dijk
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Michiel F. Reneman
- Department of Rehabilitation, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Rob J. M. Groen
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Jos M. A. Kuijlen
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| |
Collapse
|
38
|
Bunch CM, Chang E, Moore EE, Moore HB, Kwaan HC, Miller JB, Al-Fadhl MD, Thomas AV, Zackariya N, Patel SS, Zackariya S, Haidar S, Patel B, McCurdy MT, Thomas SG, Zimmer D, Fulkerson D, Kim PY, Walsh MR, Hake D, Kedar A, Aboukhaled M, Walsh MM. SHock-INduced Endotheliopathy (SHINE): A mechanistic justification for viscoelastography-guided resuscitation of traumatic and non-traumatic shock. Front Physiol 2023; 14:1094845. [PMID: 36923287 PMCID: PMC10009294 DOI: 10.3389/fphys.2023.1094845] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 02/07/2023] [Indexed: 03/03/2023] Open
Abstract
Irrespective of the reason for hypoperfusion, hypocoagulable and/or hyperfibrinolytic hemostatic aberrancies afflict up to one-quarter of critically ill patients in shock. Intensivists and traumatologists have embraced the concept of SHock-INduced Endotheliopathy (SHINE) as a foundational derangement in progressive shock wherein sympatho-adrenal activation may cause systemic endothelial injury. The pro-thrombotic endothelium lends to micro-thrombosis, enacting a cycle of worsening perfusion and increasing catecholamines, endothelial injury, de-endothelialization, and multiple organ failure. The hypocoagulable/hyperfibrinolytic hemostatic phenotype is thought to be driven by endothelial release of anti-thrombogenic mediators to the bloodstream and perivascular sympathetic nerve release of tissue plasminogen activator directly into the microvasculature. In the shock state, this hemostatic phenotype may be a counterbalancing, yet maladaptive, attempt to restore blood flow against a systemically pro-thrombotic endothelium and increased blood viscosity. We therefore review endothelial physiology with emphasis on glycocalyx function, unique biomarkers, and coagulofibrinolytic mediators, setting the stage for understanding the pathophysiology and hemostatic phenotypes of SHINE in various etiologies of shock. We propose that the hyperfibrinolytic phenotype is exemplified in progressive shock whether related to trauma-induced coagulopathy, sepsis-induced coagulopathy, or post-cardiac arrest syndrome-associated coagulopathy. Regardless of the initial insult, SHINE appears to be a catecholamine-driven entity which early in the disease course may manifest as hyper- or hypocoagulopathic and hyper- or hypofibrinolytic hemostatic imbalance. Moreover, these hemostatic derangements may rapidly evolve along the thrombohemorrhagic spectrum depending on the etiology, timing, and methods of resuscitation. Given the intricate hemochemical makeup and changes during these shock states, macroscopic whole blood tests of coagulative kinetics and clot strength serve as clinically useful and simple means for hemostasis phenotyping. We suggest that viscoelastic hemostatic assays such as thromboelastography (TEG) and rotational thromboelastometry (ROTEM) are currently the most applicable clinical tools for assaying global hemostatic function-including fibrinolysis-to enable dynamic resuscitation with blood products and hemostatic adjuncts for those patients with thrombotic and/or hemorrhagic complications in shock states.
Collapse
Affiliation(s)
- Connor M Bunch
- Department of Emergency Medicine, Henry Ford Hospital, Detroit, MI, United States.,Department of Internal Medicine, Henry Ford Hospital, Detroit, MI, United States
| | - Eric Chang
- Department of Medical Education, Indiana University School of Medicine, Notre Dame Campus, South Bend, IN, United States
| | - Ernest E Moore
- Department of Surgery, Ernest E. Moore Shock Trauma Center at Denver Health, University of Colorado, Denver, CO, United States
| | - Hunter B Moore
- Department of Surgery, Ernest E. Moore Shock Trauma Center at Denver Health, University of Colorado, Denver, CO, United States.,Department of Transplant Surgery, Denver Health and University of Colorado Health Sciences Center, Denver, CO, United States
| | - Hau C Kwaan
- Division of Hematology and Oncology, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| | - Joseph B Miller
- Department of Emergency Medicine, Henry Ford Hospital, Detroit, MI, United States.,Department of Internal Medicine, Henry Ford Hospital, Detroit, MI, United States
| | - Mahmoud D Al-Fadhl
- Department of Medical Education, Indiana University School of Medicine, Notre Dame Campus, South Bend, IN, United States
| | - Anthony V Thomas
- Department of Medical Education, Indiana University School of Medicine, Notre Dame Campus, South Bend, IN, United States
| | - Nuha Zackariya
- Department of Medical Education, Indiana University School of Medicine, Notre Dame Campus, South Bend, IN, United States
| | - Shivani S Patel
- Department of Emergency Medicine, Henry Ford Hospital, Detroit, MI, United States
| | - Sufyan Zackariya
- Department of Emergency Medicine, Henry Ford Hospital, Detroit, MI, United States
| | - Saadeddine Haidar
- Department of Emergency Medicine, Henry Ford Hospital, Detroit, MI, United States
| | - Bhavesh Patel
- Division of Critical Care, Department of Medicine, Mayo Clinic Arizona, Phoenix, AZ, United States
| | - Michael T McCurdy
- Division of Pulmonary and Critical Care, Department of Medicine, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Scott G Thomas
- Department of Trauma Surgery, Memorial Leighton Trauma Center, South Bend, IN, United States
| | - Donald Zimmer
- Department of Trauma Surgery, Memorial Leighton Trauma Center, South Bend, IN, United States
| | - Daniel Fulkerson
- Department of Trauma Surgery, Memorial Leighton Trauma Center, South Bend, IN, United States
| | - Paul Y Kim
- Department of Medicine, McMaster University, Hamilton, ON, Canada.,Thrombosis and Atherosclerosis Research Institute, Hamilton, ON, Canada
| | | | - Daniel Hake
- Departments of Emergency Medicine and Internal Medicine, Saint Joseph Regional Medical Center, Mishawaka, IN, United States
| | - Archana Kedar
- Departments of Emergency Medicine and Internal Medicine, Saint Joseph Regional Medical Center, Mishawaka, IN, United States
| | - Michael Aboukhaled
- Departments of Emergency Medicine and Internal Medicine, Saint Joseph Regional Medical Center, Mishawaka, IN, United States
| | - Mark M Walsh
- Department of Medical Education, Indiana University School of Medicine, Notre Dame Campus, South Bend, IN, United States.,Departments of Emergency Medicine and Internal Medicine, Saint Joseph Regional Medical Center, Mishawaka, IN, United States
| |
Collapse
|
39
|
Taouktsi N, Papageorgiou ST, Tousinas G, Papanikolopoulou S, Grammatikopoulou MG, Giannakoulas G, Goulis DG. Fragility of cardiovascular outcome trials (CVOTs) examining nutrition interventions among patients with diabetes mellitus: a systematic review of randomized controlled trials. Hormones (Athens) 2022; 21:665-681. [PMID: 36129664 PMCID: PMC9712353 DOI: 10.1007/s42000-022-00396-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 08/29/2022] [Indexed: 11/04/2022]
Abstract
PURPOSE There is controversy regarding the optimal statistical method to interpret how robust is a statistically significant result. The fragility index (FI) and the reverse fragility index (RFI) are quantitative measures that can facilitate the appraisal of a clinical trial's robustness. This study was performed to evaluate the FI and RFI of randomized controlled trials (RCTs) examining nutritional interventions in patients with diabetes mellitus, focusing on cardiovascular outcomes. METHODS A systematic search was conducted and relevant RCTs were identified in three databases. RCTs examining nutritional interventions (supplements or dietary patterns) in patients with DM with dichotomous primary endpoints involving cardiovascular outcomes were eligible. Data were extracted to compose 2 × 2 event tables and the FI and RFI were calculated for each comparison, using Fisher's exact test. Risk of bias (RoB) of the included RCTs was assessed with the Cochrane RoB 2.0 tool. RESULTS A total of 14,315 records were screened and 10 RCTs were included in the analyses. The median FI of the paired comparisons was 3 (IQR: 2-4) and the median RFI was 8 (IQR: 4.5-17). RoB and heterogeneity were low. CONCLUSIONS RCTs examining nutritional interventions and cardiovascular outcomes among patients with diabetes mellitus appear to be statistically fragile. Τhe FI and the RFI can be reported and interpreted as an additional perspective of a trial's robustness. HIGHLIGHTS • In the evidence-healthcare era, assessing how robust statistically significant results are remains a matter of controversy. • Recently, the fragility index (FI) and reverse fragility index (RFI) were proposed to assess the robustness of randomized controlled trials (RCTs) with 2 × 2 comparisons. • When applying the FI and RFI, RCTs examining nutritional interventions and cardiovascular outcomes among patients with diabetes mellitus (DM) appear to be statistically fragile. • Τhe FI and the RFI can be reported and interpreted as an additional perspective of a trial's robustness. • RCTs implementing nutrition interventions among patients with DM can improve their methodology.
Collapse
Affiliation(s)
- Niki Taouktsi
- Medical School, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Stefanos T Papageorgiou
- Medical School, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Georgios Tousinas
- Unit of Reproductive Endocrinology, 1st Department of Obstetrics and Gynecology, Medical School, Aristotle University of Thessaloniki, Papageorgiou General Hospital, Thessaloniki, GR-56429, Greece
| | | | - Maria G Grammatikopoulou
- Unit of Reproductive Endocrinology, 1st Department of Obstetrics and Gynecology, Medical School, Aristotle University of Thessaloniki, Papageorgiou General Hospital, Thessaloniki, GR-56429, Greece
- Department of Rheumatology and Clinical Immunology, Medical School, University of Thessaly, Larissa, Greece
| | - George Giannakoulas
- Department of Cardiology, AHEPA University Hospital, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Dimitrios G Goulis
- Unit of Reproductive Endocrinology, 1st Department of Obstetrics and Gynecology, Medical School, Aristotle University of Thessaloniki, Papageorgiou General Hospital, Thessaloniki, GR-56429, Greece.
| |
Collapse
|
40
|
Im JHB, Chow R, Prsic EH. Integrating Community Health Workers Into Care for Patients With Advanced Stages of Cancer-Fragility Index Analysis. JAMA Oncol 2022; 8:1855. [PMID: 36301584 DOI: 10.1001/jamaoncol.2022.5081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- James H B Im
- Yale Cancer Center, Yale School of Medicine, Yale University, New Haven, Connecticut
| | - Ronald Chow
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Elizabeth Horn Prsic
- Yale Cancer Center, Yale School of Medicine, Yale University, New Haven, Connecticut
| |
Collapse
|
41
|
Constant M, Trofa DP, Saltzman BM, Ahmad CS, Li X, Parisien RL. The Fragility of Statistical Significance in Patellofemoral Instability Research: A Systematic Review. Am J Sports Med 2022; 50:3714-3718. [PMID: 34633219 DOI: 10.1177/03635465211039202] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
BACKGROUND Fragility analysis is increasingly utilized to evaluate the robustness of results within the orthopaedic literature and has frequently revealed instability of reported outcomes. PURPOSE/HYPOTHESIS The purpose of this investigation was to utilize a fragility analysis to evaluate the stability of reported results in the patellofemoral instability (PFI) literature. We hypothesized the demonstration of significant fragility in patellofemoral research to be similar to that identified throughout other areas of the orthopaedic literature. STUDY DESIGN Systematic review; Level of evidence, 4. METHODS The PubMed database was queried from January 1, 2000, to October 10, 2020 for comparative trials in 10 prominent orthopaedic journals that reported dichotomous outcomes related to the management of PFI. The fragility index (FI) and the fragility quotient (FQ) were calculated for each individual outcome event, and the overall FI and FQ were determined for all included studies. RESULTS A total of 22 comparative studies comprising 11 randomized controlled trials and 11 nonrandomized trials were included for the analysis. A total of 75 outcome events underwent a fragility analysis and revealed a median FI and FQ of 3 (interquartile range [IQR], 1-5) and 0.043 (IQR, 0.018-0.081), respectively. Also 27% of included studies reported loss to follow-up greater than the overall FI, therefore suggesting the maintenance of the follow-up may have resulted in the reversal of significance. CONCLUSION The result of the comprehensive fragility analysis demonstrated a lack of robustness in PFI research with the alteration of only a few outcome events required to reverse statistical significance. We therefore recommend the triple reporting of the P value, the FI, and the FQ to aid in the interpretation of the statistical integrity of future comparative trials in the PFI literature.
Collapse
Affiliation(s)
- Michael Constant
- Department of Orthopaedics, New York-Presbyterian Hospital, Columbia University Medical Center, New York, New York, USA
| | - David P Trofa
- Department of Orthopaedics, New York-Presbyterian Hospital, Columbia University Medical Center, New York, New York, USA
| | - Bryan M Saltzman
- OrthoCarolina Sports Medicine Center, Charlotte, North Carolina, USA
| | - Christopher S Ahmad
- Department of Orthopaedics, New York-Presbyterian Hospital, Columbia University Medical Center, New York, New York, USA
| | - Xinning Li
- Department of Orthopaedics, Boston University Medical Center, Boston, Massachusetts, USA
| | - Robert L Parisien
- Department of Orthopaedic Surgery & Sports Medicine, Mount Sinai, New York, New York, USA
| |
Collapse
|
42
|
Davey MG, Kerin MJ. Evaluating the Fragility of Long-Term Outcomes for Neoadjuvant versus Adjuvant Chemotherapy Prescription in Early Breast Cancer: Pooled Data from 10 Randomised Clinical Trials. BREAST CANCER (DOVE MEDICAL PRESS) 2022; 14:343-350. [PMID: 36262332 PMCID: PMC9574564 DOI: 10.2147/bctt.s379393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 08/16/2022] [Indexed: 11/07/2022]
Abstract
Introduction Randomised clinical trials (RCTs) report similar outcomes following neoadjuvant (NAC) and adjuvant chemotherapy (AC) in breast cancer. “Fragility Indices” (FI) test significance reversal reported in RCTs. Aim To evaluate the FI of findings from RCTs assessing outcomes of NAC and AC. Methods A systematic review was performed as per PRISMA guidelines. RCTs of interest were identified and data pooled. Fisher’s exact test was used to calculate FI for reversal of statistical significance for dichotomous outcomes. “Fragility Quotient” (FQ) was calculated by division of the calculated FI by the sample size. Results Ten RCTs including 4928 patients. Mean follow-up was 8.2 years. For breast conservation surgery (BCS), the FI was 500 and FQ was 0.10781. For local recurrence (LR), the FI was 42 and FQ was 0.00852. FI and FQ varied for LR at 0–4 years (FI: 9), 5–9 years (FI: 2), 10–14 years (FI: 4), and 15+ years (FI: 3). Regarding distant recurrence (DR), the FI was 13 and FQ was 0.00264. FI and FQ trended downwards over time: 0–4 years (FI: 56), 5–9 years (FI: 18), 10–14 years (FI: 4), and 15+ years (FI: 4). For breast-cancer-specific mortality (BCSM), the overall FI was 51 and FQ was 0.01035. FI and FQ varied for BCSM at 0–4 years (FI: 5), 5–9 years (FI: 19), 10–14 years (FI: 8), and 15+ years (FI: 5). For overall survival (OS), the FI was 17 and FQ was 0.00345. FI and FQ were calculated with respect to OS at 0–4 years (FI: 19), 5–9 years (FI: 17), 10–14 years (FI: 19), and 15+ years (FI: 1). Conclusion FIs comparing survival following NAC and AC were of moderate-to-high fragility, indicating weak statistical significance. BCS eligibility following NAC was of low fragility, ratifying the oncological and surgical safety of NAC versus AC. Level of Evidence Systematic Review of Level I Randomised Control Trials.
Collapse
Affiliation(s)
- Matthew G Davey
- Department of Surgery, The Lambe Institute for Translational Research, National University of Ireland, Galway, H91YR71, Ireland,Correspondence: Matthew G Davey, Department of Surgery, The Lambe Institute for Translational Research, National University of Ireland, Galway, H91YR71, Ireland, Tel +35391524411, Email
| | - Michael J Kerin
- Department of Surgery, The Lambe Institute for Translational Research, National University of Ireland, Galway, H91YR71, Ireland
| |
Collapse
|
43
|
The Fragility of Statistical Findings in Distal Radius Fractures: A Systematic Review of Randomized Controlled Trials. Injury 2022; 53:3352-3356. [PMID: 35851474 DOI: 10.1016/j.injury.2022.07.017] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Revised: 07/06/2022] [Accepted: 07/07/2022] [Indexed: 02/02/2023]
Abstract
PURPOSE Probability analysis with reporting of P values is often used to determine the statistical significance of study findings in the orthopedic literature. The fragility index (FI) has been used to evaluate the robustness of a significant result. The purpose of this study was to determine the utility of applying a fragility analysis to randomized controlled trials (RCTs) evaluating distal radius fractures (DRFs). METHODS We identified all dichotomous outcome data for randomized controlled trials of distal radius fractures (DRFs) published in 13 orthopedic journals from 2000 to 2020. The fragility index (FI) was determined by the number of event reversals required to change a P value from less than 0.05 to greater than 0.05, or vice-versa. The associated fragility quotient (FQ) was determined by dividing the FI by the sample size. The interquartile ranges (IQR) for the FI and FQ were calculated as the difference between the 25th and 75th percentiles for the overall study. RESULTS Of the 7352 studies screened, 221 met the search criteria, with 34 randomized controlled trials evaluating distal radius fractures and 151 total outcome events included for analysis. The overall FI was 9 (IQR 6.5-11) with an associated fragility quotient of 0.097 (IQR 0.060-0.140). However, a majority of outcomes (78.8%) were not significant. Fragility analysis of statistically significant outcomes had an FI of 4 (IQR 1-10). All of the studies reported lost to follow-up (LTF) data where 20.6% reported lost to follow-up of greater or equal to 9. CONCLUSION The RCTs in distal radius fracture outcomes have comparable statistical robustness to literature in other orthopedic surgical specialties, are not as statistically stable as previously thought and should be interpreted with caution. We recommend that orthopedic literature report the FI and FQ along with the P values to help the reader draw reliable conclusions based on the fragility of outcome measures. LEVEL OF EVIDENCE Level I.
Collapse
|
44
|
Yeh DD, Hatton GE, Pedroza C, Pust G, Mantero A, Namias N, Kao LS. Complex And Simple Appendicitis: REstrictive or Liberal postoperative Antibiotic eXposure (CASA RELAX) using Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR): study protocol for a randomized controlled trial. Trauma Surg Acute Care Open 2022; 7:e000931. [PMID: 36148315 PMCID: PMC9486380 DOI: 10.1136/tsaco-2022-000931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Accepted: 08/29/2022] [Indexed: 11/04/2022] Open
Abstract
Objectives After appendectomy for simple or complicated appendicitis, the optimal duration of postoperative antibiotics (postop abx) is unclear and great practice variability exists. We propose to compare restrictive versus liberal postop abx using a hierarchical composite endpoint which includes patient-centered outcomes and accounts for duration of antibiotic exposure. Methods/Design Participants with simple or complicated appendicitis undergoing appendectomy are randomly assigned to either restricted or liberal strategy. Eligible subjects declining randomization will be recruited to enroll in an observation only cohort. The primary endpoint is an ordinal scale of mutually exclusive clinical outcomes with within-category rankings determined by duration of antibiotic exposure. Subjects in both randomized and observation only cohorts will be analyzed as intention-to-treat, per-protocol, and as-treated. Exploratory Bayesian analyses will be performed. Conclusion The complex and simple appendicitis: restrictive or liberal postoperative antibiotic exposure multicenter randomized controlled trial will enroll surgical appendectomy patients and seeks to analyze if a strategy of restricted (compared with liberal) postoperative antibiotics results in similar clinical outcomes with the benefit of reduced antibiotic exposure. Trial registration number NCT05002829.
Collapse
Affiliation(s)
- Daniel Dante Yeh
- Surgery, Ernest E. Moore Shock Trauma Center / Denver Health, Denver, Colorado, USA
| | - Gabrielle E Hatton
- Department of Surgery, McGovern Medical School at UT Health Houston, Houston, Texas, USA
| | - Claudia Pedroza
- Department of Pediatrics, McGovern Medical School at UT Health Houston, Houston, Texas, USA
| | - Gerd Pust
- Surgery, Jackson Memorial Hospital, Miami, Florida, USA
| | - Alejandro Mantero
- University of Miami Biostatistics Collaboration and Consulting Core, Miami, Florida, USA
| | | | - Lillian S Kao
- Department of Surgery, McGovern Medical School at UT Health Houston, Houston, Texas, USA
| |
Collapse
|
45
|
Morris SC, Gowd AK, Agarwalla A, Phipatanakul WP, Amin NH, Liu JN. Fragility of statistically significant findings from randomized clinical trials of surgical treatment of humeral shaft fractures: A systematic review. World J Orthop 2022; 13:825-836. [PMID: 36189338 PMCID: PMC9516622 DOI: 10.5312/wjo.v13.i9.825] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Revised: 02/28/2022] [Accepted: 08/17/2022] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Despite recent meta-analyses of randomized controlled trials (RCTs), there remains no consensus regarding the preferred surgical treatment for humeral shaft fractures. The fragility index (FI) is an emerging tool used to evaluate the robustness of RCTs by quantifying the number of participants in a study group that would need to switch outcomes in order to reverse the study conclusions.
AIM To investigate the fragility index of randomized control trials assessing outcomes of operative fixation in proximal humerus fractures.
METHODS We completed a systematic review of RCTs evaluating the surgical treatment of humeral shaft fractures. Inclusion criteria included: articles published in English; patients randomized and allotted in 1:1 ratio to 2 parallel arms; and dichotomous outcome variables. The FI was calculated for total complications, each complication individually, and secondary surgeries using the Fisher exact test, as previously published.
RESULTS Fifteen RCTs were included in the analysis comparing open reduction plate osteosynthesis with dynamic compression plate or locking compression plate, intramedullary nail, and minimally invasive plate osteosynthesis. The median FI was 0 for all parameters analyzed. Regarding individual outcomes, the FI was 0 for 81/91 (89%) of outcomes. The FI exceeded the number lost to follow up in only 2/91 (2%) outcomes.
CONCLUSION The FI shows that data from RCTs regarding operative treatment of humeral shaft fractures are fragile and does not demonstrate superiority of any particular surgical technique.
Collapse
Affiliation(s)
- Stephen Craig Morris
- Department of Orthopaedic Surgery, Loma Linda University, Loma Linda, CA 92354, United States
| | - Anirudh K Gowd
- Department of Orthopaedic Surgery, Wake Forest University Baptist Medical Center, Winston-Salem, NC 27157, United States
| | - Avinesh Agarwalla
- Department of Orthopaedic Surgery, Westchester Medical Center, Valhalla, NY 10595, United States
| | - Wesley P Phipatanakul
- Department of Orthopaedic Surgery, Loma Linda University, Loma Linda, CA 92354, United States
| | - Nirav H Amin
- Department of Orthopaedic Surgery, Premier Orthopaedic and Trauma Specialists, Pomona, CA 91767, United States
| | - Joseph N Liu
- Department of Orthopedic Surgery, USC Epstein Family Center for Sports Medicine, Los Angeles, CA 90089, United States
| |
Collapse
|
46
|
Schröder A, Muensterer OJ, Oetzmann von Sochaczewski C. Paediatric surgical trials, their fragility index, and why to avoid using it to evaluate results. Pediatr Surg Int 2022; 38:1057-1066. [PMID: 35524787 PMCID: PMC9162995 DOI: 10.1007/s00383-022-05133-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/24/2022] [Indexed: 12/30/2022]
Abstract
BACKGROUND The fragility index has been gaining ground in the evaluation of comparative clinical studies. Many scientists evaluated trials in their fields and deemed them to be fragile, although there is no consensus on the definition of fragility. We aimed to calculate the fragility index and its permutations for paediatric surgical trials. METHODS We searched pubmed for prospectively conducted paediatric surgical trials with intervention and control group without limitations and calculated their (reverse) fragility indices and respective quotients along with posthoc-power. Relationships between variables were evaluated using Spearman's ρ. We also calculated S values by negative log transformation base-2 of P values. RESULTS Of 516 retrieved records, we included 87. The median fragility index was 1.5 (interquartile range: 0-4) and the median reverse fragility index was 3 (interquartile range: 2-4), although they were statistically not different (Mood's test: χ2 = 0.557, df = 1, P = 0.4556). P values and fragility indices were strongly inversely correlated (ρ = - 0.71, 95% confidence interval: - 0.53 to - 0.85, P < 0.0001), while reverse fragility indices were moderately correlated to P values (ρ = 0.5, 95% confidence interval: 0.37-0.62, P < 0.0001). A fragility index of 1 resulted from P values between 0.039 and 0.003, which resulted in S values between 4 and 8. CONCLUSIONS Fragility indices, reverse fragility indices, and their respective fragility quotients of paediatric surgical trials are low. The fragility index can be viewed as no more than a transformed P value with even more substantial limitations. Its inherent penalisation of small studies irrespective of their clinical relevance is particularly harmful for paediatric surgery. Consequently, the fragility index should be avoided.
Collapse
Affiliation(s)
- Arne Schröder
- Klinik für Kinder- und Jugendmedizin, Klinikum Dortmund, Dortmund, Germany
| | - Oliver J Muensterer
- Kinderchirurgische Klinik und Poliklinik im Dr. von Haunerschen Kinderspital, Ludwig-Maximilians-Universität München, München, Germany
- Klinik und Poliklinik für Kinderchirurgie, Universitätsmedizin der Johannes-Gutenberg-Universität Mainz, Mainz, Germany
| | - Christina Oetzmann von Sochaczewski
- Klinik und Poliklinik für Kinderchirurgie, Universitätsmedizin der Johannes-Gutenberg-Universität Mainz, Mainz, Germany.
- Sektion Kinderchirurgie der Klinik und Poliklinik für Allgemein, Viszeral, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Venusberg-Campus 1, 53127, Bonn, Germany.
| |
Collapse
|
47
|
Xu AL, Ortiz-Babilonia C, Gupta A, Rogers D, Aiyer AA, Vulcano E. The Statistical Fragility of Platelet-Rich Plasma as Treatment for Chronic Noninsertional Achilles Tendinopathy: A Systematic Review and Meta-analysis. FOOT & ANKLE ORTHOPAEDICS 2022; 7:24730114221119758. [PMID: 36051864 PMCID: PMC9424894 DOI: 10.1177/24730114221119758] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Background: Randomized controlled trial (RCT) outcomes reaching statistical significance,
frequently determined by P <.05, are often used to guide
decision making. Noted lack of reproducibility of some RCTs has brought
special attention to the limitations of this approach. In this
meta-analysis, we assessed the robustness of RCTs evaluating platelet-rich
plasma (PRP) for the treatment of chronic noninsertional Achilles
tendinopathy (AT) by using fragility indices. Methods: The present study was a systematic review and meta-analysis of RCTs comparing
outcomes after PRP injection vs alternative treatment in patients with AT.
Representative data sets were generated for each reported continuous outcome
event using summary statistics. Fragility indices refer to the minimal
number of patients whose status would have to change from a nonevent to an
event to turn a statistically significant result into a nonsignificant
result, or vice versa. The fragility index (FI) and continuous FI (CFI) were
determined for dichotomous and continuous outcomes, respectively, by
manipulating each data set until reversal of significance (a=0.05) was
achieved. The corresponding fragility quotient (FQ) and continuous FQ (CFQ)
were calculated by dividing FI/CFI by sample size. Results: Of 432 studies screened, 8 studies (52 outcome events) were included in this
analysis. The 12 dichotomous outcomes had a median FI of 4.5 (FQ: 0.111),
and the 40 continuous outcomes had a median CFI of 5 (CFQ: 0.154). All 52
outcome events included lost-to-follow-up data, and 12 (23.1%) indicated a
greater number of patients lost to follow-up than the FI or CFI. Conclusion: Our findings suggest that RCTs evaluating PRP for AT therapy lack statistical
robustness, because changing only a small number of events may alter outcome
significance. Level of Evidence: Level II, therapeutic study.
Collapse
Affiliation(s)
- Amy L. Xu
- Department of Orthopaedic Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Carlos Ortiz-Babilonia
- Department of Orthopaedic Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Orthopaedic Surgery, University of Puerto Rico Medical Sciences Campus, San Juan, PR
| | - Arjun Gupta
- Department of Orthopaedic Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Orthopaedic Surgery, Rutgers New Jersey Medical School, Newark, NJ, USA
| | - Davis Rogers
- Department of Orthopaedic Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Amiethab A. Aiyer
- Department of Orthopaedic Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ettore Vulcano
- Department of Orthopaedic Surgery, Columbia University Mount Sinai Medical Center, Miami, FL, USA
| |
Collapse
|
48
|
Lin L, Chu H. Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package. PLoS One 2022; 17:e0268754. [PMID: 35648746 PMCID: PMC9159630 DOI: 10.1371/journal.pone.0268754] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2021] [Accepted: 05/02/2022] [Indexed: 12/01/2022] Open
Abstract
With the growing concerns about research reproducibility and replicability, the assessment of scientific results' fragility (or robustness) has been of increasing interest. The fragility index was proposed to quantify the robustness of statistical significance of clinical studies with binary outcomes. It is defined as the minimal event status modifications that can alter statistical significance. It helps clinicians evaluate the reliability of the conclusions. Many factors may affect the fragility index, including the treatment groups in which event status is modified, the statistical methods used for testing for the association between treatments and outcomes, and the pre-specified significance level. In addition to assessing the fragility of individual studies, the fragility index was recently extended to both conventional pairwise meta-analyses and network meta-analyses of multiple treatment comparisons. It is not straightforward for clinicians to calculate these measures and visualize the results. We have developed an R package called "fragility" to offer user-friendly functions for such purposes. This article provides an overview of methods for assessing and visualizing the fragility of individual studies as well as pairwise and network meta-analyses, introduces the usage of the "fragility" package, and illustrates the implementations with several worked examples.
Collapse
Affiliation(s)
- Lifeng Lin
- Department of Statistics, Florida State University, Tallahassee, FL, United States of America
| | - Haitao Chu
- Statistical Research and Innovation, Global Biometrics and Data Management, Pfizer Inc., New York, NY, United States of America
- Division of Biostatistics, University of Minnesota School of Public Health, Minneapolis, MN, United States of America
| |
Collapse
|
49
|
Fackler NP, Ehlers CB, Callan KT, Amirhekmat A, Smith EJ, Parisien RL, Wang D. Statistical Fragility of Single-Row Versus Double-Row Anchoring for Rotator Cuff Repair: A Systematic Review of Comparative Studies. Orthop J Sports Med 2022; 10:23259671221093391. [PMID: 35571970 PMCID: PMC9096204 DOI: 10.1177/23259671221093391] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Accepted: 02/17/2022] [Indexed: 01/08/2023] Open
Abstract
Background: Comparative studies and randomized controlled trials (RCTs) often use the P (probability) value to convey the statistical significance of their findings. P values are an imperfect measure, however, and are vulnerable to a small number of outcome reversals to alter statistical significance. The inclusion of a fragility index (FI) and fragility quotient (FQ) may aid in the interpretation of a study’s statistical strength. Purpose/Hypothesis: The purpose of this study was to examine the statistical stability of studies comparing single-row to double-row rotator cuff repair. It was hypothesized that the findings of these studies would be vulnerable to a small number of outcome event reversals, often fewer than the number of patients lost to follow-up. Study Design: Systematic review; Level of evidence, 3. Methods: We analyzed comparative studies and RCTs on primary single-row versus double-row rotator cuff repair that were published between 2000 and 2021 in 10 leading orthopaedic journals. Statistical significance was defined as a P < .05. The FI for each outcome was determined by the number of event reversals necessary to alter significance. The FQ was calculated by dividing the FI by the respective sample size. Results: Of 4896 studies screened, 22 comparative studies, 10 of which were RCTs, were ultimately included for analysis. A total of 74 outcomes were examined. Overall, the median FI was 2 (interquartile range [IQR], 1-3), and the median FQ was 0.035 (IQR, 0.020-0.057). The mean FI was 2.55 ± 1.29, and the mean FQ was 0.043 ± 0.027. In 64% of outcomes, the FI was less than the number of patients lost to follow-up.) Additionally, 81% of significant outcomes needed just a single outcome reversal to lose their significance. Conclusion: Over half of the studies currently used to guide clinical practice have a number of patients lost to follow-up greater than their FI. The results of these studies should be interpreted within the context of these limitations. Future analyses may benefit from the inclusion of the FI and the FQ in their statistical analyses.
Collapse
Affiliation(s)
- Nathan P. Fackler
- Department of Orthopaedic Surgery, University of California, Irvine, Irvine, California, USA
- Georgetown University School of Medicine, Washington, DC, USA
| | - Cooper B. Ehlers
- Department of Orthopaedic Surgery, University of California, San Diego, San Diego, California, USA
| | - Kylie T. Callan
- Department of Orthopaedic Surgery, University of California, Irvine, Irvine, California, USA
| | - Arya Amirhekmat
- Department of Orthopaedic Surgery, University of California, Irvine, Irvine, California, USA
| | - Eric J. Smith
- Department of Orthopaedic Surgery, University of California, Irvine, Irvine, California, USA
| | | | - Dean Wang
- Department of Orthopaedic Surgery, University of California, Irvine, Irvine, California, USA
| |
Collapse
|
50
|
Davey MS, Hurley ET, Doyle TR, Dashti H, Gaafar M, Mullett H. The Fragility Index of Statistically Significant Findings From Randomized Controlled Trials Comparing the Management Strategies of Anterior Shoulder Instability. Am J Sports Med 2022:3635465221077268. [PMID: 35414266 DOI: 10.1177/03635465221077268] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
BACKGROUND Debate centering on the management of anterior shoulder instability (ASI) in recent years has led to many randomized controlled trials (RCTs) being published on the topic. The fragility index (FI) has subsequently emerged as a novel method of assessing significant findings reported in RCTs, particularly those with small sample sizes. PURPOSE To evaluate the FI of statistically significant findings in RCTs that reported the outcomes of management strategies of patients with ASI. STUDY DESIGN Systematic review; Level of evidence, 1. METHODS Using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, 2 independent reviewers performed a systematic review of RCTs focusing on the outcomes of management strategies of patients with ASI. There were 3 main categories of RCTs included: (1) nonoperative management in internal rotation (IR) versus external rotation (ER), (2) nonoperative management versus a surgical intervention, and (3) surgical management with arthroscopic Bankart repair versus open Bankart repair. The Fisher exact test was utilized to calculate the FI for the reversal of statistical significance in all statistically significant dichotomous outcomes. RESULTS A total of 21 RCTs were included, including 1589 shoulders (mean age, 29.4 years) with a mean follow-up of 26.8 months. There were 10 RCTs (831 shoulders) that reported outcomes after the nonoperative management of ASI in IR versus ER, with a mean FI of 6.8. There were 5 RCTs (324 shoulders) that reported outcomes comparing the nonoperative and operative management of ASI, with a mean FI of 3.5. There were 6 RCTs (434 shoulders) that reported outcomes after the operative management of ASI with either arthroscopic Bankart repair or open Bankart repair, with a mean FI of 9.6. CONCLUSION The overall FI of RCTs reporting the outcomes of management strategies for patients with ASI was high, suggesting a moderate fragility of statistically significant outcomes including recurrence, revision stabilization, and return to play.
Collapse
|