1
|
Yendluri A, Gonzalez C, Cordero JK, Hayden BL, Moucha CS, Parisien RL. Statistical Outcomes Guiding Periprosthetic Joint Infection Prevention and Revision Are Fragile: A Systematic Review of Randomized Controlled Trials. J Arthroplasty 2024; 39:1869-1875. [PMID: 38331358 DOI: 10.1016/j.arth.2024.01.059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/18/2023] [Revised: 01/26/2024] [Accepted: 01/29/2024] [Indexed: 02/10/2024] Open
Abstract
BACKGROUND Dichotomous outcomes are frequently reported in orthopaedic research and have substantial clinical implications. This study utilizes the fragility index (FI) and fragility quotient (FQ) metrics to determine the statistical stability of outcomes reported in total joint arthroplasty randomized controlled trials (RCTs) relating to periprosthetic joint infection (PJI). METHODS The RCTs that reported dichotomous data related to PJI published between January 1, 2010, and December 31, 2022, were evaluated. The FI and reverse FI (RFI) were defined as the number of outcome event reversals required to reverse the significance of significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI or RFI by the respective sample size. There were 108 RCTs screened, and 17 studies included for analysis. RESULTS A total of 58 outcome events were identified, with a median FI of 4 (interquartile range [IQR] 2 to 5) and associated FQ of 0.0417 (IQR 0.0145 to 0.0602). The 13 statistically significant outcomes had a median FI of 1 (IQR 1 to 2) and FQ of 0.00935 (IQR 0.00629 to 0.01410). The 45 nonsignificant outcomes had a median RFI of 4 (IQR 3 to 5) and FQ of 0.05 (IQR 0.0361 to 0.0723). The number of patients lost to follow-up was greater than or equal to the FI in 46.6% of outcomes. CONCLUSIONS Statistical outcomes in RCTs analyzing PJI are fragile and may lack statistical integrity. We recommend a comprehensive fragility analysis, with the reporting of FI and FQ metrics, to aid in the interpretation of outcomes in the total joint arthroplasty literature.
Collapse
Affiliation(s)
- Avanish Yendluri
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Christopher Gonzalez
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, New York
| | - John K Cordero
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Brett L Hayden
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Calin S Moucha
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Robert L Parisien
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, New York
| |
Collapse
|
2
|
Proal JD, Moon AS, Kwon B. The fragility index and reverse fragility index of FDA investigational device exemption trials in spinal fusion surgery: a systematic review. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2024; 33:2594-2603. [PMID: 38802596 DOI: 10.1007/s00586-024-08317-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 04/20/2024] [Accepted: 05/16/2024] [Indexed: 05/29/2024]
Abstract
PURPOSE FDA investigational device exemption (IDE) studies are considered a gold standard of assessing safety and efficacy of novel devices through RCTs. The fragility index (FI) has emerged as a means to assess robustness of statistically significant study results and inversely, the reverse fragility index (RFI) for non-significant differences. Previous authors have defined results as fragile if loss to follow up is greater than the FI or RFI. The aim of this study was to assess the FI, RFI, and robustness of data supplied by IDE studies in spinal surgery. METHODS This was a systematic review of the literature. Inclusion criteria included randomized controlled trials with dichotomous outcome measures conducted under IDE guidelines between 2000 and 2023. FI and RFI were calculated through successively changing events to non-events until the outcome changed to non-significance or significance, respectively. The fragility quotient (FQ) and reverse fragility quotient (RFQ) were calculated by dividing the FI and RFI, respectively, by the sample size. RESULTS Thirty-two studies met inclusion criteria with a total of 40 unique outcome measures; 240 outcomes were analyzed. Twenty-six studies reported 96 statistically significant results. The median FI was 6 (IQR: 3-9.25), and patients lost to follow up was greater than the FI in 99.0% (95/96) of results. The average FQ was 0.027. Thirty studies reported 144 statistically insignificant results and a median RFI of 6 (IQR: 4-8). The average RFQ extrapolated was 0.021, and loss to follow up was greater than the RFI in 98.6% (142/144) of results. CONCLUSIONS IDE studies in spine surgery are surprisingly fragile given their reputations, large sample sizes, and intent to establish safety in investigational devices. This study found a median FI and RFI of 6. The number of patients lost to follow-up was greater than FIand RFI in 98.8% (237/240) of reported outcomes. FQ and RFQ tell us that changes of two to three patients per hundred can flip the significance of reported outcomes. This is an important reminder of the limitations of RCTs. Analysis of fragility in future studies may help clarify the strength of the relationship between reported data and their conclusions.
Collapse
Affiliation(s)
- Joshua D Proal
- Tufts University School of Medicine, 145 Harrison Ave, Boston, MA, 02111, USA.
| | - Andrew S Moon
- Department of Orthopedic Surgery, Tufts Medical Center, Tufts University School of Medicine, 800 Washington St, Tufts MC Box #306, Boston, MA, 02111, USA
| | - Brian Kwon
- New England Baptist Hospital, Department of Orthopaedic Surgery, 125 Parker Hill Ave, Boston, MA, 02120, USA
| |
Collapse
|
3
|
Yendluri A, Chiang JJ, Linden GS, Megafu MN, Galatz LM, Parsons BO, Parisien RL. The fragility of statistical findings in the reverse total shoulder arthroplasty literature: a systematic review of randomized controlled trials. J Shoulder Elbow Surg 2024; 33:1650-1658. [PMID: 38281679 DOI: 10.1016/j.jse.2023.12.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 11/14/2023] [Accepted: 12/04/2023] [Indexed: 01/30/2024]
Abstract
BACKGROUND Reverse total shoulder arthroplasty (RTSA) has seen increasing utilization as an effective intervention for a wide variety of shoulder pathologies. The scope and indications for growth are often driven by findings from randomized controlled trials (RCTs) guiding surgical decision-making for RTSA. In this study, we utilized the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to assess the robustness of outcomes reported in RCTs in the RTSA literature. METHODS PubMed, Embase, and MEDLINE were queried for RCTs (Jan. 1, 2010-Mar. 31, 2023) in the RTSA literature reporting dichotomous outcomes. The FI and rFI were defined as the number of outcome reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI by the sample size of each study. Subgroup analysis was performed based on outcome category. RESULTS One hundred seventy-six RCTs were screened with 18 studies included. The median FI across 59 total outcomes was 4 (interquartile range [IQR]: 3-5) with an associated FQ of 0.051 (IQR: 0.029-0.065). Thirteen outcomes were statistically significant with a median FI of 3 (IQR: 1-4) and FQ of 0.033 (IQR: 0.012-0.066). Forty-six outcomes were nonsignificant with a median rFI of 4 (IQR: 3-5) and FQ of 0.055 (IQR: 0.032-0.065). The most fragile outcome category was revision/reoperations with a median FI of 2.50 (IQR: 1.00-3.25), followed by clinical score/outcome (median FI: 3.00), complications (median FI: 4.00), "other" (median FI: 4.00), and radiographic findings (median FI: 5.00). Notably, the number of patients lost to follow-up was greater than or equal to the FI for 59% of outcomes. CONCLUSION The statistical findings in RTSA RCTs are fragile and should be interpreted with caution. Reversal of only a few outcomes, or maintaining postoperative follow-up, may be sufficient to alter significance of study findings. We recommend standardized reporting of P values with FI and FQ metrics to allow clinicians to effectively assess the robustness of study findings.
Collapse
Affiliation(s)
- Avanish Yendluri
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | | | | | - Michael N Megafu
- A.T. Still University Kirksville College of Osteopathic Medicine, Kirksville, MO, USA
| | - Leesa M Galatz
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Bradford O Parsons
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Robert L Parisien
- Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
4
|
Brodeur PG, Salameh M, Boulos A, Blankenhorn BD, Hsu RY. Surgical Management of Achilles Tendon Ruptures in the United States 2006-2020, an ABOS Part II Oral Examination Case List Database Study. FOOT & ANKLE ORTHOPAEDICS 2024; 9:24730114241266190. [PMID: 39091402 PMCID: PMC11292698 DOI: 10.1177/24730114241266190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/04/2024] Open
Abstract
Background In correlation with a growing body of evidence regarding nonoperative management for Achilles tendon rupture (ATR), studies from Europe and Canada have displayed a decreasing incidence in surgical management, which has not been noted in the United States. The primary objective of this study is to evaluate the US trend in ATR repair volume. Methods The American Board of Orthopaedic Surgery (ABOS) Part II Oral Examination Case List Database was used. All cases using Current Procedural Terminology codes for primary ATR repair were requested from the years 2006-2020. Total submitted Achilles repair volume, the number of candidates submitting an Achilles repair case, and the overall submitted case volume per examination year was analyzed. Poisson and linear regressions were used to determine statistically significant trends. Results The total number of Achilles repair cases submitted for the ABOS Part II Oral Examination significantly increased from 2006 to 2011 and then decreased until 2020. Taking Achilles repair cases as a proportion of total orthopaedic cases submitted, the same trend was seen. The number of candidates submitting an Achilles repair case increased from 2006 to 2009 and then decreased until 2020. Foot and Ankle fellowship-trained candidates submitted an increasing number of ATR repair cases per candidate during the time period studied. Conclusion This is the first study to demonstrate a decline in the volume of ATR repair in the United States. The decline in ATR repair volume seen in the ABOS Part II Case Lists does not match previously published US surgeon practice patterns but is not necessarily generalizable to beyond this period. Although the overall ATR repair volume in the ABOS Part II Case Lists is decreasing, we found Foot and Ankle fellowship-trained surgeons are operating on an increasing number of ATRs during their board collection period. Level of Evidence Level III, retrospective cohort study.
Collapse
Affiliation(s)
- Peter G. Brodeur
- Department of Orthopaedic Surgery, Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Motasem Salameh
- Department of Orthopaedic Surgery, Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Alexandre Boulos
- Department of Orthopaedic Surgery, Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Brad D. Blankenhorn
- Department of Orthopaedic Surgery, Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Raymond Y. Hsu
- Department of Orthopaedic Surgery, Warren Alpert Medical School of Brown University, Providence, RI, USA
| |
Collapse
|
5
|
Bullock M, Pierson Z. Achilles Tendon Rupture. Clin Podiatr Med Surg 2024; 41:535-549. [PMID: 38789169 DOI: 10.1016/j.cpm.2024.01.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/26/2024]
Abstract
There are many high-level studies comparing nonoperative treatment, open repair, and minimally invasive repair for Achilles tendon ruptures. This article summarizes the most up-to-date literature comparing these treatment options. The authors' preferred protocol for nonoperative treatment is discussed. Preferred techniques for open repair and chronic Achilles repair are discussed with reference to the literature.
Collapse
Affiliation(s)
- Mark Bullock
- Department of Orthopedics, Covenant Healthcare, Saginaw, MI, USA; Department of Podiatric Medicine and Surgery, Central Michigan University, Saginaw, MI, USA.
| | - Zachary Pierson
- Carolina Foot and Ankle Specialists, 1505 SW Cary Parkway, Suite 200, Cary, NC 27511, USA
| |
Collapse
|
6
|
Yendluri A, Megafu MN, Wang A, Cordero JK, Podolnick JD, Forsh DA, Tornetta P, Parisien RL. The Fragility of Statistical Findings in the Femoral Neck Fracture Literature: A Systematic Review of Randomized Controlled Trials. J Orthop Trauma 2024; 38:e230-e237. [PMID: 38442195 DOI: 10.1097/bot.0000000000002793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/27/2024] [Indexed: 03/07/2024]
Abstract
OBJECTIVES Randomized controlled trials (RCTs) in the femoral neck fracture literature frequently report P -values for outcomes, which have substantial implications in guiding surgical management. This study used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to assess the statistical stability of outcomes reported in RCTs evaluating the management and treatment of femoral neck fractures. METHODS DATA SOURCES DESIGN PubMed, Embase, and MEDLINE were queried for RCTs (January 1, 2010 to February 28, 2023). SETTING RCTs that evaluated surgical management or treatment of femoral neck fractures were included. STUDY SELECTION CRITERIA RCTs with 2 treatment arms reporting categorical dichotomous outcomes were included. Non-RCT studies, RCTs with greater than 2 treatment arms, and RCTs without a femoral neck fracture cohort were excluded. DATA EXTRACTION AND SYNTHESIS OUTCOME MEASURES AND COMPARISONS The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant ( P < 0.05) and nonsignificant ( P ≥ 0.05) outcomes, respectively. The FQ was calculated by dividing the FI by the sample size for the study. RESULTS Nine hundred eighty-five articles were screened, with 71 studies included for analysis. The median FI across a total of 197 outcomes was 4 [interquartile range (IQR) 2-5] with an associated FQ of 0.033 (IQR 0.017-0.060). Forty-seven outcomes were statistically significant with a median FI of 2 (IQR 1-4) and associated FQ of 0.02 (IQR 0.014-0.043). One hundred fifty outcomes were statistically nonsignificant with a median rFI of 4 (IQR 3-5) and associated FQ of 0.037 (IQR 0.019-0.065). CONCLUSIONS Statistical findings in femoral neck fracture RCTs are fragile, with reversal of a median 4 outcomes altering significance of study findings. The authors thus recommend standardized reporting of P -values with FI and FQ metrics to aid in interpreting the robustness of outcomes in femoral neck fracture RCTs. LEVEL OF EVIDENCE Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.
Collapse
Affiliation(s)
| | | | - Anya Wang
- Icahn School of Medicine at Mount Sinai, New York, NY
| | | | | | - David A Forsh
- Icahn School of Medicine at Mount Sinai, New York, NY
| | - Paul Tornetta
- Chobanian and Avedisian School of Medicine, Boston, MA
| | | |
Collapse
|
7
|
Brown AN, Yendluri A, Lawrence KW, Cordero JK, Moucha CS, Hayden BL, Parisien RL. The Statistical Fragility of Tranexamic Acid Use in the Orthopaedic Surgery Literature: A Systematic Review of Randomized Controlled Trials. J Am Acad Orthop Surg 2024; 32:508-515. [PMID: 38574390 DOI: 10.5435/jaaos-d-23-00503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/06/2023] [Accepted: 02/15/2024] [Indexed: 04/06/2024] Open
Abstract
INTRODUCTION Randomized controlled trials (RCTs) represent the highest level of evidence in orthopaedic surgery literature, although the robustness of statistical findings in these trials may be unreliable. We used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate the statistical stability of outcomes reported in RCTs that assess the use of tranexamic acid (TXA) across orthopaedic subspecialties. METHODS PubMed, EMBASE, and MEDLINE were queried for RCTs (2010-present) reporting dichotomous outcomes with study groups stratified by TXA administration. The FI and rFI were defined as the number of outcome event reversals needed to alter the significance level of significant and nonsignificant outcomes, respectively. FQ was determined by dividing the FI or rFI by sample size. Subgroup analyses were conducted based on orthopaedic subspecialty. RESULTS Six hundred five RCTs were screened with 108 studies included for analysis comprising 192 total outcomes. The median FI of the 192 outcomes was 4 (IQR 2 to 5) with an associated FQ of 0.03 (IQR 0.019 to 0.050). 45 outcomes were reported as statistically significant with a median FI of 1 (IQR 1 to 5) and associated FQ of 0.02 (IQR 0.011 to 0.034). 147 outcomes were reported as nonsignificant with a median rFI of 4 (IQR 3 to 5) and associated FQ of 0.04 (IQR 0.023 to 0.051). The adult reconstruction, trauma, and spine subspecialties had a median FI of 4. Sports had a median FI of 3. Shoulder and elbow and foot and ankle had median FIs of 6. DISCUSSION Statistical outcomes reported in RCTs on the use of TXA in orthopaedic surgery are fragile. Reversal of a few outcomes is sufficient to alter statistical significance. We recommend reporting FI, rFI, and FQ metrics to aid in interpreting the outcomes reported in comparative trials.
Collapse
Affiliation(s)
- Ashley N Brown
- From the Icahn School of Medicine at Mount Sinai, New York, NY (Brown, Yendluri, Cordero, Moucha, Hayden, Parisien), and the Boston University School of Medicine, Boston, MA (Lawrence)
| | | | | | | | | | | | | |
Collapse
|
8
|
Oeding JF, Krych AJ, Camp CL, Varady NH. The Number of Patients Lost to Follow-Up May Exceed the Fragility Index of a Randomized Controlled Trial Without Reversing Statistical Significance: A Systematic Review and Statistical Model. Arthroscopy 2024:S0749-8063(24)00366-9. [PMID: 38777001 DOI: 10.1016/j.arthro.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 04/21/2024] [Accepted: 05/02/2024] [Indexed: 05/25/2024]
Abstract
PURPOSE To (1) analyze trends in the publishing of statistical fragility index (FI)-based systematic reviews in the orthopaedic literature, including the prevalence of misleading or inaccurate statements related to the statistical fragility of randomized controlled trials (RCTs) and patients lost to follow-up (LTF), and (2) determine whether RCTs with relatively "low" FIs are truly as sensitive to patients LTF as previously portrayed in the literature. METHODS All FI-based studies published in the orthopaedic literature were identified using the Cochrane Database of Systematic Reviews, Web of Science Core Collection, PubMed, and MEDLINE databases. All articles involving application of the FI or reverse FI to study the statistical fragility of studies in orthopaedics were eligible for inclusion in the study. Study characteristics, median FIs and sample sizes, and misleading or inaccurate statements related to the FI and patients LTF were recorded. Misleading or inaccurate statements-defined as those basing conclusions of trial fragility on the false assumption that adding patients LTF back to a trial has the same statistical effect as existing patients in a trial experiencing the opposite outcome-were determined by 2 authors. A theoretical RCT with a sample size of 100, P = .006, and FI of 4 was used to evaluate the difference in effect on statistical significance between flipping outcome events of patients already included in the trial (FI) and adding patients LTF back to the trial to show the true sensitivity of RCTs to patients LTF. RESULTS Of the 39 FI-based studies, 37 (95%) directly compared the FI with the number of patients LTF. Of these 37 studies, 22 (59%) included a statement regarding the FI and patients LTF that was determined to be inaccurate or misleading. In the theoretical RCT, a reversal of significance was not observed until 7 patients LTF (nearly twice the FI) were added to the trial in the distribution of maximal significance reversal. CONCLUSIONS The claim that any RCT in which the number of patients LTF exceeds the FI could potentially have its significance reversed simply by maintaining study follow-ups is commonly inaccurate and prevalent in orthopaedic studies applying the FI. Patients LTF and the FI are not equivalent. The minimum number of patients LTF required to flip the significance of a typical RCT was shown to be greater than the FI, suggesting that RCTs with relatively low FIs may not be as sensitive to patients LTF as previously portrayed in the literature; however, only a holistic approach that considers the context in which the trial was conducted, potential biases, and study results can determine the merits of any particular RCT. CLINICAL RELEVANCE Surgeons may benefit from re-examining their interpretation of prior FI reviews that have made claims of substantial RCT fragility based on comparisons between the FI and patients LTF; it is possible the results are more robust than previously believed.
Collapse
Affiliation(s)
- Jacob F Oeding
- School of Medicine, Mayo Clinic Alix School of Medicine, Rochester, Minnesota, U.S.A.; Department of Orthopaedics, Institute of Clinical Sciences, The Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
| | - Aaron J Krych
- Department of Orthopaedic Surgery, Mayo Clinic, Rochester, Minnesota, U.S.A
| | - Christopher L Camp
- Department of Orthopaedic Surgery, Mayo Clinic, Rochester, Minnesota, U.S.A
| | - Nathan H Varady
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, U.S.A
| |
Collapse
|
9
|
Ahn BJ, Quinn M, Zhao L, He EW, Dworkin M, Naphade O, Byrne RA, Molino J, Blankenhorn B. Statistical Fragility Analysis of Open Reduction Internal Fixation vs Primary Arthrodesis to Treat Lisfranc Injuries: A Systematic Review. Foot Ankle Int 2024; 45:298-308. [PMID: 38327213 DOI: 10.1177/10711007231224797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
BACKGROUND There is a lack of consensus in the use of open reduction internal fixation (ORIF) vs primary arthrodesis (PA) in the management of Lisfranc injuries. Statistical fragility represents the number of events needed to flip statistical significance and provides context to interpret P values of outcomes from conflicting studies. The current study evaluates the statistical fragility of existing research with an outcome-specific approach to provide statistical clarity to the ORIF vs PA discussion. We hypothesized that statistical fragility analysis would offer clinically relevant insight when interpreting conflicting outcomes regarding ORIF vs PA management of Lisfranc injuries. METHODS All comparative studies, RCTs, and case-series investigating ORIF vs PA management of Lisfranc injuries published through October 5, 2023, were identified. Descriptive characteristics, dichotomous outcomes, and continuous outcomes were extracted. Fragility index and continuous fragility index were calculated by the number of event reversals needed to alter significance. Outcomes were categorized by clinical relevance, and median FI and CFI were reported. RESULTS A total of 244 studies were screened. Ten studies and 67 outcomes (44 dichotomous, 23 continuous) were included in the fragility analysis. Of the 10 studies, 4 studies claimed PA to correlate with superior outcomes compared to ORIF with regard to functional scores and return to function outcomes. Of these 4 studies, 3 were statistically robust. Six studies claimed PA and ORIF to have no differences in outcomes, in which only 2 studies were statistically robust. CONCLUSION The overall research regarding ORIF vs PA is relatively robust compared with other orthopaedic areas of controversy. Although the full statistical context of each article must be considered, studies supporting PA superiority with regard to functional scores and return to function metrics were found to be statistically robust. Outcome-specific analysis revealed moderate fragility in several clinically relevant outcomes such as functional score, return to function, and wound complications.
Collapse
Affiliation(s)
- Benjamin J Ahn
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Matthew Quinn
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Leon Zhao
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Elaine W He
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Myles Dworkin
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Om Naphade
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Rory A Byrne
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Janine Molino
- The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Brad Blankenhorn
- Department of Orthopaedic Surgery, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| |
Collapse
|
10
|
Parsons N, Whitehouse MR, Costa ML. What is a fragility index? Bone Joint J 2024; 106-B:319-322. [PMID: 38555942 DOI: 10.1302/0301-620x.106b4.bjj-2023-1043.r1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/02/2024]
Affiliation(s)
- Nick Parsons
- Warwick Clinical Trials Unit, Warwick Medical School, University of Warwick, Coventry, UK
| | | | - Matthew L Costa
- Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
11
|
Megafu M, Megafu E, Mian H, Singhal S, Lee A, Gladstone JN, Parisien RL. Fragile Statistical Findings in Randomized Controlled Trials Evaluating Autograft Versus Allograft Use in Anterior Cruciate Ligament Reconstruction: A Systematic Review. Arthroscopy 2024; 40:1009-1018. [PMID: 37579956 DOI: 10.1016/j.arthro.2023.07.055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 07/25/2023] [Accepted: 07/28/2023] [Indexed: 08/16/2023]
Abstract
PURPOSE To analyze the statistical stability of randomized controlled trials (RCTs) evaluating the surgical management of autografts versus allografts in the anterior cruciate ligament reconstruction (ACLR) literature and calculate the fragility index (FI) and fragility quotient and explore a subgroup analysis by calculating the proportion of outcome events where the FI was less than the number of patients lost to follow-up. METHODS Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic search in the PubMed and Cochrane databases to identify RCTs published between 2000 and 2022 that investigated the use of autografts versus allografts in ACLR literature and reported dichotomous data. The fragility index of each dichotomous variable was calculated through the reversal of a single outcome event until significance was reversed. The fragility quotient was calculated by dividing each fragility index by the study sample size. The interquartile range also was calculated. RESULTS Of the 4407 articles screened, 23 met the search criteria, with 11 RCTs evaluating ALCR using autografts and allografts included for analysis. Two hundred and 18 outcome events with 32 significant (P < .05) outcomes and 186 nonsignificant (P ≥ .05) outcomes were identified. The overall fragility index and fragility quotient for all 218 outcomes were 6 subjects (interquartile range 5-8) and 0.058 (interquartile range 0.039-0.077). Fragility analysis of statistically significant outcomes and nonsignificant outcomes had a fragility index of 3.5 (interquartile range 1-5.5) and 6 (interquartile range 5-8), respectively. All of the studies reported a loss to follow-up where 45.5% (5) reported a loss to follow-up greater or equal to 6. CONCLUSIONS The RCTs in the ACLR peer-reviewed literature evaluating autograft versus allograft use are vulnerable to a small number of outcome event reversals and exemplify significant statistical fragility in statistically significant findings. LEVEL OF EVIDENCE Level I, systematic review of Level I studies.
Collapse
Affiliation(s)
- Michael Megafu
- A.T. Still University, Kirksville College of Osteopathic Medicine, Kirksville, Missouri, U.S.A..
| | - Emmanuel Megafu
- Geisinger Commonwealth School of Medicine, Scranton, Pennsylvania, U.S.A
| | - Hassan Mian
- University of Minnesota Medical School, Twin Cities Campus, Minneapolis, Minnesota, U.S.A
| | - Sulabh Singhal
- Drexel University College of Medicine, Philadelphia, Pennsylvania, U.S.A
| | - Alexander Lee
- University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, U.S.A
| | - James N Gladstone
- Mount Sinai Hospital, Department of Orthopedic Surgery and Sports Medicine, New York, New York, U.S.A
| | - Robert L Parisien
- Mount Sinai Hospital, Department of Orthopedic Surgery and Sports Medicine, New York, New York, U.S.A
| |
Collapse
|
12
|
Lawrence KW, Okewunmi JO, Chakrani Z, Cordero JK, Li X, Parisien RL. Randomized Controlled Trials Comparing Bone-Patellar Tendon-Bone Versus Hamstring Tendon Autografts in Anterior Cruciate Ligament Reconstruction Surgery Are Statistically Fragile: A Systematic Review. Arthroscopy 2024; 40:998-1005. [PMID: 37543146 DOI: 10.1016/j.arthro.2023.07.039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Revised: 06/07/2023] [Accepted: 07/27/2023] [Indexed: 08/07/2023]
Abstract
PURPOSE To assess the statistical fragility of recently published randomized controlled trials (RCTs) comparing the use of hamstring tendon autograft with bone-patellar tendon-bone autograft for anterior cruciate ligament (ACL) reconstruction. METHODS The PubMed, Embase, and MEDLINE databases were queried for RCTs published since 2010 comparing autograft type (bone-patellar tendon-bone vs hamstring tendon) in ACL reconstruction surgery. The fragility index (FI) and reverse FI (rFI) were determined for significant and nonsignificant outcomes, respectively, as the number of outcome reversals required to change statistical significance. The fragility quotient (FQ) and reverse FQ, representing fragility as a proportion of the study population, were calculated by dividing the FI and rFI, respectively, by the sample size. RESULTS We identified 19 RCTs reporting 55 total dichotomous outcomes. The median FI of the 55 total outcomes was 5 (interquartile range [IQR], 4-7), meaning a median of 5 outcome event reversals would alter the outcomes' significance. Five outcomes were reported as statistically significant with a median FI of 4 (IQR, 2-6), meaning a median of 4 outcome event reversals would change outcomes to be nonsignificant. Fifty outcomes were reported as nonsignificant with a median rFI of 5 (IQR, 4-7), meaning a median of 5 outcome event reversals would change outcomes to be significant. The FQ and reverse FQ for significant and nonsignificant outcomes were 0.025 (IQR, 0.018-0.045) and 0.082 (IQR, 0.041-0.106), respectively. For 61.8% of outcomes, patients lost to follow-up exceeded the corresponding FI or rFI. CONCLUSIONS There is substantial statistical fragility in recent RCTs on autograft choice in ACL reconstruction surgery given that altering a few outcome events is sufficient to reverse study findings. For over half of outcomes, maintaining patients lost to follow-up may have been sufficient to reverse study conclusions. CLINICAL RELEVANCE We recommend co-reporting FIs and P values to provide a more comprehensive representation of a study's conclusions when conducting an RCT.
Collapse
Affiliation(s)
- Kyle W Lawrence
- Boston University School of Medicine, Boston, Massachusetts, U.S.A..
| | | | - Zakaria Chakrani
- Icahn School of Medicine at Mount Sinai, New York, New York, U.S.A
| | - John K Cordero
- Icahn School of Medicine at Mount Sinai, New York, New York, U.S.A
| | - Xinning Li
- Boston University School of Medicine, Boston, Massachusetts, U.S.A
| | | |
Collapse
|
13
|
Cote MP, Asnis P, Hutchinson ID, Berkson E. Editorial Commentary: The Statistical Fragility Index of Medical Trials Is Low By Design: Critical Evaluation of Confidence Intervals Is Required. Arthroscopy 2024; 40:1006-1008. [PMID: 38219106 DOI: 10.1016/j.arthro.2023.10.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 10/12/2023] [Indexed: 01/15/2024]
Abstract
The Fragility Index (FI) provides the number of patients whose outcome would need to have changed for the results of a clinical trial to no longer be statistically significant. Although it's a well-intended and easily interpreted metric, its calculation is based on reversing a significant finding and therefore its interpretation is only relevant in the domain of statistical significance. Its interpretation is only relevant in the domain of statistical significance. A well-designed clinical trial includes an a priori sample size calculation that aims to find the bare minimum of patients needed to obtain statistical significance. Such trials are fragile by design! Examining the robustness of clinical trials requires an estimation of uncertainty, rather than a misconstrued, dichotomous focus on statistical significance. Confidence intervals (CIs) provide a range of values that are compatible with a study's data and help determine the precision of results and the compatibility of the data with different hypotheses. The width of the CI speaks to the precision of the results, and the extent to which the values contained within have potential to be clinically important. Finally, one should not assume that a large FI indicates robust findings. Poorly executed trials are prone to bias, leading to large effects, and therefore, small P values, and a large FI. Let's move our future focus from the FI toward the CI.
Collapse
Affiliation(s)
| | | | | | - Eric Berkson
- Boston, Massachusetts, U.S.A.; Foxborough, Massachusetts, U.S.A
| |
Collapse
|
14
|
Yendluri A, Alexanian A, Chari RR, Corvi JJ, Namiri NK, Song J, Alaia MJ, Li X, Parisien RL. The Statistical Fragility of Marrow Stimulation for Cartilage Defects of the Knee: A Systematic Review of Randomized Controlled Trials. Cartilage 2024:19476035241233441. [PMID: 38403983 DOI: 10.1177/19476035241233441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/27/2024] Open
Abstract
OBJECTIVE Marrow stimulation is used to address knee cartilage defects. In this study, we used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate statistical fragility of outcomes reported in randomized controlled trials (RCTs) evaluating marrow stimulation. DESIGN PubMed, Embase, and MEDLINE were queried for recent RCTs (January 1, 2010-September 5, 2023) assessing marrow stimulation for cartilage defects of the knee. The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI by the study sample size. RESULTS Across 155 total outcomes from 21 RCTs, the median FI was 3 (interquartile range [IQR], 2-5), with an associated median FQ of 0.067 (IQR, 0.033-0.010). Thirty-two outcomes were statistically significant, with a median FI of 2 (IQR, 1-3.25) and FQ of 0.050 (IQR, 0.025-0.069). Ten of the 32 (31.3%) outcomes reported as statistically significant had an FI of 1. In total, 123 outcomes were nonsignificant, with a median rFI of 3 (IQR, 2-5). Studies assessing stem cell augments were the most fragile, with a median FI of 2. In 55.5% of outcomes, the number of patients lost to follow-up was greater than or equal to the FI. CONCLUSION Statistical findings in RCTs evaluating marrow stimulation for cartilage defects of the knee are statistically fragile. We recommend combined reporting of P-values with FI and FQ metrics to aid in the interpretation of clinical findings in comparative trials assessing cartilage restoration.
Collapse
Affiliation(s)
- Avanish Yendluri
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Rohit R Chari
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - John J Corvi
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Nikan K Namiri
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Junho Song
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Michael J Alaia
- Department of Orthopaedic Surgery, New York University Langone Health, New York, NY, USA
| | - Xinning Li
- Department of Orthopaedic Surgery, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
| | - Robert L Parisien
- Department of Orthopedic Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
15
|
Megafu M, Megafu E, Mian H, Singhal S, Nietsch K, Yendluri A, Tornetta P, Parisien RL. The statistical fragility of outcomes in calcaneus fractures: A systematic review of randomized controlled trials. Foot (Edinb) 2023; 57:102047. [PMID: 37672893 DOI: 10.1016/j.foot.2023.102047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 08/23/2023] [Indexed: 09/08/2023]
Abstract
INTRODUCTION The purpose of this study was to utilize the fragility index to assess the robustness of randomized controlled trials (RCTs) evaluating the management of calcaneus fractures. We hypothesize that the dichotomous outcomes in calcaneus fracture literature will be statistically fragile and comparable to other orthopedic specialties. METHODS We performed a PubMed search for calcaneus fracture RCTs from 2000 to 2022 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The fragility index (FI) of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The fragility quotient (FQ) was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ. RESULTS Of the 3003 studies screened, 97 met the search criteria, with 19 RCTs evaluating calcaneus fractures included in the analysis. Seventy-nine dichotomous outcomes with 30 significant (P < 0.05) outcomes and 49 with nonsignificant (P> 0.05) outcomes were identified. The overall FI and FQ of all outcomes were 6 (IQR 3-8) and 0.067 (IQR 0.032-0.100), respectively. CONCLUSIONS The literature surrounding calcaneus fractures may not be as statistically stable as previously thought. The sole reliance on the P value may depict misleading results. We, therefore, recommend reporting the P value in conjunction with the FI and FQ to give a robust contextualization of clinical findings in the calcaneus fracture literature.
Collapse
Affiliation(s)
- Michael Megafu
- A.T. Still University Kirksville College of Osteopathic Medicine, Kirksville, MO, USA.
| | - Emmanuel Megafu
- Geisinger Commonwealth School of Medicine, Scranton, PA, USA
| | - Hassan Mian
- University of Minnesota Medical School, Twin Cities Campus, Minneapolis, MN, USA
| | - Sulabh Singhal
- Drexel University College of Medicine, Philadelphia, PA, USA
| | | | | | - Paul Tornetta
- Boston University School of Medicine, Department of Orthopedic Surgery, Boston, MA, USA
| | - Robert L Parisien
- Ichan School of Medicine at Mount Sinai, New York, NY, USA; Mount Sinai Hospital, Department of Orthopedic Surgery, New York, NY, USA
| |
Collapse
|
16
|
Bleakley CM, Wagemans J, Schurz AP, Smoliga JM. How robust are clinical trials in primary and secondary ankle sprain prevention? Phys Ther Sport 2023; 64:85-90. [PMID: 37801794 DOI: 10.1016/j.ptsp.2023.08.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 08/24/2023] [Accepted: 08/25/2023] [Indexed: 10/08/2023]
Abstract
OBJECTIVES Determine the statistical stability of RCTs examining primary and secondary prevention of ankle sprains. METHODS Databases were searched to August 2023. We included parallel design RCTs, using conservative interventions for preventing ankle sprain, reporting dichotomous injury event outcomes. Statistical stability was quantified using Fragility Index (FI) and Fragility Quotient (FQ). Subgroup analyses were undertaken to test if FI varied based on by study objective, original approach to analysis (frequency vs time to event), follow-up duration, and pre-registration. RESULTS 3559 studies were screened with 45 RCTs included. The median number of events required to change the statistical significance (FI) was 4 (IQR 1-6). FI was similar regardless of study objective, original analysis, follow-up duration, and pre-registration status. Median (IQR) FQ was 0.015 (0.005-0.046), therefore reversing events <2 patients/100 would alter significance. In 80% of studies the number of patients lost to follow-up was greater than the FI. CONCLUSION RCTs informing primary and secondary prevention of ankle sprain are fragile. Only a small percentage of outcome event reversals would reverse study significance, and this is often exceeded by the number of drop outs. Robust reporting of dichotomous outcomes requires the use P values and key metrics such as FI or FQ.
Collapse
Affiliation(s)
- C M Bleakley
- Faculty of Life and Health Sciences, Ulster University, Belfast, United Kingdom.
| | - J Wagemans
- Faculty of Medicine and Health Sciences, University of Antwerp, Belgium
| | - A P Schurz
- Department of Health Professions, Bern University of Applied Sciences, Switzerland; Faculty of Physical Education and Physiotherapy, Vrije Universiteit Brussels, Belgium
| | - J M Smoliga
- Department of Physical Therapy, High Point University, United States; School of Medicine, Tufts University, United States
| |
Collapse
|
17
|
Cordero JK, Lawrence KW, Brown AN, Li X, Hayden BL, Parisien RL. The Fragility of Tourniquet Use in Total Knee Arthroplasty: A Systematic Review of Randomized Controlled Trials. J Arthroplasty 2023; 38:1177-1183. [PMID: 36566999 DOI: 10.1016/j.arth.2022.12.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/07/2022] [Revised: 12/17/2022] [Accepted: 12/18/2022] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND Physicians utilize P-values to interpret clinical trial data and guide patient-care decisions. Fragility analysis assesses the stability of statistical findings in relation to outcome event reversals. This study assessed the statistical fragility of recent randomized controlled trials (RCTs) investigating tourniquet use in total knee arthroplasty (TKA). METHODS We queried PubMed, EMBASE, and MEDLINE for RCTs comparing outcomes in TKA based on tourniquet use. Fragility index (FI) and reverse fragility index (reverse FI) were calculated - for significant and nonsignificant outcomes, respectively - as the number of outcome reversals required to change statistical significance. The fragility quotient (FQ) was calculated by dividing the FI or reverse FI by the sample size. Median overall FI and FQ were calculated for all included outcomes, and sub-analyses were performed by reported significance. The literature search yielded 23 studies reporting 91 total dichotomous outcomes. RESULTS Overall median FI was 4 with an interquartile range (IQR) of 3 to 6. Overall median FQ was 0.0476 (IQR 0.0291 to 0.0867). A total of 11 outcomes were statistically significant with a median FI and FQ of 2 (IQR 1.5 to 5) and 0.0200 (IQR 0.0148 to 0.0484), respectively. There were 80 outcomes that were nonsignificant with a median reverse FI of 4 (IQR 3 to 6). Loss to follow-up was greater than the median FI in 17.6% of outcomes. CONCLUSION Altering a small number of outcomes is often sufficient to reverse findings in RCTs evaluating tourniquet use in TKA. We recommend including fragility analyses to increase reliability in the interpretation of study conclusions.
Collapse
Affiliation(s)
- John K Cordero
- Icahn School of Medicine at Mount Sinai, New York, New York
| | | | - Ashley N Brown
- Icahn School of Medicine at Mount Sinai, New York, New York
| | - Xinning Li
- Boston University School of Medicine, Boston, Massachusetts
| | - Brett L Hayden
- Icahn School of Medicine at Mount Sinai, New York, New York
| | | |
Collapse
|
18
|
Finni T, Vanwanseele B. Towards modern understanding of the Achilles tendon properties in human movement research. J Biomech 2023; 152:111583. [PMID: 37086579 DOI: 10.1016/j.jbiomech.2023.111583] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 03/21/2023] [Accepted: 04/04/2023] [Indexed: 04/24/2023]
Abstract
The Achilles tendon (AT) is the strongest tendon in humans, yet it often suffers from injury. The mechanical properties of the AT afford efficient movement, power amplification and power attenuation during locomotor tasks. The properties and the unique structure of the AT as a common tendon for three muscles have been studied frequently in humans using in vivo methods since 1990's. As a part of the celebration of 50 years history of the International Society of Biomechanics, this paper reviews the history of the AT research focusing on its mechanical properties in humans. The questions addressed are: What are the most important mechanical properties of the Achilles tendon, how are they studied, what is their significance to human movement, and how do they adapt? We foresee that the ongoing developments in experimental methods and modeling can provide ways to advance knowledge of the complex three-dimensional structure and properties of the Achilles tendon in vivo, and to enable monitoring of the loading and recovery for optimizing individual adaptations.
Collapse
Affiliation(s)
- Taija Finni
- Faculty of Sport and Health Sciences, Neuromuscular Research Center, University of Jyväskylä, Finland.
| | - Benedicte Vanwanseele
- Faculty of Movement and Rehabilitation Science, Human Movement Biomechanics Research Group, KU Leuven, Belgium
| |
Collapse
|
19
|
Megafu MN, Megafu EC, Nguyen JT, Mian HS, Singhal SS, Parisien RL. The Statistical Fragility of Orbital Fractures: A Systematic Review of Randomized Controlled Trials. J Oral Maxillofac Surg 2023:S0278-2391(23)00209-4. [PMID: 36931316 DOI: 10.1016/j.joms.2023.02.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 02/14/2023] [Indexed: 03/15/2023]
Abstract
BACKGROUND The P value has often been used as a tool to determine the statistical significance and evaluate the statistical robustness of study findings in orthopedic literature. The purpose of this study is to apply both the fragility index (FI) and the fragility quotient (FQ) to evaluate the degree of statistical fragility in orbital fracture literature. We hypothesized that the dichotomous outcomes within the orbital fracture literature will be vulnerable to a small number of outcome event reversals and will be statistically fragile. METHODS Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), the authors identified all dichotomous data for randomized controlled trials (RCTs) in orbital fracture literature and performed a PubMed search from 2000 to 2022. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each FI by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ. RESULTS Of the 3,329 studies screened, 28 met the criteria with 10 RCTs evaluating orbital fractures included for analysis. A total of 58 outcome events with 22 significant (P < .05) outcomes and 36 nonsignificant (P ≥ .05) outcomes were identified. The overall FI and FQ for all 58 outcomes was 5 (IQR: 4 to 5) and 0.140 (IQR: 0.075 to 0.250), respectively. Fragility analysis of statistical significant outcomes and nonsignificant outcomes had an FI of 3.5 with no IQR and 5 (IQR 4-5), respectively. All of the studies reported a loss to follow-up data, where 20% (2) was greater than the overall FI of 5. CONCLUSION The orbital fracture literature provides treatment guidance by relying on statistical significant results from RCTs. However, the RCTs in the orbital fracture peer-reviewed literature may not be statistically stable as previously thought. The sole reliance of the P value may depict misleading results. Thus, we recommend standardizing the reporting of the P value, FI, and FQ in the orbital fracture literature to aid readers in reliably drawing conclusions based on fragility outcome measures impacting clinical decision-making.
Collapse
Affiliation(s)
- Michael N Megafu
- A.T. Still University, Kirksville College of Osteopathic Medicine, Kirksville, MO.
| | | | | | - Hassan S Mian
- University of Minnesota Medical School, Twin Cities Campus, Minneapolis, MN
| | | | - Robert L Parisien
- Mount Sinai Hospital, Department of Orthopedic Surgery, New York, NY
| |
Collapse
|
20
|
Megafu M, Mian H, Megafu E, Singhal S, Lee A, Cassie R, Tornetta P, Parisien R. The fragility of statistical significance in distal femur fractures: systematic review of randomized controlled trials. EUROPEAN JOURNAL OF ORTHOPAEDIC SURGERY & TRAUMATOLOGY : ORTHOPEDIE TRAUMATOLOGIE 2022:10.1007/s00590-022-03452-3. [PMID: 36461949 DOI: 10.1007/s00590-022-03452-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Accepted: 11/27/2022] [Indexed: 06/17/2023]
Abstract
PURPOSE The purpose of this study was to apply both the fragility index (FI) and fragility quotient (FQ) to evaluate the degree of statistical fragility in the distal femur fracture (DFF) literature. We hypothesized that the dichotomous outcomes within the DFF literature are statistically fragile. METHODS Using preferred reporting items for systematic reviews and meta-analyses, we performed a PubMed search for distal femur fractures clinical trials from 2000 to 2022 reporting dichotomous outcomes. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ. RESULTS Of the 4258 articles screened, 92 met the search criteria, with eleven RCTs included for analysis. Ninety eight outcome events with 25 significant (P < 0.05) outcomes and 73 nonsignificant (P > 0.05) outcomes were identified. The overall FI and FQ for all 98 outcomes were 5 (IQR 4-6) and 0.130 (IQR 0.087-0.174), respectively. Three studies (33.3%) reported loss to follow (LTF) greater than 5. CONCLUSIONS The randomized controlled trials in the peer-reviewed distal femur fracture literature may not be as robust as previously thought, as incorporating statistical analyses solely on a P value threshold is misleading. Standardized reporting of the P value, FI and FQ can help the clinician reliably draw conclusions based on the fragility of outcome measures.
Collapse
Affiliation(s)
- Michael Megafu
- Kirksville College of Osteopathic Medicine, A.T. Still University, Kirksville, MO, USA.
| | - Hassan Mian
- University of Minnesota Medical School, Twin Cities Campus, Minneapolis, MN, USA
| | | | - Sulabh Singhal
- Drexel University College of Medicine, Philadelphia, PA, USA
| | - Alexander Lee
- University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Richawna Cassie
- Department of Orthopedic Surgery, Mount Sinai Hospital, New York, NY, USA
| | - Paul Tornetta
- Department of Orthopedic Surgery, Boston University School of Medicine, Boston, MA, USA
| | - Robert Parisien
- Department of Orthopedic Surgery, Mount Sinai Hospital, New York, NY, USA
| |
Collapse
|