The Fragility of Statistically Significant Findings From Randomized Controlled Trials in Hip and Knee Arthroplasty

Randomized Controlled Trials Assessing Continuous Outcomes for the use of Platelet-Rich Plasma in Knee Osteoarthritis Are Statistically Fragile: A Systematic Review

PURPOSE

The aim of this study is to assess the statistical fragility of randomized controlled trials that assess the use of platelet rich plasma (PRP) for the treatment of knee osteoarthritis (OA) and report a continuous primary outcome measure with statistical significance.

METHODS

A systematic electronic search of MEDLINE, EMBASE, and Cochrane database was performed on July 26, 2023. All randomized controlled trials addressing the use of PRP for the treatment of symptomatic knee osteoarthritis were included that reported statistically significant primary continuous outcomes. The continuous fragility index (CFI) and continuous fragility quotient (CFQ) was calculated using approximative method as previously described using the mean and standard deviation of the outcomes of interest.

RESULTS

There was a total of 34 eligible outcomes for analysis. The overall median CFI across all included studies was 5.7 (IQR, 4.8 - 9.9). The overall median CFQ across all included studies was 0.131 (IQR, 0.055 - 0.243). Loss to follow-up was greater than the CFI in only 3 of 34 eligible outcomes (8.8%). The most analyzed outcome was the WOMAC total score (n=9) with a median CFI of 6.6 and median CFQ of 0.250. The outcome with the highest median CFI was the WOMAC stiffness subscale at 93. Spearman correlation analysis demonstrated a non-statistically significant trend towards decreasing CFI (-0.497) and a statistically significant decrease in CFQ (-0.681, p = 0.03) with increasing grades of osteoarthritis.

CONCLUSION

The was an overall median CFI of 5.7 and CFQ of 0.131 for RCTs that report statistically significant continuous outcomes for the use of PRP for symptomatic knee OA. Although there are no current guidelines regarding statistical fragility of continuous outcomes, these results can be considered fragile given statistical significance may be reversed with only a few changes in patient outcomes.

LEVEL OF EVIDENCE

2; systematic review of Level I and II studies.

Platelet-Rich Plasma Versus Alternative Injections for Osteoarthritis of the Knee: A Systematic Review and Statistical Fragility Index-Based Meta-analysis of Randomized Controlled Trials

BACKGROUND

Based in part on the results of randomized controlled trials (RCTs) that suggest a beneficial effect over alternative treatment options, the use of platelet-rich plasma (PRP) for the management of knee osteoarthritis (OA) is widespread and increasing. However, the extent to which these studies are vulnerable to slight variations in the outcomes of patients remains unknown.

PURPOSE

To evaluate the statistical fragility of conclusions from RCTs that reported outcomes of patients with knee OA who were treated with PRP versus alternative nonoperative management strategies.

STUDY DESIGN

Systematic review and meta-analysis; Level of evidence, 2.

METHODS

All RCTs comparing PRP with alternative nonoperative treatment options for knee OA were identified. The fragility index (FI) and reverse FI were applied to assess the robustness of conclusions regarding the efficacy of PRP for knee OA. Meta-analyses were performed to determine the minimum number of patients from ≥1 trials included in the meta-analysis for which a modification on the event status would change the statistical significance of the pooled treatment effect.

RESULTS

In total, this analysis included outcomes from 1993 patients with a mean ± SD age of 58.0 ± 3.8 years. The mean number of events required to reverse significance of individual RCTs (FI) was 4.57 ± 5.85. Based on random-effects meta-analyses, PRP demonstrated a significantly higher rate of successful outcomes when compared with hyaluronic acid (P = .002; odds ratio [OR], 2.19; 95% CI, 1.33-3.62), as well as higher rates of patient-reported symptom relief (P = .019; OR, 1.55; 95% CI, 1.07-2.24), not requiring a reintervention after the initial injection treatment (P = .002; OR, 2.17; 95% CI, 1.33-3.53), and achieving the minimal clinically important difference (MCID) for pain improvement (P = .007; OR, 6.19; 95% CI, 1.63-23.42) when compared with all alternative nonoperative treatments. Overall, the mean number of events per meta-analysis required to change the statistical significance of the pooled treatment effect was 8.67 ± 4.50.

CONCLUSION

Conclusions drawn from individual RCTs evaluating PRP for knee OA demonstrated slight robustness. On meta-analysis, PRP demonstrated a significant advantage over hyaluronic acid as well as improved symptom relief, lower rates of reintervention, and more frequent achievement of the MCID for pain improvement when compared with alternative nonoperative treatment options. Statistically significant pooled treatment effects evaluating PRP for knee OA are more robust than approximately half of all comparable meta-analyses in medicine and health care. Future RCTs and meta-analyses should consider reporting FIs and fragility quotients to facilitate interpretation of results in their proper context.

The Statistical Fragility of Patellar Resurfacing in Total Knee Arthroplasty: A Systematic Review of Randomized Controlled Trials

BACKGROUND

Randomized controlled trials (RCTs) evaluating patellar resurfacing in total knee arthroplasty (TKA) have conflicting findings, with some reporting its efficacy and others demonstrating no clinical significance. The fragility index (FI), reverse FI (rFI), and fragility quotient (FQ) evaluated statistical fragility of outcomes in RCTs evaluating patellar resurfacing in TKA.

METHODS

The PubMed, Embase, and MEDLINE databases were systematically searched for RCTs (January 1, 2000, to August 1, 2023) assessing patellar resurfacing in TKA. Of the 226 RCTs screened, 19 studies were included for analysis. We calculated FI and rFI, which represent the number of outcome event reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The outcome categories of interest included anterior knee pain, complications/adverse events, crepitus, reoperation, patient satisfaction, and clinical improvement. The FQ was determined by dividing the FI by the study sample size.

RESULTS

Across 46 outcomes, the median FI was 5 (interquartile range (IQR) 3 to 8) with a median FQ of 0.041 (IQR 0.025 to 0.063). There were nine outcomes that were statistically significant, with a median FI of 3 (IQR 2 to 8) and a FQ of 0.011 (IQR 0.0044 to 0.039). There were 37 outcomes that were nonsignificant, with a median rFI of 5 (IQR 4 to 7) and FQ of 0.043 (IQR 0.031 to 0.062). Notably, in 47.8% of all outcomes, the number of patients lost to follow-up was greater than the outcome's respective FI or rFI. Outcomes regarding patient satisfaction (FI 4.5) and anterior knee pain (FI 5) were most fragile.

CONCLUSIONS

The outcomes of interest regarding patellar resurfacing from RCTs are statistically fragile, particularly significant outcomes and patient satisfaction outcomes. Data surrounding patellar resurfacing remain inconclusive, and combining P values with FI/FQ metrics may aid in interpreting patellar resurfacing findings. Future studies may mitigate fragility by obtaining higher follow-up rates and sample sizes.

LEVEL OF EVIDENCE

I.

Fragility Index Analysis of the 2018 Clinical Practice Guidelines on Tranexamic Acid Use in Total Joint Arthroplasty

BACKGROUND

The 2018 American Association of Hip and Knee Surgeons clinical practice guideline (CPG) 'tranexamic acid use in total joint arthroplasty' evaluated the efficacy and safety of tranexamic acid in primary total joint arthroplasty. The following review assessed the statistical fragility of the randomized controlled trial (RCT) outcomes on which the CPG recommendations were based using a fragility analysis.

METHODS

All dichotomous outcomes from the RCTs used to guide the CPG from its associated network, and direct meta-analyses were analyzed. Fragility and reverse fragility indices (FI and rFI) and quotients were calculated for each outcome. The mean indices and quotients were calculated for each guideline question, outcome category, and comparison of tranexamic dose, formulation, and administration timing.

RESULTS

This review evaluated 403 dichotomous outcomes on transfusion and complication rates associated with tranexamic acid (TXA) administration. The mean FI of significant outcomes of the CPG was 5.23, and the mean rFI of nonsignificant outcomes was 5.80. Outcomes assessing complication rates had a mean rFI of 6.48. Most outcomes on transfusion in categories comparing TXA to placebo administration had higher mean FIs than rFIs, and all outcomes comparing transfusion risk associated with different TXA formulations and doses had higher mean rFIs than FI or no associated significant outcomes.

CONCLUSIONS

The rFI and FIs calculated for this CPG are comparable to or higher than mean values reported across orthopaedic literature, indicating the relative statistical stability of its included outcomes. As we learn more about fragility analyses and their potential applications, this type of statistical analysis shows promise as a useful tool to incorporate into future guidelines to assess the quality of RCTs and evaluate the strength of recommendations.

The fragility index and reverse fragility index of FDA investigational device exemption trials in spinal fusion surgery: a systematic review

PURPOSE

FDA investigational device exemption (IDE) studies are considered a gold standard of assessing safety and efficacy of novel devices through RCTs. The fragility index (FI) has emerged as a means to assess robustness of statistically significant study results and inversely, the reverse fragility index (RFI) for non-significant differences. Previous authors have defined results as fragile if loss to follow up is greater than the FI or RFI. The aim of this study was to assess the FI, RFI, and robustness of data supplied by IDE studies in spinal surgery.

METHODS

This was a systematic review of the literature. Inclusion criteria included randomized controlled trials with dichotomous outcome measures conducted under IDE guidelines between 2000 and 2023. FI and RFI were calculated through successively changing events to non-events until the outcome changed to non-significance or significance, respectively. The fragility quotient (FQ) and reverse fragility quotient (RFQ) were calculated by dividing the FI and RFI, respectively, by the sample size.

RESULTS

Thirty-two studies met inclusion criteria with a total of 40 unique outcome measures; 240 outcomes were analyzed. Twenty-six studies reported 96 statistically significant results. The median FI was 6 (IQR: 3-9.25), and patients lost to follow up was greater than the FI in 99.0% (95/96) of results. The average FQ was 0.027. Thirty studies reported 144 statistically insignificant results and a median RFI of 6 (IQR: 4-8). The average RFQ extrapolated was 0.021, and loss to follow up was greater than the RFI in 98.6% (142/144) of results.

CONCLUSIONS

IDE studies in spine surgery are surprisingly fragile given their reputations, large sample sizes, and intent to establish safety in investigational devices. This study found a median FI and RFI of 6. The number of patients lost to follow-up was greater than FIand RFI in 98.8% (237/240) of reported outcomes. FQ and RFQ tell us that changes of two to three patients per hundred can flip the significance of reported outcomes. This is an important reminder of the limitations of RCTs. Analysis of fragility in future studies may help clarify the strength of the relationship between reported data and their conclusions.

The Number of Patients Lost to Follow-Up May Exceed the Fragility Index of a Randomized Controlled Trial Without Reversing Statistical Significance: A Systematic Review and Statistical Model

PURPOSE

To (1) analyze trends in the publishing of statistical fragility index (FI)-based systematic reviews in the orthopaedic literature, including the prevalence of misleading or inaccurate statements related to the statistical fragility of randomized controlled trials (RCTs) and patients lost to follow-up (LTF), and (2) determine whether RCTs with relatively "low" FIs are truly as sensitive to patients LTF as previously portrayed in the literature.

METHODS

All FI-based studies published in the orthopaedic literature were identified using the Cochrane Database of Systematic Reviews, Web of Science Core Collection, PubMed, and MEDLINE databases. All articles involving application of the FI or reverse FI to study the statistical fragility of studies in orthopaedics were eligible for inclusion in the study. Study characteristics, median FIs and sample sizes, and misleading or inaccurate statements related to the FI and patients LTF were recorded. Misleading or inaccurate statements-defined as those basing conclusions of trial fragility on the false assumption that adding patients LTF back to a trial has the same statistical effect as existing patients in a trial experiencing the opposite outcome-were determined by 2 authors. A theoretical RCT with a sample size of 100, P = .006, and FI of 4 was used to evaluate the difference in effect on statistical significance between flipping outcome events of patients already included in the trial (FI) and adding patients LTF back to the trial to show the true sensitivity of RCTs to patients LTF.

RESULTS

Of the 39 FI-based studies, 37 (95%) directly compared the FI with the number of patients LTF. Of these 37 studies, 22 (59%) included a statement regarding the FI and patients LTF that was determined to be inaccurate or misleading. In the theoretical RCT, a reversal of significance was not observed until 7 patients LTF (nearly twice the FI) were added to the trial in the distribution of maximal significance reversal.

CONCLUSIONS

The claim that any RCT in which the number of patients LTF exceeds the FI could potentially have its significance reversed simply by maintaining study follow-ups is commonly inaccurate and prevalent in orthopaedic studies applying the FI. Patients LTF and the FI are not equivalent. The minimum number of patients LTF required to flip the significance of a typical RCT was shown to be greater than the FI, suggesting that RCTs with relatively low FIs may not be as sensitive to patients LTF as previously portrayed in the literature; however, only a holistic approach that considers the context in which the trial was conducted, potential biases, and study results can determine the merits of any particular RCT.

CLINICAL RELEVANCE

Surgeons may benefit from re-examining their interpretation of prior FI reviews that have made claims of substantial RCT fragility based on comparisons between the FI and patients LTF; it is possible the results are more robust than previously believed.

Applying the fragility index to randomized controlled trials evaluating total neoadjuvant therapy for rectal cancer: A methodological survey

BACKGROUND

Recently, there has been significant interest in, and adoption of, total neoadjuvant therapy (TNT) for locally advanced rectal cancer (LARC). We designed the present study to assess the robustness of the randomized controlled trials (RCTs) evaluating contemporary TNTs for LARC using the fragility index (FI).

MATERIALS AND METHODS

Relevant articles were identified through a review article by Johnson et al. in the Canadian Journal of Surgery. Dichotomous outcomes within these RCTs were eligible for inclusion if the reported effect size had a p-value < 0.05. The main outcome was FI for each included outcome. Walsh et al.'s method of calculating FI was utilized. Correlations between FI and research characteristics were assessed using the Spearman's rank correlation coefficients. Risk of bias was assessed using Cochrane recommended tools.

RESULTS

Ten RCTs were identified with 25 outcomes having statistically significant differences between groups. Eleven outcomes were time-to-event outcomes, while the remainder were dichotomous outcomes. Approximately half (n = 13) were oncologic outcomes. The median FI was 2 (interquartile range [IQR] 1-16). The number of patients lost to follow-up exceeded the FI in 17 outcomes (68.0 %) and thus these results were considered "fragile". Lower FI was associated with high risk of bias (rho = -0.5594) and greater loss to follow-up (rho = -0.4394), while higher FI was associated with large study size (rho = 0.5120).

CONCLUSIONS

The robustness of outcomes from trials assessing TNT for LARC was found to be questionable. Most outcomes were fragile, as determined by the FI. This survey is limited by the number of included studies.

Fragility of Statistically Significant Outcomes in Colonic Diverticular Disease Randomized Trials: A Systematic Review

BACKGROUND

The p value has been criticized as an oversimplified determination of whether a treatment effect exists. One alternative is the fragility index. It is a representation of the minimum number of nonevents that would need to be converted to events to increase the p value above 0.05.

OBJECTIVE

To determine the fragility index of randomized controlled trials assessing the efficacy of interventions for patients with diverticular disease since 2010 to assess the robustness of current evidence.

DESIGN

MEDLINE, Embase, and Cochrane Central Register of Controlled Trials were searched from inception to August 2022.

SETTINGS

Articles were eligible for inclusion if they were randomized trials conducted between 2010 and 2022 with parallel, superiority designs evaluating interventions in patients with diverticular disease. Only randomized trials with dichotomous primary outcomes with an associated p value of <0.05 were considered for inclusion.

PARTICIPANTS

Any surgical or medical intervention for patients with diverticular disease.

MAIN OUTCOME MEASURES

The fragility index was determined by adding events and subtracting nonevents from the groups with the smaller number of events. Events were added until the p value exceeded 0.05. The smallest number of events required was considered the fragility index.

RESULTS

After screening 1271 citations, 15 randomized trials met the inclusion criteria. Nine of the studies evaluated surgical interventions and 6 evaluated medical interventions. The mean number of patients randomly assigned and lost to follow-up per randomized controlled trial was 92 (SD 35.3) and 9 (SD 11.4), respectively. The median fragility index was 1 (range, 0-5). The fragility indices for the included studies did not correlate significantly with any study characteristics.

LIMITATIONS

Small sample, heterogeneity, and lack of inclusion of studies with continuous outcomes.

CONCLUSIONS

The randomized trials evaluating surgical and medical interventions for diverticular disease are not robust. Changing a single-outcome event in most studies was sufficient to make a statistically significant study finding not significant. See Video Abstract .

FRAGILIDAD DE LOS RESULTADOS ESTADSTICAMENTE SIGNIFICATIVOS EN ENSAYOS ALEATORIOS DE ENFERMEDAD DIVERTICULAR DEL COLON UNA REVISIN SISTEMTICA

ANTECEDENTES:El valor p ha sido criticado por una determinación demasiado simplificada de si existe un efecto del tratamiento. Una alternativa es el Índice de Fragilidad. Es una representación del número mínimo de no eventos que deberían convertirse en eventos para aumentar el valor p por encima de 0,05.OBJETIVO:Determinar el IF de ensayos controlados aleatorios que evalúan la eficacia de las intervenciones para pacientes con enfermedad diverticular desde 2010 para evaluar la solidez de la evidencia actual.FUENTES DE DATOS:Se realizaron búsquedas en MEDLINE, Embase y CENTRAL desde el inicio hasta agosto de 2022.SELECCIÓN DE ESTUDIOS:Los artículos eran elegibles para su inclusión si eran ensayos aleatorizados realizados entre 2010 y 2022 con diseños paralelos de superioridad que evaluaran intervenciones en pacientes con enfermedad diverticular. Sólo se consideraron para su inclusión los ensayos aleatorizados con resultados primarios dicotómicos con un valor de p asociado menor que 0,05.INTERVENCIÓNES:Cualquier intervención quirúrgica o médica para pacientes con enfermedad diverticular.PRINCIPALES MEDIDAS DE VALORACIÓN:El índice de fragilidad se determinó sumando eventos y restando no eventos de los grupos con el menor número de eventos. Se agregaron eventos hasta que el valor p superó 0,05. El menor número de eventos requeridos se consideró índice de fragilidad.RESULTADOS:Después de examinar 1271 citas, 15 ensayos aleatorios cumplieron los criterios de inclusión. Nueve de los estudios evaluaron intervenciones quirúrgicas y seis evaluaron intervenciones médicas. El número medio de pacientes aleatorizados y perdidos durante el seguimiento por ECA fue 92 (DE 35,3) y 9 (DE 11,4), respectivamente. La mediana del índice de fragilidad fue 1 (rango: 0-5). Los índices de fragilidad de los estudios incluidos no se correlacionaron significativamente con ninguna característica del estudio.LIMITACIONES:Muestra pequeña, heterogeneidad y falta de inclusión de estudios con resultados continuos.CONCLUSIONES:Los ensayos aleatorios que evalúan las intervenciones quirúrgicas y médicas para la enfermedad diverticular no son sólidos. Cambiar un solo evento de resultado en la mayoría de los estudios fue suficiente para que un hallazgo estadísticamente significativo del estudio no fuera significativo. (Traducción- Dr. Ingrid Melo ).

The statistical fragility of intramedullary reaming in tibial nail fixation: a systematic review

PURPOSE

To report the statistical stability of prospective clinical trials evaluating the effect of intramedullary reaming on rates of non-union in tibial fractures through calculation of the fragility metrics for non-union rates and all other dichotomous outcomes.

METHODS

Literature search was conducted for prospective clinical trials evaluating the effect of intramedullary reaming on non-union rates in tibial nailing. All dichotomous outcomes were extracted from the manuscripts. The fragility index (FI) and reverse fragility index (RFI) were calculated by determining the number of event reversals required for a statistically significant outcome to lose significance and vice-versa. The fragility quotient (FQ) and reverse fragility quotient (RFQ) were calculated by dividing the FI or RFI by the sample size, respectively. Outcomes were defined as "fragile" if the FI or RFI was found to be less than or equal to the number of patients lost to follow-up.

RESULTS

Literature search identified 579 results which produced ten studies meeting the criteria for review. There were 111 outcomes identified for analysis, of which 89 (80%) exhibited statistical fragility. For reported outcomes across the studies the median and mean FI was 2, the median FQ was 0.019, the mean FQ was 0.030, the median RFI was 4, the mean RFI was 3.95, the median RFQ was 0.045, and the mean RFQ was 0.030. Four studies reported outcomes which were found to have an FI of 0.

CONCLUSIONS

The studies evaluating the effect of intramedullary reaming on tibial nail fixation demonstrate considerable fragility. On average, two event reversals for significant findings, and four event reversals for insignificant findings are sufficient to alter statistical significance.

LEVEL OF EVIDENCE

Level II, systematic review of Level I and Level II studies.

The Fragility of Tourniquet Use in Total Knee Arthroplasty: A Systematic Review of Randomized Controlled Trials

BACKGROUND

Physicians utilize P-values to interpret clinical trial data and guide patient-care decisions. Fragility analysis assesses the stability of statistical findings in relation to outcome event reversals. This study assessed the statistical fragility of recent randomized controlled trials (RCTs) investigating tourniquet use in total knee arthroplasty (TKA).

METHODS

We queried PubMed, EMBASE, and MEDLINE for RCTs comparing outcomes in TKA based on tourniquet use. Fragility index (FI) and reverse fragility index (reverse FI) were calculated - for significant and nonsignificant outcomes, respectively - as the number of outcome reversals required to change statistical significance. The fragility quotient (FQ) was calculated by dividing the FI or reverse FI by the sample size. Median overall FI and FQ were calculated for all included outcomes, and sub-analyses were performed by reported significance. The literature search yielded 23 studies reporting 91 total dichotomous outcomes.

RESULTS

Overall median FI was 4 with an interquartile range (IQR) of 3 to 6. Overall median FQ was 0.0476 (IQR 0.0291 to 0.0867). A total of 11 outcomes were statistically significant with a median FI and FQ of 2 (IQR 1.5 to 5) and 0.0200 (IQR 0.0148 to 0.0484), respectively. There were 80 outcomes that were nonsignificant with a median reverse FI of 4 (IQR 3 to 6). Loss to follow-up was greater than the median FI in 17.6% of outcomes.

CONCLUSION

Altering a small number of outcomes is often sufficient to reverse findings in RCTs evaluating tourniquet use in TKA. We recommend including fragility analyses to increase reliability in the interpretation of study conclusions.

Statistical Fragility of Venous Thromboembolism Prophylaxis Following Total Joint Arthroplasty

Background

Statistical fragility is a quantitative measure of the robustness of the statistical conclusions drawn in a study. Although statistical fragility has been comprehensively evaluated in the arthroplasty literature, the statistical fragility of large-scale randomized trials evaluating venous thromboembolism (VTE) prophylaxis has not been evaluated. The purpose of this study was to determine the utility of applying the fragility index (FI) and the fragility quotient (FQ) analysis to randomized controlled trials (RCTs) evaluating VTE prophylaxis following total joint arthroplasty.

Methods

A systematic review was performed by searching multiple databases to identify RCTs that evaluated VTE prophylaxis following total joint arthroplasty from 2000 to 2020. The FI was determined by manipulating each reported dichotomous outcome event until a reversal of significance was appreciated with 2 × 2 contingency tables. The associated FQ was determined by dividing the FI by the sample size.

Results

Thirty-two RCTs were ultimately included for analysis. The overall FI incorporating all 32 RCTs was only 7 (interquartile range 3-9), suggesting that the reversal of only 7 events is required to change study significance. The associated FQ was determined to be 0.01. Of the RCTs that reported lost-to-follow-up data, the majority of studies had lost-to-follow-up numbers greater than 7.

Conclusions

Our findings suggest that RCTs evaluating VTE prophylaxis following total hip arthroplasty and total knee arthroplasty may lack statistical stability as few outcome events are required to reverse the significance of outcomes. Future randomized trials should consider reporting FI and FQ along with the P value analysis to provide better context to the integrity of statistical stability.

Failure to disclose industry funding impacts outcomes in randomized controlled trials of platelet-rich plasma

PURPOSE

Platelet-rich plasma (PRP) represents a highly profitable biological therapy. Platelet-rich plasma is widely used to treat musculoskeletal disorders despite mixed evidence of its efficacy. As evidenced by literature from other domains, industry funding may influence the results of clinical trials. The objective of the current study was to determine the association between industry funding and positive results for randomized controlled trials (RCTs) assessing the efficacy of PRP in musculoskeletal disorders.

METHODS

A search of four databases was conducted. Included studies were RCTs comparing PRP to any non-PRP comparator in adults (18 years old or over) with musculoskeletal disorders and had full text available in English. Studies were excluded if they were published before 2016 or were non-human trials. A multivariate binomial logistic regression model was created to explore predictors of statistically significant findings. Covariates included the presence of industry funding, sample size, and length of study follow-up. 1440 records were screened with 87 trials included in the final analysis.

RESULTS

Of the 87 studies, 61 (70%) reported a statistically significant primary outcome. The presence of industry funding was not predictive of a statistically significant primary outcome [OR = 0.36, 95% CI 0.096-1.36, (n.s.)]. Studies that did not state whether industry funding was present had a higher chance of reporting a statistically significant primary outcome (OR = 3.61, 95% CI 1.1-11.9, p = 0.035). Sample size and length of follow-up were not predictive of a statistically significant primary outcome.

CONCLUSION

The results of the current study conclude that industry funding had no impact on the reporting of positive results for RCTs investigating PRP in musculoskeletal disorders. However, not disclosing sources of funding was associated with a higher likelihood of reporting positive results. The results of trials that fail to disclose funding sources should be interpreted with caution in the PRP literature.

LEVEL OF EVIDENCE

I.

The Statistical Fragility of Foot and Ankle Surgery Randomized Controlled Trials

Fragility index (FI) is a metric used to interpret the results of randomized controlled trials (RCTs), and describes the number of subjects that would need to be switched from event to non-event for a result to no longer be significant. Studies that analyze FI of RCTs in various orthopedic subspecialties have shown the RCTs to be largely underpowered and highly fragile. However, FI has not been assessed in foot and ankle RCTs. The MEDLINE and Embase online databases were searched from 1/1/2011 through 11/19/2021 for RCTs involving foot and ankle conditions. FI, fragility quotient (FQ), and difference between the FI and number of subjects lost to follow-up was calculated. Spearman correlation was performed to determine the relationship between sample size and FI. Overall, 1262 studies were identified of which 18 were included in the final analysis. The median sample size was 65 (interquartile range [IQR] 57-95.5), the median FI was 2 (IQR 1-2.5), and the median FQ was 0.026 (IQR 0.012-0.033). Ten of 15 (67%) studies with non-zero FI values had FI values less than the number of subjects lost to follow-up. There was linear association between FI and sample size (R² = 0.495, p-value: .031). This study demonstrates that RCTs in the field of foot and ankle surgery are highly fragile, similar to other orthopedic subspecialties.

The Statistical Fragility of Operative vs Nonoperative Management for Achilles Tendon Rupture: A Systematic Review of Comparative Studies

BACKGROUND

The statistical significance of randomized controlled trials (RCTs) and comparative studies is often conveyed utilizing the P value. However, P values are an imperfect measure and may be vulnerable to a small number of outcome reversals to alter statistical significance. The interpretation of the statistical strength of these studies may be aided by the inclusion of a Fragility Index (FI) and Fragility Quotient (FQ). This study examines the statistical stability of studies comparing operative vs nonoperative management for Achilles tendon rupture.

METHODS

A systematic search was performed of 10 orthopaedic journals between 2000 and 2021 for comparative studies focusing on management of Achilles tendon rupture reporting dichotomous outcome measures. FI for each outcome was determined by the number of event reversals necessary to alter significance (P < .05). FQ was calculated by dividing the FI by the respective sample size. Additional subgroup analyses were performed.

RESULTS

Of 8020 studies screened, 1062 met initial search criteria with 17 comparative studies ultimately included for analysis, 10 of which were RCTs. A total of 40 outcomes were examined. Overall, the median FI was 2.5 (interquartile range [IQR] 2-4), the mean FI was 2.90 (±1.58), the median FQ was 0.032 (IQR 0.012-0.069), and the mean FQ was 0.049 (±0.062). The FI was less than the number of patients lost to follow-up for 78% of outcomes.

CONCLUSION

Studies examining the efficacy of operative vs nonoperative management of Achilles tendon rupture may not be as statistically stable as previously thought. The average number of outcome reversals needed to alter the significance of a given study was 2.90. Future analyses may benefit from the inclusion of a fragility index and a fragility quotient in their statistical analyses.

Short-term outcomes vary by surgical approach in total hip arthroplasty: a network meta-analysis

BACKGROUND

The direct anterior approach (DAA) has increased in popularity in recent years. Proponents cite its muscle-sparing approach and purported reduction in pain and improvement in function when compared to the traditional surgical approaches. There remains controversy surrounding the validity of these claims. The objective of this study was to compare the common total hip surgical approaches in terms of pain scores, functional outcomes, opioid use and complications within the first 12 weeks postoperatively.

METHODS

A network meta-analysis of randomized controlled trials (RCT) comparing postoperative outcomes of different surgical approaches in primary THA up to 12 weeks was performed. PubMed, MEDLINE, Embase, Web of Science and SCOPUS were systematically searched from inception to May 2020. Outcomes included pain scores, functional outcome scores, length of stay (LOS), complications and opioid consumption.

RESULTS

Twenty-five RCTs (n = 2339) were included. The DAA demonstrated statistically significant improvement in Harris Hip Scores at 6 weeks when compared to the posterior and direct lateral approaches. The DAA reduced pain scores on postoperative day 2 and at 2 weeks compared to the direct lateral approach. The anterolateral approach was found to have a significantly shorter LOS compared to the other major surgical approaches. The differences in functional outcomes or pain scores did not surpass conventional cutoffs for a minimal clinically important difference.

CONCLUSION

The DAA led to functional improvements at 6 weeks compared to the posterior and direct lateral approaches and reduced postoperative pain compared to the direct lateral approach. However, these improvements failed to reach clinical significance. All major surgical approaches led to large improvements in function by 12 weeks with relatively low complication rates. Whether a short-term statistically significant improvement in function is sufficiently patient important to recommend DAA as a standard remains uncertain.

The fragility of statistically significant results in gynecologic surgery: A systematic review

OBJECTIVE

To use the fragility index (FI) to evaluate the robustness of gynaecologic surgery trials that report statistically significant results. The FI defines the minimum number of patients who must have an alternative outcome to alter statistical significance.

DATA SOURCES

We searched MEDLINE, Web of Science, Embase, and ClinicalTrials.gov from 2011 to 2021 to identify gynaecologic surgery randomized controlled trials (RCTs).

STUDY SELECTION

A total of 4775 trials were screened for eligibility. All included studies evaluated benign gynaecologic surgery interventions or peri-operative medical interventions. Only two-arm RCTs with statistically significant dichotomous primary outcomes were included. Ninety-three trials were ultimately included for analysis.

DATA EXTRACTION AND SYNTHESIS

Data from the included studies, including sample size, loss to follow-up, and number of events, were recorded. The FI of each study was calculated using a predefined technique. The overall FI and FIs by subgroup (clinical subspecialty, country of origin, and statistical test used) are reported as medians alongside their interquartile ranges (IQRs). The Kruskal-Wallis test was applied to find possible statistically significant relationships between FI and the nominal subgroups. Among this cohort, the median FI was 3 (IQR 1-7). The FI was 0 in 13 trials (14%), and in 39 trials (42%), the number of patients lost to follow-up was greater than the FI. The median FI within clinical subspecialty groups (general gynaecology, anaesthesia, urogynaecology, and fertility) did not differ (P = 0.122).

CONCLUSION

Statistically significant results of RCTs in gynaecologic surgery are fragile, suggesting that clinicians should interpret results with caution. This is particularly true when the number of patients lost to follow-up is greater than the FI. The FI serves as a quality metric that can be used to evaluate robustness of results when applying the outcomes of RCTs to clinical practice or guideline development.

Static versus dynamic fixation of distal tibiofibular syndesmosis: a systematic review of overlapping meta-analyses

PURPOSE

Multiple Level I meta-analyses were conducted comparing traditional static vs. more recently introduced dynamic strategies of fixation for injuries of the distal tibiofibular syndesmosis (TFS). The aim of this review was to assess their robustness and methodological quality, providing support in the choice of a treatment strategy in case of TFS injury using the highest level of evidence.

METHODS

In this systematic review, conducted in accordance with the PRISMA guidelines, meta-analyses/systematic reviews comparing static and dynamic fixation methods after acute TFS injury were identified. The robustness of studies was evaluated using the fragility index (FI) for meta-analysis and the fragility quotient (FQ). The risk of bias was evaluated using the Assessment of Multiple Systematic Reviews (AMSTAR) instrument. Finally, the Jadad was applied to select the study which provided the highest quality of evidence to develop recommendations for the fixation strategy of these lesions.

RESULTS

Out of 1.302 records, four Level I meta-analyses were included in this study. Analyzing the statistically significant dichotomous outcomes, the median FI was 3.5 (IQR, 2 to 5.5; range, 1 to 9), while the median FQ was 1.9% (IQR, 1 to 3.5; range 0.35 to 4.4). In total, 37% had an FI of 2 or less and 75% of outcomes had a FI of 4 or less. According to the AMSTAR score and Jadad algorithm, the largest meta-analysis was selected as the highest evidence provided so far.

CONCLUSION

The meta-analyses with statistically significant dichotomous outcomes comparing dynamic and static fixation for treating injuries of the distal tibiofibular syndesmosis are fragile, with a change in less than four patients or less than 2% of the study population sufficient to reverse a significant outcome to nonsignificant.

LEVEL OF EVIDENCE

Level I.

Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review

Background

The Fragility Index (FI) and Reverse Fragility Index are powerful tools to supplement the P value in evaluation of randomized clinical trial (RCT) outcomes. These metrics are defined as the number of patients needed to change the significance level of an outcome. The purpose of this study was to calculate these metrics for published RCTs in total joint arthroplasty (TJA).

Methods

We performed a systematic review of RCTs in TJA over the last decade. For each study, we calculated the FI (for statistically significant outcomes) or Reverse Fragility Index (for nonstatistically significant outcomes) for all dichotomous, categorical outcomes. We also used the Pearson correlation coefficient to evaluate publication-level variables.

Results

We included 104 studies with 473 outcomes; 92 were significant, and 381 were nonstatistically significant. The median FI was 6 overall and 4 and 7 for significant and nonsignificant outcomes, respectively. There was a positive correlation between FI and sample size (R = 0.14, P = .002) and between FI and P values (R = 0.197, P = .000012).

Conclusions

This study is the largest evaluation of FI in orthopedics literature to date. We found a median FI that was comparable to or higher than FIs calculated in other orthopedic subspecialties. Although the mean and median FIs were greater than the 2 recommended by the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines to demonstrate strong evidence, a large percentage of studies have an FI < 2. This suggests that the TJA literature is on par or slightly better than other subspecialties, but improvements must be made.

Level of Evidence

Level I; Systematic Review.

Assessment of Risk of Bias in Osteosarcoma and Ewing's Sarcoma Randomized Controlled Trials: A Systematic Review

Aim: The aim of this study was to systematically assess the risk of bias in osteosarcoma and Ewing’s sarcoma (ES) randomized controlled trials (RCT) and to examine the relationships between bias and conflict of interest/industry sponsorship. Methods: An OVID-MEDLINE search was performed (1976–2019). Using the Cochrane Collaboration guidelines, two reviewers independently assessed the prevalence of risk of bias in different RCT design domains. The relationship between conflicts of interest and industry funding with the frequency of bias was examined. Results: 73 RCTs met inclusion criteria. Prevalence of low-risk bias domains was 47.3%, unclear-risk domains 47.8%, and 4.9% of the domains had a high-risk of bias. Domains with the highest risk of bias were blinding of participants/personnel and outcome assessors, followed by randomization and allocation concealment. Overtime, frequency of unclear-risk of bias domains decreased (χ² = 5.32, p = 0.02), whilst low and high-risk domains increased (χ² = 8.13, p = 0.004). Studies with conflicts of interest and industry sponsorships were 4.2 and 3.1 times more likely to have design domains with a high-risk of bias (p < 0.05). Conclusion: This study demonstrates that sources of potential bias are prevalent in both osteosarcoma and ES RCTs. Studies with financial conflicts of interest and industry sponsors were significantly more likely to have domains with a high-risk of bias. Improvements in reporting and adherence to proper methodology will reduce the risk of bias and improve the validity of the results of RCTs in osteosarcoma and ES.