Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-analyses Covering Treatments for Achilles Tendon Ruptures

Reporting Bias in Systematic Reviews and Meta-Analyses Related to the Treatment of Distal Radius Fractures: The Presence of Spin in the Abstract

BACKGROUND

Spin is a form of reporting bias which suggests a treatment is beneficial despite a statistically nonsignificant difference in outcomes. Our purpose was to define the prevalence of spin within the abstracts of distal radius fracture (DRF) systematic reviews (SRs) and meta-analyses (MA). We also sought to identify article characteristics that were more likely to contain spin.

METHODS

We performed a SR of multiple databases to identify DRF SRs and MAs. Articles were screened and analyzed by 3 reviewers. We recorded article and journal characteristics including adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, funding disclosures, methodologic quality (AMSTAR 2 instrument), impact factor, and country of origin. Presence of the 9 most severe types of spin in abstracts were recorded. Unadjusted odds ratios (ORs) were calculated to analyze the association between article characteristics and the presence of spin.

RESULTS

A total of 112 articles were included. Spin was present in 46% of abstracts, with type 1 spin ("conclusions not supported by findings") most frequent (19%). Spin was present in 43% of abstracts in PRISMA-adhering journals compared to 49% in journals that did not (OR = 0.79, 95% confidence interval [CI] = 0.37-1.68). For articles originating from China, spin was present in 61% of abstracts compared to 39% of abstracts from other countries (OR = 2.55, 95% CI = 1.13-5.75).

CONCLUSIONS

In addition to low article quality, there are high rates of spin within the abstracts of SRs and MAs related to treatment of DRF. Articles within journals that adhere to PRISMA do not appear to contain less spin.

Evaluation of Spin in Systematic Reviews and Meta-Analyses of Minimally Invasive Surgical Techniques and Standard Microdiscectomies for Treating Lumbar Disc Herniation

STUDY DESIGN

Systematic review.

OBJECTIVES

Spin in scientific literature is defined as bias that overstates efficacy and/or underestimates harms of procedures undergoing review. While lumbar microdiscectomies (MD) are considered the gold standard for treating lumbar disc herniations (LDH), outcomes of novel procedures are being weighed against open MD. This study identifies the quantity and type of spin in systematic reviews and meta-analyses of LDH interventions.

METHODS

A search was conducted on the PubMed, Scopus, and SPORTDiscus databases for systematic reviews and meta-analyses evaluating the outcomes of MD against other LDH interventions. Each included study's abstract was assessed for the presence of the 15 most common types of spin, with full texts reviewed during cases of disagreement or for clarification. Full texts were used in the assessment of study quality per AMSTAR 2.

RESULTS

All 34 included studies were observed to have at least 1 form of spin, in either the abstract or full text. The most common type of spin identified was type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies"), which was observed in ten studies (10/34, 29.4%). There was a statistically significant association between studies not registered with PROSPERO and the failure to satisfy AMSTAR type 2 (P < .0001).

CONCLUSION

Misleading reporting is the most common category of spin in literature related to LDH. Spin overwhelmingly tends to go in the positive direction, with results inappropriately favoring the efficacy or safety of an experimental intervention.

Evaluation of Spin Bias in Systematic Reviews and Meta-analyses of Rotator Cuff Repair With Platelet-Rich Plasma

BACKGROUND

The use of platelet-rich plasma (PRP) in orthopaedics continues to increase. One common use of PRP is as an adjunct in rotator cuff repair surgery. Multiple systematic reviews and meta-analyses have summarized the data on PRP use in rotator cuff repair surgery. However, systematic reviews and meta-analyses are subject to spin bias, where authors' interpretations of results influence readers' interpretations.

PURPOSE

To evaluate spin in the abstracts of systematic reviews and meta-analyses of PRP with rotator cuff repair surgery.

STUDY DESIGN

Systematic review; Level of evidence, 3.

METHODS

A PubMed and Embase search was conducted using the terms rotator cuff repair and PRP and systematic review or meta-analysis. After review of 74 initial studies, 25 studies met the inclusion criteria. Study characteristics were documented, and each study was evaluated for the 15 most common forms of spin and using the AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews, Version 2) rating system. Correlations between spin types and study characteristics were evaluated using binary logistic regression for continuous independent variables and a chi-square test or Fisher exact test for categorical variables.

RESULTS

At least 1 form of spin was found in 56% (14/25) of the included studies. In regard to the 3 different categories of spin, a form of misleading interpretation was found in 56% (14/25) of the studies. A form of misleading reporting was found in 48% (12/25) of the studies. A form of inappropriate extrapolation was found in 16% (4/25) of the studies. A significant association was found between misleading interpretation and publication year (odds ratio [OR], 1.41 per year increase in publication; 95% CI, 1.04-1.92; P = .029) and misleading reporting and publication year (OR, 1.41 per year increase in publication; 95% CI, 1.02-1.95; P = .037). An association was found between inappropriate extrapolation and journal impact factor (OR, 0.21 per unit increase in impact factor; 95% CI, 0.044-0.99; P = .048).

CONCLUSION

A significant amount of spin was found in the abstracts of systematic reviews and meta-analyses of PRP use in rotator cuff repair surgery. Given the increasing use of PRP by clinicians and interest among patients, spin found in these studies may have a significant effect on clinical practice.

Evaluation of spin in reviews of biodegradable balloon spacers for massive irreparable rotator cuff tears

HYPOTHESIS

Clinical studies are often at risk of spin, a form of bias where beneficial claims are overstated while negative findings are minimized or dismissed. Spin is often more problematic in abstracts given their brevity and can result in the misrepresentation of a study's actual findings. The goal of this study is to aggregate primary and secondary studies reporting the clinical outcomes of the use of subacromial balloon spacers in the treatment of massive irreparable rotator cuff tears to identify the incidence of spin and find any significant association with study design parameters.

MATERIALS AND METHODS

This study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Independent searches were completed on 2 databases (PubMed and Embase) for primary studies, systematic and current concepts reviews, and meta-analyses and the results were compiled. Two authors independently screened the studies using a predetermined inclusion criteria and aggregated data including titles, publication journals and years, authors, study design, etc. Each study was independently assessed for the presence of 15 different types of spin. Statistical analysis was conducted to identify associations between study characteristics and spin.

RESULTS

Twenty-nine studies met the inclusion criteria for our analysis, of which 10 were reviews or meta-analyses and 19 were primary studies. Spin was identified in every study except for 2 (27/29, 93.1%). Type 3 spin, "Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention" and type 9 spin, "Conclusion claims the beneficial effect of the experimental treatment despite reporting bias" were most frequently noted in our study, both observed in 12/29 studies (41.4%). Date of publication, and adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses or "The International Prospective Register of Systematic Reviews" were study characteristics associated with a higher rate of certain types of spin. There was a statistically significant association between disclosure of external study funding source and the presence of spin type 4, but none of the other forms of spin.

CONCLUSION

Spin is highly prevalent in the abstracts of primary studies, systematic reviews, and meta-analyses discussing the use of subacromial balloon spacer technology in the treatment of massive irreparable rotator cuff tears. Our findings revealed that spin in the abstract tended to favor the balloon spacer intervention. Further efforts are required in the future to mitigate spin within the abstracts of published manuscripts.

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses of Ulnar Collateral Ligament Reconstruction

Purpose

To identify the quantity and types of spin present in systematic reviews and meta-analyses of ulnar collateral ligament reconstruction (UCLR) outcomes and to characterize the studies with spin to determine if any patterns exist.

Methods

This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In August 2022, PubMed, Scopus, and SportDiscus databases were searched using the terms "ulnar collateral ligament reconstruction" AND "systematic review" OR "meta-analysis." Each abstract was assessed for the presence of the 15 most common types of spin derived from a previously established methodology. General data that were extracted included study title, authors, publication year, journal, level of evidence, study design, funding source, reported adherence to PRISMA guidelines, preregistration of the study protocol, and methodologic quality per A Measurement Tool to Assess Systematic Reviews Version 2 (AMSTAR 2).

Results

In total, 122 studies were identified during the preliminary search, of which 19 met the inclusion criteria. Each study had at least 1 form of spin. The most common type of spin identified was type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") (7/19, 36.8%). AMSTAR type 9 ("Did the review authors use a satisfactory technique for assessing the RoB [risk of bias] in individual studies that were included in the review?") was associated with both a lower Clarivate Impact Factor (P = .001) and a lower Scopus CiteScore (P = .015). Studies receiving external funding were associated with the failure to satisfy AMSTAR type 3 ("Did the review authors explain their selection of the study designs for inclusion in the review?") (P = .047).

Conclusions

Spin is highly prevalent in the abstracts of systematic reviews and meta-analyses that investigate the outcomes of UCLR.

Clinical Relevance

Spin has been identified in peer-reviewed articles published on various topics, including many in orthopaedics. Systematic reviews and meta-analyses contain the most comprehensive evidence regarding a clinical question, so it is important to identify spin that may be included in these reports. Greater efforts are needed to ensure that the abstracts of papers accurately represent the results in the full text.

Lower re-rupture rates but higher complication rates following surgical versus conservative treatment of acute achilles tendon ruptures: a systematic review of overlapping meta-analyses

PURPOSE

To systematically review and evaluate the current meta-analyses for the treatment of acute Achilles tendon rupture (AATR). This study can provide clinicians with a clear overview of the current literature to aid clinical decision-making and the optimal formulation of treatment plans for AATR.

METHODS

Two independent reviewers searched PubMed and Embase on June 2, 2022 based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Assessment of evidence was twofold: level of evidence (LoE) and quality of evidence (QoE). LoE was evaluated using published criteria by The Journal of Bone and Joint Surgery and the QoE by the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) scale. Pooled complication rates were highlighted for significance in favour of one treatment arm or no significance.

RESULTS

There were 34 meta-analyses that met the eligibility criteria, with 28 studies of LoE 1, and the mean QoE was 9.8 ± 1.2. Significantly lower re-rupture rates were reported with surgical (2.3-5%) versus conservative treatment (3.9-13%), but conservative treatment was favoured in terms of lower complication rates. The re-rupture rates were not significantly different between percutaneous repair or minimally invasive surgery (MIS) compared to open repair, but MIS was favoured in terms of lower complication rates (7.5-10.4%). When comparing rehabilitation protocols following open repair (four studies), conservative treatment (nine studies), or combined (three studies), there was no significant difference in terms of re-rupture or obvious advantage in terms of lower complication rates between early versus later rehabilitation.

CONCLUSION

This systematic review found that surgical treatment was significantly favoured over conservative treatment for re-rupture, but conservative treatment had lower complication rates other than re-rupture, notably for infections and sural nerve injury. Open repair had similar re-rupture rates to MIS, but lower complication rates; however, the rate of sural nerve injuries was lower in open repair. When comparing earlier versus later rehabilitation, there was no difference in re-rupture rates or obvious advantage in complications between open repair, conservative treatment, or when combined. The findings of this study will allow clinicians to effectively counsel their patients on the postoperative outcomes and complications associated with different treatment approaches for AATR.

LEVEL OF EVIDENCE

IV.

Evaluation of Spin in the Clinical Literature of Suture Tape Augmentation for Ankle Instability

Background

Spin is defined as the use of specific reporting strategies to highlight the beneficial effect of a treatment despite nonsignificant results. The presence of spin in peer-reviewed literature can negatively impact clinical and research practices. The purpose of this study was to identify the quantity and types of spin present in primary studies and systematic reviews using suture tape augmentation for ankle instability as a model.

Methods

This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Each abstract was assessed for the presence of the 15 most common types of spin. Extracted data included study title, authors, publication year, journal, level of evidence, study design, funding, reported adherence to PRISMA guidelines, and PROSPERO registration. Full texts of systematic reviews were used in the assessment of study quality per A Measurement Tool to Assess Systematic Reviews Version 2 (AMSTAR 2).

Results

Nineteen studies were included in the final sample. At least 1 type of spin was identified in each study except one (18 of 19, 94.7%). The most common type of spin observed was type 3 ("selective reporting or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention") (6 of 19, 31.6%), The second most reported category of spin was type 4 ("the conclusion claims safety based on non-statistically significant results with a wide confidence interval") (4 of 19, 21.1%). Among systematic reviews, we identified type 5 ("the conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") in 4 out of 6 (66.7%) of the articles that were included. No significant associations were found between study characteristics and type of spin.

Conclusion

In this exploration of the introduction of a new technology, we identified spin to be highly present in the abstracts of primary studies and systematic reviews concerning suture tape augmentation for ankle instability. Steps should be taken by scientific journals to ensure that spin is minimized in the abstract to accurately reflect the quality of the intervention.

Assessment of Spin in the Abstracts of Systematic Reviews and Meta-analyses on Platelet-Rich Plasma Treatment in Orthopaedics: A Cross-sectional Analysis

Background

Systematic reviews on the use of platelet-rich plasma (PRP) in orthopaedic surgery are abundant in current published literature. However, a beautification of results (referred to as spin) has been noted in abstracts across various aspects of medicine.

Purpose

To determine the prevalence of spin in systematic reviews of PRP-related orthopaedic surgery abstracts.

Study Design

Cross-sectional study.

Methods

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and Murad and Wang guidelines, we conducted a search in Medline, Embase, and the Cochrane Database for reviews on PRP-related orthopaedic surgery. The search included studies published from inception until June 30, 2021. Included were systematic reviews written in English that involved the use of PRP in the treatment of orthopaedic injuries in human participants. The abstracts of the included reviews were evaluated for the top 9 types of spin as described by Yavchitz et al in 2016. We determined the relationship between spin and study characteristics using odds ratios.

Results

Of an initial 1560 studies, 176 were included. We found that 50 studies (28.4%) contained at least 1 form of spin. The 2 most common forms of spin found in our sample were type 5 ("Conclusion claims the beneficial effect of treatment despite high risk of bias"; n = 27 [15.3%]) and type 3 ("Selective reporting or overemphasis of efficacy in outcomes favoring beneficial effect of intervention"; n = 18 [10.2%]). No statistical significance was found between study characteristics and the presence of spin.

Conclusion

Spin was present in 28% of the systematic reviews that covered PRP-related orthopaedic treatments. Spin was not associated with general study characteristics, including adherence to PRISMA guidelines or funding. Journals and authors should be aware of spin in articles and avoid its usage.