Evaluation of spin in the abstracts of orthopedic trauma literature: A cross-sectional review

Reporting Bias in Systematic Reviews and Meta-Analyses Related to the Treatment of Distal Radius Fractures: The Presence of Spin in the Abstract

BACKGROUND

Spin is a form of reporting bias which suggests a treatment is beneficial despite a statistically nonsignificant difference in outcomes. Our purpose was to define the prevalence of spin within the abstracts of distal radius fracture (DRF) systematic reviews (SRs) and meta-analyses (MA). We also sought to identify article characteristics that were more likely to contain spin.

METHODS

We performed a SR of multiple databases to identify DRF SRs and MAs. Articles were screened and analyzed by 3 reviewers. We recorded article and journal characteristics including adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, funding disclosures, methodologic quality (AMSTAR 2 instrument), impact factor, and country of origin. Presence of the 9 most severe types of spin in abstracts were recorded. Unadjusted odds ratios (ORs) were calculated to analyze the association between article characteristics and the presence of spin.

RESULTS

A total of 112 articles were included. Spin was present in 46% of abstracts, with type 1 spin ("conclusions not supported by findings") most frequent (19%). Spin was present in 43% of abstracts in PRISMA-adhering journals compared to 49% in journals that did not (OR = 0.79, 95% confidence interval [CI] = 0.37-1.68). For articles originating from China, spin was present in 61% of abstracts compared to 39% of abstracts from other countries (OR = 2.55, 95% CI = 1.13-5.75).

CONCLUSIONS

In addition to low article quality, there are high rates of spin within the abstracts of SRs and MAs related to treatment of DRF. Articles within journals that adhere to PRISMA do not appear to contain less spin.

Evaluation of spin in reviews of biodegradable balloon spacers for massive irreparable rotator cuff tears

HYPOTHESIS

Clinical studies are often at risk of spin, a form of bias where beneficial claims are overstated while negative findings are minimized or dismissed. Spin is often more problematic in abstracts given their brevity and can result in the misrepresentation of a study's actual findings. The goal of this study is to aggregate primary and secondary studies reporting the clinical outcomes of the use of subacromial balloon spacers in the treatment of massive irreparable rotator cuff tears to identify the incidence of spin and find any significant association with study design parameters.

MATERIALS AND METHODS

This study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Independent searches were completed on 2 databases (PubMed and Embase) for primary studies, systematic and current concepts reviews, and meta-analyses and the results were compiled. Two authors independently screened the studies using a predetermined inclusion criteria and aggregated data including titles, publication journals and years, authors, study design, etc. Each study was independently assessed for the presence of 15 different types of spin. Statistical analysis was conducted to identify associations between study characteristics and spin.

RESULTS

Twenty-nine studies met the inclusion criteria for our analysis, of which 10 were reviews or meta-analyses and 19 were primary studies. Spin was identified in every study except for 2 (27/29, 93.1%). Type 3 spin, "Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention" and type 9 spin, "Conclusion claims the beneficial effect of the experimental treatment despite reporting bias" were most frequently noted in our study, both observed in 12/29 studies (41.4%). Date of publication, and adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses or "The International Prospective Register of Systematic Reviews" were study characteristics associated with a higher rate of certain types of spin. There was a statistically significant association between disclosure of external study funding source and the presence of spin type 4, but none of the other forms of spin.

CONCLUSION

Spin is highly prevalent in the abstracts of primary studies, systematic reviews, and meta-analyses discussing the use of subacromial balloon spacer technology in the treatment of massive irreparable rotator cuff tears. Our findings revealed that spin in the abstract tended to favor the balloon spacer intervention. Further efforts are required in the future to mitigate spin within the abstracts of published manuscripts.

Spin in the abstracts of randomized controlled trials of nurse-led care: A cross-sectional study

BACKGROUND

Nurse-led models of care had been reported as effective intervention approaches for improving health management and reducing hospitalizations of target patients in a number of studies. However, the reporting quality of studies in the field varied and there was a lack of literature evaluation.

OBJECTIVE

The objective of the study was to assess the reporting quality and spin of abstracts of published randomized controlled trials which had statistically not significant primary outcomes. Moreover, potential factors associated with the presence of spin were also assessed.

METHODS

Studies on nurse-led care were retrieved from PubMed from January 1st, 2017, to December 31st, 2021. Only randomized controlled trials with statistically not significant primary outcomes were included. Study screening and data extraction were carried out by two reviewers independently. The reporting quality of each abstract was evaluated by the Consolidated Standards of Reporting Trials statement, and spin strategies were analyzed using a pre-designed assessment form. Potential predictors for the presence of spin were analyzed by multivariate logistic regressions.

RESULTS

The overall reporting quality of the included 75 randomized controlled trial abstracts was not satisfying, with a median score of 16-item Consolidated Standards of Reporting Trials statement at 6 (IQR 5, 8). Forty abstracts used at least one spin strategy in abstracts. Among them, 18 (45.0 %) used spin strategies in the result section and 39 (97.5 %) had spin in the conclusion section. The most common spin strategy identified in abstracts was focusing on statistically significant secondary outcomes (12/40, 30.0 %) in the result section and claiming benefit with no consideration of statistically not significant results for the primary outcomes (32/40, 80.0 %) in the conclusion section. Based on the definition, 29 (72.5 %) abstracts were assessed to have high level of spin in the conclusions of abstracts. By multivariate logistic regression analyses, it was found that only geographic origin (reference: studies from Asian countries, OR = 0.118, 95 % CI 0.027 to 0.511, P = 0.004) and the Consolidated Standards of Reporting Trials statement score (reference: lower score, OR = 0.625, 95 % CI 0.470 to 0.829, P = 0.001) were significantly associated with the presence of spin in abstracts.

CONCLUSION

Among the randomized controlled trials with statistically not significant primary outcomes in the field of nurse-led care, the reporting quality of abstracts needs to be improved. Trials from Asian countries and with lower Consolidated Standards of Reporting Trials statement scores are more likely to present spin in abstracts. Findings reported in the result and conclusion sections of these abstracts need to be interpreted with caution.

Spin in the Abstracts of Meta-analyses and Systematic Reviews: Quadriceps Tendon Graft for Anterior Cruciate Ligament Reconstruction

BACKGROUND

Spin is a reporting bias that misrepresents research. Ultimately it can affect surgeon decision making and patient care. Anterior cruciate ligament (ACL) reconstruction is common, but debate continues over optimal treatment modalities.

PURPOSE

To identify the prevalence of spin in meta-analysis and systematic review abstracts regarding the treatment of ACL injuries with quadriceps tendon graft.

STUDY DESIGN

Cross-sectional study.

METHODS

Electronic libraries (MEDLINE, Embase, Web of Science, Google Scholar) were searched for meta-analyses and systematic reviews regarding the treatment of ACL tears with quadriceps tendon graft. The 9 most severe types of spin commonly found in abstracts were used as an evaluation tool to assess the articles. Two reviewers each performed a blinded assessment of each article for spin. A third reviewer helped after review was done to address any discrepancies between the original reviewers. Further evaluation included year of publication, number of citations, journal impact factor, and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) score.

RESULTS

The electronic database search resulted in 986 articles, of which 13 met our inclusion criteria. After review, we found that 53.8% (7/13) of the included articles contained 1 of the 9 most severe forms of spin. Of the 13 articles, 15.4% (n = 2) contained 2 types of spin, and 38.5% (n = 5) contained 1 type of spin. No studies contained ≥3 types of spin. Of the types of spin evaluated, the most prevalent (n = 4; 30.8%) was type 3 ("selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention"). All studies, regardless of the presence of spin, were found to be low or critically low quality according to the AMSTAR-2 assessment.

CONCLUSION

This study demonstrated the presence of spin in 53.8% of meta-analysis and systematic review abstracts pertaining to quadriceps tendon graft for ACL reconstruction. Orthopaedic surgeons should learn to recognize spin as they review articles when deciding the treatment course for ACL injuries. Additionally, strict criteria should be considered to reduce the prevalence of spin in orthopaedic literature.

Evaluation of Spin in the Clinical Literature of Suture Tape Augmentation for Ankle Instability

Background

Spin is defined as the use of specific reporting strategies to highlight the beneficial effect of a treatment despite nonsignificant results. The presence of spin in peer-reviewed literature can negatively impact clinical and research practices. The purpose of this study was to identify the quantity and types of spin present in primary studies and systematic reviews using suture tape augmentation for ankle instability as a model.

Methods

This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Each abstract was assessed for the presence of the 15 most common types of spin. Extracted data included study title, authors, publication year, journal, level of evidence, study design, funding, reported adherence to PRISMA guidelines, and PROSPERO registration. Full texts of systematic reviews were used in the assessment of study quality per A Measurement Tool to Assess Systematic Reviews Version 2 (AMSTAR 2).

Results

Nineteen studies were included in the final sample. At least 1 type of spin was identified in each study except one (18 of 19, 94.7%). The most common type of spin observed was type 3 ("selective reporting or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention") (6 of 19, 31.6%), The second most reported category of spin was type 4 ("the conclusion claims safety based on non-statistically significant results with a wide confidence interval") (4 of 19, 21.1%). Among systematic reviews, we identified type 5 ("the conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") in 4 out of 6 (66.7%) of the articles that were included. No significant associations were found between study characteristics and type of spin.

Conclusion

In this exploration of the introduction of a new technology, we identified spin to be highly present in the abstracts of primary studies and systematic reviews concerning suture tape augmentation for ankle instability. Steps should be taken by scientific journals to ensure that spin is minimized in the abstract to accurately reflect the quality of the intervention.

'Spin' among abstracts of randomised controlled trials in sleep medicine: A research-on-research study

STUDY OBJECTIVES

'Spin', using reporting strategies to distort study results, can mislead readers of medical research. This study aimed to evaluate the prevalence and characteristics of 'spin' among randomised controlled trial (RCT) abstracts published in sleep medicine journals, and to identify factors associated with its presence and severity.

METHODS

The search for RCTs published between 2010 and 2020 were conducted in seven reputable journals of sleep medicine. Abstracts of RCTs with statistically nonsignificant primary outcomes were included and analysed for 'spin', according to pre-determined 'spin' strategies. Chi-square tests or logistic regression analyses were performed to detect the association between characteristics of included abstracts and the presence and severity of 'spin'.

RESULTS

A total of 114 RCT abstracts were included in this study, of which 89 (78.1%) were identified as having at least one type of 'spin' strategy. Sixty-six abstracts (57.9%) had 'spin' in the Results section, 82 (71.9%) abstracts presented with 'spin' in the Conclusions section. The presence of 'spin' varied significantly among RCTs based on the different categories of research area (P=0.047) and the statistician involvement (P=0.045). Furthermore, research area (P=0.019) and funding status (P=0.033) were significant factors associated with the severity of 'spin'.

CONCLUSIONS

The prevalence of 'spin' is high among RCT abstracts in sleep medicine. This raise the need for researchers, editors and other stakeholders to be aware of the issue of 'spin' and make joint efforts to eliminate it in future publications.

Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance

The power of language to modify the reader's perception of interpreting biomedical results cannot be underestimated. Misreporting and misinterpretation are pressing problems in randomized controlled trials (RCT) output. This may be partially related to the statistical significance paradigm used in clinical trials centered around a P value below 0.05 cutoff. Strict use of this P value may lead to strategies of clinical researchers to describe their clinical results with P values approaching but not reaching the threshold to be "almost significant." The question is how phrases expressing nonsignificant results have been reported in RCTs over the past 30 years. To this end, we conducted a quantitative analysis of English full texts containing 567,758 RCTs recorded in PubMed between 1990 and 2020 (81.5% of all published RCTs in PubMed). We determined the exact presence of 505 predefined phrases denoting results that approach but do not cross the line of formal statistical significance (P < 0.05). We modeled temporal trends in phrase data with Bayesian linear regression. Evidence for temporal change was obtained through Bayes factor (BF) analysis. In a randomly sampled subset, the associated P values were manually extracted. We identified 61,741 phrases in 49,134 RCTs indicating almost significant results (8.65%; 95% confidence interval (CI): 8.58% to 8.73%). The overall prevalence of these phrases remained stable over time, with the most prevalent phrases being "marginally significant" (in 7,735 RCTs), "all but significant" (7,015), "a nonsignificant trend" (3,442), "failed to reach statistical significance" (2,578), and "a strong trend" (1,700). The strongest evidence for an increased temporal prevalence was found for "a numerical trend," "a positive trend," "an increasing trend," and "nominally significant." In contrast, the phrases "all but significant," "approaches statistical significance," "did not quite reach statistical significance," "difference was apparent," "failed to reach statistical significance," and "not quite significant" decreased over time. In a random sampled subset of 29,000 phrases, the manually identified and corresponding 11,926 P values, 68,1% ranged between 0.05 and 0.15 (CI: 67. to 69.0; median 0.06). Our results show that RCT reports regularly contain specific phrases describing marginally nonsignificant results to report P values close to but above the dominant 0.05 cutoff. The fact that the prevalence of the phrases remained stable over time indicates that this practice of broadly interpreting P values close to a predefined threshold remains prevalent. To enhance responsible and transparent interpretation of RCT results, researchers, clinicians, reviewers, and editors may reduce the focus on formal statistical significance thresholds and stimulate reporting of P values with corresponding effect sizes and CIs and focus on the clinical relevance of the statistical difference found in RCTs.