Evaluation of spin in the abstracts of emergency medicine systematic reviews and meta-analyses

A guide to evaluating systematic reviews for the busy clinicians or reluctant readers

Systematic reviews (SRs) provide a solution to handle information overload for busy clinicians by summarising and synthesizing studies on a specific issue. However, because SRs are complicated and often boring to read, the busy or reluctant reader may make do with the abstract. When, as it has been shown, many authors overstate efficacy or understate harm in their abstracts, not consulting the underlying article could be misleading. This means that the prudent reader must have the ability to identify the 'tender points' of SRs to avoid falling for 'spin'. To this end we briefly review the method of SRs and ways to relatively quickly determine trustworthiness.

Spin in Abstracts of Systematic Reviews and Meta-analyses of Melanoma Therapies: Cross-sectional Analysis

BACKGROUND

Spin is defined as the misrepresentation of a study's results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various nondermatological conditions.

OBJECTIVE

The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles.

METHODS

We used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic.

RESULTS

A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38% (n=76) of the abstracts. The most common type of spin found was type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention), occurring 40 times; the least common was type 2 (title claims or suggests a beneficial effect of the experimental intervention not supported by the findings), which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin. The likelihood of an article containing spin has decreased annually (adjusted odds ratio 0.91, 95% CI 0.84-0.99). No significant correlation between funding source or other study characteristics and the presence of spin was identified.

CONCLUSIONS

We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but the prevalence of spin in these abstracts has been declining from 1992-2020.