Study design and quality of reporting of randomized controlled trials of chronic idiopathic or autoimmune urticaria: review

Spin in Published Reports of Tinnitus Randomized Controlled Trials: Evidence of Overinterpretation of Results

Background: Spin refers to reporting practices that could distort the interpretation and mislead readers by being more optimistic than the results justify, thereby possibly changing the perception of clinicians and influence their decisions. Because of the clinical importance of accurate interpretation of results and the evidence of spin in other research fields, we aim to identify the nature and frequency of spin in published reports of tinnitus randomized controlled trials (RCTs) and to assess possible determinants and effects of spin. Methods: We searched PubMed systematically for RCTs with tinnitus-related outcomes published from 2015 to 2019. All eligible articles were assessed on actual and potential spin using prespecified criteria. Results: Our search identified 628 studies, of which 87 were eligible for evaluation. A total of 95% of the studies contained actual or potential spin. Actual spin was found mostly in the conclusion of articles, which reflected something else than the reported point estimate (or CI) of the outcome (n = 34, 39%) or which was selectively focused (n = 49, 56%). Linguistic spin ("trend," "marginally significant," or "tendency toward an effect") was found in 17% of the studies. We were not able to assess the association between study characteristics and the occurrence of spin due to the low number of trials for some categories of the study characteristics. We found no effect of spin on type of journal [odds ratio (OR) -0.13, 95% CI -0.56-0.31], journal impact factor (OR 0.17, 95% CI -0.18-0.51), or number of citations (OR 1.95, CI -2.74-6.65). Conclusion: There is a large amount of spin in tinnitus RCTs. Our findings show that there is room for improvement in reporting and interpretation of results. Awareness of different forms of spin must be raised to improve research quality and reduce research waste.

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses of Atopic Dermatitis Treatments and Interventions

BACKGROUND

Spin - the misrepresentation of a study's results - has been identified in abstracts of studies focused on a variety of disorders from multiple fields of medicine.

OBJECTIVES

This study's primary objective was to evaluate the abstracts of systematic reviews and meta-analyses focused on the treatment of atopic dermatitis for the nine most severe forms of spin.

METHODS

We systematically searched Embase and MEDLINE for systematic reviews of atopic dermatitis therapies. Screening and data extraction occurred in a masked, duplicate fashion. Each included study was evaluated for the nine most severe types of spin and other study characteristics.

RESULTS

Our searches retrieved 2,456 studies, of which 113 were included for data extraction. Spin was found in 74.3% of our included studies (84/113). Spin type 6 occurred most frequently (68/113, 60.2%). Spin types 1, 2, and 9 were not identified. All industry-funded systematic reviews contained spin in their abstract. The presence of spin was not associated with any specific study characteristics, including the methodological quality of the study.

CONCLUSIONS

Severe forms of spin were found in the majority of abstracts for systematic reviews of atopic dermatitis treatments. Steps should be taken to prevent spin to improve the quality of reporting in abstracts.

Evaluation of Spin in the Abstracts of Emergency Medicine Randomized Controlled Trials

STUDY OBJECTIVE

We aim to investigate spin in emergency medicine abstracts, using a sample of randomized controlled trials from high-impact-factor journals with statistically nonsignificant primary endpoints.

METHODS

This study investigated spin in abstracts of emergency medicine randomized controlled trials from emergency medicine literature, with studies from 2013 to 2017 from the top 5 emergency medicine journals and general medical journals. Investigators screened records for inclusion and extracted data for spin. We considered evidence of spin if trial authors focused on statistically significant results, interpreted statistically nonsignificant results as equivalent or noninferior, used favorable rhetoric in the interpretation of nonsignificant results, or claimed benefit of an intervention despite statistically nonsignificant results.

RESULTS

Of 772 abstracts screened, 114 randomized controlled trials reported statistically nonsignificant primary endpoints. Spin was found in 50 of 114 abstracts (44.3%). Industry-funded trials were more likely to have evidence of spin in the abstract (unadjusted odds ratio 3.4; 95% confidence interval 1.1 to 11.9). In the abstracts' results, evidence of spin was most often due to authors' emphasizing a statistically significant subgroup analysis (n=9). In the abstracts' conclusions, spin was most often due to authors' claiming they accomplished an objective that was not a prespecified endpoint (n=14).

CONCLUSION

Spin was prevalent in the selected randomized controlled trial, emergency medicine abstracts. Authors most commonly incorporated spin into their reports by focusing on statistically significant results for secondary outcomes or subgroup analyses when the primary outcome was statistically nonsignificant. Spin was more common in studies that had some component of industry funding.

Misrepresentation and distortion of research in biomedical literature

Publication in peer-reviewed journals is an essential step in the scientific process. However, publication is not simply the reporting of facts arising from a straightforward analysis thereof. Authors have broad latitude when writing their reports and may be tempted to consciously or unconsciously "spin" their study findings. Spin has been defined as a specific intentional or unintentional reporting that fails to faithfully reflect the nature and range of findings and that could affect the impression the results produce in readers. This article, based on a literature review, reports the various practices of spin from misreporting by "beautification" of methods to misreporting by misinterpreting the results. It provides data on the prevalence of some forms of spin in specific fields and the possible effects of some types of spin on readers' interpretation and research dissemination. We also discuss why researchers would spin their reports and possible ways to avoid it.

'Spin' in published biomedical literature: A methodological systematic review

In the scientific literature, spin refers to reporting practices that distort the interpretation of results and mislead readers so that results are viewed in a more favourable light. The presence of spin in biomedical research can negatively impact the development of further studies, clinical practice, and health policies. This systematic review aims to explore the nature and prevalence of spin in the biomedical literature. We searched MEDLINE, PreMEDLINE, Embase, Scopus, and hand searched reference lists for all reports that included the measurement of spin in the biomedical literature for at least 1 outcome. Two independent coders extracted data on the characteristics of reports and their included studies and all spin-related outcomes. Results were grouped inductively into themes by spin-related outcome and are presented as a narrative synthesis. We used meta-analyses to analyse the association of spin with industry sponsorship of research. We included 35 reports, which investigated spin in clinical trials, observational studies, diagnostic accuracy studies, systematic reviews, and meta-analyses. The nature of spin varied according to study design. The highest (but also greatest) variability in the prevalence of spin was present in trials. Some of the common practices used to spin results included detracting from statistically nonsignificant results and inappropriately using causal language. Source of funding was hypothesised by a few authors to be a factor associated with spin; however, results were inconclusive, possibly due to the heterogeneity of the included papers. Further research is needed to assess the impact of spin on readers’ decision-making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research in order to ensure accurate interpretation and dissemination of research.

In the scientific literature, spin refers to reporting practices that distort the interpretation of results and mislead readers so that results are viewed in a more favourable light. The presence of spin in biomedical research can negatively impact the development of further studies, clinical practice, and health policies. We conducted a systematic review to explore the nature and prevalence of spin in the biomedical literature. We included 35 reports, which investigated spin in clinical trials, observational studies, diagnostic accuracy studies, systematic reviews, and meta-analyses. The nature of spin varied according to study design. The highest (but also greatest) variability in the prevalence of spin was present in trials. Some of the common practices used to spin results included detracting from statistically nonsignificant results and inappropriately using causal language. Source of funding was hypothesised by a few authors to be a factor associated with spin; however, results were inconclusive, possibly due to the heterogeneity of the included papers. Further research is needed to assess the impact of spin on readers’ decision-making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research in order to ensure accurate interpretation and dissemination of research.

Overinterpretation of Research Findings: Evidence of "Spin" in Systematic Reviews of Diagnostic Accuracy Studies

BACKGROUND

We wished to assess the frequency of overinterpretation in systematic reviews of diagnostic accuracy studies.

METHODS

MEDLINE was searched through PubMed from December 2015 to January 2016. Systematic reviews of diagnostic accuracy studies in English were included if they reported one or more metaanalyses of accuracy estimates. We built and piloted a list of 10 items that represent actual overinterpretation in the abstract and/or full-text conclusion, and a list of 9 items that represent potential overinterpretation. Two investigators independently used the items to score each included systematic review, with disagreements resolved by consensus.

RESULTS

We included 112 systematic reviews. The majority had a positive conclusion regarding the accuracy or clinical usefulness of the investigated test in the abstract (n = 83; 74%) and full-text (n = 83; 74%). Of the 112 reviews, 81 (72%) contained at least 1 actual form of overinterpretation in the abstract, and 77 (69%) in the full-text. This was most often a "positive conclusion, not reflecting the reported summary accuracy estimates," in 55 (49%) abstracts and 56 (50%) full-texts and a "positive conclusion, not taking high risk of bias and/or applicability concerns into account," in 47 abstracts (42%) and 26 full-texts (23%). Of these 112 reviews, 107 (96%) contained a form of potential overinterpretation, most frequently "nonrecommended statistical methods for metaanalysis performed" (n = 57; 51%).

CONCLUSIONS

Most recent systematic reviews of diagnostic accuracy studies present positive conclusions and a majority contain a form of overinterpretation. This may lead to unjustified optimism about test performance and erroneous clinical decisions and recommendations.

If the results of an article are noteworthy, read the entire article; do not rely on the abstract alone

Clinicians typically update their knowledge by reading articles on the Internet. Easy access to the articles' abstracts and a lack of time to access other information sources creates a risk that therapeutic or diagnostic decisions will be made after reading just the abstracts. Occasionally, however, the abstracts of articles from clinical trials that have not obtained statistically significant differences in the primary study endpoint have reported other positive results, for example, of a secondary endpoint or a subgroup analysis. The article, however, correctly reports all results, including those of the primary endpoint. In the abstract, the safety information of the experimental treatment is usually deficient. The whole article should be read if a clinical decision is to be made.