1
|
Qureshi R, Naaman K, Quan NG, Mayo-Wilson E, Page MJ, Cornelius V, Chou R, Boutron I, Golder S, Bero L, Doshi P, Vassar M, Meursinge Reynders R, Li T. Development and Evaluation of a Framework for Identifying and Addressing Spin for Harms in Systematic Reviews of Interventions. Ann Intern Med 2024; 177:1089-1098. [PMID: 39008854 DOI: 10.7326/m24-0771] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/17/2024] Open
Abstract
"Spin" refers to misleading reporting, interpretation, and extrapolation of findings in primary and secondary research (such as in systematic reviews). The study of spin primarily focuses on beneficial outcomes. The objectives of this research were threefold: first, to develop a framework for identifying spin associated with harms in systematic reviews of interventions; second, to apply the framework to a set of reviews, thereby pinpointing instances where spin may be present; and finally, to revise the spin examples, offering guidance on how spin can be rectified. The authors developed their framework through an iterative process that engaged an international group of researchers specializing in spin and reporting bias. The framework comprises 12 specific types of spin for harms, grouped by 7 categories across the 3 domains (reporting, interpretation, and extrapolation). The authors subsequently gathered instances of spin from a random sample of 100 systematic reviews of interventions. Of the 58 reviews that assessed harm and the 42 that did not, they found that 28 (48%) and 6 (14%), respectively, had at least 1 of the 12 types of spin for harms. Inappropriate extrapolation of the results and conclusions for harms to populations, interventions, outcomes, or settings not assessed in a review was the most common category of spin in 17 of 100 reviews. The authors revised the examples to remove spin, taking into consideration the context (for example, medical discipline, source population), findings for harms, and methodological limitations of the original reviews. They provide guidance for authors, peer reviewers, and editors in recognizing and rectifying or (preferably) avoiding spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications.
Collapse
Affiliation(s)
- Riaz Qureshi
- University of Colorado Anschutz Medical Campus, Denver, Colorado (R.Q., N.G.Q., L.B., T.L.)
| | - Kevin Naaman
- Indiana University Bloomington, Bloomington, Indiana (K.N.)
| | - Nicolas G Quan
- University of Colorado Anschutz Medical Campus, Denver, Colorado (R.Q., N.G.Q., L.B., T.L.)
| | - Evan Mayo-Wilson
- University of North Carolina, Chapel Hill, North Carolina (E.M.)
| | - Matthew J Page
- Monash University, Melbourne, Victoria, Australia (M.J.P.)
| | | | - Roger Chou
- Oregon Health & Science University, Portland, Oregon (R.C.)
| | | | - Su Golder
- University of York, York, United Kingdom (S.G.)
| | - Lisa Bero
- University of Colorado Anschutz Medical Campus, Denver, Colorado (R.Q., N.G.Q., L.B., T.L.)
| | - Peter Doshi
- University of Maryland, Baltimore, Maryland (P.D.)
| | - Matt Vassar
- Oklahoma State University, Tulsa, Oklahoma (M.V.)
| | | | - Tianjing Li
- University of Colorado Anschutz Medical Campus, Denver, Colorado (R.Q., N.G.Q., L.B., T.L.)
| |
Collapse
|
2
|
Nguyen J, Li A, Tam DY, Forbes TL. ANALYSIS OF SPIN IN VASCULAR SURGERY RANDOMIZED CONTROLLED TRIALS WITH NONSIGNIFICANT OUTCOMES. J Vasc Surg 2021; 75:1074-1080.e17. [PMID: 34923067 DOI: 10.1016/j.jvs.2021.09.051] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Accepted: 09/25/2021] [Indexed: 12/17/2022]
Abstract
OBJECTIVES Spin is the manipulation of language that distorts the interpretation of objective findings. The purpose of this study is to describe the characteristics of spin found in statistically nonsignificant randomized controlled trials (RCT) comparing carotid endarterectomy (CEA) to carotid artery stenting (CAS) for carotid stenosis (CS), and endovascular repair (EVAR) to open repair (OR) for abdominal aortic aneurysms (AAA). METHODS A search of MEDLINE, EMBASE, and the Cochrane Controlled Register of Trials was performed in June 2020 for studies published describing AAA or CS. All phase three RCTs with nonsignificant primary outcomes comparing OR to EVAR or CEA to CAS were included. Studies were appraised for the characteristics and severity of spin using a validated tool. Binary logistic regression was performed to assess the association of spin grade to (1) funding source (commercial vs non-commercial) and (2) the publishing journal's impact factor. RESULTS Thirty-one of 355 articles captured were included for analysis. Spin was identified in nine abstracts (9/18) and 13 main texts (13/18) of AAA articles and seven abstracts (7/13) and ten main texts (10/13) of CS articles. For both AAA and CS articles, spin was most commonly found in the manuscript discussion section, with the most commonly employed strategy being the interpretation of statistically nonsignificant primary results to show treatment equivalence or rule out adverse treatment effects. Increasing journal impact factor was associated with a statistically significant lower likelihood of spin in the study title or abstract conclusion (βOR = 0.96, 95% CI: 0.94 - 0.98, p < 0.01) while no significant association could be found with funding source (βOR = 1.33, 95% CI: 0.30-5.92, p = 0.71). CONCLUSIONS A large proportion of statistically nonsignificant RCTs contain interpretations that are inconsistent with their results. These findings should prompt authors and readers to appraise study findings independently and to limit the use of spin in study interpretations.
Collapse
Affiliation(s)
| | - Allen Li
- University of Ottawa, Faculty of Medicine
| | - Derrick Y Tam
- Division of Cardiac Surgery, University of Toronto; Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Thomas L Forbes
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network; Department of Surgery, University of Toronto, Toronto, ON, Canada.
| |
Collapse
|