1
|
O'Leary R, La Rosa GRM, Vernooij R, Polosa R. Identifying spin bias of nonsignificant findings in biomedical studies. BMC Res Notes 2023; 16:50. [PMID: 37131244 PMCID: PMC10155298 DOI: 10.1186/s13104-023-06321-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 03/30/2023] [Indexed: 05/04/2023] Open
Abstract
OBJECTIVE The purpose of this research note is to share a technique for the identification of spin bias that we developed as part of a living systematic review on the cardiovascular testing of e-cigarette substitution for cigarette smoking. While some researchers have remarked on the subjective nature of ascertaining spin bias, our technique objectively documents forms of spin bias arising from the misrepresentation of nonsignificant findings and from the omission of data. RESULTS We offer a two-step process for the identification of spin bias consisting of tracking data and findings and recording of data discrepancies by describing how the spin bias was produced in the text. In this research note, we give an example of the documentation of spin bias from our systematic review. Our experience was that nonsignificant results were presented as causal or even as significant in the Discussion of studies. Spin bias distorts scientific research and misleads readers; therefore it behooves peer reviewers and journal editors to make the effort to detect and correct it.
Collapse
Affiliation(s)
- Renée O'Leary
- Center of Excellence for the Acceleration of Harm Reduction, University of Catania, Via Santa Sofia, 89 Torre Biologica 11 piano, 95123, Catania, Italy.
| | | | - Robin Vernooij
- Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, The Netherlands
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Riccardo Polosa
- Center of Excellence for the Acceleration of Harm Reduction, University of Catania, Via Santa Sofia, 89 Torre Biologica 11 piano, 95123, Catania, Italy
- Department of Clinical and Experimental Medicine, University of Catania, Catania, Italy
| |
Collapse
|
2
|
Nowlin R, Wirtz A, Wenger D, Ottwell R, Cook C, Arthur W, Sallee B, Levin J, Hartwell M, Wright D, Sealey M, Zhu L, Vassar M. Spin in Abstracts of Systematic Reviews and Meta-analyses of Melanoma Therapies: Cross-sectional Analysis. JMIR DERMATOLOGY 2022; 5:e33996. [PMID: 37632865 PMCID: PMC10334896 DOI: 10.2196/33996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Revised: 01/01/2022] [Accepted: 01/03/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Spin is defined as the misrepresentation of a study's results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various nondermatological conditions. OBJECTIVE The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles. METHODS We used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic. RESULTS A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38% (n=76) of the abstracts. The most common type of spin found was type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention), occurring 40 times; the least common was type 2 (title claims or suggests a beneficial effect of the experimental intervention not supported by the findings), which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin. The likelihood of an article containing spin has decreased annually (adjusted odds ratio 0.91, 95% CI 0.84-0.99). No significant correlation between funding source or other study characteristics and the presence of spin was identified. CONCLUSIONS We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but the prevalence of spin in these abstracts has been declining from 1992-2020.
Collapse
Affiliation(s)
- Ross Nowlin
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - Alexis Wirtz
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - David Wenger
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - Ryan Ottwell
- Department of Internal Medicine, University of Oklahoma College of Community Medicine, Tulsa, OK, United States
- Department of Dermatology, St. Joseph Mercy Hospital, Ann Arbor, MI, United States
| | - Courtney Cook
- Department of Dermatology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States
| | - Wade Arthur
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Fayetteville, AR, United States
| | - Brigitte Sallee
- Department of Dermatology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States
| | - Jarad Levin
- Department of Dermatology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States
| | - Micah Hartwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - Drew Wright
- Samuel J. Wood Library and C.V. Starr Biomedical Information Center, Weill Cornell Medical College, New York, NY, United States
| | - Meghan Sealey
- Department of Statistics, Oklahoma State University, Stillwater, OK, United States
| | - Lan Zhu
- Department of Statistics, Oklahoma State University, Stillwater, OK, United States
| | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| |
Collapse
|
3
|
Ottwell R, Esmond L, Rea W, Hartwell M, Som M, Harris R, Miao Z, Zhu L, Arthur W, Brachtenbach T, Wright DN, Vassar M. Spin Infrequently Occurs in Abstracts of Systematic Reviews For The Pharmacological Treatment of Type 2 Diabetes Mellitus. Diabet Med 2021; 38:e14653. [PMID: 34289158 DOI: 10.1111/dme.14653] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Accepted: 07/13/2021] [Indexed: 12/19/2022]
Abstract
AIMS Currently, there is a growing body of research demonstrating that spin - the misinterpretation and distortion of a study's findings - is common in different fields of medicine. To our knowledge, no study has investigated its presence in systematic reviews focused on diabetic therapies. METHODS We performed a cross-sectional study by searching MEDLINE and Embase for systematic reviews focused on pharmacologic treatments for type 2 diabetes mellitus. Our search retrieved 26,490 records, from which 199 studies were extracted in a masked, duplicate fashion. Each study was evaluated for the nine most severe types of spin and other study design parameters. Spin was presented as frequencies and odds ratios to identify associations between study characteristics. RESULTS Spin was identified in the abstracts of 15 systematic reviews (15/199, 7.5%). Spin type 5 was the most common type identified (7/199, 3.5%). Spin types 1, 2, 4, and 8 were not identified. In the last 5 years (2016-2021), 7 systematic reviews contained spin within their abstract. There was no association between spins presence and any extracted study characteristic . CONCLUSIONS Our findings show that spin infrequently occurs in abstracts of systematic reviews focused on pharmacologic therapies for type 2 diabetes mellitus. However, any amount of spin can lead to the distortion of a reader's interpretation of the study's findings. Thus, we provide recommendations with rationale to prevent spin in future systematic reviews.
Collapse
Affiliation(s)
- Ryan Ottwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Internal Medicine, University of Oklahoma, School of Community Medicine, Tulsa, OK, USA
- Samuel J. Wood Library & C.V. Starr Biomedical Information Center, Weill Cornell Medical College, New York, NY, USA
| | - Lindy Esmond
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - William Rea
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Kansas City University of Medicine and Biosciences, Joplin, MO, USA
| | - Micah Hartwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Mousumi Som
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Rachael Harris
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Zhuqi Miao
- Spears School of Business, Oklahoma State University, Stillwater, OK, USA
| | - Lan Zhu
- Department of Statistics, Oklahoma State University, Stillwater, OK, USA
| | - Wade Arthur
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Travis Brachtenbach
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Drew N Wright
- Spears School of Business, Oklahoma State University, Stillwater, OK, USA
| | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| |
Collapse
|
4
|
Cole WT, Wittl P, Arthur W, Ottwell R, Greiner B, Koshy G, Chronister J, Hartwell M, Staheli J, Wright DN, Sealey M, Zhu L, Vassar M. Spin in the abstracts of systematic reviews and metaanalyses focused on percutaneous coronary intervention. J Osteopath Med 2021; 121:723-731. [PMID: 34213843 DOI: 10.1515/jom-2021-0085] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2021] [Accepted: 05/25/2021] [Indexed: 01/10/2023]
Abstract
CONTEXT "Spin" is a form of bias that involves highlighting study results in a way that presents the conclusions about benefit or efficacy beyond the scope of the data. Spin in the abstract of published studies has the potential to affect patient care, making investigations about its presence and prevalence important for readers. OBJECTIVES To evaluate the most severe types of spin in the abstracts of systematic reviews and metaanalyses focused on percutaneous coronary intervention. METHODS Using a cross sectional study design, the authors searched MEDLINE and Embase with the terms "percutaneous coronary intervention," "percutaneous coronary revascularization," "PCI," "systematic review," "meta analysis," and "meta-analysis." To be considered for this study, the article must have (1) focused on PCI; (2) had either a systematic review or metaanalysis study design; (3) been conducted on human subjects; and (4) been available in English. Reviews were excluded if these criteria were not met. Each included article was assessed for the nine most severe types of spin as defined in a previously published article, as well as other study characteristics (type of intervention being compared, date the review was received, adherence of systematic review and/or meta-analysis to Preferred Reporting for Systematic Reviews or Metanalyses (PRISMA) guidelines, requirement of PRISMA guidelines by the publishing journal, the publishing journal's five-year impact factor, and sources of funding). RESULTS Our database search retrieved 7,038 records; 2,190 duplicates were removed. Initial title and abstract screening led to the exclusion of 4,367 records, and an additional 281 records were excluded during full text screening. An arbitrary limit of 200 articles was applied for this analysis; five additional articles were excluded for ineligible study design, so 195 were included in our final analysis. Spin was present in the abstracts of 43 studies from that pool (22.1%). Spin type 3-selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention-occurred most frequently (29; 14.8%). The presence of spin was not associated with any of the extracted study characteristics. CONCLUSIONS Our data showed that spin occurred in more than one in every five systematic reviews or metaanalyses of PCI. Spin has the potential to distort a reader's ability to translate the true findings of a study; therefore, efforts are needed to prevent spin from appearing in article summaries.
Collapse
Affiliation(s)
- Wesley Tanner Cole
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Peter Wittl
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Wade Arthur
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Ryan Ottwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Benjamin Greiner
- Department of Internal Medicine, University of Texas Medical Branch, Galveston, TX, USA
| | - Gershon Koshy
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Justin Chronister
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Micah Hartwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Jonathan Staheli
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Drew N Wright
- Samuel J. Wood Library & C. V. Starr Biomedical Information Center, Weill Cornell Medical College, New York, NY, USA
| | - Meghan Sealey
- Department of Statistics, Oklahoma State University, Stillwater, OK, USA
| | - Lan Zhu
- Department of Statistics, Oklahoma State University, Stillwater, OK, USA
| | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| |
Collapse
|