1
|
Mayo-Wilson E, Qureshi R, Li T. Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses. Syst Rev 2023; 12:67. [PMID: 37061724 PMCID: PMC10105415 DOI: 10.1186/s13643-023-02234-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 04/10/2023] [Indexed: 04/17/2023] Open
Abstract
Guidance for systematic reviews of interventions recommends both benefits and harms be included. Systematic reviews may reach conclusions about harms (or lack of harms) that are not true when reviews include only some relevant studies, rely on incomplete data from eligible studies, use inappropriate methods for synthesizing data, and report results selectively. Separate reviews about harms could address some of these problems, and we argue that conducting separate reviews of harms is a feasible alternative to current standards and practices. Systematic reviews of potential benefits could be organized around the use of interventions for specific health problems. Systematic reviews of potential harms could be broader, including more diverse study designs and including all people at risk of harms (who might use the same intervention to treat different health problems). Multiple reviews about benefits could refer to a single review of harms. This approach could improve the reliability, completeness, and efficiency of systematic reviews.
Collapse
Affiliation(s)
- Evan Mayo-Wilson
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, 27599, USA.
| | - Riaz Qureshi
- Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Tianjing Li
- Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|
2
|
Sargeant J, O'Connor A, O'Sullivan T, Ramirez A. Maximizing value and minimizing waste in clinical trials in swine: Selecting outcomes to build an evidence base. JOURNAL OF SWINE HEALTH AND PRODUCTION 2023. [DOI: 10.54846/jshap/1300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
Researchers planning clinical trials should identify the primary trial outcome and adequately power the trial to detect clinically meaningful differences in this outcome. All primary and secondary outcomes and their measurement should be comprehensively described, and their results reported. There is evidence that trials on the same subject use different outcomes or measure the same outcome in different ways, making it difficult to compare intervention effectiveness across clinical trials. Consensus development of core outcome sets could improve consistency in outcome measures used across trials and aid in development of an evidence-based body of literature on intervention effectiveness in swine populations.
Collapse
|
3
|
Guo CC, Chiesa PA, de Moor C, Fazeli MS, Schofield T, Hofer K, Belachew S, Scotland A. Digital Devices for Assessing Motor Functions in Mobility-Impaired and Healthy Populations: Systematic Literature Review. J Med Internet Res 2022; 24:e37683. [DOI: 10.2196/37683] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Revised: 07/18/2022] [Accepted: 10/11/2022] [Indexed: 11/22/2022] Open
Abstract
Background
With the advent of smart sensing technology, mobile and wearable devices can provide continuous and objective monitoring and assessment of motor function outcomes.
Objective
We aimed to describe the existing scientific literature on wearable and mobile technologies that are being used or tested for assessing motor functions in mobility-impaired and healthy adults and to evaluate the degree to which these devices provide clinically valid measures of motor function in these populations.
Methods
A systematic literature review was conducted by searching Embase, MEDLINE, CENTRAL (January 1, 2015, to June 24, 2020), the United States and European Union clinical trial registries, and the United States Food and Drug Administration website using predefined study selection criteria. Study selection, data extraction, and quality assessment were performed by 2 independent reviewers.
Results
A total of 91 publications representing 87 unique studies were included. The most represented clinical conditions were Parkinson disease (n=51 studies), followed by stroke (n=5), Huntington disease (n=5), and multiple sclerosis (n=2). A total of 42 motion-detecting devices were identified, and the majority (n=27, 64%) were created for the purpose of health care–related data collection, although approximately 25% were personal electronic devices (eg, smartphones and watches) and 11% were entertainment consoles (eg, Microsoft Kinect or Xbox and Nintendo Wii). The primary motion outcomes were related to gait (n=30), gross motor movements (n=25), and fine motor movements (n=23). As a group, sensor-derived motion data showed a mean sensitivity of 0.83 (SD 7.27), a mean specificity of 0.84 (SD 15.40), a mean accuracy of 0.90 (SD 5.87) in discriminating between diseased individuals and healthy controls, and a mean Pearson r validity coefficient of 0.52 (SD 0.22) relative to clinical measures. We did not find significant differences in the degree of validity between in-laboratory and at-home sensor-based assessments nor between device class (ie, health care–related device, personal electronic devices, and entertainment consoles).
Conclusions
Sensor-derived motion data can be leveraged to classify and quantify disease status for a variety of neurological conditions. However, most of the recent research on digital clinical measures is derived from proof-of-concept studies with considerable variation in methodological approaches, and much of the reviewed literature has focused on clinical validation, with less than one-quarter of the studies performing analytical validation. Overall, future research is crucially needed to further consolidate that sensor-derived motion data may lead to the development of robust and transformative digital measurements intended to predict, diagnose, and quantify neurological disease state and its longitudinal change.
Collapse
|
4
|
Chevance A, Ravaud P, Cornelius V, Mayo-Wilson E, Furukawa TA. Designing clinically useful psychopharmacological trials: challenges and ways forward. Lancet Psychiatry 2022; 9:584-594. [PMID: 35525252 DOI: 10.1016/s2215-0366(22)00041-4] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Revised: 01/15/2022] [Accepted: 02/01/2022] [Indexed: 12/23/2022]
Abstract
The clinical guidelines that underpin the use of drugs for mental disorders are informed by evidence from randomised controlled trials (RCTs). RCTs are performed to obtain marketing authorisation from regulators. The methods used in these RCTs could be appropriate for early phases of drug development because they identify drugs with important harms and drugs that are efficacious for specific health problems and populations. RCTs done before marketing authorisation do not tend to address clinical questions that concern the effectiveness of a drug in heterogeneous and comorbid populations, the optimisation of drug sequencing and discontinuation, or the comparative benefits and harms of different drugs that could be used for the same health problem. This Review proposes an overview of some shortcomings of RCTs, at an individual level and at the whole portfolio level, and identifies some methods in planning, conducting, and carrying out analyses in RCTs that could enhance their ability to support therapeutic decisions. These suggestions include: identifying patient-important questions to be investigated by psychopharmacological RCTs; embedding pragmatic RCTs within clinical practice to improve generalisability to target populations; collecting evidence about drugs in overlooked populations; developing methods to facilitate the recruitment of patients with mental disorders and to reduce the number of patients who drop out, using specific methods; using core outcome sets to standardise the assessment of benefits and harms; and recording systematically serious objective outcomes, such as suicide or hospitalisation, to be evaluated in meta-analyses. This work is a call to address questions relevant to patients using diverse design of RCTs, thus contributing to the development of a patient-centred, evidence-based psychiatry.
Collapse
Affiliation(s)
- Astrid Chevance
- Université Paris Cité, CRESS, INSERM, INRAE, Paris, France; Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu, AP-HP, Paris, France.
| | - Philippe Ravaud
- Université Paris Cité, CRESS, INSERM, INRAE, Paris, France; Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu, AP-HP, Paris, France
| | - Victoria Cornelius
- Imperial Clinical Trials Unit, School of Public Health, Imperial College London, London, UK
| | - Evan Mayo-Wilson
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, IN, USA
| | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior and Department of Clinical Epidemiology, School of Public Health, Kyoto University Graduate School of Medicine, Kyoto, Japan
| |
Collapse
|
5
|
Grant S, Gardner F, Bradshaw CP. Moving Toward Transparent, Open, and Reproducible Prevention Science: Introduction to the Special Issue. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2022; 23:697-700. [PMID: 35698022 DOI: 10.1007/s11121-022-01393-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/02/2022] [Indexed: 11/30/2022]
Affiliation(s)
- Sean Grant
- Department of Social & Behavioral Sciences, Richard M. Fairbanks School of Public Health, Indiana University, 1050 Wishard Blvd, Indianapolis, IN, 46202, USA.
| | - Frances Gardner
- Department of Social Policy and Intervention, University of Oxford, Oxford, UK
| | - Catherine P Bradshaw
- School of Education & Human Development, University of Virginia, Charlottesville, VA, USA
| |
Collapse
|
6
|
Qureshi R, Mayo-Wilson E, Rittiphairoj T, McAdams-DeMarco M, Guallar E, Li T. Harms in Systematic Reviews Paper 3: Given the same data sources, systematic reviews of gabapentin have different results for harms. J Clin Epidemiol 2022; 143:224-241. [PMID: 34742790 PMCID: PMC9875741 DOI: 10.1016/j.jclinepi.2021.10.025] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 10/27/2021] [Accepted: 10/29/2021] [Indexed: 01/27/2023]
Abstract
OBJECTIVE In this methodologic study (Part 2 of 2), we examined the overlap in sources of evidence and the corresponding results for harms in systematic reviews for gabapentin. STUDY DESIGN & SETTING We extracted all citations referenced as sources of evidence for harms of gabapentin from 70 systematic reviews, as well as the harms assessed and numerical results. We assessed consistency of harms between pairs of reviews with a high degree of overlap in sources of evidence (>50%) as determined by corrected covered area (CCA). RESULTS We found 514 reports cited across 70 included reviews. Most reports (244/514, 48%) were not cited in more than one review. Among 18 pairs of reviews, we found reviews had differences in which harms were assessed and their choice to meta-analyze estimates or present descriptive summaries. When a specific harm was meta-analyzed in a pair of reviews, we found similar effect estimates. CONCLUSION Differences in harms results across reviews can occur because the choice of harms is driven by reviewer preferences, rather than standardized approaches to selecting harms for assessment. A paradigm shift is needed in the current approach to synthesizing harms.
Collapse
Affiliation(s)
- Riaz Qureshi
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Evan Mayo-Wilson
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health, Bloomington, ID, USA
| | - Thanitsara Rittiphairoj
- Cochrane Eyes and Vision United States, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Mara McAdams-DeMarco
- Department of Surgery, Department of Epidemiology, Johns Hopkins School of Medicine and School of Public Health, Baltimore, MD, USA
| | - Eliseo Guallar
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Tianjing Li
- Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
| |
Collapse
|
7
|
Qureshi R, Mayo-Wilson E, Li T. Harms in Systematic Reviews Paper 1: An introduction to research on harms. J Clin Epidemiol 2022; 143:186-196. [PMID: 34742788 PMCID: PMC9126149 DOI: 10.1016/j.jclinepi.2021.10.023] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 10/27/2021] [Accepted: 10/29/2021] [Indexed: 11/15/2022]
Abstract
OBJECTIVE Most systematic reviews of interventions focus on potential benefits. Common methods and assumptions that are appropriate for assessing benefits can be inappropriate for harms. This paper provides a primer on researching harms, particularly in systematic reviews. STUDY DESIGN AND SETTING Commentary describing challenges with assessing harm. RESULTS Investigators should be familiar with various terminologies used to describe, classify, and group harms. Published reports of clinical trials include limited information about harms, so systematic reviewers should not depend on these studies and journal articles to reach conclusions about harms. Visualizations might improve communication of multiple dimensions of harms such as severity, relatedness, and timing. CONCLUSION The terminology, classification, detection, collection, and reporting of harms create unique challenges that take time, expertise, and resources to navigate in both primary studies and evidence syntheses. Systematic reviewers might reach incorrect conclusions if they focus on evidence about harms found in published reports of randomized trials of a particular health problem. Systematic reviews could be improved through better identification and reporting of harms in primary studies and through better training and uptake of appropriate methods for synthesizing evidence about harms.
Collapse
Affiliation(s)
- Riaz Qureshi
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Evan Mayo-Wilson
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health, Bloomington, ID, USA
| | - Tianjing Li
- Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
| |
Collapse
|
8
|
Sendyk DI, Souza NV, César Neto JB, Tatakis DN, Pannuti CM. Selective outcome reporting in root coverage randomized clinical trials. J Clin Periodontol 2021; 48:867-877. [PMID: 33745136 DOI: 10.1111/jcpe.13451] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2020] [Revised: 12/31/2020] [Accepted: 02/16/2021] [Indexed: 12/28/2022]
Abstract
BACKGROUND Outcome discrepancies between protocols and respective publications represent a concerning bias. The purpose of this study was to assess the prevalence of selective outcome reporting (SOR) in root coverage randomized clinical trials (RCTs). METHODS Published root coverage RCTs (July 2005 to March 2020) were included if a corresponding protocol could be identified in a public registry. Discrepancies between protocol and its correspondent publication(s) were compared regarding primary and secondary outcomes and other study characteristics. Associations between trial characteristics and SOR were evaluated. RESULTS Forty four studies (54 publications) were included. The majority of studies (77.3%) were retrospectively registered. SOR was frequent (40.9% of trials) and consisted of primary outcome downgrade (22.7%); secondary outcome upgrade (11.4%); new primary outcome introduced in publication (25%); protocol primary outcome omitted from publication (13.6%) and discrepancy in primary outcome timing (18.2%). SOR was unclear in 20.5% of studies and favoured statistical significance in 12 studies (27.3%). SOR was significantly associated with study significance (p < 0.001) and unclear outcome definition in the publication (p < 0.001). Only a third (32.8%) of primary outcomes were completely defined. CONCLUSIONS The present study identified high prevalence of SOR in root coverage RCTs.
Collapse
Affiliation(s)
- Daniel Isaac Sendyk
- Division of Periodontology, College of Dentistry, University of São Paulo, São Paulo, Brazil
| | - Nathalia Vilela Souza
- Division of Periodontology, College of Dentistry, University of São Paulo, São Paulo, Brazil
| | - João Batista César Neto
- Division of Periodontology, College of Dentistry, University of São Paulo, São Paulo, Brazil
| | - Dimitris N Tatakis
- Division of Periodontology, College of Dentistry, The Ohio State University, Columbus, OH, USA
| | - Cláudio Mendes Pannuti
- Division of Periodontology, College of Dentistry, University of São Paulo, São Paulo, Brazil
| |
Collapse
|
9
|
Goodman M, Li J, Flanders WD, Mahood D, Anthony LG, Zhang Q, LaKind JS. Epidemiology of PCBs and neurodevelopment: Systematic assessment of multiplicity and completeness of reporting. GLOBAL EPIDEMIOLOGY 2020. [DOI: 10.1016/j.gloepi.2020.100040] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
|
10
|
Page MJ, Bero L, Kroeger CM, Dai Z, McDonald S, Forbes A, McKenzie JE. Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol. F1000Res 2019. [PMID: 32117567 DOI: 10.12688/f1000research.20726.1] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/20/2023] Open
Abstract
Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the 'index meta-analysis') will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.
Collapse
Affiliation(s)
- Matthew J Page
- School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 3004, Australia
| | - Lisa Bero
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Cynthia M Kroeger
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Zhaoli Dai
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Sally McDonald
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Andrew Forbes
- School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 3004, Australia
| | - Joanne E McKenzie
- School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 3004, Australia
| |
Collapse
|
11
|
Page MJ, Bero L, Kroeger CM, Dai Z, McDonald S, Forbes A, McKenzie JE. Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol. F1000Res 2019; 8:1760. [PMID: 32117567 PMCID: PMC7025772 DOI: 10.12688/f1000research.20726.2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/14/2019] [Indexed: 12/21/2022] Open
Abstract
Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the 'index meta-analysis') will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.
Collapse
Affiliation(s)
- Matthew J. Page
- School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 3004, Australia
| | - Lisa Bero
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Cynthia M. Kroeger
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Zhaoli Dai
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Sally McDonald
- Charles Perkins Centre, School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, 2006, Australia
| | - Andrew Forbes
- School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 3004, Australia
| | - Joanne E. McKenzie
- School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC, 3004, Australia
| |
Collapse
|
12
|
Increase value and reduce waste in research on psychological therapies. Behav Res Ther 2019; 123:103479. [PMID: 31639527 DOI: 10.1016/j.brat.2019.103479] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2019] [Revised: 07/25/2019] [Accepted: 09/09/2019] [Indexed: 12/27/2022]
Abstract
A seminal Lancet series focused on increasing value and reducing waste in biomedical research, providing a transferrable template to diagnose problems in research. Our goal was to document how some of these sources of waste apply to mental health and particularly psychological treatments research. We synthesize and critically evaluate empirical findings in relation to four major sources: i) defining research priorities; ii) research design, methods and analysis; iii) accessibility of research information; iv) accuracy and usability of research reports. We demonstrate that each source of waste considered is well-represented and amply documented within this field. We describe hype and insufficient consideration of what is known in defining research priorities, persistent risk of bias, particularly due to selective outcome reporting, for psychotherapy trials across mental disorders, intellectual and financial biases, direct and indirect evidence of publication bias, largely inexistent adoption of data sharing, issues of multiplicity and fragmentation of data and findings, and insufficient adoption of reporting guidelines. We expand on a few general solutions, including supporting meta-research, properly testing interventions to increase research quality, placing open science at the center of psychological treatment research and remaining vigilant particularly regarding the strains of research currently prioritized, such as experimental psychopathology.
Collapse
|
13
|
Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K, Bertizzolo L, Cowley T, Doshi P, Ehmsen J, Gresham G, Guo N, Haythornthwaite J, Heyward J, Pham D, Payne J, Rosman L, Stuart E, Suarez-Cuervo C, Tolbert E, Twose C, Vedula S. Harms are assessed inconsistently and reported inadequately part 1: systematic adverse events. J Clin Epidemiol 2019; 113:20-27. [DOI: 10.1016/j.jclinepi.2019.04.022] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2018] [Revised: 03/04/2019] [Accepted: 04/05/2019] [Indexed: 12/21/2022]
|
14
|
Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K, Bertizzolo L, Cowley T, Doshi P, Ehmsen J, Gresham G, Guo N, Haythornthwaite J, Heyward J, Pham D, Payne J, Rosman L, Stuart E, Suarez-Cuervo C, Tolbert E, Twose C, Vedula S. Harms are assessed inconsistently and reported inadequately Part 2: nonsystematic adverse events. J Clin Epidemiol 2019; 113:11-19. [DOI: 10.1016/j.jclinepi.2019.04.020] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2018] [Revised: 03/04/2019] [Accepted: 04/05/2019] [Indexed: 01/08/2023]
|
15
|
Sendyk DI, Rovai ES, Souza NV, Deboni MCZ, Pannuti CM. Selective outcome reporting in randomized clinical trials of dental implants. J Clin Periodontol 2019; 46:758-765. [PMID: 31077411 DOI: 10.1111/jcpe.13128] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2019] [Revised: 04/19/2019] [Accepted: 05/01/2019] [Indexed: 11/30/2022]
Abstract
AIM Selective outcome reporting (SOR) is a type of bias that occurs when the primary outcome of a trial protocol is changed or omitted in the paper. The purpose of this study was to evaluate the prevalence of SOR in publications of randomized clinical trials (RCT) concerning dental implants. MATERIALS AND METHODS Two reviewers independently screened protocols registered at ClinicalTrials.gov until February/2019. If the protocol met the eligibility criteria, the reviewers tried to identify the corresponding publication. Data extraction was carried out by the same reviewers. Primary and secondary outcomes were recorded for each trial and compared to outcomes previously described in protocols. RESULTS A total of 49 protocols were included. SOR was identified in 27 (55.1%) trials. The major discrepancies were as follows: protocol-defined primary outcome omitted in the publication (n = 6, 12.2%), new primary outcome introduced (n = 8, 16.3%), discrepancy in the primary outcome time frame (n = 17, 34.7%) and new secondary outcome introduced (n = 31, 63.3%). SOR was significantly associated with industry funding (p = 0.04) and timing of registration (p = 0.04). CONCLUSIONS Our results indicate a high rate of SOR in dental implants clinical trials. Use of registry data during the peer-review process may help decreasing SOR.
Collapse
Affiliation(s)
- Daniel Isaac Sendyk
- Discipline of Periodontics, School of Dentistry, University of São Paulo, São Paulo, Brazil
| | - Emanuel Silva Rovai
- Discipline of Periodontics, School of Dentistry, University of São Paulo, São Paulo, Brazil
| | - Nathalia Vilela Souza
- Discipline of Periodontics, School of Dentistry, University of São Paulo, São Paulo, Brazil
| | | | - Claudio Mendes Pannuti
- Discipline of Periodontics, School of Dentistry, University of São Paulo, São Paulo, Brazil
| |
Collapse
|