1
|
Page MJ, Moher D. Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Syst Rev 2017; 6:263. [PMID: 29258593 PMCID: PMC5738221 DOI: 10.1186/s13643-017-0663-8] [Citation(s) in RCA: 344] [Impact Index Per Article: 49.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/02/2017] [Accepted: 12/08/2017] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations. METHODS We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated). RESULTS We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions. CONCLUSIONS Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.
Collapse
Affiliation(s)
- Matthew J Page
- School of Public Health and Preventive Medicine, Monash University, 553 St Kilda Road, Melbourne, VIC, 3004, Australia.
| | - David Moher
- Centre for Journalology and Canadian EQUATOR Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, K1H 8L6, Canada.,School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1H 8M5, Canada
| |
Collapse
|
2
|
Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A, Boutron I, Sarkis-Onofre R, Bjerre LM, Hróbjartsson A, Altman DG, Moher D. Systematic review adherence to methodological or reporting quality. Syst Rev 2017; 6:131. [PMID: 28720117 PMCID: PMC5516390 DOI: 10.1186/s13643-017-0527-2] [Citation(s) in RCA: 151] [Impact Index Per Article: 21.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/07/2016] [Accepted: 06/16/2017] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews. METHODS The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively. RESULTS Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the SRs assessed risk of bias (validity) in the included studies, while 80% (1112/1387) reported the criteria for study selection. CONCLUSIONS Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.
Collapse
Affiliation(s)
- Kusala Pussegoda
- Ottawa Methods Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada
| | - Lucy Turner
- Ottawa Methods Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada
| | - Chantelle Garritty
- Ottawa Methods Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada
- Translational Research in Biomedicine (TRIBE) Program, University of Split School of Medicine, Split, Croatia
| | - Alain Mayhew
- Ottawa Methods Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada
- Bruyère Research Institute, Ottawa, ON, Canada
| | - Becky Skidmore
- Ottawa Methods Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada
| | - Adrienne Stevens
- Ottawa Methods Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada
- Translational Research in Biomedicine (TRIBE) Program, University of Split School of Medicine, Split, Croatia
| | - Isabelle Boutron
- Paris Descartes University, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité (CRESS), UMR 1153, INSERM, Paris, France
| | | | - Lise M Bjerre
- Bruyère Research Institute, Ottawa, ON, Canada
- Department of Family Medicine, University of Ottawa, Ottawa, ON, Canada
- School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Asbjørn Hróbjartsson
- Center for Evidence-Based Medicine, University of Southern Denmark/Odense University Hospital, Odense, Denmark
| | - Douglas G Altman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - David Moher
- Centre for Journalology; Canadian EQUATOR Centre, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Centre for Practice-Changing Research, Ottawa, ON, Canada.
| |
Collapse
|
3
|
Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A, Boutron I, Sarkis-Onofre R, Bjerre LM, Hróbjartsson A, Altman DG, Moher D. Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study. Syst Rev 2017; 6:117. [PMID: 28629396 PMCID: PMC5477124 DOI: 10.1186/s13643-017-0507-6] [Citation(s) in RCA: 38] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2016] [Accepted: 05/31/2017] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time. METHODS The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports. RESULTS Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ. CONCLUSIONS The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.
Collapse
Affiliation(s)
- Kusala Pussegoda
- Ottawa Methods Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Lucy Turner
- Ottawa Methods Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Chantelle Garritty
- Ottawa Methods Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Translational Research in Biomedicine (TRIBE) Program, University of Split School of Medicine, Split, Croatia
| | - Alain Mayhew
- Ottawa Methods Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Becky Skidmore
- Ottawa Methods Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Adrienne Stevens
- Ottawa Methods Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Translational Research in Biomedicine (TRIBE) Program, University of Split School of Medicine, Split, Croatia
| | - Isabelle Boutron
- INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité, University Paris Descartes, Paris, France
| | - Rafael Sarkis-Onofre
- Graduate Program in Dentistry, Federal University of Pelotas, Pelotas, RS, Brazil
| | - Lise M Bjerre
- Department of Family Medicine, University of Ottawa, Ottawa, Ontario, Canada
- Bruyère Research Institute, Ottawa, Ontario, Canada
- School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Asbjørn Hróbjartsson
- Center for Evidence-Based Medicine, University of Southern Denmark & Odense University Hospital, Odense, Denmark
| | - Douglas G Altman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - David Moher
- Centre for Journalology, Canadian EQUATOR Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
| |
Collapse
|