Reporting and methodological quality of systematic reviews in the orthopaedic literature

A Review of Meta-Analyses in Plastic Surgery: Need for Adequate Assessment of Publication Bias

INTRODUCTION

Publication bias describes a phenomenon in which significant positive results have a higher likelihood of being published compared to negative or nonsignificant results. Publication bias can confound the estimated therapeutic effect in meta-analyses and needs to be adequately assessed in the surgical literature.

METHODS

A review of meta-analyses published in five plastic surgery journals from 2002 to 2022 was conducted. The inclusion criteria for meta-analyses were factors that demonstrated an obligation to assess publication bias, such as interventions with comparable treatment groups and enough power for statistical analysis. Acknowledgment of publication bias risk, quality of bias assessment, methods used in assessment, and individual article factors were analyzed.

RESULTS

318 unique meta-analyses were identified in literature search, and after full-text reviews, 143 met the inclusion criteria for obligation to assess publication bias. 64% of eligible meta-analyses acknowledged the confounding potential of publication bias, and only 46% conducted a formal assessment. Of those who conducted an assessment, 49% used subjective inspection of funnel plots alone, while 47% used any statistical testing in analysis. Overall, only 9/143 (6.3%) assessed publication bias and attempted to correct for its effect. Journals with a higher average impact factor were associated with mention and assessment of publication bias, but more recent publication year and higher number of primary articles analyzed were not.

CONCLUSIONS

This review identified low rates of proper publication bias assessment in meta-analyses published in five major plastic surgery journals. Assessment of publication bias using objective statistical testing is necessary to ensure quality literature within surgical disciplines.

Evaluating the quality of systematic reviews of comparative studies in autograft-based anterior cruciate ligament reconstruction using the AMSTAR-2 tool: A systematic umbrella review

PURPOSE

There remains a lack of consensus around autograft selection in anterior cruciate ligament reconstruction (ACLR), though there is a large body of overlapping systematic reviews and meta-analyses. Systematic reviews and their methodological quality were aimed to be further assessed, using a validated tool known as assessing the methodological quality of systematic reviews (AMSTAR-2).

METHODS

MEDLINE, Embase and CENTRAL were searched from inception to 23 April 2023 for systematic reviews (with/without meta-analysis) comparing primary ACLR autografts. A final quality rating from AMSTAR-2 was provided for each study ('critically low', 'low', 'moderate' or 'high' quality). Correlational analyses were conducted for ratings in relation to study characteristics.

RESULTS

Two thousand five hundred and ninety-eight studies were screened, and 50 studies were ultimately included. Twenty-four studies (48%) were rated as 'critically low', 17 (34%) as 'low', seven (14%) as 'moderate' and two (4%) as 'high' quality. The least followed domains were reporting on sources of funding (1/50 studies), the impact of risk of bias on results of meta-analyses (11/36 studies) and publication bias (17/36 studies). There was a significant increase in the frequency of studies graded as 'moderate' compared to 'low' or 'critically low' quality over time (p = 0.020).

CONCLUSION

The methodological quality of systematic reviews comparing autografts in ACLR is low, with many studies being rated lower due to commonly absent aspects of systematic review methodology such as investigating sources of funding and publication bias. More recent studies were generally more likely to be of higher quality. Authors are advised to consult AMSTAR-2 prior to conducting systematic reviews in ACLR.

LEVEL OF EVIDENCE

Level IV.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Impacts of accreditation on the performance of primary health care centres: A systematic review

Introduction

Evidence on the impacts of accreditation on primary health care (PHC) services is inconsistent. Thus, this study aimed to assess the impacts of accreditation on the performance of PHC centres.

Method

This study systematically reviewed articles published from 2000 to 2019 in the Web of Science, Scopus, ScienceDirect, Springer, PubMed and ProQuest. The following keywords were used: ((primary care OR primary health care) AND (accreditation) AND (impact OR effect OR output OR outcome OR influence OR result OR consequences)). The database search yielded a total of41256 articles, among which 30 articles were finally included in the review.

Results

Accreditation showed the most positive impacts on the quality, effectiveness, human resource management and strategic management of PHC services. Accreditation also positively affected safety, responsiveness, accessibility, customer satisfaction, documentation, leadership, efficiency and continuity of care. Few negative impacts were noted, including the possibility of accreditation being used as a bureaucratic tool, high cost of acquiring accreditation, difficulties in understanding the accreditation process, high staff turnover rate in accredited PHC centres and weak sustainability of some accreditation programmes.

Conclusion

Given its numerous positive impacts, accreditation could be used to effectively improve the performance of PHC centres.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to Best Tools and Practices for Systematic Reviews

» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews1

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

PROTOCOL: Critical appraisal of methodological quality and reporting items of systematic reviews with meta-analysis in evidence-based social science in China: A systematic review

This is the protocol for a Campbell systematic review. The objectives are as follows: (1) To evaluate the reporting quality of systematic reviews published in Chinese social science journals against the PRISMA and MOOSE standards; (2) To evaluate the methodology quality of systematic reviews published in Chinese social science journals against the AMSTAR-2 and DART standards; and (3) To analyze other characteristics of systematic reviews published in Chinese social science journals using content analysis.

Meningioma systematic reviews and meta-analyses: an assessment of reporting and methodological quality

INTRODUCTION

Systematic reviews (SR) and systematic reviews with meta-analysis (SRMA) can constitute the highest level of research evidence. Such evidence syntheses are relied upon heavily to inform the clinical knowledge base and to guide clinical practice for meningioma. This review evaluates the reporting and methodological quality of published meningioma evidence syntheses to date.

METHODS

Eight electronic databases/registries were searched to identify eligible meningioma SRs with and without meta-analysis published between January 1990 and December 2020. Articles concerning spinal meningioma were excluded. Reporting and methodological quality were assessed against the following tools: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), A MeaSurement Tool to Assess systematic Reviews (AMSTAR 2), and Risk Of Bias in Systematic reviews (ROBIS).

RESULTS

116 SRs were identified, of which 57 were SRMAs (49.1%). The mean PRISMA score for SRMA was 20.9 out of 27 (SD 3.9, 77.0% PRISMA adherence) and for SR without meta-analysis was 13.8 out of 22 (SD 3.4, 63% PRISMA adherence). Thirty-eight studies (32.8%) achieved greater than 80% adherence to PRISMA. Methodological quality assessment against AMSTAR 2 revealed that 110 (94.8%) studies were of critically low quality. Only 21 studies (18.1%) were judged to have a low risk of bias against ROBIS.

CONCLUSION

The reporting and methodological quality of meningioma evidence syntheses was poor. Established guidelines and critical appraisal tools may be used as an adjunct to aid methodological conduct and reporting of such reviews, in order to improve the validity and transparency of research which may influence clinical practice.

Trends in Level of Evidence of Systematic Reviews in Sports Medicine, 2010-2020 : A Systematic Review and Meta-analysis

Background:

Popularization of systematic reviews has been met with controversy because of concerns that the primary literature for certain topics may not be suited for systematic review and meta-analysis.

Purpose:

To assess the rate of publication of systematic reviews based on their level of evidence (LOE) in influential orthopaedic sports medicine journals and commonly studied topics in sports medicine.

Study Design:

Systematic review.

Methods:

An electronic search was performed using the PubMed database of studies published from January 2010 to December 2020. The advanced search function was used to identify systematic reviews from the Journal of Shoulder and Elbow Surgery (JSES), American Journal of Sports Medicine (AJSM), Arthroscopy, British Journal of Sports Medicine (BJSM), Journal of Bone and Joint Surgery–American Volume (JBJS), and Sports Medicine (SM Auckland), as well as reviews of the most common areas of sports medicine research, including rotator cuff repair (RCR), shoulder instability (SI), anterior cruciate ligament reconstruction (ACLR), and meniscal repair. The LOE was assigned to each included study according to the Oxford Centre for Evidence-Based Medicine. Studies were grouped as LOE 1-2, LOE 3-5, and nonclinical systematic reviews. A negative binomial regression was used to determine the changes in publication rate over time.

Results:

A total of 2162 systematic reviews were included in this study. From 2010 to 2020, the rate of publication of LOE 3-5 systematic reviews increased significantly among most of the surveyed journals (AJSM, P < .0001; Arthroscopy, P = .01; BJSM, P < .0001; JSES, P < .0001; SM Auckland, P < .0001), with the exception of JBJS (P = .57). The rate of publication of LOE 1-2 systematic reviews increased in AJSM (P < .0001), Arthroscopy (P = .02), BJSM (P < .0001), and SM Auckland (P < .0001); however, no significant changes were seen in JBJS (P = .08) or JSES (P = .15). The publication rate of LOE 3-5 systematic reviews increased for all sports medicine topics surveyed (meniscal repair, P < .0001; RCR, P < .0001; SI, P < .0001; ACLR, P < .0001). However, the publication rate of LOE 1-2 studies only increased for RCR (P = .0003) and ACLR (P < .0001).

Conclusion:

The rate of publication of LOE 3-5 systematic reviews exponentially increased in orthopaedic sports medicine journals over the past decade, outpacing the publication rate of LOE 1-2 systematic reviews.

Quality of the Systematic Reviews in Cochrane Gynecological Cancer Group and Their Understudied RCTs

Purpose

Gynecological cancers are common neoplasms in clinical settings with a high impact on the economy of communities. The medical literature is an essential resource to guide clinical decision-making, and misconduct in researches undermines the credibility and integrity of research in general. We aimed to evaluate the quality of Cochrane gynecological cancers reviews and their understudies RCTS among the different biases dimensions.

Methods

This cross-sectional analytical study was performed on 118 systematic reviews published by the Cochrane gynecological cancers Group up to June 2021. The risk of bias was assessed in each Cochrane survey using the Joanna Bridges Institute (JBI) critical assessment tool consisting of 11 questions. The JBI checklist for systematic reviews and research syntheses is available at https://jbi.global/critical-appraisal-tools. After a systematic critical evaluation of the reviews and meta-analysis, we extracted a different bias from all of their understudied RCTs examined in these systematic reviews, which were evaluated by systematic review authors using a standard bias risk tool developed by the Cochrane Group.

Results

Cochrane gynecological cancers reviews had high quality based on appraise results using the JBI appraisal checklist. In addition, all of the included studies used PRISMA standards for reporting their results. However, in their understudied RCTs, the most prevalent risk of bias was unclear selection bias (allocation concealment) and performance bias (blinding of participants and personnel). Also, the highest risk of bias was blinding participants and personnel (performance bias) and incomplete outcome data (attrition bias). Our results showed that the lowest risk of bias was incomplete outcome data (attrition bias) and random sequence generation (selection bias).

Conclusion

Although most Cochrane gynecological cancers reviews had high quality, unclear performance bias was the highest in their understudied RCTs, indicating structural deficiencies.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13224-022-01655-6.

Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention

Systematic reviews/meta-analyses (SR/MAs) are considered a reliable source of information in healthcare. We aimed to explore the association of several characteristics of SR/MAs addressing nutrition in cancer prevention and their quality/risk of bias (using assessments from AMSTAR-2 and ROBIS tools). The analysis included 101 SR/MAs identified in a systematic survey. Associations of each specified characteristic (e.g., information about the protocol, publication year, reported use of GRADE, or other methods for assessing overall certainty of evidence) with the number of AMSTAR-2 not met (‘No’ responses) and the number of ROBIS items met (‘Probably Yes’ or “Yes’ responses) were examined. Poisson regression was used to identify predictors of the number of ‘No’ answers (indicating lower quality) for all AMSTAR-2 items and the number of ‘Yes’ or ‘Probably Yes’ answers (indicating higher quality/lower concern for bias) for all ROBIS items. Logistic regression was used to identify variables associated with at least one domain assessed as ‘low concern for bias’ in the ROBIS tool. In multivariable analysis, SR/MAs not reporting use of any quality/risk of bias assessment instrument for primary studies were associated with a higher number of ‘No’ answers for all AMSTAR-2 items (incidence rate ratio (IRR) 1.26, 95% confidence interval (CI) 1.09–1.45), and a lower number of ‘Yes’ or ‘Probably Yes’ answers for all ROBIS items (IRR 0.76, 95% CI 0.66–0.87). Providing information about the protocol and search for unpublished studies was associated with a lower number of ‘No’ answers (IRR 0.73, 95% CI 0.56–0.97 and IRR 0.75, 95% CI 0.59–0.95, respectively) and a higher number of ‘Yes’ or ‘Probably Yes’ answers (IRR 1.43, 95% CI 1.17–1.74 and IRR 1.28, 95% CI 1.07–1.52, respectively). Not using at least one quality/risk of bias assessment tool for primary studies within an SR/MA was associated with lower odds that a study would be assessed as ‘low concern for bias’ in at least one ROBIS domain (odds ratio 0.061, 95% CI 0.007–0.527). Adherence to methodological standards in the development of SR/MAs was associated with a higher overall quality of SR/MAs addressing nutrition for cancer prevention.

Adherence to the PRISMA statement and its association with risk of bias in systematic reviews published in rehabilitation journals: A meta-research study

•

The mean overall adherence to the PRISMA checklist across the sample of systematic reviews published in rehabilitation journals was 61.4%.

•

A high overall risk of bias was a significant predictor of lower adherence (B=−7.1%; 95%CI −12.1, −2.0).

•

Studies published in fourth quartile journals displayed a lower overall adherence than those published in the first quartile.

•

No association between adherence and publication options and publication year was found.

•

The overall adherence increased (B= 11.9%; 95%CI 5.9, 18.0) when the SR protocol was registered

Background

Systematic reviews (SRs) and meta-analyses are essential resources for the clinicians. They allow to evaluate the strengths and the weaknesses of the evidence to support clinical decision-making if they are adequately reported. Little is known in the rehabilitation field about the completeness of reporting of SRs and its relationship with the risk of bias (ROB).

Objectives

Primary: 1) To evaluate the completeness of reporting of systematic reviews (SRs) published in rehabilitation journals by evaluating their adherence to the PRISMA 2009 checklist, 2) To investigate the relationship between ROB and completeness of reporting. Secondary: To study the association between completeness of reporting and journals and study characteristics.

Methods

A random sample of 200 SRs published between 2011 and 2020 in 68 rehabilitation journals was indexed under the “rehabilitation” category in the InCites database. Two independent reviewers evaluated adherence to the PRISMA checklist and assessed ROB using the ROBIS tool. Overall adherence and adherence to each PRISMA item and section were calculated. Regression analyses investigated the association between completeness of reporting, ROB, and other characteristics (impact factor, publication options, publication year, and study protocol registration).

Results

The mean overall PRISMA adherence across the 200 studies considered was 61.4%. Regression analyses show that having a high overall ROB is a significant predictor of lower adherence (B=-7.1%; 95%CI -12.1, -2.0). Studies published in fourth quartile journals displayed a lower overall adherence (B= -7.2%; 95%CI -13.2, -1.3) than those published in first quartile journals; the overall adherence increased (B= 11.9%; 95%CI 5.9, 18.0) if the SR protocol was registered. No association between adherence, publication options, and publication year was found.

Conclusion

Reporting completeness in rehabilitation SRs is suboptimal and is associated with ROB, impact factor, and study registration. Authors of SRs should improve adherence to the PRISMA guideline, and journal editors should implement strategies to optimize the completeness of reporting.

Are systematic reviews addressing nutrition for cancer prevention trustworthy? A systematic survey of quality and risk of bias

Context

The last 30 years have yielded a vast number of systematic reviews and/or meta-analyses addressing the link between nutrition and cancer risk.

Objective

The aim of this survey was to assess overall quality and potential for risk of bias in systematic reviews and meta-analyses (SRMAs) that examined the role of nutrition in cancer prevention.

Data Sources

MEDLINE, Embase, and the Cochrane Library databases were searched (last search performed November 2018).

Study Selection

Studies identified as SRMAs that investigated a nutritional or dietary intervention or exposure for cancer prevention in the general population or in people at risk of cancer and in which primary studies had a comparison group were eligible for inclusion. Screening, data extraction, and quality assessment were conducted independently by 2 reviewers.

Data Extraction

Altogether, 101 studies were randomly selected for analysis. The methodological quality and risk of bias were evaluated using the AMSTAR-2 and ROBIS tools, respectively.

Results

Most SRMAs included observational studies. Less than 10% of SRMAs reported a study protocol, and only 51% of SRMAs assessed the risk of bias in primary studies. Most studies conducted subgroup analyses, but only a few reported tests of interaction or specified subgroups of interest a priori. Overall, according to AMSTAR-2, only 1% of SRMAs were of high quality, while 97% were of critically low quality. Only 3% had a low risk of bias, according to ROBIS.

Conclusions

This systematic survey revealed substantial limitations with respect to quality and risk of bias of SRMAs. SRMAs examining nutrition and cancer prevention cannot be considered trustworthy, and results should be interpreted with caution. Peer reviewers as well as users of SRMAs should be advised to use the AMSTAR-2 and/or ROBIS instruments to help to determine the overall quality and risk of bias of SRMAs.

Systematic Review Registration

PROSPERO registration number CRD42019121116.

Nearly One-Third of Published Systematic Reviews and Meta-analyses Yield Inconclusive Conclusions: A Systematic Review

PURPOSE

To perform a systematic review that determines the percentage of published orthopedic surgery and sports medicine systematic reviews and meta-analyses that have a conclusive conclusion.

METHODS

A systematic review was performed using PRISMA guidelines. Six high-quality orthopedics journals were chosen for analysis over a 10-year eligibility period. Systematic reviews and meta-analyses published in these journals were included in the investigation. Narrative, scoping, and umbrella reviews were excluded. A systematic review or meta-analysis was defined as having an inconclusive conclusion if the conclusion in the manuscript body or abstract was stated directly as inconclusive, indeterminate, unknown, or having a lack of evidence (or no evidence). A conclusive conclusion stated a direct answer to the study's primary and/or accessory outcomes. Due to the categorical nature of the data, comparisons were made using χ² test and logistic regression.

RESULTS

There were 1,108 systematic reviews/meta-analyses analyzed (30.9 ± 70.3 studies analyzed per review). More reviews (69.9%) were published with conclusive conclusions rather than without (30.1%). More reviews were surgical (73%) rather than nonsurgical. The United States and North America published the most reviews by country and continent, respectively. There were statistically significant differences between countries (highest proportion with China) and continents (highest proportion with Asia) based on the number of conclusive conclusions in published reviews, respectively. There were no significant differences in the proportion of conclusive conclusion reviews between the 6 analyzed journals. Australia published the largest proportion on nonsurgical reviews. The British Journal of Sports Medicine published a significantly higher proportion of nonsurgical reviews than the other 5 journals. There was no temporal relationship with the proportion of conclusive conclusion reviews.

CONCLUSIONS

This systematic review observed that only 70% of orthopedic systematic reviews and meta-analyses published in 6 high-quality orthopedic journals over a 10-year eligibility period had conclusive conclusions.

LEVEL OF EVIDENCE

Level IV, systematic review and/or meta-analysis of studies with Levels I to IV.

Clinical Epidemiology in China series. Paper 3: The methodological and reporting quality of systematic reviews and meta-analyses published by China' researchers in English-language is higher than those published in Chinese-language

OBJECTIVE

To assess the methodological and reporting quality of Chinese- and English -language systematic reviews and meta-analyses (SRs/MAs) published by Chinese authors between 2016 and 2018.

STUDY DESIGN AND SETTING

We searched MEDLINE and Chinese Science Citation Database (CSCD) for SRs/MAs led by Chinese authors published between 2016 and 2018. We used random sampling to select 10% of the eligible SRs/MAs published in each year from CSCD, and then matched the same number of SRs/MAs in MEDLINE. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR-2) tool. Stratified analyses were conducted to compare the differences of quality between Chinese- and English language SRs/MAs.

RESULTS

We identified 336 SRs/MAs (168 in Chinese and 168 in English). The reporting quality in Chinese-language SRs/MAs was slightly lower than English-language SRs/MAs (mean PRISMA scores: 20.58 vs. 21.71 in 2016, 19.87 vs. 21.24 in 2017, and 21.29 vs. 22.38 in 2018). Less than half of both Chinese- and English-language SRs/MAs complied with item 5 (protocol and registration), item 7 (information sources), item 8 (search) and item 27 (funding)). The methodological quality in Chinese -language SRs/MAs was also slightly lower than English -language SRs/MAs (mean AMSTAR-2 scores: 8.07 vs. 9.36 in 2016; 9.21 vs. 10.26 in 2017; 8.86 vs. 9.28 in 2018). Three items (item 2: established a protocol; item 4: use a comprehensive literature search; and item 10: report the sources of funding) were adhered to by less than 10% of both Chinese- and English -language SRs/MAs. Only one (0.6%) Chinese-language SRs/MA and nine (5.4%) English-language SRs/MAs were rated as high methodological quality.

CONCLUSION

The reporting and methodological quality of English-language SRs/MAs conducted by authors from China between 2016 and 2018 were slightly better than those of Chinese -language SRs/MAs.

Compliance With Preferred Reporting Items for Systematic Review and Meta-Analysis Individual Participant Data Statement for Meta-Analyses Published for Stroke Studies

[Figure: see text].

Systematic Reviews and Meta-Analysis in Spine Surgery-How Good Are They in Methodological Quality? A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVES

To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades.

MATERIALS AND METHODS

We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria.

RESULTS

A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality.

CONCLUSION

Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

Mapping the characteristics of meta-analyses of pharmacy services: a systematic review

BACKGROUND

Suboptimal meta-analyses with misleading conclusions are frequently published in the health areas, and they can compromise decision making in clinical practice.

AIM OF THE REVIEW

This systematic review aimed to map the characteristics of published meta-analyses of pharmacy services and their association with the study conclusions.

METHOD

We searched electronic databases (PubMed, Scopus, and Web of Science) to identify published meta-analyses of pharmacy services up to January 2019. Components of meta-analyses were extracted (i.e. studies' metadata; methods used in the systematic review; description of the statistical model used for the meta-analysis; main results; conflict of interest and funding source). The methodological quality was evaluated using the R-AMSTAR tool.

RESULTS

A total of 85 meta-analyses were included, with 2016 as the median publication year. Overall, the methodological quality of meta-analyses of pharmacy services was considered suboptimal. Only one-third of authors registered a protocol; complete search strategy and raw data were provided by 55.3% and 9.4% of studies, respectively. Evidence strength (GRADE) was evaluated in only 19.2% of studies. PRISMA and Cochrane recommendations were stated to be followed in 60% and 27.4% of articles, respectively. Around half of studies performed sensitivity analysis, however, the prediction interval was presented by only one meta-analysis. Studies that favoured the pharmacists' interventions poorly discussed the methodological quality and heterogeneity of primary trials.

CONCLUSION

Poor conduction and reporting were observed in meta-analyses of pharmacy services, especially in those that favoured the pharmacist's interventions. Reproducibility and transparency should be rigorously ensured by journal editors and peer-reviewers.

The quality of systematic reviews/meta-analyses published in the field of bariatrics: A cross-sectional systematic survey using AMSTAR 2 and ROBIS

High-quality systematic reviews (SR) and meta-analyses (MA) are considered to be reliable sources of information. This study aims to assess the quality of studies published as SR or MA in the field of bariatrics in 2016 and 2017. We identified SR and MA in the field of bariatrics by searching electronic databases (MEDLINE, Embase, and Cochrane Database of Systematic Reviews). Eligible studies were those identified as SR/MA in the title/abstract, which aimed to assess any outcome in patients with morbid obesity undergoing or scheduled to undergo bariatric surgery. Two authors independently reviewed all titles and abstracts, assessed full texts of potentially eligible studies, and assessed the quality of included studies. Any discrepancies were resolved by the third reviewer. We evaluated the quality and risk of bias of each SR/MA using AMSTAR 2 checklist and ROBIS tool, respectively. Seventy-eight of 4236 references met inclusion criteria and were assessed for their quality/risk of bias. The methodological quality of 99% of all papers was classified as "critically low." A total of 6% of the studies were at low risk of bias, and 78% were assessed as being at high risk of bias. The methodological quality of studies published in 2016 and 2017 as SR/MA is highly unsatisfactory.

Quality of reporting of systematic reviews and meta-analyses of surgical randomized clinical trials

Background

Well designed and conducted systematic reviews are essential to clinical practice. Surgical intervention is more complex than medical intervention when considering special items related to procedures. There has been no cross‐sectional study of the reporting quality of systematic reviews of surgical randomized trials focused on special items relating to surgical interventions.

Methods

A cross‐sectional survey of systematic reviews of surgical randomized trials published in 2007 and 2017 was undertaken via a PubMed search. Quality of reporting was assessed by the PRISMA checklist, with intervention details containing 27 items. Univariable and multivariable linear regression was used to explore factors in the checklist as indicators of reporting quality.

Results

A total of 204 systematic reviews were identified. The median score for the PRISMA checklist was 22 (i.q.r. 20–24), and systematic reviews published in 2017 had a significantly higher median score than those from 2007 (22 (i.q.r. 21–24) versus 20 (17–22); P < 0·001). Among the 27 items, 15 were reported adequately and three were reported poorly (in less than 50 per cent of reports). The proportion of other items reported ranged from 54·4 to 77·9 per cent. In multivariable analysis, systematic reviews published in 2017 (coefficient 0·59, 95 per cent c.i. 0·50 to 0·69) and Cochrane reviews (coefficient 0·67, 0·55 to 0·81) were associated with better reporting.

Conclusion

The quality of reporting of systematic reviews of surgical randomized trials has improved in the past 10 years. Some information relating to specific surgical interventions is, however, still reported poorly.

Reporting quality in systematic reviews of in vitro studies: a systematic review

Background: Systematic reviews (SRs) and/or meta-analyses of in vitro research have an important role in establishing the foundation for clinical studies. In this study, we aimed to evaluate the reporting quality of SRs of in vitro studies using the PRISMA checklist.Method: Four databases were searched including PubMed, Virtual Health Library (VHL), Web of Science (ISI) and Scopus. The search was limited from 2006 to 2016 to include all SRs and/or meta-analyses (MAs) of pure in vitro studies. The evaluation of reporting quality was done using the PRISMA checklist.Results: Out of 7702 search results, 65 SRs were included and evaluated with the PRISMA checklist. Overall, the mean overall quality score of reported items of the PRISMA checklist was 68%. We have noticed an increasing pattern in the numbers of published SRs of in vitro studies over the last 10 years. In contrast, the reporting quality was not significantly improved over the same period (p = .363). There was a positive but not significant correlation between the overall quality score and the journal impact factor of the included studies.Conclusions: The adherence of SRs of in vitro studies to the PRISMA guidelines was poor. Therefore, we believe that using reporting guidelines and journals paying attention to this fact will improve the quality of SRs of in vitro studies.

Meta-analyses indexed in PsycINFO had a better completeness of reporting when they mention PRISMA

OBJECTIVES

To investigate the effect of the explicit mention of PRISMA, a statement designed to help authors report meta-analyses (MAs), on the reporting completeness of MAs.

STUDY DESIGN AND SETTING

Two investigators evaluated a random sample of 206 MAs indexed in PsycINFO in 2016; 100 explicitly mentioned PRISMA and 106 did not. Two authors independently evaluated the 27 PRISMA items and extracted factors that could potentially be associated with reporting completeness. The data were analyzed descriptively.

RESULTS

Among our 206 MAs, perfect adherence to PRISMA was found in less than 4%, of which 87% explicitly followed PRISMA. The following items were encountered significantly more frequently in MAs that explicitly mentioned PRISMA than in those that did not: summary, protocol, information sources, search strategy, study characteristics, results of individual studies, funding, study selection, risk of bias in individual studies, and bias across studies. The journal's impact factor, endorsement of PRISMA by the journal, number of authors, country of author, open access, and design of the included studies were significantly and positively associated with the explicit mention of PRISMA.

CONCLUSIONS

Even if far from optimal, the explicit mention of PRISMA has a positive influence on the reporting completeness of MAs from PsycINFO.

Reporting quality of systematic reviews of interventions aimed at improving vaccination coverage: compliance with PRISMA guidelines

Systematic reviews have become increasingly important for informing clinical practice and policy; however, little is known about the reporting characteristics and quality of SRs of interventions to improve immunization coverage in different settings. The aim of this study was to assess the reporting quality of systematic reviews of interventions aimed at improving vaccination coverage using the recommended Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guideline.PubMed and Cochrane Library were searched to identify SRs of interventions to improve immunization coverage, indexed up to May 2016. Two authors independently screened the search output, assessed study eligibility, and extracted data from eligible SRs using a 27-item data collection form derived from PRISMA. Discrepancies in reviews assessments were resolved by discussion and consensus.A total of 57 reviews were included in this study with a mean percentage of applicable PRISMA items that were met across all studies of 66% (range 19-100%) and median compliance of 70%. 39 out of the 57 reviews were published after the release of the PRISMA statement in 2009. Highest compliance was observed in items related to the "description of rational", "description of eligibility criteria", "synthesis of results" and "provision of a general interpretation of the results" (items #3, #6, #14 and #26, respectively). Compliance was poorest in the items "describing summary of evidence" (item 24, 19%), "describing indication of review protocol and registration" (item 5, 26%) and "describing results of risk of bias across studies (item 22, 33%).The overall reporting quality of systematic reviews of interventions to improve vaccination coverage requires significant improvement. There remains a need for additional research targeted at addressing potential barriers to compliance and strategies to improve compliance with PRISMA guideline.

Characteristics, trend, and methodological quality of systematic reviews and meta-analyses in nuclear medicine: A bibliometric analysis of studies published between 2005 and 2016

To evaluate the characteristics, trend, and quality of systematic reviews and meta-analyses in nuclear medicine.We performed a PubMed search to identify systematic reviews and meta-analyses published between 2005 and 2016 in the field of nuclear medicine. The following data were extracted: journal name, impact factor, type of study, topics with cancer type, imaging modalities, authors (number, country, affiliation, presence of nuclear medicine specialists and statisticians, discordance between the first and corresponding authors), funding, methodological quality, methods used for quality assessment, and statistical methods.We included 185 nuclear medicine articles. Meta-analyses (n = 164; 88.6%) were published about 7 times more frequently than systematic reviews. Oncology was the most commonly studied topic (n = 125, 67.6%). The first authors were most frequently located in China (n = 73; 39.5%). PET was the most commonly used modality (n = 150; 81.1%). Both the number of authors and the ratio of discordance between the first and corresponding authors tended to progressively increase over time.The mean AMSTAR score increased over time (5.77 in 2005-2008, 6.71 in 2009-2012, and 7.44 in 2013-2016). The proportion of articles with quality assessment increased significantly (20/26 in 2005-2008, 54/65 in 2009-2012, and 79/94 in 2013-2016). The most commonly used assessment tool was quality assessment of diagnostic accuracy studies (n = 85; 54.9%).The number and quality of systematic reviews and meta-analyses in nuclear medicine have significantly increased over the review period; however, the quality of these articles varies. Efforts to overcome specific weaknesses of the methodologies can provide opportunities for quality improvement.

An Evaluation of Publication Bias in High-Impact Orthopaedic Literature

Background:

Statistical analysis of systematic reviews allows the results of previous studies to be combined and synthesized to assess the overall health effect of the intervention in question. Systematic reviews can also be used to guide the creation of clinical practice guidelines and are considered to have a high level of evidence. Thus, it is important that their methodological quality is of the highest standard. Publication bias presents 2 problems: (1) studies with significant results may be overrepresented in systematic reviews and meta-analyses (“false positives”) and (2) studies without significant results may not be included in systematic reviews and meta-analyses (“false negatives”) because each study, on its own, was underpowered, meaning that some treatment options that may have clinical benefit will not be adopted.

Methods:

We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups. Two authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We assessed publication bias in the systematic reviews that did not assess publication bias themselves.

Results:

Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria. Our review included 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%). Of the studies that assessed publication bias, 31.9% demonstrated evidence of publication bias. Only 43% and 22% of studies that involved use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines discussed and assessed publication bias, respectively.

Conclusions:

Publication bias is infrequently discussed and assessed in the high-impact orthopaedic literature. Furthermore, nearly one-third of the studies that assessed for publication bias demonstrated evidence of publication bias. In addition to these shortcomings, fewer than half of these studies involved use of the PRISMA guidelines and yet only one-fourth of the studies assessed for publication bias.

Clinical Relevance:

By understanding the degree to which publication bias is discussed and presented in high-impact orthopaedic literature, changes can be made by journals and researchers alike to improve the overall quality of research produced and reported.

Replicate systematic review and meta-analyses on robotic surgery: a quality appraisal and overlap investigation

BACKGROUND

The number of publications of systematic reviews and meta-analyses (MAs) on robotic surgery have been increasing, including many investigating the same topic. Their quality and extent of overlap remains unclear. We assessed the quality of the MAs in this area and investigated the extent of their overlap.

METHODS

Relevant studies were identified by searching the MEDLINE, EMBASE, and Cochrane Library databases up to August 1, 2017. Reporting and methodological quality levels were assessed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Assessment of Multiple Systematic Reviews (AMSTAR) checklists. A thorough investigation of the extent of overlap was performed.

RESULTS

In total, 90 MAs in 5 surgical subspecialties were included after full-text review. The mean reporting and methodological quality scores were 22.5 (83.2%) and 7.6 (69.2%), respectively. Authors from university-affiliated institutions and the presence of statistician or epidemiologist coauthors were associated with better-reporting quality scores. The topics with the most overlapping MAs (all ≥ 6) were robot-assisted thyroidectomy, prostatectomy, gastrectomy, colectomy, and fundoplication. 36 (40%) of the included MAs cited previous MAs on the same topic. Among the 7 MAs comparing robot-assisted radical prostatectomy to the open procedure, most (6/7) drew the same conclusion. 50 to 86% of MAs on this topic included the same trials as primary studies.

CONCLUSION

Conducting multiple overlapping MAs with identical conclusions on the same topic that are of suboptimal quality may be a waste of resource and effort. Authors from university-affiliated institutes and experts in epidemiology and statistics are more likely to conduct MAs that have better quality. More guidelines and registries are needed to avoid overlapping MAs.

Hypothyroidism and related diseases: a methodological quality assessment of meta-analysis

OBJECTIVES

There is an increasing prevalence of hypothyroidism and there is a growing body of meta-analyses (MAs) on the association between hypothyroidism and other diseases. However, the methodological quality of the MAs significantly varies. Thus, this study aimed to evaluate and summarise data on the methodological quality of MAs on the associations between hypothyroidism and other diseases using the Assessment of Multiple Systematic Reviews (AMSTAR) scale, providing suggestions for clinical decision-making processes.

DESIGN

To assess the methodological quality of MAs using the AMSTAR scale.

DATA SOURCES

A systematic literature search was performed in PubMed, EMBASE, the Cochrane Library, web of science and Chinese Biomedicine Literature Database.

ELIGIBILITY CRITERIA

We included MAs that had assessed the association between hypothyroidism and other diseases in humans and that had full texts regardless of the publication status. No restriction applied on language or date.

DATA EXTRACTION AND SYNTHESIS

Two reviewers independently screened the titles and abstracts of all searched literature to acquire potentially eligible publications. The full texts of possible eligible publications were downloaded and assessed. Inconsistent comments were resolved through discussions with a third reviewer.

RESULTS

52 studies were included. The average AMSTAR score of the included articles was 8.6 (range: 5-10), and those of English and Chinese MAs were 8.8 and 7.0, respectively. A total of 52 MAs were evaluated, and 19 (36.5%) and 33 (63.5%) of these MAs were of moderate and high quality, respectively. None of the MAs were of low quality. Only two MAs had an a priori design. Items 3, 5 and 9 had the highest compliance (50/52, 96.2%), and aside from item 1, items 7 and 8 had the lowest compliance (33/52,63.5%). According to the results of these MAs, hypothyroidism was significantly associated with cardiovascular diseases, metabolic diseases, neuropsychiatric disorders, breast cancer and pregnancy outcome.

CONCLUSIONS

The methodological quality of the included MAs on the association between hypothyroidism and other diseases was moderate to high. MAs with high qualities confirmed that hypothyroidism was significantly associated with cardiovascular diseases, metabolic syndrome, preterm birth and neonatal outcomes. Consideration of scientific quality when formulating conclusions should be made explicit and more attention should be paid to improving the methodological quality of MAs, and increasing their applicability for clinical decision-making.

Improving the quality of reporting of systematic reviews of dose-response meta-analyses: a cross-sectional survey

Background

Dose-response meta-analysis (DRMA) is a useful tool to investigate potential dose-response relationship between certain exposure or intervention and the outcome of interest. A large number of DRMAs have been published in the past several years. However, the standard of reporting for such studies is not known.

Methods

Medline, Embase, and Wiley Library were searched for systematic reviews with DRMAs (SR-DRMAs) published from January 2011 to July 2017. We used the combination of PRISMA and MOOSE statements, containing 33 items, to assess the reporting of included SR-DRMAs. The adherence of reporting was defined as the proportion of SR-DRMAs meeting the reporting requirement of an item. We explored the association between five pre-specified variables with the total score of reporting on both fully as well as each domain of the checklist.

Results

In total, 529 SR-DRMAs were eligible. Ten out of 33 items were under reported, and this mainly refers to the methods domain: only a small proportion of SR-DRMAs stated whether a review protocol existed (45, 8.5%); clarified the qualifications of searchers (1.7%); presented full electronic search strategy (25.9%); described any effort to include all available studies (22.9%), described methods for languages other than English (27.4%), and stated the process for selecting studies (20.2%). Multiple regression analysis suggested that studies with more authors (regression coefficient = 0.78; 95% CI: 0.35, 1.20; P <  0.001), published more recently (regression coefficient = 0.38; 95% CI: 0.28 to 0.47; trend P <  0.001), used reporting guideline (regression coefficient = 0.98; 95% CI: 0.68 to 1.32; P <  0.001), and involvement of methodologist (regression coefficient = 0.86; 95% CI: 0.42 to 1.32; P <  0.001) were associated with higher score of reporting. Further regression suggested that the improvement on the quality mainly concentrated on the methods and results domains.

Conclusions

The reporting of SR-DRMAs needs to be further improved, particularly in the issues refer to the methods. The quality of reporting may improve when involving more authors and methodologists and employing any reporting guidelines.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0623-6) contains supplementary material, which is available to authorized users.

Publication of Podium Presentations at the Orthopaedic Trauma Association Annual Meetings: 2008-2012

OBJECTIVE

To determine journal publication rates of podium presentations from the OTA Annual Meetings between 2008 and 2012.

METHODS

Podium presentations from the 2008 to 2012 OTA annual meeting were compiled from the Annual Meeting archives. During December 2016, and Google Scholar were performed using individual keywords in the abstract title and content. The results were reviewed for matches to the meeting abstracts with regard to the title, authors, and abstract content. Yearly publication rates were calculated, along with time to publication and common journals for publication.

RESULTS

The publication rate for the 357 podium abstracts presented at the OTA between 2008 and 2012 was 72.8%. Eighty-one percent of abstracts were from the US institutions. The mean time to publication from podium presentation was 23.4 months, and the most common journals of publication were Journal of Orthopaedic Trauma (45.4%) and The Journal of Bone & Joint Surgery (15.3%).

CONCLUSIONS

The publication rate of the podium presentations at the OTA Annual Meeting from 2008 to 2012 has increased since previous years. Compared with other orthopaedic subspecialty and nonorthopaedic specialty meetings, the OTA publication rate is among the highest in the medical field. OTA annual meetings are an opportunity for early access to high-quality research in the area of orthopaedic trauma.

Reproducible research practices are underused in systematic reviews of biomedical interventions

OBJECTIVES

To evaluate how often reproducible research practices, which allow others to recreate the findings of studies, given the original data, are used in systematic reviews (SRs) of biomedical research.

STUDY DESIGN AND SETTING

We evaluated a random sample of SRs indexed in MEDLINE during February 2014, which focused on a therapeutic intervention and reported at least one meta-analysis. Data on reproducible research practices in each SR were extracted using a 26-item form by one author, with a 20% random sample extracted in duplicate. We explored whether the use of reproducible research practices was associated with an SR being a Cochrane review, as well as with the reported use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.

RESULTS

We evaluated 110 SRs of therapeutic interventions, 78 (71%) of which were non-Cochrane SRs. Across the SRs, there were 2,139 meta-analytic effects (including subgroup meta-analytic effects and sensitivity analyses), 1,551 (73%) of which were reported in sufficient detail to recreate them. Systematic reviewers reported the data needed to recreate all meta-analytic effects in 72 (65%) SRs only. This percentage was higher in Cochrane than in non-Cochrane SRs (30/32 [94%] vs. 42/78 [54%]; risk ratio 1.74, 95% confidence interval 1.39-2.18). Systematic reviewers who reported imputing, algebraically manipulating, or obtaining some data from the study author/sponsor infrequently stated which specific data were handled in this way. Only 33 (30%) SRs mentioned access to data sets and statistical code used to perform analyses.

CONCLUSION

Reproducible research practices are underused in SRs of biomedical interventions. Adoption of such practices facilitates identification of errors and allows the SR data to be reanalyzed.

Compliance of systematic reviews in ophthalmology with the PRISMA statement

Background

Systematic reviews and meta-analyses are becoming increasingly important methods to summarize published research. Studies of ophthalmology may present additional challenges because of their potentially complex study designs. The aim of this study was to evaluate the reporting quality of systematic reviews and meta-analyses on topics in ophthalmology to determine compliance with the PRISMA guidelines. We assessed articles published between 2010 and 2015 in the five major relevant journals with the highest impact factors.

Methods

The MEDLINE and EMBASE databases were searched to identify systematic reviews published between January 2010 and December 2015 in the following 5 major ophthalmology journals: Progress in Retinal and Eye Research, Ophthalmology, Archives of Ophthalmology, American Journal of Ophthalmology, and Survey of Ophthalmology. The screening, identification, and scoring of articles were independently performed by two teams, and the results were submitted to statistical analysis to determine medians, ranges, and 95% CIs.

Results

A total of 115 articles were included. The median compliance was 15 out of 27 items (56%), the range was 5–26 (26–96%), and the inter-quartile range was 10 (37%). Compliance was highest in items related to the ‘description of rationale’ (item 3, 100%) and sequentially lower in ‘the general interpretation of results’ (item 26, 96%) and ‘the inclusion of a structured summary in the abstract’ (item 2, 90%). Compliance was poorest in the items ‘indication of review protocol and registration’ (item 5, 9%), ‘specification of risk of biases that may affect the cumulative evidence’ (item 15, 24%), and ‘description of clear objectives in the introduction’ (item 4, 26%).

Conclusion

The reporting quality of systematic reviews and meta-analyses in ophthalmology should be significantly improved. While we recommend the use of the PRISMA criteria as a guideline before journal submission, additional research aimed at identifying potential barriers to compliance may be required to improve compliance with PRISMA guidelines.

Electronic supplementary material

The online version of this article (10.1186/s12874-017-0450-1) contains supplementary material, which is available to authorized users.

A survey of prevalence of narrative and systematic reviews in five major medical journals

Background

Systematic reviews may provide less biased evidence than narrative reviews because they observe a strict methodology, similarly to primary studies. Hence, for clinical research questions, systematic reviews should be the study design of choice. It would be important to evaluate the prevalence and characteristics of narrative and systematic reviews published in prominent medical journals. Researchers and clinicians give great value to articles published in such scientific journals. This study sought to evaluate the prevalence and characteristics of narrative and systematic reviews in the five highest-ranked general medical journals and investigate the associations among type of review, number of citations, and impact factor (IF).

Methods

We surveyed the five highest-ranked medical journals (The New England Journal of Medicine, The Lancet, The Journal of the American Medical Association, The BMJ, and Annals of Internal Medicine) for narrative and systematic reviews published between June 2015 and June 2016. We independently selected and extracted the data from the reviews by strictly following the pre-determined eligibility criteria (Systematic and narrative reviews that focused on the management of diseases). We conducted regression analyses to investigate the associations among review type, number of citations, and IF. We also descriptively reported narrative reviews containing some methodology that might be reproducible.

Results

Two hundred seventy-five reviews were included: 75 (27%) systematic; 126 (46%) narrative with some methodology reported, and 74 (27%) narrative reviews. In comparison to systematic reviews, narrative reviews were more frequently published in journals with higher IF (risk ratio [RR] = 1.114 (95% CI 1.080 to 1.149). Systematic reviews received more citations than narrative reviews (group formed by narrative and narrative with some methodology reported (RR = 0.985 95% CI 0.978 to 0.991).

Conclusions

Non-systematic evidence is the most prevalent type of evidence in reviews published in the five highest-ranked general medical journals. Narrative reviews were more frequently published in journals with higher IF. We recommend that journals limit their space for narrative information, and to address clinical research questions, these journals consider publishing systematic evidence exclusively.

Electronic supplementary material

The online version of this article (10.1186/s12874-017-0453-y) contains supplementary material, which is available to authorized users.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Intra-articular hyaluronic acid injection in treating knee osteoarthritis: assessing risk of bias in systematic reviews with ROBIS tool

OBJECTIVES

Intra-articular injection of hyaluronic acid (HA) is a common, yet controversial therapeutic option in treating knee osteoarthritis (OA). The purpose of the present study was to assess the risk of bias (RoB) of systematic reviews (SRs) and to summarize available evidence of HA in treating knee OA.

METHODS

A systematic search of SRs published through to December 2016 was conducted using the MEDLINE, EMBASE and Cochrane Library. The RoB of included SRs was assessed by ROBIS tool. In addition, the methodological quality of primary studies in SRs with low RoB was evaluated according to the Cochrane Handbook. The evidence quality of each primary outcome of SRs with low RoB was determined by the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) system.

RESULTS

Thirty-one SRs were eligible for inclusion. According to the ROBIS tool, there were 13 SRs with low RoB, 16 with high RoB and two with unclear RoB. The methodological quality of a total of 135 primary studies was evaluated and summarized. Forty-two outcomes from these 13 SRs were classified into the four following quality levels based on the GRADE approach: three outcomes with high quality, eight with moderate quality, 12 with low quality and 19 with very low quality.

CONCLUSIONS

This study evaluated RoB in SRs for managing knee OA with HA and assessed the evidence quality of each primary outcome in SRs with low RoB. These results can help users of SRs to improve the process of SR assessment in developing overviews or guidelines, leading to more reliable recommendations for improvements in treating knee OA. Registration: PROSPERO ((http://www.crd.york.ac.uk/PROSPERO) [CRD42017057384].

Poor methodological quality and reporting standards of systematic reviews in burn care management

The methodological and reporting quality of burn-specific systematic reviews has not been established. The aim of this study was to evaluate the methodological quality of systematic reviews in burn care management. Computerised searches were performed in Ovid MEDLINE, Ovid EMBASE and The Cochrane Library through to February 2016 for systematic reviews relevant to burn care using medical subject and free-text terms such as 'burn', 'systematic review' or 'meta-analysis'. Additional studies were identified by hand-searching five discipline-specific journals. Two authors independently screened papers, extracted and evaluated methodological quality using the 11-item A Measurement Tool to Assess Systematic Reviews (AMSTAR) tool and reporting quality using the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Characteristics of systematic reviews associated with methodological and reporting quality were identified. Descriptive statistics and linear regression identified features associated with improved methodological quality. A total of 60 systematic reviews met the inclusion criteria. Six of the 11 AMSTAR items reporting on 'a priori' design, duplicate study selection, grey literature, included/excluded studies, publication bias and conflict of interest were reported in less than 50% of the systematic reviews. Of the 27 items listed for PRISMA, 13 items reporting on introduction, methods, results and the discussion were addressed in less than 50% of systematic reviews. Multivariable analyses showed that systematic reviews associated with higher methodological or reporting quality incorporated a meta-analysis (AMSTAR regression coefficient 2.1; 95% CI: 1.1, 3.1; PRISMA regression coefficient 6·3; 95% CI: 3·8, 8·7) were published in the Cochrane library (AMSTAR regression coefficient 2·9; 95% CI: 1·6, 4·2; PRISMA regression coefficient 6·1; 95% CI: 3·1, 9·2) and included a randomised control trial (AMSTAR regression coefficient 1·4; 95%CI: 0·4, 2·4; PRISMA regression coefficient 3·4; 95% CI: 0·9, 5·8). The methodological and reporting quality of systematic reviews in burn care requires further improvement with stricter adherence by authors to the PRISMA checklist and AMSTAR tool.

"Assessing the methodological quality of systematic reviews in radiation oncology: A systematic review"

OBJECTIVE

The objective of our study was to evaluate the methodological quality of systematic reviews and meta-analyses in Radiation Oncology.

METHODS

A systematic literature search was conducted for all eligible systematic reviews and meta-analyses in Radiation Oncology from 1966 to 2015. Methodological characteristics were abstracted from all works that satisfied the inclusion criteria and quality was assessed using the critical appraisal tool, AMSTAR. Regression analyses were performed to determine factors associated with a higher score of quality.

RESULTS

Following exclusion based on a priori criteria, 410 studies (157 systematic reviews and 253 meta-analyses) satisfied the inclusion criteria. Meta-analyses were found to be of fair to good quality while systematic reviews were found to be of less than fair quality. Factors associated with higher scores of quality in the multivariable analysis were including primary studies consisting of randomized control trials, performing a meta-analysis, and applying a recommended guideline related to establishing a systematic review protocol and/or reporting.

CONCLUSIONS

Systematic reviews and meta-analyses may introduce a high risk of bias if applied to inform decision-making based on AMSTAR. We recommend that decision-makers in Radiation Oncology scrutinize the methodological quality of systematic reviews and meta-analyses prior to assessing their utility to inform evidence-based medicine and researchers adhere to methodological standards outlined in validated guidelines when embarking on a systematic review.

Systematic review adherence to methodological or reporting quality

BACKGROUND

Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively.

RESULTS

Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the SRs assessed risk of bias (validity) in the included studies, while 80% (1112/1387) reported the criteria for study selection.

CONCLUSIONS

Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.

Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study

BACKGROUND

The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.

RESULTS

Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.

CONCLUSIONS

The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

A Systematic Review of Measurement Properties of Patient-Reported Outcome Measures Used in Patients Undergoing Total Knee Arthroplasty

BACKGROUND

While clinical research on total knee arthroplasty (TKA) outcomes is prevalent in the literature, studies often have poor methodological and reporting quality. A high-quality patient-reported outcome instrument is reliable, valid, and responsive. Many studies evaluate these properties, but none have done so with a systematic and accepted method. The objectives of this study were to identify patient-reported outcome measures (PROMs) for TKA, and to critically appraise, compare, and summarize their psychometric properties using accepted methods.

METHODS

MEDLINE, EMBASE, SCOPUS, Web of Science, PsycINFO, and SPORTDiscus were systematically searched for articles with the following inclusion criteria: publication before December 2014, English language, non-generic PRO, and evaluation in the TKA population. Methodological quality and evidence of psychometric properties were assessed with the COnsensus-based standards for the selection of health Status Measurement INstruments (COSMIN) checklist and criteria for psychometric evidence proposed by the COSMIN group and Terwee et al.

RESULTS

One-hundred fifteen studies on 32 PROMs were included in this review. Only the Work, Osteoarthritis or joint-Replacement Questionnaire, the Oxford Knee Score, and the Western Ontario and McMaster Universities Arthritis Index had 4 or more properties with positive evidence.

CONCLUSION

Most TKA PROMs have limited evidence for their psychometric properties. Although not all the properties were studied, the Work, Osteoarthritis or joint-Replacement Questionnaire, with the highest overall ratings, could be a useful PROM for evaluating patients undergoing TKA. The methods and reporting of this literature can improve by following accepted guidelines.

Reporting and methodological quality of meta-analyses in urological literature

Purpose

To assess the overall quality of published urological meta-analyses and identify predictive factors for high quality.

Materials and Methods

We systematically searched PubMed to identify meta-analyses published from January 1st, 2011 to December 31st, 2015 in 10 predetermined major paper-based urology journals. The characteristics of the included meta-analyses were collected, and their reporting and methodological qualities were assessed by the PRISMA checklist (27 items) and AMSTAR tool (11 items), respectively. Descriptive statistics were used for individual items as a measure of overall compliance, and PRISMA and AMSTAR scores were calculated as the sum of adequately reported domains. Logistic regression was used to identify predictive factors for high qualities.

Results

A total of 183 meta-analyses were included. The mean PRISMA and AMSTAR scores were 22.74 ± 2.04 and 7.57 ± 1.41, respectively. PRISMA item 5, protocol and registration, items 15 and 22, risk of bias across studies, items 16 and 23, additional analysis had less than 50% adherence. AMSTAR item 1, “a priori” design, item 5, list of studies and item 10, publication bias had less than 50% adherence. Logistic regression analyses showed that funding support and “a priori” design were associated with superior reporting quality, following PRISMA guideline and “a priori” design were associated with superior methodological quality.

Conclusions

Reporting and methodological qualities of recently published meta-analyses in major paper-based urology journals are generally good. Further improvement could potentially be achieved by strictly adhering to PRISMA guideline and having “a priori” protocol.

Are child-centric aspects in newborn and child health systematic review and meta-analysis protocols and reports adequately reported?-two systematic reviews

BACKGROUND

Evidence suggests that newborn and child health systematic reviews and meta-analyses exhibit poor quality in reporting. The "Preferred Reporting Items in Systematic Review and Meta-Analysis" (PRISMA) and PRISMA-Protocols (PRISMA-P) checklists have been developed to improve the reporting of systematic review results and protocols, respectively. We aimed to evaluate the clarity and transparency in reporting of child-centric items in child health systematic reviews (SRs) and SR protocols and to identify areas where reporting could be strengthened.

METHODS

Two preliminary lists of potential child-centric reporting items were used to examine current reporting. The Cochrane, DARE, MEDLINE, and EMBASE libraries were searched from 2010 to 2014 for systematic reviews that included children. Each report and protocol that met the inclusion criteria had their quality of reporting assessed by their reporting of child-centric items. Quality of reporting was assessed per whether one third, one to two thirds, or more than two thirds of papers complied with potential child-centric potential modifications/extensions to PRISMA and were analyzed by the following: (i) paper type (i.e., report vs. protocol), (ii) publication type (i.e., Cochrane vs. non-Cochrane), and (iii) population type (i.e., child-only vs. mixed populations vs. family/maternal).

RESULTS

Of the 414 eligible articles, 248 reports and 76 protocols were included. In 21 of 24 potential SR reporting items and 13 of 14 potential SR protocol reporting items, less than two thirds of papers met the child-centric reporting item requirements. Mixed population studies displayed significantly poorer reporting in comparison to child-only and family/maternal intervention studies for 11 potential SR reporting items (p < 0.05) and five potential SR protocol items (p < 0.05). When comparing non-Cochrane to Cochrane reports and protocols, five items in both lists were found to perform significantly poorer in non-Cochrane reports (p < 0.05). Significant differences in reporting quality were found in three of 14 items shared between the potential SR reporting items and potential SR protocol reporting items (p < 0.05).

CONCLUSIONS

Newborn and child health systematic reviews and meta-analyses exhibit incomplete reporting, thereby hindering prudent decision-making by healthcare providers and policy makers. These results provide a rationale for the implementation of child-centric extensions and modifications to current PRISMA and PRISMA-P, such as to improve reporting in this population.

Do systematic reviews on pediatric topics need special methodological considerations?

BACKGROUND

Systematic reviews are key tools to enable decision making by healthcare providers and policymakers. Despite the availability of the evidence based Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA-2009 and PRISMA-P 2015) statements that were developed to improve the transparency and quality of reporting of systematic reviews, uncertainty on how to deal with pediatric-specific methodological challenges of systematic reviews impairs decision-making in child health. In this paper, we identify methodological challenges specific to the design, conduct and reporting of pediatric systematic reviews, and propose a process to address these challenges.

DISCUSSION

One fundamental decision at the outset of a systematic review is whether to focus on a pediatric population only, or to include both adult and pediatric populations. Both from the policy and patient care point of view, the appropriateness of interventions and comparators administered to pre-defined pediatric age subgroup is critical. Decisions need to be based on the biological plausibility of differences in treatment effects across the developmental trajectory in children. Synthesis of evidence from different trials is often impaired by the use of outcomes and measurement instruments that differ between trials and are neither relevant nor validated in the pediatric population. Other issues specific to pediatric systematic reviews include lack of pediatric-sensitive search strategies and inconsistent choices of pediatric age subgroups in meta-analyses. In addition to these methodological issues generic to all pediatric systematic reviews, special considerations are required for reviews of health care interventions' safety and efficacy in neonatology, global health, comparative effectiveness interventions and individual participant data meta-analyses. To date, there is no standard approach available to overcome this problem. We propose to develop a consensus-based checklist of essential items which researchers should consider when they are planning (PRISMA-PC-Protocol for Children) or reporting (PRISMA-C-reporting for Children) a pediatric systematic review. Available guidelines including PRISMA do not cover the complexity associated with the conduct and reporting of systematic reviews in the pediatric population; they require additional and modified standards for reporting items. Such guidance will facilitate the translation of knowledge from the literature to bedside care and policy, thereby enhancing delivery of care and improving child health outcomes.