Reporting and methodologic quality of Cochrane Neonatal review group systematic reviews

Diabetes and anti-diabetic interventions and the risk of gynaecological and obstetric morbidity: an umbrella review of the literature

BACKGROUND

Diabetes has reached epidemic proportions in recent years with serious health ramifications. The aim of this study was to evaluate the strength and validity of associations between diabetes and anti-diabetic interventions and the risk of any type of gynaecological or obstetric conditions.

METHODS

Design: Umbrella review of systematic reviews and meta-analyses.

DATA SOURCES

PubMed, Medline, Embase, Cochrane Database of Systematic Reviews, manual screening of references.

ELIGIBILITY CRITERIA

Systematic reviews and meta-analyses of observational and interventional studies investigating the relationship between diabetes and anti-diabetic interventions with gynaecological or obstetric outcomes. Meta-analyses that did not include complete data from individual studies, such as relative risk, 95% confidence intervals, number of cases/controls, or total population were excluded.

DATA ANALYSIS

The evidence from meta-analyses of observational studies was graded as strong, highly suggestive, suggestive or weak according to criteria comprising the random effects estimate of meta-analyses and their largest study, the number of cases, 95% prediction intervals, I2 heterogeneity index between studies, excess significance bias, small study effect and sensitivity analysis using credibility ceilings. Interventional meta-analyses of randomised controlled trials were assessed separately based on the statistical significance of reported associations, the risk of bias and quality of evidence (GRADE) of included meta-analyses.

RESULTS

A total of 117 meta-analyses of observational cohort studies and 200 meta-analyses of randomised clinical trials that evaluated 317 outcomes were included. Strong or highly suggestive evidence only supported a positive association between gestational diabetes and caesarean section, large for gestational age babies, major congenital malformations and heart defects and an inverse relationship between metformin use and ovarian cancer incidence. Only a fifth of the randomised controlled trials investigating the effect of anti-diabetic interventions on women's health reached statistical significance and highlighted metformin as a more effective agent than insulin on risk reduction of adverse obstetric outcomes in both gestational and pre-gestational diabetes.

CONCLUSIONS

Gestational diabetes appears to be strongly associated with a high risk of caesarean section and large for gestational age babies. Weaker associations were demonstrated between diabetes and anti-diabetic interventions with other obstetric and gynaecological outcomes.

TRIAL REGISTRATION

Open Science Framework (OSF) (Registration https://doi.org/10.17605/OSF.IO/9G6AB ).

Quality of reporting among systematic reviews underpinning the ESC/ACC guidelines on ventricular arrhythmias and sudden cardiac death

OBJECTIVES

The main objective of this study was to assess the methodological and reporting quality of the systematic reviews (SRs) supporting the European Society of Cardiology (ESC) and the American College of Cardiology (ACC) clinical practice guidelines (CPGs) recommendations for the management of patients with ventricular arrhythmias and sudden cardiac death (SCD). As a secondary objective, we sought to determine: (1) the proportion of Cochrane SRs were cited; and (2) whether Cochrane SRs scored higher on Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) appraisals.

DESIGN

Cross-sectional analysis.

MAIN OUTCOME MEASURES

We searched for CPGs published by the ESC and the ACC from 2010 to 2020. We selected the CPGs for ventricular arrhythmias and the prevention of SCD. The reference sections were searched for SRs. Two independent investigators evaluated eligible SR using the PRISMA checklist and the AMSTAR-2 assessment tool.

RESULTS

Two CPGs for ventricular arrhythmia and SCD were included in this study. Fifty-five SRs were included in our analysis. Across all SRs, the mean PRISMA score was 0.70. The lowest scoring PRISMA item related to the presence of a pre-published protocol (item 5, score 0.17). Overall, 40% of included SRs were found to have 'critically low' AMSTAR-2 ratings. One of the lowest scoring items for AMSTAR-2 was reporting of sources of funding (item 10). The 4 Cochrane SRs that were included scored higher on both assessment tools than non-Cochrane studies, specifically in PRISMA overall completion (88.7% vs 69.7%).

CONCLUSION

Our study suggests the methodological and reporting quality of SRs used within ESC and ACC CPGs is insufficient, as demonstrated by the lack of adherence to both AMSTAR-2 and PRISMA checklists. Given the importance of CPGs on clinical decision making, and ultimately patient care, the methodological rigour and quality reporting within SRs used in CPGs should be held to the highest standard within the field of cardiology.

Clinical Trial Registry Use in Orthopaedic Surgery Systematic Reviews

BACKGROUND

Results from systematic reviews and meta-analyses, which have the highest level of evidence (Level I), often drive clinical decision-making and health policy. Often, unpublished trial data are omitted from systematic reviews, raising concerns about the extent of the reliability and validity of results that have been drawn from systematic reviews. We aimed to determine the extent to which systematic review authors include searches of clinical trial registries for unpublished data when conducting systematic reviews in orthopaedic surgery.

METHODS

Systematic reviews and/or meta-analyses were gathered from the top 5 orthopaedic surgery journals based on the h5-index from Google Scholar Metrics. Systematic reviews that had been published in the Cochrane Database of Systematic Reviews, which requires the inclusion of a clinical trial registry search, served as controls. For the primary outcome, each systematic review from the top 5 orthopaedic journals was screened to determine whether the authors of each study searched for unpublished data in clinical trial registries. We then compared the rate of registry searches with those in the control group. For the secondary analysis, a search of ClinicalTrials.gov was performed for unpublished trial data for 100 randomized systematic reviews.

RESULTS

All 38 of the Cochrane systematic reviews (100%) included clinical trial registry searches, while the top 5 orthopaedic journals had only 31 of 480 studies (6.5%) that looked at clinical trial registries. The secondary analysis yielded 59 of 100 systematic review articles (59.0%) that could have included unpublished clinical trial data from ≥1 studies to their sample.

CONCLUSIONS

Systematic reviews that have been published in the top orthopaedic surgery journals seldom included a search for unpublished clinical trial data.

CLINICAL RELEVANCE

The exclusion of clinical trial registry searches potentially contributes to publication bias within the orthopaedic literature. Moving forward, systematic review authors should include clinical trial registry searches for unpublished clinical trial data to provide the most accurate representation of the available evidence for systematic reviews and meta-analyses.

Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? - A systematic review of reviews: an update

PURPOSE

Reporting guidelines (eg, Consolidated Standards of Reporting Trials [CONSORT] statement) are intended to improve reporting standards and enhance the transparency and reproducibility of research findings. Despite accessibility of such guidelines, researchers are not required to adhere to them. Our goal was to determine the current status of reporting quality in the medical literature and examine whether adherence of reporting guidelines has improved since the inception of reporting guidelines.

MATERIALS AND METHODS

Eight reporting guidelines, such as CONSORT, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), STrengthening the Reporting of OBservational studies in Epidemiology (STROBE), Quality of Reporting of Meta-analysis (QUOROM), STAndards for Reporting of Diagnostic accuracy (STARD), Animal Research: Reporting In Vivo Experiments (ARRIVE), Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and Meta-analysis of Observational Studies in Epidemiology (MOOSE) were examined. Our inclusion criteria included reviews published between January 1996 to September 2016 which investigated the adherence to reporting guidelines in the literature that addressed clinical trials, systematic reviews, observational studies, meta-analysis, diagnostic accuracy, economic evaluations, and preclinical animal studies that were in English. All reviews were found on Web of Science, Excerpta Medical Database (EMBASE), MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL).

RESULTS

Among the general searching of 26,819 studies by using the designed searching method, 124 studies were included post screening. We found that 87.9% of the included studies reported suboptimal adherence to reporting guidelines. Factors associated with poor adherence included non-pharmacological interventions, year of publication, and trials concluding with significant results. Improved adherence was associated with better study designs such as allocation concealment, random sequence, large sample sizes, adequately powered studies, multiple authorships, and being published in journals endorsing guidelines.

CONCLUSION

We conclude that the level of adherence to reporting guidelines remains suboptimal. Endorsement of reporting guidelines by journals is important and recommended.

Safety of Moderate Hypothermia for Perinatal Hypoxic-Ischemic Encephalopathy: A Meta-analysis

BACKGROUND

We investigated the safety of therapeutic hypothermia during intervention in infants with hypoxic-ischemic encephalopathy (HIE).

METHODS

The MEDLINE, EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials from onset to November 30, 2016 were searched for studies on perinatal HIE. Randomized controlled trials comparing the use of therapeutic hypothermia with normothermia for perinatal HIE were included in the study. Safety and efficacy data for therapeutic hypothermia in 1806 infants with HIE were included in this meta-analysis. The primary outcomes were safety variables, and the secondary outcomes were efficacy variables. A fixed-effect model was used to perform the meta-analysis. Risk ratios (RR), risk differences (RD), and 95% confidence intervals (CI) were calculated.

RESULTS

Thirteen trials, including 1806 infants, contained information on safety and efficacy variables. Moderate hypothermia significantly increased the risk of thrombocytopenia (RR 1.18, 95% CI 1.02 to 1.37, P = 0.03; RD 0.06, 95% CI -0.02 to 0.09) and cardiac arrhythmia (RR 2.52, 95% CI 1.62 to 3.93, P < 0.0001; RD 0.19, 95% CI 0.09 to 0.03) during intervention.

CONCLUSIONS

In infants with HIE, the application of therapeutic hypothermia increases the risk of thrombocytopenia and cardiac arrhythmia during intervention.

Systematic review adherence to methodological or reporting quality

BACKGROUND

Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively.

RESULTS

Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the SRs assessed risk of bias (validity) in the included studies, while 80% (1112/1387) reported the criteria for study selection.

CONCLUSIONS

Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.

Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study

BACKGROUND

The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.

RESULTS

Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.

CONCLUSIONS

The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

Therapeutic hypothermia increases the risk of cardiac arrhythmia for perinatal hypoxic ischaemic encephalopathy: A meta-analysis

Objective

To determine whether therapeutic hypothermia after hypoxic ischaemic encephalopathy (HIE) in neonates increases the risk of cardiac arrhythmia during intervention.

Design

A meta-analysis was conducted using a fixed-effect model. Risk ratios, risk differences, and 95% confidence intervals, were measured.

Data sources

Studies identified from the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, Google Scholar, previous reviews, and abstracts from onset to August, 2016.

Review methods

Reports that compared therapeutic hypothermia with normal care for neonates with HIE and that included data on safety or cardiac arrhythmia, which is of interest to patients and clinicians, were selected.

Results

We found seven trials, encompassing 1322 infants that included information on safety or cardiac arrhythmia during intervention. Therapeutic hypothermia considerably increased the combined rate of cardiac arrhythmia in the seven trials (risk ratio 2.42, 95% confidence interval 1.23 to 4.76. p = 0.01; risk difference 0.02, 95% CI 0.01 to 0.04) during intervention.

Conclusions

In infants with hypoxic ischaemic encephalopathy, therapeutic hypothermia is associated with a consistent increase in cardiac arrhythmia during intervention.

Are child-centric aspects in newborn and child health systematic review and meta-analysis protocols and reports adequately reported?-two systematic reviews

BACKGROUND

Evidence suggests that newborn and child health systematic reviews and meta-analyses exhibit poor quality in reporting. The "Preferred Reporting Items in Systematic Review and Meta-Analysis" (PRISMA) and PRISMA-Protocols (PRISMA-P) checklists have been developed to improve the reporting of systematic review results and protocols, respectively. We aimed to evaluate the clarity and transparency in reporting of child-centric items in child health systematic reviews (SRs) and SR protocols and to identify areas where reporting could be strengthened.

METHODS

Two preliminary lists of potential child-centric reporting items were used to examine current reporting. The Cochrane, DARE, MEDLINE, and EMBASE libraries were searched from 2010 to 2014 for systematic reviews that included children. Each report and protocol that met the inclusion criteria had their quality of reporting assessed by their reporting of child-centric items. Quality of reporting was assessed per whether one third, one to two thirds, or more than two thirds of papers complied with potential child-centric potential modifications/extensions to PRISMA and were analyzed by the following: (i) paper type (i.e., report vs. protocol), (ii) publication type (i.e., Cochrane vs. non-Cochrane), and (iii) population type (i.e., child-only vs. mixed populations vs. family/maternal).

RESULTS

Of the 414 eligible articles, 248 reports and 76 protocols were included. In 21 of 24 potential SR reporting items and 13 of 14 potential SR protocol reporting items, less than two thirds of papers met the child-centric reporting item requirements. Mixed population studies displayed significantly poorer reporting in comparison to child-only and family/maternal intervention studies for 11 potential SR reporting items (p < 0.05) and five potential SR protocol items (p < 0.05). When comparing non-Cochrane to Cochrane reports and protocols, five items in both lists were found to perform significantly poorer in non-Cochrane reports (p < 0.05). Significant differences in reporting quality were found in three of 14 items shared between the potential SR reporting items and potential SR protocol reporting items (p < 0.05).

CONCLUSIONS

Newborn and child health systematic reviews and meta-analyses exhibit incomplete reporting, thereby hindering prudent decision-making by healthcare providers and policy makers. These results provide a rationale for the implementation of child-centric extensions and modifications to current PRISMA and PRISMA-P, such as to improve reporting in this population.

Methodological quality, completeness of reporting and use of systematic reviews as evidence in clinical practice guidelines for paediatric overweight and obesity

Paediatric obesity rates remain high despite extensive efforts to prevent and treat obesity in children. We investigated the quality of the methodology and reporting within systematic reviews (SRs) underpinning paediatric content in US clinical practice guidelines (CPGs). In June 2016 we searched guideline clearinghouses and professional organization websites for guidelines published by national or professional organizations in the United States from January 2007 onwards. In our primary, a priori analysis, we used PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and AMSTAR (A Measurement Tool to Assess Systematic Reviews) instruments to score SRs and meta-analyses that included paediatric populations and were cited by included CPGs. In a secondary, post hoc analysis, we determined the extent to which US CPGs use available, relevant SRs and meta-analyses compared with non-US CPGs. Eight US-based CPGs with 27 references to 22 unique SRs were found. AMSTAR and PRISMA scores were low overall, with only three SRs having 'high' methodological quality. Items dealing with bias assessments and search strategies had especially low scores. US CPGs were also older on average and cited fewer SRs than their international counterparts. Low quality scores and dated guidelines should be a cause for concern among practicing clinicians and a call to action for future guideline developers, publishers and research institutions.

An Overview of Systematic Reviews of Ginkgo biloba Extracts for Mild Cognitive Impairment and Dementia

Ginkgo biloba extracts (GBEs) have been recommended to improve cognitive function and to prevent cognitive decline, but earlier evidence was inconclusive. Here, we evaluated all systematic reviews of GBEs for prevention of cognitive decline, and intervention of mild cognitive impairment (MCI) and dementia. Six databases from their inception to September 2015 were searched. Ten systematic reviews were identified, including reviews about Alzheimer's disease (n = 3), about vascular dementia (n = 1), about both Alzheimer's disease and vascular dementia (n = 2), about Alzheimer's disease, vascular dementia and mixed dementia (n = 3), and a review about MCI (n = 1). Based on the overview quality assessment questionnaire, eight studies were scored with at least 5 points, while the other two scored 4 points and 3 points, respectively. Medication with GBEs showed improvement in cognition, neuropsychiatric symptoms, and daily activities, and the effect was dose-dependent. Efficacy was convincingly demonstrated only when high daily dose (240 mg) was applied. Compared with placebo, overall adverse events and serious adverse events were at the same level as placebo, with less adverse events in favor of GBE in the subgroup of Alzheimer's disease patients, and fewer incidences in vertigo, tinnitus, angina pectoris, and headache. In conclusion, there is clear evidence to support the efficacy of GBEs for MCI and dementia, whereas the question on efficacy to prevent cognitive decline is still open. In addition, GBEs seem to be generally safe.

Clinical trials registries are underused in the pregnancy and childbirth literature: a systematic review of the top 20 journals

Background

Systematic reviews and meta-analyses that do not include unpublished data in their analyses may be prone to publication bias, which in some cases has been shown to have deleterious consequences on determining the efficacy of interventions.

Methods

We retrieved systematic reviews and meta-analyses published in the past 8 years (January 1, 2007–December 31, 2015) from the top 20 journals in the Pregnancy and Childbirth literature, as rated by Google Scholar’s h5-index. A meta-epidemiologic analysis was performed to determine the frequency with which authors searched clinical trials registries for unpublished data.

Results

A PubMed search retrieved 372 citations, 297 of which were deemed to be either a systematic review or a meta-analysis and were included for analysis. Twelve (4 %) of these searched at least one WHO-approved clinical trials registry or clinicaltrials.gov.

Conclusion

Systematic reviews and meta-analyses published in pregnancy and childbirth journals do not routinely report searches of clinical trials registries. Including these registries in systematic reviews may be a promising avenue to limit publication bias if registry searches locate unpublished trial data that could be used in the systematic review.

Quality assessment and factor analysis of systematic reviews and meta-analyses of endoscopic ultrasound diagnosis

Background

Comprehensive monitoring of the quality of systematic reviews (SRs) and meta-analyses (MAs) of endoscopic ultrasound (EUS) requires complete and accurate reporting and methodology.

Objective

To assess the reporting and methodological quality of SRs/MAs on EUS diagnosis and to explore the potential factors influencing articles’ quality.

Methods

The quality of the reporting and methodology was evaluated in relation to the adherence of papers to the PRISMA checklist and the AMSTAR quality scale. The total scores for every criterion and for every article on the two standards were calculated. Data were evaluated and analyzed using SPSS17.0 and RevMan 5.1 in terms of publication time, category of reviews, category of journals, and funding resource.

Results

A total of 72 SRs/MAs was included, but no Cochrane Systematic Reviews (CSRs) were obtained. The number of SRs/MAs ranged from 1 in 1998 to 15 in 2013; 88.1% used the QUADAS tool; the average overall scores by PRISMA statement and AMSTAR tool were 19.9 and 5.4, respectively. Scores on some items showed substantial improvement after publication of PRISMA and AMSTAR. However, no reviews followed the criterion of protocol and registration, and only 11.1% of articles fulfilled the criterion of literature search. SRs/MAs from the Science Citation Index (SCI) were of better quality than non-SCI studies. Funding resource made no difference to quality. Regression analysis showed that time of publication and inclusion in the SCI were significantly correlated with total scores on the two standards.

Conclusion

The reporting and methodological quality of SRs/MAs on EUS diagnosis has improved measurably since PRISMA and AMSTAR checklists released. It is hoped that CSR in this field will be produced. Literature searching and protocol criteria, as well as QUADAS-2 tool need to be addressed more in the future. Time of publication and SCI relate more to the overall quality of SRs/MAs than does funding resource.

Buyang huanwu decoction for healthcare: evidence-based theoretical interpretations of treating different diseases with the same method and target of vascularity

Buyang Huanwu Decoction (BHD) is a famous herbal prescription that has been used to treat stroke for centuries. Recent studies reported that the use of BHD had been extended to treat various kinds of disorders according to the TCM syndrome theory of Treating Different Diseases with the Same Method (TDDSM). Here, an overview of systematic reviews (SRs) of BHD for healthcare was conducted to interpret the TCM theory of TDDSM and its target of vascularity in an evidence-based manner. Literature searches were carried out in 5 databases to search SRs of BHD for any indication up to August 2013. Thirteen eligible SRs were identified which reported a wide range of vascular conditions. Based on the Overview Quality Assessment Questionnaire scores, the quality of included SRs was varied, with an average score of 4 points. We found that there is premature evidence for the use of BHD for healthcare, whereas BHD was well tolerable in all patients. BHD can be used to treat many disorders with the same therapeutic principle of invigorating Qi to activate blood circulation, which is essentially a manifestation of the TDDSM and is likely to account for targeting the specific pathogenesis of vascular diseases.

An overview of systematic reviews of shenmai injection for healthcare

Shenmai injection (SMI) is widely applied in clinical practice as an organ protector. This overview is to evaluate the current evidence from systematic reviews (SRs) of SMI for healthcare. The literature searches were carried out in 6 databases without language restrictions until December 2012. The quality of the primary studies from the respective SRs was evaluated by using Jadad score. The overview quality assessment questionnaire (OQAQ) was used to evaluate the methodological quality of all included SRs. Twenty eligible SRs were identified. They reported a wide range of conditions, including SMI for cardio/cerebrovascular diseases, viral myocarditis, tumor chemotherapy, and adverse drug reactions. Most of the primary studies were of good quality only in 1 SR of non-small-cell lung cancer. According to the OQAQ scores, the quality of included SRs was variable and six reviews were of high quality with a score of 5 points. Two SRs showed that SMI had low adverse drug reaction occurrence. In conclusion, there is mixed evidence to support efficacy of SMI for an adjunct therapy to tumor chemotherapy and premature evidence for the use of SMI for cardio/cerebrovascular disorders and viral myocarditis. SMI seems generally safe for clinical application. Further large sample-size and well-designed RCTs are needed.

GAP-REACH: a checklist to assess comprehensive reporting of race, ethnicity, and culture in psychiatric publications

Growing awareness of health and health care disparities highlights the importance of including information about race, ethnicity, and culture (REC) in health research. Reporting of REC factors in research publications, however, is notoriously imprecise and unsystematic. This article describes the development of a checklist to assess the comprehensiveness and the applicability of REC factor reporting in psychiatric research publications. The 16-item GAP-REACH checklist was developed through a rigorous process of expert consensus, empirical content analysis in a sample of publications (N = 1205), and interrater reliability (IRR) assessment (N = 30). The items assess each section in the conventional structure of a health research article. Data from the assessment may be considered on an item-by-item basis or as a total score ranging from 0% to 100%. The final checklist has excellent IRR (κ = 0.91). The GAP-REACH may be used by multiple research stakeholders to assess the scope of REC reporting in a research article.

Systematic Cochrane reviews in neonatology: a critical appraisal

BACKGROUND

There is a lack of up-to-date, systematic reviews that critically assess the role and potential limitations of evidence-based medicine (EBM) and systematic reviews in neonatology.

METHODS

We performed a systematic literature review of all Cochrane reviews published between 1996 and 2010 by the Cochrane Neonatal Review Group (CNRG). Main outcome parameter: assessment of the percentage of reviews that concluded that a certain intervention provides a benefit, the percentage of reviews that concluded that no benefit was seen, and the percentage of studies that concluded that the current level of evidence is inconclusive.

RESULTS

In total, 262 reviews were assessed, most of which included exclusively preterm infants (146/262). The majority of reviews assessed pharmacological interventions (145/262); other important fields included nutritional (46/262), and ventilatory issues (27/262). In 42/262 reviews, a clear recommendation in favor of a specific intervention was given, whereas 98/262 reviews concluded that certain interventions should not be performed. However, the largest proportion of reviews was inconclusive (122/262) and did not issue specific recommendations. The proportion of inconclusive reviews increased from 30% (1996-2000), to 50% (2001-2005), and finally to 58% for the years 2006-2010. Common reasons for inconclusive reviews were the small number of patients (105), insufficient data (94), insufficient methodological quality (87), and heterogeneity of studies (69).

CONCLUSION

There is an ongoing need for high-quality research in order to reduce the proportion of inconclusive meta-analyses in the field of neonatology. Funding and research agencies will play a vital role in selecting the most appropriate research programs.

Systematic reviews of surgical procedures in children: quantity, coverage and quality

AIM

Systematic reviews have the potential to map those areas where children are under-represented in surgical research. We aimed to describe and evaluate the quantity, coverage and the quality of conduct and reporting of systematic reviews of surgical procedures in children.

METHODS

We searched four biomedical databases, a systematic review register, reference lists and conducted hand searching to identify relevant reviews. Two reviewers worked independently to critically appraise included studies and abstract data. We assessed reporting quality using the preferred reporting items for systematic reviews and meta-analysis statement and methodological quality using the Assessment of Multiple SysTemAtic Reviews tool.

RESULTS

Fifteen systematic reviews were identified, representing 0.01% of all paediatric surgical citations in MEDLINE and Embase. Thirteen of the reviews were Cochrane reviews, and most reviews (12/15) addressed subspecialty interests such as otorhinolaryngology. The median number of included trials per systematic review was four (interquartile range 1 to 9.5), the median number of primary outcomes was 5.5 (interquartile range 3.5 to 7.5). In general, reporting and methodological quality was good although there were several omissions, particularly around completeness of reporting of statistical methods used, and utilisation of quality assessments in analyses. Outcomes were often not clearly defined and descriptions of procedures lacked sufficient detail to determine the similarities and differences among surgical procedures within the contributing trials.

CONCLUSION

Systematic reviews of surgical procedures in children are rarely published. To improve the evidence base and guide research agendas, more systematic reviews should be conducted, using standard guidelines for conduct and reporting.

Methodological quality and descriptive characteristics of prosthodontic-related systematic reviews

Ideally, healthcare systematic reviews (SRs) should be beneficial to practicing professionals in making evidence-based clinical decisions. However, the conclusions drawn from SRs are directly related to the quality of the SR and of the included studies. The aim was to investigate the methodological quality and key descriptive characteristics of SRs published in prosthodontics. Methodological quality was analysed using the Assessment of Multiple Reviews (AMSTAR) tool. Several electronic resources (MEDLINE, EMBASE, Web of Science and American Dental Association's Evidence-based Dentistry website) were searched. In total 106 SRs were located. Key descriptive characteristics and methodological quality features were gathered and assessed, and descriptive and inferential statistical testing performed. Most SRs in this sample originated from the European continent followed by North America. Two to five authors conducted most SRs; the majority was affiliated with academic institutions and had prior experience publishing SRs. The majority of SRs were published in specialty dentistry journals, with implant or implant-related topics, the primary topics of interest for most. According to AMSTAR, most quality aspects were adequately fulfilled by less than half of the reviews. Publication bias and grey literature searches were the most poorly adhered components. Overall, the methodological quality of the prosthodontic-related systematic was deemed limited. Future recommendations would include authors to have prior training in conducting SRs and for journals to include a universal checklist that should be adhered to address all key characteristics of an unbiased SR process.

Quality and transparency of overviews of systematic reviews

OBJECTIVE

To evaluate the reporting and methodological quality of overviews of systematic reviews.

METHOD

We developed an 18-item assessment tool for overviews of systematic reviews. We then performed a systematic search for such overviews using the terms ('overview' AND ('meta analys*' OR 'systematic review*')) OR 'umbrella review' in the title. We only included those overviews that were limited to systematic reviews or meta-analyses. Their methodological and reporting quality were assessed by two independent reviewers using the checklist, and differences were resolved by a third reviewer. Data analyses was conducted by SPSS version 15.0 for Windows.

RESULTS

We identified 41 overviews of systematic reviews whose mean total reporting score was 10.78 (SD 3.84) and methodological score 3.05 (SD 2.09). Some important items were not adequately reported: only 69% reported defined eligibility criteria, 76% reported search strategy, 49% reported the process of review selection, 44% reported the data collection process, 5% reported evaluating the reporting quality, 46% reported evaluating methodological quality, and 20% reported assessing the evidence level for each outcome.

CONCLUSION

The reporting and methodological quality of overviews of systematic reviews was very poor, and there is still much room for improvement. A checklist for overviews of systematic reviews should be developed and used.

A PRISMA assessment of the reporting quality of systematic reviews in orthodontics

OBJECTIVES

To assess the reporting quality of Cochrane and non-Cochrane systematic reviews (SR) in orthodontics and to compare the reporting quality (PRISMA score) with methodological quality (AMSTAR criteria).

MATERIALS AND METHODS

Systematic reviews (n  =  109) published between January 2000 and July 2011 in five leading orthodontic journals were identified and included. The quality of reporting of the included reviews was assessed by two authors in accordance with the PRISMA guidelines. Each article was assigned a cumulative grade based on fulfillment of the applicable criteria, and an overall percentage score was assigned. Descriptive statistics and simple and multiple linear regression analyses were undertaken.

RESULTS

The mean overall PRISMA score was 64.1% (95% confidence interval [CI], 62%-65%). The quality of reporting was considerably better in reviews published in the Cochrane Database of Systematic Reviews (P < .001) than in non-Cochrane reviews. Both multivariable and univariable analysis indicated that journal of publication and number of authors was significantly associated with the PRISMA score. The association between AMSTAR score and modified PRISMA score was also found to be highly statistically significant.

CONCLUSION

Compliance of orthodontic SRs published in orthodontic journals with PRISMA guidelines was deficient in several areas. The quality of reporting assessed using PRISMA guidelines was significantly better in orthodontic SRs published in the Cochrane Database of Systematic Reviews.

A descriptive analysis of child-relevant systematic reviews in the Cochrane Database of Systematic Reviews

Background

Systematic reviews (SRs) are considered an important tool for decision-making. There has been no recent comprehensive identification or description of child-relevant SRs. A description of existing child-relevant SRs would help to identify the extent of available child-relevant evidence available in SRs and gaps in the evidence base where SRs are required. The objective of this study was to describe child-relevant SRs from the Cochrane Database of Systematic Reviews (CDSR, Issue 2, 2009).

Methods

SRs were assessed for relevance using pre-defined criteria. Data were extracted and entered into an electronic form. Univariate analyses were performed to describe the SRs overall and by topic area.

Results

The search yielded 1666 SRs; 793 met the inclusion criteria. 38% of SRs were last assessed as up-to-date prior to 2007. Corresponding authors were most often from the UK (41%). Most SRs (59%) examined pharmacological interventions. 53% had at least one external source of funding. SRs included a median of 7 studies (IQR 3, 15) and 679 participants (IQR 179, 2833). Of all studies, 48% included only children, and 27% only adults. 94% of studies were published in peer-reviewed journals. Primary outcomes were specified in 72% of SRs. Allocation concealment and the Jadad scale were used in 97% and 25% of SRs, respectively. Adults and children were analyzed separately in 12% of SRs and as a subgroup analysis in 14%. Publication bias was assessed in only 14% of SRs. A meta-analysis was conducted in 68% of SRs with a median of 5 trials (IQR 3, 9) each. Variations in these characteristics were observed across topic areas.

Conclusions

We described the methodological characteristics and rigour of child-relevant reviews in the CDSR. Many SRs are not up-to-date according to Cochrane criteria. Our study describes variation in conduct and reporting across SRs and reveals clinicians' ability to access child-specific data.