Methodological Concerns and Quality Appraisal of Contemporary Systematic Reviews and Meta-Analyses in Pediatric Urology

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Reporting standards for child health research were few and poorly implemented

OBJECTIVES

This study aims to identify existing reporting standards for child health research, assess the robustness of the standards development process, and evaluate the dissemination of these standards.

STUDY DESIGN AND SETTING

We searched MEDLINE, the EQUATOR Network Library, and Google to identify reporting standards for child health research studies. We assessed the adherence of the Guidance for Developers of Health Research Reporting Guidelines (GDHRG) by the identified reporting standards. We also assessed the use of the identified reporting standards by primary research studies, and the endorsement of the included reporting standards by journals.

RESULTS

We identified six reporting standards for child health research, including two under development. Among the four available standards their median adherence to the 18 main steps of the GDHRG was 58.35% (range: 27.8%-83.3%). None of these four reporting standards had been endorsed by pediatric journals indexed by the Science Citation Index. Only 26 primary research studies declared that they followed one of the reporting standards.

CONCLUSION

There is a quantitative and qualitative paucity of well-developed reporting standards for child health research. The available standards are also poorly implemented. This situation demands an urgent need to develop robust standards and ensure their implementation.

Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist

Objective

We appraised the reporting quality of abstracts of systematic reviews/meta-analyses (SR/MAs) published in one urology journal and explored associations between abstract characteristics and completeness of reporting.

Methods

The Arab Journal of Urology (AJU) was searched for SR/MAs published between January 2011 and 31 May 2022. SR/MAs with structured abstract and quantitative synthesis were eligible. Two reviewers simultaneously together selected the SR/MAs by title, screened the abstracts, and included those based on inclusion/exclusion criteria. Data of a range of characteristics were extracted from each SR/MAs into a spreadsheet. To gauge completeness of reporting, the PRISMA-Abstract checklist (12 items) was used to appraise the extent to which abstracts adhered to the checklist. For each abstract, we computed item, section, and overall adherence. Chi-square and t-tests compared the adherence scores. Univariate and multivariate analyses identified the abstract characteristics associated with overall adherence.

Results

In total, 66 SR/MAs published during the examined period; 62 were included. Partial reporting was not uncommon. In terms of adherence to the 12 PRISMA-A items were: two items exhibited 100% adherence (title, objectives); five items had 80% to <100% adherence (interpretation, included studies, synthesis of results, eligibility criteria, and information sources); two items displayed 40% to <80% adherence (description of the effect, strengths/limitations of evidence); and three items had adherence that fell between 0% and 1.6% (risk of bias, funding/conflict of interest, registration). Multivariable regression revealed two independent predictors of overall adherence: single-country authorship (i.e. no collaboration) was associated with higher overall adherence (P = 0.046); and abstracts from South America were associated with lower overall adherence (P = 0.04).

Conclusion

This study is the first to appraise abstracts of SR/MAs in urology. For high-quality abstracts, improvements are needed in the quality of reporting. Adoption/better adherence to PRISMA-A checklist by editors/authors could improve the reporting quality and completeness of SR/MAs abstracts.

Quality of reporting for pilot randomized controlled trials in the pediatric urology literature-A systematic review

BACKGROUND

The conduct and reporting of pilot studies is important to assess the feasibility of future randomized controlled trials (RCT). The Consolidated Standards of Reporting Trials (CONSORT) statement extension to pilot/feasibility studies addresses the reporting quality of the pilot studies (Summary Table 1). The aims of this systematic review are (1) to assess the reporting quality of pilot studies in pediatric urology and (2) to explore the factors that are associated with the reporting quality of these studies.

METHODS

A comprehensive search was conducted through MEDLINE® and EMBASE® to identify pilot RCTs from 2005 to 2018. Two reviewers independently performed title and abstract screening and full text review, with discrepancies resolved by consensus. CONSORT extension reported items were summarized and overall proportion of reported items for each article was estimated. A linear regression model was conducted to determine factors associated with higher reporting quality. Publication year, biostatistician/epidemiologist support, sample size justification and journal impact factor were collected.

RESULTS

Of the 1463 titles duplicates were removed and 1347 were screened, 36 studies were included. Overall, 36 pilot studies reported about 8-9 of 17 items [51% (95% CI: 46 - 56%)]. The most reported items were contact details for the corresponding author (97%), title identification of study as randomised pilot or feasibility trial (95%), eligibility criteria and setting (81%), both interventions (78%), and specific objectives of the pilot trial (75%). Less fulfilled items were blinding (11%), registration of the trial (11%), randomization details (28%), detailing recruitment status in the pilot study (19%), trial design (31%), and source of funding for pilot trial (34%). Interpretation of the results of pilot trial and their implications for the future definitive trial was reported by 34% of the studies. Factors associated with higher reporting quality were the presence of biostatistician or epidemiologist (P = 0.004), and if the sample size for the pilot study was justified (P = 0.002).

DISCUSSION

Overall reporting quality of pilot studies in pediatric urology literature from 2005-2018 was suboptimal. The quality of pilot RCTs included in the present review were lower than that observed in the orthopedic literature, however, it appears to be consistent with the trends regarding OQS in chronic kidney disease and allopathic medicine. While we endeavoured to maintain utmost rigidity of this systematic review, there are inherent limitations. The CONSORT 2010 extension for pilot RCTs was published in 2016. Clinical trials can take several years, many pilot studies published pre-2016 would not have had the guidance of the extension during designing phases. Not all pilot RCTs are published, so this could potentially reduce the generalizability of the findings from this review. Only studies in English, published in full peer-reviewed journals were included, and this review only addressed the reporting quality of pilot studies in pediatric urology.

CONCLUSION

This review demonstrated that reporting quality of pilot studies in pediatric urology is currently suboptimal. Including biostatistician and/or epidemiologist, can ameliorate the quality of future pilot studies. Implementing CONSORT 2010 extension by journals as a prerequisite for submission of pilot or feasibility trials is recommended to improve the robustness and transparency of future pilot studies.

Prophylactic central neck dissection for well-differentiated thyroid carcinoma: results and methodological assessment of systematic reviews

OBJECTIVE

The aim was to assess the methodological quality of systematic reviews evaluating the effectivness of prophylactic central neck dissection plus thyroidectomy to decrease locoregional recurrence in patients with thyroid carcinoma.

INTRODUCTION

Many systematic reviews have been published concerning prophylactic central neck dissection in well-differentiated thyroid carcinoma, finding inconsistent results regarding the risk of locoregional recurrence. Because systematic reviews are considered the best source on which to base clinical decisions, the assessment of methodological quality is important.

INCLUSION CRITERIA

This paper included studies that mentioned that a systematic review was performed in adult patients with confirmed diagnosis of well-differentiated thyroid carcinoma without evidence of nodal metastatic disease who underwent total thyroidectomy. Reviews evaluated prophylactic central neck dissection compared with no neck diessection for decreasing locoregional recurrence.

METHODS

A systematic search was performed using MEDLINE, Embase, the Cochrane Database of Systematic Reviews, LILACS, Web of Science, CNKI, and Google Scholar. PROSPERO was searched for unpublished studies and gray literature. Data regarding the study characteristics, patient types, numbers of patients, and locoregional recurrence by group were extracted. Methodological characteristics, type of statistical analysis and summary estimator, heterogeneity, and publication bias were recorded. The methodological quality was measured using the AMSTAR 2 tool.

RESULTS

The search identified 12 systematic reviews. Only four systematic reviews reported adherence to methodological guidelines. The AMSTAR 2 critical criteria classified all the studies as critically low quality. The pooled risk difference for locoregional recurrence varied among studies from -5% to 0%. Most of the studies were published in high-impact journals.

CONCLUSIONS

The published systematic reviews that assessed prophylactic central neck dissection as an intervention to decrease the rate of locoregional recurrence have a critically low methodological quality. The results and recommendations based on these studies should be used with caution. The evaluation of methodological quality by peer reviewers must be improved.

A Reporting Quality Assessment of Systematic Reviews and Meta-Analyses in Sports Physical Therapy: A Review of Reviews

This review of reviews aimed to evaluate the reporting quality of published systematic reviews and meta-analyses in the field of sports physical therapy using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. This review of reviews included a literature search; in total, 2047 studies published between January 2015 and December 2020 in the top three journals related to sports physical therapy were screened. Among the 125 identified articles, 47 studies on sports physical therapy were included in the analysis (2 systematic reviews and 45 meta-analyses). There were several problems areas, including a lack of reporting for key components of the structured summary (10/47, 21.3%), protocol and registration (18/47, 38.3%), risk of bias in individual studies (28/47, 59.6%), risk of bias across studies (24/47, 51.1%), effect size and variance calculations (5/47, 10.6%), additional analyses (25/47, 53.2%), and funding (10/47, 21.3%). The quality of the reporting of systematic reviews and meta-analyses of studies on sports physical therapy was low to moderate. For better evidence-based practice in sports physical therapy, both authors and readers should examine assumptions in more detail, and report valid and adequate results. The PRISMA guideline should be used more extensively to improve reporting practices in sports physical therapy.

Evaluating the Quality of Systematic Reviews and Meta-Analyses About Breast Augmentation Using AMSTAR

BACKGROUND

Breast augmentation is one of the most commonly performed cosmetic surgeries worldwide. Therefore, it is imperative to have evidence with high methodological quality to guide clinical decision making.

OBJECTIVES

To evaluate the methodological quality of the systematic reviews (SRs) focused on breast augmentation.

METHODS

A comprehensive search of MEDLINE, Embase, and the Cochrane Library of Systematic Reviews was performed. SRs that have a particular focus on breast augmentation and were published in the top 15 plastic and reconstructive surgery journals were included. Quality assessment was performed using a measurement tool to assess systematic reviews (AMSTAR). Study characteristics were extracted including journal and impact factor, year of publication, country affiliation of the corresponding author, reporting adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, number of citations, and number of studies included.

RESULTS

Among the 22 studies included for analysis, the mean AMSTAR score was moderate (5.55), with no SR achieving good quality (AMSTAR score of ≥9). There were no significant associations between AMSTAR score and journal impact factor, number of citations, year of publication, or number of included studies. Studies that reported adherence to PRISMA guidelines on average scored higher on the AMSTAR tool (P = 0.03).

CONCLUSIONS

The methodological quality of reviews about breast augmentation was found to be moderate, with no significant increase in studies or quality over time. Adherence to PRISMA guidelines and increased appraisal of SRs about breast augmentation using methodological assessment tools would further strengthen methodological quality and confidence in study findings.

Appraising the quality of meta-analysis for breast cancer treatment in the adjuvant setting: A systematic review

INTRODUCTION

Breast cancer is the tumor with highest incidence in women worldwide and adjuvant treatment is extremely important to achieve disease control. Given the relevance of systematic reviews, their rigor should be warranted to avoid biased conclusions. Our objective was to investigate the methodological quality of meta-analysis of early breast cancer adjuvant treatment.

MATERIAL AND METHODS

Comprehensive searches were performed using electronic databases from 1/1/2007 to 11/12/2018. All studies identified as a systematic review with meta-analysis investigating the efficacy of breast cancer adjuvant treatments were included. Two reviewers independently assessed titles and abstracts, then full-texts for eligibility. Quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) version 2 tool.

RESULTS

Of 950 citations retrieved, 66 studies (7.0%) were deemed eligible. Methodological quality was highly variable, median AMSTAR score 8.5 (IQR 7-9.5) and range 0-16. There was a weak positive correlation between journal impact factor and AMSTAR score (r = 0.17) and citation rate and AMSTAR score (r = 0.16). Cochrane Systematic Reviews were of higher quality than reviews from other journals. Overall confidence was critically low for 61 (92.4%) studies, and the least well-reported domains were the statement of conflict of interest and funding source for the included studies (4.6%), the report of a pre-defined study protocol (15.2%), and the description of details of excluded studies (6.1%).

CONCLUSIONS

Our findings reinforce concerns about the design, conduction and interpretation of meta-analysis in current literature. Methodological quality should be carefully considered and journal editors, decision makers and readers in general, must follow a critical approach to this studies.

The quality of systematic reviews and meta-analyses assessing the treatment and management of male infertility

BACKGROUND

Male factor infertility (MFI) is a common medical condition which requires high-quality research to guide clinical practice; however, systematic reviews (SRs) and meta-analyses (MAs) often vary in quality, raising concerns regarding the validity of their results. We sought to perform an objective analysis of SRs and MAs in MFI treatment and management and to report on the quality of published literature.

METHODS

A comprehensive search in PubMed/MEDLINE and Embase was used to identify relevant publications. Primary search terms were male infertility, male sterility, and male subfertility. Two authors independently performed searches, screened citations for eligibility, extracted data for analysis, and graded methodological quality using the validated AMSTAR (A Measurement Tool to Assess Systematic Reviews) instrument, a validated tool used in the critical appraisal of SRs/MAs.

RESULTS

Of 27 publications met inclusion criteria and were included in the analysis. Mean AMSTAR score (± SD) among all publications was 7.4 (1.9) out of 11, reflecting "fair to good" quality. Non-pharmacological medical treatment for MFI was the most commonly assessed intervention (n = 13, 48.1%). No publications met all AMSTAR criteria. While the number of SRs/MAs has increased over time (P = 0.037), the quality of publications has not significantly changed (P = 0.72). SRs/MAs of the Cochrane Library had higher AMSTAR score than non-Cochrane SRs/MAs (8.5 vs 6.3, P = 0.002).

CONCLUSIONS

The methodological quality of SRs/MAs should be assessed to ensure high-quality evidence for clinical practice guidelines in MFI treatment and management. This review highlights a need for increased effort to publish high-quality studies in MFI treatment and management.

Assessing the methodological and reporting quality of clinical systematic reviews and meta-analyses in paediatric urology: can practices on contemporary highest levels of evidence be built?

INTRODUCTION

Systematic reviews and meta-analyses provide a comprehensive summary of research studies and are used to assess clinical evidence, form policy and construct guidelines. This is pertinent to childhood surgery with issues of consent and condition prevalence. The aims of this study were to evaluate the methodological and reporting quality of these reviews and to identify how these reviews might guide clinical practice amongst those conditions most commonly encountered and managed by practicing paediatric urologists.

METHODS

A systematic search of the English literature was performed to identify systematic reviews and meta-analyses focusing on clinical paediatric urology (1/1/1992-1/12/2018) to include common paediatric urological conditions managed by paediatric urology residents/fellows. To these reviews, Assessing the Methodological Quality of Systematic Reviews (AMSTAR)-2 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scores were applied. Univariate linear regression and descriptive statistical methods were performed.

RESULTS

From an initial literature review of 1723 articles, 227 were included in the analysis. Inter-reviewer agreement was high amongst 3 independent reviewers (κ = 0.92). Eighty-four percent of systematic reviews and meta-analyses were published since 2009 following publication of the PRISMA guidelines. The overall impact factor was 3.38 (0.83-17.58), with adherence to AMSTAR-2 criteria 48.46% and PRISMA criteria 70.1%. From a methodological perspective, 15% of reviews were of moderate quality, 65% were of low quality and 20% reviews were of critically low quality, with none found to have good quality reporting.

CONCLUSIONS

Despite the continued increase of systematic reviews and meta-analyses in paediatric urology from which many guidelines are based, a significant number of reviews contain poor methodology and, to a lesser extent, poor reporting quality. Journals should consider having specific 'a priori' criteria based on checklists before publication of manuscripts to ensure the highest possible reporting quality.

Medical and surgical interventions for the treatment of urinary stones in children

BACKGROUND

Urolithiasis is a condition where crystalline mineral deposits (stones) form within the urinary tract. Urinary stones can be located in any part of the urinary tract. Affected children may present with abdominal pain, blood in the urine or signs of infection. Radiological evaluation is used to confirm the diagnosis, to assess the size of the stone, its location, and the degree of possible urinary obstruction.

OBJECTIVES

To assess the effects of different medical and surgical interventions in the treatment of urinary tract stones of the kidney or ureter in children.

SEARCH METHODS

We searched the Cochrane Register of Controlled Trials (CENTRAL), MEDLINE (Ovid), Embase (Ovid) as well as the World Health Organization International Clinical Trials Registry Platform Search Portal and ClinicalTrials.gov. We searched reference lists of retrieved articles and conducted an electronic search for conference abstracts for the years 2012 to 2017. The date of the last search of all electronic databases was 31 December 2017 and we applied no language restrictions.

SELECTION CRITERIA

We included all randomised controlled trials (RCTs) and quasi-RCTs looking at interventions for upper urinary tract stones in children. These included shock wave lithotripsy, percutaneous nephrolithotripsy, ureterorenoscopy, open surgery and medical expulsion therapy for upper urinary tract stones in children aged 0 to 18 years.

DATA COLLECTION AND ANALYSIS

We used standard methodological procedures according to Cochrane guidance. Two review authors independently searched and assessed studies for eligibility and conducted data extraction. 'Risk of bias' assessments were completed by three review authors independently. We used Review Manager 5 for data synthesis and analysis. We used the GRADE approach to assess the quality of evidence.

MAIN RESULTS

We included 14 studies with a total of 978 randomised participants in our review, informing eight comparisons. The studies contributing to most comparisons were at high or unclear risk of bias for most domains.Shock wave lithotripsy versus dissolution therapy for intrarenal stones: based on one study (87 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on stone-free rate (SFR), serious adverse events or complications of treatment and secondary procedures for residual fragments.Slow shock wave lithotripsy versus rapid shock wave lithotripsy for renal stones: based on one study (60 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on SFR, serious adverse events or complications of treatment and secondary procedures for residual fragments.Shock wave lithotripsy versus ureteroscopy with holmium laser or pneumatic lithotripsy for renal and distal ureteric stones: based on three studies (153 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on SFR, serious adverse events or complications of treatment and secondary procedures.Shock wave lithotripsy versus mini-percutaneous nephrolithotripsy for renal stones: based on one study (212 participants), SWL likely has a lower SFR (RR 0.88, 95% CI 0.80 to 0.97; moderate quality evidence); this corresponds to 113 fewer stone-free patients per 1000 (189 fewer to 28 fewer). SWL may reduce severe adverse events (RR 0.13, 95% CI 0.02 to 0.98; low quality evidence); this corresponds to 66 fewer serious adverse events or complications per 1000 (74 fewer to 2 fewer). Rates of secondary procedures may be higher (RR 2.50, 95% CI 1.01 to 6.20; low-quality evidence); this corresponds to 85 more secondary procedures per 1000 (1 more to 294 more).Percutaneous nephrolithotripsy versus tubeless percutaneous nephrolithotripsy for renal stones: based on one study (23 participants) and consistently very low quality evidence, we are uncertain about the effects of percutaneous nephrolithotripsy on SFR, serious adverse events or complications of treatment and secondary procedures.Percutaneous nephrolithotripsy versus tubeless mini-percutaneous nephrolithotripsy for renal stones: based on one study (70 participants), SFR are likely similar (RR 1.03, 95% CI 0.93 to 1.14; moderate-quality evidence); this corresponds to 28 more per 1,000 (66 fewer to 132 more). We did not find any data relating to serious adverse events. Based on very low quality evidence we are uncertain about secondary procedures.Alpha-blockers versus placebo with or without analgesics for distal ureteric stones: based on six studies (335 participants), alpha-blockers may increase SFR (RR 1.34, 95% CI 1.16 to 1.54; low quality evidence); this corresponds to 199 more stone-free patients per 1000 (94 more to 317 more). Based on very low quality evidence we are uncertain about serious adverse events or complications and secondary procedures.

AUTHORS' CONCLUSIONS

Based on mostly very low-quality evidence for most comparisons and outcomes, we are uncertain about the effect of nearly all medical and surgical interventions to treat stone disease in children.Common reasons why we downgraded our assessments of the quality of evidence were: study limitations (risk of bias), indirectness, and imprecision. These issues make it difficult to draw clinical inferences. It is important that affected individuals, clinicians, and policy-makers are aware of these limitations of the evidence. There is a critical need for better quality trials assessing patient-important outcomes in children with stone disease to inform future guidelines on the management of this condition.

Reporting quality and statistical analysis of published dose-response meta-analyses was suboptimal: a cross-sectional literature survey

OBJECTIVES

The objective of this study was to investigate the characteristics, methodological quality, and reporting of statistical analyses of published dose-response meta-analyses (DRMAs).

STUDY DESIGN AND SETTING

We searched PubMed to identify DRMAs published in 2017. The reporting characteristics and methodological qualities were assessed by the PRISMA (27 items) and AMSTAR (11 items), respectively. We also summarized the reporting of statistical analyses of included DRMAs.

RESULTS

We identified 93 DRMAs, most of which (59/93) were conducted by Chinese researchers and the main outcome was the incidence of cancers. Of the PRISMA and AMSTAR items, twenty and five were well complied (80% or more), respectively. The compliance rates of several PRISMA checklist items, such as structured summary, objectives, protocol and registration, and funding, were less than 50%. There were no criteria to estimate the doses for the open-ended intervals of exposure or intervention doses. When the restricted cubic splines were used to fit nonlinear dose-response relationships, there were also no criteria to determine the fixed knots.

CONCLUSION

The adherence to the methodological items of reporting guidelines and statistical analysis of published DRMAs were suboptimal. Development of reporting guidelines to assist authors in writing and readers in critically appraising the reports of DRMAs is timely.

Methodological Quality of Systematic Reviews of Intraoperative Neuromonitoring in Thyroidectomy: A Systematic Review

Importance

Systematic reviews and meta-analyses are considered the best evidence for clinical decision making. Many reviews of intraoperative neuromonitoring (IONM) in thyroidectomy have conflicting results, owing in large part to methodological quality.

Objective

To assess the methodological quality and the causes of heterogeneous results of systematic reviews that compare routine IONM vs visual identification of the recurrent laryngeal nerve (RLN) in patients undergoing thyroidectomy.

Data Sources

A systematic search was performed of MEDLINE (PubMed), Embase, the Cochrane Library, LILACS (Literatura Latino Americana e do Caribe em Ciências da Saúde), Web of Science, and Google from January 1, 1968, through June 30, 2018. Data were analyzed from July 17 to November 30, 2018.

Study Selection

Studies that mentioned performance of a systematic review/meta-analysis during the search period.

Data Extraction and Synthesis

Data including study characteristics, type of patients, numbers of nerves at risk, and temporary and definitive RLN paralysis by group were extracted. Data about methodological characteristics, type of statistical analysis and summary estimator, endorsement of systematic review/meta-analysis guidelines, heterogeneity, publication bias, funding, conflict of interest, and statistical analysis were also recorded. The methodological quality was measured with the AMSTAR2 (A Measurement Tool to Assess Systematic Reviews) tool by 2 independent evaluators.

Main Outcomes and Measures

Methodological quality.

Results

The search identified 13 systematic reviews that included patients who underwent open or minimally invasive thyroidectomy, second operations, and a mixture of low- and high-risk procedures. The mean compliance with the AMSTAR2 overall criteria was 53% (range, 11%-83%); with critical criteria, 71% (range, 50%-94%). The percentage of nerves at risk from RCTs was 4.8%. The mean (SD) crude rate of definitive RLN paralysis was 0.81% (0.22%; median, 0.75% [range, 0.53%-1.30%]) in the monitoring group and 1.14% (0.56%; median, 0.96% [range, 0.57%-2.56%]) in the control group.

Conclusions and Relevance

A substantial number of systematic reviews of IONM in thyroidectomy have conflicting results, but their mean methodological quality is critically low. Design of a systematic review should comply with methodological standards and recommendations to offer relevant and practical information for decision making.

Quality of reporting and fragility index for randomized controlled trials in the vesicoureteral reflux literature: where do we stand?

BACKGROUND/INTRODUCTION

Randomized controlled trials (RCTs) are considered the "gold standard" methodology for examining the effects of clinical interventions, yet only 1% of urology literature employs this design. The Consolidated Standards of Reporting Trials (CONSORT) statement contains a standardized checklist of 37 items to be included when reporting RCTs to ensure transparency and completeness of information [2]. Despite the robust design of RCTs, the number of events can greatly change the significance of the results, which can be represented by the fragility index (FI).

OBJECTIVE

The objective was to assess the quality of reporting of RCTs in the pediatric vesicoureteral reflux (VUR) literature using the 2010 CONSORT statement and, for studies with significant positive findings, to determine the FI as a measure of robustness of the results.

STUDY DESIGN

A comprehensive search was conducted through MEDLINE® and Embase® to identify RCTs in VUR literature from 2000 to 2016. Two reviewers independently selected articles, which were evaluated using the CONSORT checklist. An overall quality of reporting score (OQS) (%) was calculated by dividing the number of checklist items present in each study by the maximum possible score (34) and expressed as a percentage. Studies were classified as low (<40%), moderate (40-70%) and high quality (>70%) based on the modified assessing the methodological quality of systematic reviews (AMSTAR) checklist. Of the 2052 initial matches, 98% were excluded due to methodology or content, a further 28 studies were found not to meet inclusion criteria after full text review. The FI was calculated for the 7 studies that met inclusion criteria with significantly different results by manually adding events to the study groups until p>0.05.

RESULTS

Twenty-two studies met inclusion criteria. The mean OQS was 46+17% with 9 (41%) identified as low quality (score <40%), 11 (50%) as moderate (40-70%) and 2 (9%) as high quality (>70%). There was no significant difference in OQS between RCTs with a sample size > 100 (n=15) versus <100 patients (n=7) (45+17% vs. 47+17%, p=0.7). However, we noted a difference when we compared RCTs with biostatistician support (n=4) vs. those without (n=18) (59+20% vs. 43+15%, p<0.05). Seven studies reported significant positive results making calculation of FI possible. The mean FI was 5.8+5.1 indicating that most studies were fragile. There was no correlation between the OQS and FI.

DISCUSSION AND CONCLUSION

The mean OQS of VUR RCTs was suboptimal (46%), with most studies having low FI scores indicating the instability of the findings. The only variable that significantly impacted the OQS was biostatistician support. Implementation of the CONSORT checklist with a minimum of 50% inclusion as a prerequisite for submission of manuscripts may improve the quality and transparency of reporting. Calculation of the FI could provide readers with an objective measure of robustness for the published trials, allowing for appropriate interpretation of the results.

The quality of systematic reviews addressing peripheral nerve repair and reconstruction

While systematic reviews are regarded as the strongest level of medical evidence, inconsistency in the quality and rigor of systematic reviews raises concerns about their use as a tool in guiding quality delivery in evidence-based clinical practice. The objective of this present study was to assess methodological soundness of systematic reviews with a particular focus on peripheral nerve repair and reconstruction. We performed a comprehensive search using PubMed and Scopus to identify all systematic reviews published on peripheral nerve reconstruction in 9 high-impact surgical journals. Two authors independently performed literature searches, screened abstracts, and extracted data. Discrepancies were resolved by discussion and consensus. The quality of systematic reviews was assessed using AMSTAR criteria. Initial search retrieved 184 articles. After screening duplicates, titles, abstracts, and conducting full text reviews, 26 studies met inclusion criteria. Of those, 18 (65%) were published by Plastic Surgery, 7 (27%) by Orthopedic Surgery, and 1 (4%) by Occupational Therapy. The total number of systematic reviews published on peripheral nerves each year has shown an increasing trend from 2004 through 2015. The overall median AMSTAR score was 5, reflecting a "fair" quality. There was no evidence of AMSTAR score improvement over time. Although the number of systematic reviews published on peripheral nerve repair has risen over the last decade, their quality has not exhibited the same increase. This highlights the necessity to increase familiarity with and conform to methodological quality criteria in order to improve the integrity of evidence-based medicine in peripheral nerve repair and reconstruction.

Medical and surgical interventions for the treatment of urinary stones in children

BACKGROUND

Urolithiasis is a condition where crystalline mineral deposits (stones) form within the urinary tract. Urinary stones can be located in any part of the urinary tract. Affected children may present with abdominal pain, blood in the urine or signs of infection. Radiological evaluation is used to confirm the diagnosis, to assess the size of the stone, its location, and the degree of possible urinary obstruction.

OBJECTIVES

To assess the effects of different medical and surgical interventions in the treatment of urinary tract stones of the kidney or ureter in children.

SEARCH METHODS

We searched the Cochrane Register of Controlled Trials (CENTRAL), MEDLINE (Ovid), Embase (Ovid) as well as the World Health Organization International Clinical Trials Registry Platform Search Portal and ClinicalTrials.gov. We searched reference lists of retrieved articles and conducted an electronic search for conference abstracts for the years 2012 to 2017. The date of the last search of all electronic databases was 31 December 2017 and we applied no language restrictions.

SELECTION CRITERIA

We included all randomised controlled trials (RCTs) and quasi-RCTs looking at interventions for upper urinary tract stones in children. These included shock wave lithotripsy, percutaneous nephrolithotripsy, ureterorenoscopy, open surgery and medical expulsion therapy for upper urinary tract stones in children aged 0 to 18 years.

DATA COLLECTION AND ANALYSIS

We used standard methodological procedures according to Cochrane guidance. Two review authors independently searched and assessed studies for eligibility and conducted data extraction. 'Risk of bias' assessments were completed by three review authors independently. We used Review Manager 5 for data synthesis and analysis. We used the GRADE approach to assess the quality of evidence.

MAIN RESULTS

We included 14 studies with a total of 978 randomised participants in our review, informing eight comparisons. The studies contributing to most comparisons were at high or unclear risk of bias for most domains.Shock wave lithotripsy versus dissolution therapy for intrarenal stones: based on one study (87 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on stone-free rate (SFR), serious adverse events or complications of treatment and secondary procedures for residual fragments.Slow shock wave lithotripsy versus rapid shock wave lithotripsy for renal stones: based on one study (60 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on SFR, serious adverse events or complications of treatment and secondary procedures for residual fragments.Shock wave lithotripsy versus ureteroscopy with holmium laser or pneumatic lithotripsy for renal and distal ureteric stones: based on three studies (153 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on SFR, serious adverse events or complications of treatment and secondary procedures.Shock wave lithotripsy versus mini-percutaneous nephrolithotripsy for renal stones: based on one study (212 participants), SWL likely has a lower SFR (RR 0.88, 95% CI 0.80 to 0.97; moderate quality evidence); this corresponds to 113 fewer stone-free patients per 1000 (189 fewer to 28 fewer). SWL may reduce severe adverse events (RR 0.13, 95% CI 0.02 to 0.98; low quality evidence); this corresponds to 66 fewer serious adverse events or complications per 1000 (74 fewer to 2 fewer). Rates of secondary procedures may be higher (RR 2.50, 95% CI 1.01 to 6.20; low-quality evidence); this corresponds to 85 more secondary procedures per 1000 (1 more to 294 more).Percutaneous nephrolithotripsy versus tubeless percutaneous nephrolithotripsy for renal stones: based on one study (23 participants) and consistently very low quality evidence, we are uncertain about the effects of SWL on SFR, serious adverse events or complications of treatment and secondary procedures.Percutaneous nephrolithotripsy versus tubeless mini-percutaneous nephrolithotripsy for renal stones: based on one study (70 participants), SFR are likely similar (RR 1.03, 95% CI 0.93 to 1.14; moderate-quality evidence); this corresponds to 28 more per 1,000 (66 fewer to 132 more). We did not find any data relating to serious adverse events. Based on very low quality evidence we are uncertain about secondary procedures.Alpha-blockers versus placebo with or without analgesics for distal ureteric stones: based on six studies (335 participants), alpha-blockers may increase SFR (RR 1.34, 95% CI 1.16 to 1.54; low quality evidence); this corresponds to 199 more stone-free patients per 1000 (94 more to 317 more). Based on very low quality evidence we are uncertain about serious adverse events or complications and secondary procedures.

AUTHORS' CONCLUSIONS

Based on mostly very low-quality evidence for most comparisons and outcomes, we are uncertain about the effect of nearly all medical and surgical interventions to treat stone disease in children.Common reasons why we downgraded our assessments of the quality of evidence were: study limitations (risk of bias), indirectness, and imprecision. These issues make it difficult to draw clinical inferences. It is important that affected individuals, clinicians, and policy-makers are aware of these limitations of the evidence. There is a critical need for better quality trials assessing patient-important outcomes in children with stone disease to inform future guidelines on the management of this condition.

Quality assessment of systematic reviews on total hip or knee arthroplasty using mod-AMSTAR

Background

Increasing numbers of systematic reviews (SRs) on total knee arthroplasty (TKA) and total hip arthroplasty (THA) have been published in recent years, but their quality has been unclear. The purpose of this study is to evaluate the methodological quality of SRs on TKA and THA.

Methods

We searched Ovid-Medline, Ovid-Embase, Cochrane Databases (including HTA, DARE, and CDSR), CBM, CNKI, Wang Fang, and VIP, from January 2014 to December 2015 for THA and TKA. The quality of SRs was assessed using the modified 25-item “Assessment of Multiple Systematic Reviews” (mod-AMSTAR) tool, which was based on the AMSTAR scale. A T-test, nonparametric test, and linear regression were conducted to assess the relationship between bibliographical characteristics and methodological quality.

Results

Sixty-three SRs were included, from which the majority of SRs (50, 79.4%) were conducted in Asia. Only 4 reviews were rated as high quality, and most were weak in providing a priori design (6, 9.5%), not limiting the publication type (8, 13%), providing an excluded primary studies list (4, 6.3%) and reporting support for the included primary studies (1, 1.6%). Reviews published in English journals performed better than did Chinese journals in duplicate data extraction (81.3% vs 46.7%, p = 0.017; 70.8% vs 33.3%, p = 0.009) and providing source of support for the SR (87.5% vs 33.3%, P < 0.001). Reviews published in journals with a higher impact factor were associated with a higher mod-AMSTAR score (regression coefficient: 0.38, 95%CI: 0.11–0.65; P = 0.006).

Conclusion

The methodological quality of the included SRs is far from satisfactory. Authors of SRs should conform to the recommendations outlined in the mod-AMSTAR items. Areas needing improvement were providing a priori design, not limiting the publication type, providing an excluded primary studies list, and reporting conflicts of interest.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0488-8) contains supplementary material, which is available to authorized users.

The Quality of Systematic Reviews in Head and Neck Microsurgery: A Perspective from Plastic Surgery and Otolaryngology

INTRODUCTION

In recent years, there has been a push to publish higher level of evidence studies in medicine, particularly in plastic surgery. Well-conducted systematic reviews are considered the strongest level of evidence in medicine, recently becoming the key process indicators for quality delivery. A varying quality of systematic reviews, however, has led to concerns of their validity in clinical decision-making. We perform a quality analysis of systematic reviews published in head and neck microsurgery by the surgical specialties of plastic surgery and otolaryngology.

MATERIALS AND METHODS

An evaluation of systematic reviews published on microsurgery in 13 high-impact surgical journals was conducted by searching PubMed and Scopus. Two authors independently performed searches, screened for eligibility, and extracted data from included articles. Discrepancies were resolved by discussion and consensus. Assessment of Multiple Systematic Reviews (AMSTAR) criteria were used to assess methodological quality.

RESULTS

The initial database search retrieved 166 articles. After removing duplicates, screening titles and abstracts, 26 articles remained for full text review. Seven did not focus on head and neck microsurgery and were further excluded, leaving 19 systematic reviews for final analysis. Of those, 10 systematic reviews were published by otolaryngology, and 9 were published by plastic surgery. Median AMSTAR score was 8 for otolaryngology, 7 for plastic surgery, and 8 overall, reflecting "fair to good" quality. The number of systematic reviews on head and neck microsurgery markedly increased over time. Of note, both the AMSTAR score and the number of systematic reviews published by plastic surgery have steadily increased from 2014 to 2016, whereas those published by otolaryngology have remained relatively stable since 2010.

CONCLUSIONS

Our review shows a trend toward publishing more systematic reviews. The increasing quantity and quality of systematic reviews published by plastic surgeons indicates recognition in the need for higher levels of evidence in plastic surgery, as well as growing interest and advances in microsurgery. Given these trends, familiarity with quality assessment guidelines, such as AMSTAR, will remain important in providing a basis for building relevant value-based quality measures.

Quality of systematic reviews and meta-analyses published in pediatric surgery

AIMS

Systematic reviews (SRs) and meta-analyses (MAs) studies may influence and direct surgical practice. Against this background we have analyzed the quality of systematic reviews and meta-analyses published in the specialty field of pediatric surgery using the Assessment of Multiple Systematic Reviews 11-item tool (AMSTAR).

METHODS

MEDLINE and EMBASE databases and the three major journals in pediatric surgery were searched for SRs and MAs in pediatric surgery. Studies involving predominantly adult populations were excluded. Two reviewers independently scored included studies against AMSTAR criteria and disagreements were resolved by consensus. A total rating of 4 or less was considered 'poor' methodological quality, 5-8 as 'fair to good' and 9 or greater as 'good'.

RESULTS

Original searching retrieved 1,281 articles. 126 articles were included for final analysis. Examining recent trends, 4 studies were published in 1995-2000 compared to 78 in 2011-2014. Using AMSTAR scoring criteria, 35 reviews (28%) were regarded as 'poor' in terms of methodological quality, 59 (47%) 'fair', and 32 (25%) 'good' quality. We observed no improvement in AMSTAR score before and after the development of the tool (mean score pre-2008 6.8, post-2008 5.9, p = 0.136).

CONCLUSIONS

Despite an increase in the number of SRs / MAs published in pediatric surgery, a quarter of all studies were considered poor in terms of their quality and scientific validity. Journals must define and apply minimum criteria to ensure pediatric surgeons seeking to publish high quality SRs / MAs achieve these requirements.

Quality of reporting for randomized controlled trials in the hypospadias literature: Where do we stand?

INTRODUCTION

To assess the quality of reporting of randomized controlled trials (RCTs) in the hypospadias literature using the 2010 Consolidated Standards of Reporting Trials (CONSORT) statement. It was hypothesized that hypospadias RCTs that contained clear descriptions of key methodological items, allocation concealment, blinding, and sample size justification would have higher overall quality of reporting scores (OQS).

MATERIALS AND METHODS

A comprehensive search was conducted through MEDLINE to identify RCTs in hypospadias surgical techniques and postoperative management during the period 1990-2014. Two reviewers independently selected articles, which were evaluated using the CONSORT checklist. An overall quality score (%) was calculated to assess the quality of reporting. In addition, a methodological index score out of 4 was calculated based on the following items: use of intention to treat/sample size justification, allocation concealment, specification of randomization type, and blinding of outcome assessors.

RESULTS

Of the 76 initial results, 39 (51%) were excluded due to their predominant focus on anesthesia. After full-text screening, 10 (13%) citations were further excluded because they were case control studies or did not focus on hypospadias techniques, resulting in 27 (36%) studies included for analysis. The mean overall quality score was 37 ± 12% and a median of 36% (range: 14-61%). Fifteen (56%) studies were identified as low quality (score <40%) and 12 (44%) as moderate quality (40-70%). No studies were classified as high quality (>70%). Hypospadias RCTs published between 2007 and 2014 versus those reported before 2007 (44 ± 9% vs 33 ± 11%, P = 0.01), RCTs with a sample size >100 patients versus those <100 (47 ± 8% vs 36 ± 11%, P = 0.01), RCTs that disclosed having received funding versus those that did not (56 ± 4% vs 38 ± 10%, P < 0.01) and RCTs that had proof of biostatistician/epidemiologist support versus those that did not (58 ± 5% vs 36 ± 11%, P = 0.01) had a higher mean OQS. The number of articles that met specific 2010 CONSORT criteria is illustrated in Summary Fig.

DISCUSSION

It was found that the contemporary hypospadias literature continues to suffer from suboptimal reporting standards. There seems to be an improvement in the OQS for studies published after 2007 and those with larger sample sizes, usually >100 patients. Nevertheless, none of the studies obtained high quality of reporting (OQS >70%) as per the CONSORT statement checklist. The inadequacies in reporting were related to sample size justifications, randomization method, allocation concealment strategy, blinding, description of subjects lost to follow-up and Intention To Treat (ITT) analysis. These findings were consistent with the poor quality of reporting observed in other surgical fields.

CONCLUSIONS

The current overall quality score in hypospadias literature is suboptimal and efforts must be made to improve quality.

"Assessing the methodological quality of systematic reviews in radiation oncology: A systematic review"

OBJECTIVE

The objective of our study was to evaluate the methodological quality of systematic reviews and meta-analyses in Radiation Oncology.

METHODS

A systematic literature search was conducted for all eligible systematic reviews and meta-analyses in Radiation Oncology from 1966 to 2015. Methodological characteristics were abstracted from all works that satisfied the inclusion criteria and quality was assessed using the critical appraisal tool, AMSTAR. Regression analyses were performed to determine factors associated with a higher score of quality.

RESULTS

Following exclusion based on a priori criteria, 410 studies (157 systematic reviews and 253 meta-analyses) satisfied the inclusion criteria. Meta-analyses were found to be of fair to good quality while systematic reviews were found to be of less than fair quality. Factors associated with higher scores of quality in the multivariable analysis were including primary studies consisting of randomized control trials, performing a meta-analysis, and applying a recommended guideline related to establishing a systematic review protocol and/or reporting.

CONCLUSIONS

Systematic reviews and meta-analyses may introduce a high risk of bias if applied to inform decision-making based on AMSTAR. We recommend that decision-makers in Radiation Oncology scrutinize the methodological quality of systematic reviews and meta-analyses prior to assessing their utility to inform evidence-based medicine and researchers adhere to methodological standards outlined in validated guidelines when embarking on a systematic review.

Promoting physical activity in the workplace: A systematic meta-review

INTRODUCTION

Physically active (PA) people have a lower risk of various diseases, compared to those with sedentary lifestyles. Evidence on the effects of PA promoting programs in the workplace is large, and several systematic reviews (SR) and/or meta-analyses (MA) have been published. However, they have failed to consider factors that could influence interventions. This paper aimed to classify and describe interventions to promote PA in the workplace based on evidence from SR/MA.

METHOD

A literature search for SR/MA was done using PubMed, Web of Science, and Science Direct (January 2006-February 2015). Quality assessment of SR/MA was performed using AMSTAR. The PRECEDE-PROCEED model was used for classifying the interventions into predisposing, enabling, reinforcing, environment, and policy domains of focus.

RESULTS

Eleven SR/MA included 220 primary studies, of which 139 (63%) were randomized controlled trials. Of 48 interventions identified, 22 (46%) and 17 (35%) focused on predisposing or enabling employees to have more PA, respectively. Of the 22 predisposing factors, 6 were information delivery, 5 were self-motivation, and 11 were program training. The enabling approaches were 12 instrument resources and 5 health service facilities. The reinforcing approaches were 4 incentive and 3 social support. The remaining interventions focused on the environmental development and policy regulation.

CONCLUSIONS

This systematic meta-review classified interventions using appropriate framework and described the intervention pattern.

Systematic review adherence to methodological or reporting quality

BACKGROUND

Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively.

RESULTS

Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the SRs assessed risk of bias (validity) in the included studies, while 80% (1112/1387) reported the criteria for study selection.

CONCLUSIONS

Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.

Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study

BACKGROUND

The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.

RESULTS

Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.

CONCLUSIONS

The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

A systematic review of the quality of conduct and reporting of systematic reviews and meta-analyses in paediatric surgery

OBJECTIVE

Our objective was to evaluate quality of conduct and reporting of published systematic reviews and meta-analyses in paediatric surgery. We also aimed to identify characteristics predictive of review quality.

BACKGROUND

Systematic reviews summarise evidence by combining sources, but are potentially prone to bias. To counter this, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was published to aid in reporting. Similarly, the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) measurement tool was designed to appraise methodology. The paediatric surgical literature has seen an increasing number of reviews over the past decade, but quality has not been evaluated.

METHODS

Adhering to PRISMA guidelines, we performed a systematic review with a priori design to identify systematic reviews and meta-analyses of interventions in paediatric surgery. From 01/2010 to 06/2016, we searched: MEDLINE, EMBASE, Cochrane, Centre for Reviews and Dissemination, Web of Science, Google Scholar, reference lists and journals. Two reviewers independently selected studies and extracted data. We assessed conduct and reporting using AMSTAR and PRISMA. Scores were calculated as the sum of reported items. We also extracted author, journal and article characteristics, and used them in exploratory analysis to determine which variables predict quality.

RESULTS

112 articles fulfilled eligibility criteria (53 systematic reviews; 59 meta-analyses). Overall, 68% AMSTAR and 56.8% PRISMA items were reported adequately. Poorest scores were identified with regards a priori design, inclusion of structured summaries, including the grey literature, citing excluded articles and evaluating bias. 13 reviews were pre-registered and 6 in PRISMA-endorsing journals. The following predicted quality in univariate analysis:, word count, Cochrane review, journal h-index, impact factor, journal endorses PRISMA, PRISMA adherence suggested in author guidance, article mentions PRISMA, review includes comparison of interventions and review registration. The latter three variables were significant in multivariate regression.

CONCLUSIONS

There are gaps in the conduct and reporting of systematic reviews in paediatric surgery. More endorsement by journals of the PRISMA guideline may improve review quality, and the dissemination of reliable evidence to paediatric clinicians.

Are child-centric aspects in newborn and child health systematic review and meta-analysis protocols and reports adequately reported?-two systematic reviews

BACKGROUND

Evidence suggests that newborn and child health systematic reviews and meta-analyses exhibit poor quality in reporting. The "Preferred Reporting Items in Systematic Review and Meta-Analysis" (PRISMA) and PRISMA-Protocols (PRISMA-P) checklists have been developed to improve the reporting of systematic review results and protocols, respectively. We aimed to evaluate the clarity and transparency in reporting of child-centric items in child health systematic reviews (SRs) and SR protocols and to identify areas where reporting could be strengthened.

METHODS

Two preliminary lists of potential child-centric reporting items were used to examine current reporting. The Cochrane, DARE, MEDLINE, and EMBASE libraries were searched from 2010 to 2014 for systematic reviews that included children. Each report and protocol that met the inclusion criteria had their quality of reporting assessed by their reporting of child-centric items. Quality of reporting was assessed per whether one third, one to two thirds, or more than two thirds of papers complied with potential child-centric potential modifications/extensions to PRISMA and were analyzed by the following: (i) paper type (i.e., report vs. protocol), (ii) publication type (i.e., Cochrane vs. non-Cochrane), and (iii) population type (i.e., child-only vs. mixed populations vs. family/maternal).

RESULTS

Of the 414 eligible articles, 248 reports and 76 protocols were included. In 21 of 24 potential SR reporting items and 13 of 14 potential SR protocol reporting items, less than two thirds of papers met the child-centric reporting item requirements. Mixed population studies displayed significantly poorer reporting in comparison to child-only and family/maternal intervention studies for 11 potential SR reporting items (p < 0.05) and five potential SR protocol items (p < 0.05). When comparing non-Cochrane to Cochrane reports and protocols, five items in both lists were found to perform significantly poorer in non-Cochrane reports (p < 0.05). Significant differences in reporting quality were found in three of 14 items shared between the potential SR reporting items and potential SR protocol reporting items (p < 0.05).

CONCLUSIONS

Newborn and child health systematic reviews and meta-analyses exhibit incomplete reporting, thereby hindering prudent decision-making by healthcare providers and policy makers. These results provide a rationale for the implementation of child-centric extensions and modifications to current PRISMA and PRISMA-P, such as to improve reporting in this population.

Do systematic reviews on pediatric topics need special methodological considerations?

BACKGROUND

Systematic reviews are key tools to enable decision making by healthcare providers and policymakers. Despite the availability of the evidence based Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA-2009 and PRISMA-P 2015) statements that were developed to improve the transparency and quality of reporting of systematic reviews, uncertainty on how to deal with pediatric-specific methodological challenges of systematic reviews impairs decision-making in child health. In this paper, we identify methodological challenges specific to the design, conduct and reporting of pediatric systematic reviews, and propose a process to address these challenges.

DISCUSSION

One fundamental decision at the outset of a systematic review is whether to focus on a pediatric population only, or to include both adult and pediatric populations. Both from the policy and patient care point of view, the appropriateness of interventions and comparators administered to pre-defined pediatric age subgroup is critical. Decisions need to be based on the biological plausibility of differences in treatment effects across the developmental trajectory in children. Synthesis of evidence from different trials is often impaired by the use of outcomes and measurement instruments that differ between trials and are neither relevant nor validated in the pediatric population. Other issues specific to pediatric systematic reviews include lack of pediatric-sensitive search strategies and inconsistent choices of pediatric age subgroups in meta-analyses. In addition to these methodological issues generic to all pediatric systematic reviews, special considerations are required for reviews of health care interventions' safety and efficacy in neonatology, global health, comparative effectiveness interventions and individual participant data meta-analyses. To date, there is no standard approach available to overcome this problem. We propose to develop a consensus-based checklist of essential items which researchers should consider when they are planning (PRISMA-PC-Protocol for Children) or reporting (PRISMA-C-reporting for Children) a pediatric systematic review. Available guidelines including PRISMA do not cover the complexity associated with the conduct and reporting of systematic reviews in the pediatric population; they require additional and modified standards for reporting items. Such guidance will facilitate the translation of knowledge from the literature to bedside care and policy, thereby enhancing delivery of care and improving child health outcomes.

Quality of reporting of the literature on gastrointestinal reflux after repair of esophageal atresia-tracheoesophageal fistula

OBJECTIVES

There is variation in the management of postoperative gastroesophageal reflux (GER) in esophageal atresia-tracheoesophageal fistula (EA-TEF). Well-reported literature is important for clinical decision-making. We assessed the quality of reporting (QOR) of postoperative GER management in EA-TEF.

METHODS

A comprehensive search of MEDLINE, EMBASE, CINHAL, CENTRAL databases and gray literature was conducted. Included articles reported a primary diagnosis of EA-TEF, a secondary diagnosis of postoperative GER, and primary treatment of GER with antireflux medications. The QOR was assessed using the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) checklist.

RESULTS

Retrieval of 2910 articles resulted in 48 relevant articles (N=2592 patients) with an overall quality percentage score of 48%-95% (median=65%). The best reported items were "participants" and "outcome data" (93.8% each), "generalisability" (91.7%) and "background/rationale" (89.6%). Less than 20% of studies provided detailed "main results"; less than 5% of studies reported adequately on "bias" or "funding." Sample size calculation and study limitations were included in 17 (35.4%) and 16 (33.3%) studies respectively. Follow-up time was inconsistently reported.

CONCLUSIONS

Although the overall QOR is moderate using STROBE, important areas are underreported. Inadequate methodological reporting may lead to inappropriate clinical decisions. Awareness of STROBE, emphasizing proper reporting is needed.

Systematic reviews addressing microsurgical head and neck reconstruction

BACKGROUND

Systematic reviews frequently form the basis for clinical decision making and guideline development. Yet, the quality of systematic reviews has been variable, thus raising concerns about the validity of their conclusions. In the current study, a quality analysis of systematic reviews was performed, addressing microsurgical head and neck reconstruction.

MATERIALS AND METHODS

A PubMed search was performed to identify all systematic reviews published up to and including December 2012 in 12 surgical journals. Two authors independently reviewed the literature and extracted data from the included reviews. Discrepancies were resolved by consensus. Quality assessment was performed using AMSTAR.

RESULTS

The initial search retrieved 1020 articles. After screening titles and abstracts, 987 articles were excluded. Full-text review of the remaining 33 articles resulted in further exclusion of 18 articles, leaving 15 systematic reviews for final analysis. A marked increase in the number of published systematic reviews over time was noted (P = 0.07). The median AMSTAR score was 5, thus reflecting a "fair" quality. No evidence for improvement in methodological quality over time was noted.

CONCLUSIONS

The trend to publish more systematic reviews in microsurgical head and neck reconstruction is encouraging. However, efforts are indicated to improve the methodological quality of systematic reviews. Familiarity with criteria of methodological quality is critical to ensure future improvements in the quality of systematic reviews conducted in microsurgery.

Risk of cardiovascular disease from antiretroviral therapy for HIV: a systematic review

Background

Recent studies suggest certain antiretroviral therapy (ART) drugs are associated with increases in cardiovascular disease.

Purpose

We performed a systematic review and meta-analysis to summarize the available evidence, with the goal of elucidating whether specific ART drugs are associated with an increased risk of myocardial infarction (MI).

Data Sources

We searched Medline, Web of Science, the Cochrane Library, and abstract archives from the Conference on Retroviruses and Opportunistic Infections and International AIDS Society up to June 2011 to identify published articles and abstracts.

Study Selection

Eligible studies were comparative and included MI, strokes, or other cardiovascular events as outcomes.

Data Extraction

Eligibility screening, data extraction, and quality assessment were performed independently by two investigators.

Data Synthesis

Random effects methods and Fisher’s combined probability test were used to summarize evidence.

Findings

Twenty-seven studies met inclusion criteria, with 8 contributing to a formal meta-analysis. Findings based on two observational studies indicated an increase in risk of MI for patients recently exposed (usually defined as within last 6 months) to abacavir (RR 1.92, 95% CI 1.51–2.42) and protease inhibitors (PI) (RR 2.13, 95% CI 1.06–4.28). Our analysis also suggested an increased risk associated with each additional year of exposure to indinavir (RR 1.11, 95% CI 1.05–1.17) and lopinavir (RR 1.22, 95% CI 1.01–1.47). Our findings of increased cardiovascular risk from abacavir and PIs were in contrast to four published meta-analyses based on secondary analyses of randomized controlled trials, which found no increased risk from cardiovascular disease.

Conclusion

Although observational studies implicated specific drugs, the evidence is mixed. Further, meta-analyses of randomized trials did not find increased risk from abacavir and PIs. Our findings that implicate specific ARTs in the observational setting provide sufficient evidence to warrant further investigation of this relationship in studies designed for that purpose.