Need for quality improvement in renal systematic reviews

Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018

INTRODUCTION

The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals.

METHODS

This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics.

RESULTS

The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting.

CONCLUSION

Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.

Effect of premedication on the success of inferior alveolar nerve block in patients diagnosed with irreversible pulpitis: An umbrella review

Successful management of pain during endodontic treatment is essential for both patients and dentists. Achieving adequate pulp anesthesia in mandibular molars is a significant concern for patients with irreversible pulpitis during endodontic treatment. The increased sensitization of nociceptors due to inflammation decreases the success of inferior alveolar nerve block (IANB). The main focus is on reducing inflammation before delivery of local anesthesia to increase the success of anesthetic drugs. This umbrella review aimed to revise, qualify and summarize the existing body of evidence on the effect of premedication on IANB in patients with irreversible pulpitis. A literature search was conducted using electronic databases (PubMed, Scopus, the Web of Science, and the Cochrane Library) with no date restriction until September 2021 to identify the relevant studies. All the cross-references of the selected studies and grey literature were also screened. Four systematic reviews assessing the effect of premedication on the success of IANB were selected. A conclusion was drawn that premedication with >400 mg of ibuprofen can positively affect the success of IANB.

Reporting and methodological quality of systematic literature reviews evaluating the associations between e-cigarette use and cigarette smoking behaviors: a systematic quality review

Introduction

Several published systematic reviews have examined the potential associations between e-cigarette use and cigarette smoking, but their methodological and/or reporting quality have not yet been assessed. This systematic quality review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 to evaluate the quality of systematic reviews investigating potential associations between e-cigarette use and cigarette smoking.

Materials and methods

PubMed/MEDLINE, Embase, and PsycINFO were searched from 01 January 2007 to 24 June 2020. Methodological quality was assessed using AMSTAR 2, and reporting quality was assessed using PRISMA guidelines.

Results

Of 331 potentially relevant systematic reviews, 20 met predefined inclusion criteria. Most reviews (n = 15; 75%) reported on e-cigarette use and cigarette smoking cessation, while three reported on e-cigarette use and cigarette smoking initiation (15%); and two reported on cigarette smoking initiation and cessation (10%). According to AMSTAR 2 guidelines, 18 of the 20 reviews (90%) were “critically low” in overall confidence of the results, while two were ranked “low.” Additionally, reporting quality varied across the reviews, with only 60% reporting at least half of the PRISMA items.

Discussion

Methodological limitations were identified across reviews examining potential associations between e-cigarette use and cigarette smoking behaviors, indicating that findings from these reviews should be interpreted with caution.

Conclusions

Future systematic reviews in this field should strive to adhere to AMSTAR 2 and PRISMA guidelines, to provide high quality syntheses of the available data with transparent and complete reporting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12954-021-00570-9.

Systematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017

To gain insight into changes of scholarly journals' recommendations, we conducted a systematic review of studies that analysed journals' Instructions to Authors (ItAs). We summarised results of 153 studies, and meta-analysed how often ItAs addressed: 1) authorship, 2) conflicts of interest, 3) data sharing, 4) ethics approval, 5) funding disclosure, and 6) International Committee of Medical Journal Editors' Uniform Requirements for Manuscripts. For each topic we found large between-study heterogeneity. Here, we show six factors that explained most of that heterogeneity: 1) time (addressing of topics generally increased over time), 2) country (large differences found between countries), 3) database indexation (large differences found between databases), 4) impact factor (topics were more often addressed in highest than in lowest impact factor journals), 5) discipline (topics were more often addressed in Health Sciences than in other disciplines), and 6) sub-discipline (topics were more often addressed in general than in sub-disciplinary journals).

Associations and risk factors for dental trauma: A systematic review of systematic reviews

OBJECTIVE

We sought to revise, qualify and summarize the body of evidence based on current information on the known associations and risk factors for traumatic dental injury (TDI) occurrence with an overview of systematic reviews.

METHODS

Electronic searches were performed with no language nor date restrictions.According to the PECO strategy, systematic reviews that investigated participants (P) with (E) and without (O) the presentation of factors relating to TDI and these factors' association with TDI episodes (O) as primary or secondary outcomes were included. Quality assessment and bias control were carried out according to the AMSTAR 2 checklist. In cases of systematic review results of discordance, the Jadad decision algorithm was applied.

RESULTS

After titles, abstracts and full texts were read, 19 systematic reviews were included in this overview. Four were classified with critically low, eleven with low and four with moderate methodological quality, respectively. In all studies, 249 cross-sectional, 34 epidemiological surveys, 22 cohort, 9 case-control, 4 longitudinal, 3 ecological and 30 unspecified studies were included. Male gender, child age, greater overjet, inadequate lip coverage, anterior open bite, caries in the permanent dentition, overweight, a previous history of TDI, tongue piercing, the use of alcoholic beverages and participation in sports were all associated with a greater chance of suffering TDI.

CONCLUSIONS

Sociodemographic, clinical and environmental factors are associated with a greater chance of TDI occurrence. However, most of the systematic reviews included were of a low quality and may not provide an accurate and comprehensive summary of the available research that addresses the question of interest. Well-designed primary studies on different aspects of TDI are encouraged to provide higher quality scientific evidence.

Con: Meta-analysis: some key limitations and potential solutions

Meta-analysis, a statistical combination of results of several trials to produce a summary effect, has been subject to criticism in the past, mainly for the reasons of poor quality of included studies, heterogeneity between studies meta-analyzed and failing to address publication bias. These limitations can cause the results to be misleading, which is important if policy and practice decisions are based on systematic reviews and meta-analyses. We elaborate on these limitations and illustrate them with examples from the nephrology literature. Finally, we present some potential solutions, notably, education in meta-analysis for evidence producers and consumers as well as the use of individual patient data for meta-analyses.

Moderator's view: Meta-analysis: the best knowledge but not always shining gold

Meta-analysis has gained top status in medicine. Correctly so, well-performed meta-analysis is perceived as an unbeatable method for distilling first class medical knowledge. However, such a high status should in no way be considered as a guarantee that all information derived from meta-analyses is pure gold. Systematic reviews and meta-analyses not performed with due methodological attention abound and nephrology is not at all a protected territory. Herein I give concrete examples of meta-analyses published in major journals whose findings are not pure gold.

Systematic review adherence to methodological or reporting quality

BACKGROUND

Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively.

RESULTS

Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the SRs assessed risk of bias (validity) in the included studies, while 80% (1112/1387) reported the criteria for study selection.

CONCLUSIONS

Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.

Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study

BACKGROUND

The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.

METHODS

The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.

RESULTS

Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.

CONCLUSIONS

The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

Epidemiology, methodological and reporting characteristics of systematic reviews of nursing interventions published in China

The importance of systematic reviews (SRs) of nursing interventions' impact on practice makes their methodological quality and reporting characteristics especially important as it directly influence their utility for clinicians, patients and policy makers.The study aims to assess the methodological quality and reporting characteristics of SRs of nursing interventions in Chinese nursing journals. Three Chinese databases were searched for SRs of nursing interventions from inception to October 2011. The assessment of multiple systematic reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) statements were used to assess methodological quality and reporting characteristics. Seventy-four SRs were included. The proportion of SRs complying with AMSTAR checklist items ranged from 0% to 82.4%. No SRs reported an 'a priori' design or conflict of interest. Only four items were found to be reported in more than 50% of the SRs: a list of included and excluded studies, the scientific quality of included studies, the appropriate use of methods to combine findings, and formulating conclusions appropriately. The majority of SRs of nursing interventions in China had major methodological and reporting flaws that limited their value to guide decisions. Chinese authors and journals should adopt and keep up with the AMSTAR and PRISMA statements to improve the quality of SRs in this field.

A PRISMA assessment of the reporting quality of systematic reviews in orthodontics

OBJECTIVES

To assess the reporting quality of Cochrane and non-Cochrane systematic reviews (SR) in orthodontics and to compare the reporting quality (PRISMA score) with methodological quality (AMSTAR criteria).

MATERIALS AND METHODS

Systematic reviews (n  =  109) published between January 2000 and July 2011 in five leading orthodontic journals were identified and included. The quality of reporting of the included reviews was assessed by two authors in accordance with the PRISMA guidelines. Each article was assigned a cumulative grade based on fulfillment of the applicable criteria, and an overall percentage score was assigned. Descriptive statistics and simple and multiple linear regression analyses were undertaken.

RESULTS

The mean overall PRISMA score was 64.1% (95% confidence interval [CI], 62%-65%). The quality of reporting was considerably better in reviews published in the Cochrane Database of Systematic Reviews (P < .001) than in non-Cochrane reviews. Both multivariable and univariable analysis indicated that journal of publication and number of authors was significantly associated with the PRISMA score. The association between AMSTAR score and modified PRISMA score was also found to be highly statistically significant.

CONCLUSION

Compliance of orthodontic SRs published in orthodontic journals with PRISMA guidelines was deficient in several areas. The quality of reporting assessed using PRISMA guidelines was significantly better in orthodontic SRs published in the Cochrane Database of Systematic Reviews.

Finding and evaluating renal evidence: bridging the knowledge gap

Nephrologists often search for clinical information to guide patient care, especially in the chronic kidney disease patient population where information needs are complex. Applying evidence-based principles while searching for current high-quality information can be challenging. This article details the "6S" framework to approaching evidence-based information resources popularized by Haynes and colleagues. This hierarchal structure of resources includes "systems," "summaries," "synopses of syntheses," "syntheses," "synopses of studies," and finally, "studies" with the intent of providing nephrologists with an efficient method of retrieving accurate, high-quality information. Accurate clinical decision support systems are ideal but are yet to reach their full potential. Summaries of best evidence for individual clinical problems are useful (UpToDate, DynaMed, Best Practice, and PIER). Syntheses of articles are an efficient way to understand the available evidence for a clinical question. The Cochrane Renal Review Group is a helpful resource which provides many systematic reviews in renal research. When using bibliographic databases, such PubMed, nephrologists can improve the precision of their search results with information filters. The application of accurate and current clinical knowledge is increasingly important in renal patient care. Better ways of achieving this will have large benefits.

Do journals publishing in the field of urology endorse reporting guidelines? A survey of author instructions

INTRODUCTION

Reporting guidelines aim to ensure adequate and complete reporting of clinical studies and are an indispensable tool to translate scientific results into clinical practice. The extent to which reporting guidelines are incorporated into the author instructions of journals publishing in the field of urology remained unclear.

MATERIALS AND METHODS

We assessed the author instructions of uro-nephrological journals indexed in 'Journal Citation Reports 2009'. Two authors independently assessed the author guidelines. We evaluated additional information including whether a journal was published by or in association with a medical association. Discrepancies were resolved by re-checking the respective author instructions and by discussion with a third author.

RESULTS

The recommendations of the International Committee of Journal Editors were endorsed by 32 journals (58.2%) but were mentioned in 12 (37.5%) only to give general advice about manuscript preparation. Fourteen journals (25.5%) mentioned at least one reporting guideline, with CONSORT the most frequently cited. Journals with high impact factors were more likely to endorse CONSORT (p < 0.009). Other reporting guidelines were mentioned by <6% of the journals.

CONCLUSION

All key stakeholders involved in the publication process should more frequently promote the awareness and use of reporting guidelines.

Quality of systematic reviews of observational nontherapeutic studies

INTRODUCTION

High-quality epidemiologic research is essential in reducing chronic diseases. We analyzed the quality of systematic reviews of observational nontherapeutic studies.

METHODS

We searched several databases for systematic reviews of observational nontherapeutic studies that examined the prevalence of or risk factors for chronic diseases and were published in core clinical journals from 1966 through June 2008. We analyzed the quality of such reviews by using prespecified criteria and internal quality evaluation of the included studies.

RESULTS

Of the 145 systematic reviews we found, fewer than half met each quality criterion; 49% reported study flow, 27% assessed gray literature, 2% abstracted sponsorship of individual studies, and none abstracted the disclosure of conflict of interest by the authors of individual studies. Planned, formal internal quality evaluation of included studies was reported in 37% of systematic reviews. The journal of publication, topic of review, sponsorship, and conflict of interest were not associated with better quality. Odds of formal internal quality evaluation (odds ratio [OR], 1.10 per year; 95% confidence interval [CI], 1.02-1.19) and either planned, formal internal quality evaluation or abstraction of quality criteria of included studies (OR, 1.17 per year; 95% CI, 1.08-1.26) increased over time, without positive trends in other quality criteria from 1990 through June 2008. Systematic reviews with internal quality evaluation did not meet other quality criteria more often than those that ignored the quality of included studies.

CONCLUSION

Collaborative efforts from investigators and journal editors are needed to improve the quality of systematic reviews.

Do meta-analyses in nephrology change the way we treat patients?

There has been an exponential increase in the number of nephrological meta-analyses published, but their relative contribution to the nephrology literature is unclear and their influence on physician behavior and evidence-based patient care is poorly understood. We studied the nephrology literature, point-of-care resources, guidelines, and a questionnaire survey of the New York Society of Nephrology membership to understand the role and perception of meta-analyses in nephrology. We discuss our results in the context of the strengths and limitations of meta-analyses and their relatively limited, albeit increasing influence on published guidelines and on point-of-care references. The results of our practitioner survey and our review of the nephrology literature suggest an increasing influence at the level of the individual practitioner of meta-analyses. This underlines the need to develop a better understanding of the contributions and role of meta-analyses in the literature.

AHRQ series paper 2: principles for developing guidance: AHRQ and the effective health-care program

OBJECTIVE

This article describes some of the fundamental principles that have been developed to guide the work of producing comparative effectiveness reviews (CERs).

STUDY DESIGN AND SETTING

We briefly describe the role stakeholders play in providing important insights that inform the evidence-gathering process, and discuss the critical role of analytic frameworks in illuminating the relationship between surrogate measures and health outcomes, providing an understanding of the context in which clinical decisions are made and the uncertainties that underlie clinical controversies.

RESULTS

We describe the Effective Health Care program conceptual model for considering different types of evidence that emphasizes minimizing the risk of bias, but places high-quality, highly applicable evidence about effectiveness at the top of the hierarchy. Finally, we briefly describe areas of future methodological research.

CONCLUSION

CERs have become a foundation for decision-making in clinical practice and health policy. To be useful, CERs must approach the evidence from a patient-centered perspective; explore the clinical logic underlying the rationale for a service; cast a broad net with respect to types of evidence, placing a high value on effectiveness and applicability, in addition to internal validity; and, present benefits and harms for treatments and tests in a consistent way.