The quality assessment of clinical practice guidelines for intracranial aneurysms: a systematic appraisal

A quality assessment of evidence-based guidelines for the prevention and management of ventilator-associated pneumonia: a systematic review

Background

Numerous evidence-based guidelines (EBGs) pertaining to ventilator-associated pneumonia (VAP) have been published by domestic and international organizations, but their qualities have not been reported.

Methods

A systematic search of the literature was performed up to July 2018 for relevant guidelines. Guidelines were eligible for inclusion if they incorporated recommendation statements for prevention and/or management in adults or children with VAP and were developed on a systematic evidence-based method. Four reviewers evaluated each guideline using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument, which comprises 23 items organized into six domains in addition to two overall items.

Results

Thirteen EBGs were identified for review. An overall high degree of agreement among reviewers was reached [intra-class correlation coefficient (ICC), 0.885; 95% CI, 0.862-0.905] during their review. The scores (mean, range) for the six AGREE domains were: scope and purpose (61%, 51-74%), stakeholder involvement (36%, 18-68%), rigor of development (41%, 22-59%), clarity and presentation (56%, 47-71%), applicability (38%, 21-59%) and editorial independence (50%, 0-77%). Only two EBGs (15.4%) were rated "recommended" for clinical practice. Approximately 86% of recommendations were based on moderate or low levels of evidence (levels B-D were 46.2%, 19.0%, and 21.2%, respectively). The recommendations for prevention and management of VAP were similar among the different EBGs.

Conclusions

The overall quality of the identified EBGs pertaining to VAP was classified as moderate. The management of VAP varied by guideline. More high-quality evidence is needed to improve guideline recommendations.

An appraisal of emergency medicine clinical practice guidelines: Do we agree?

BACKGROUND

Clinical practice guidelines (CPGs) have been published by the American College of Emergency Physicians (ACEP) since 1990 to advance evidence-based emergency care. ACEP clinical policies have drawn anecdotal criticism for bias, yet the overall quality of these guidelines has not previously been quantified. We sought to examine ACEP clinical policies using a recognised, validated appraisal instrument: Appraisal of Guidelines for Research & Evaluation (AGREE II).

METHODS

Systematic assessment of current ACEP clinical policies was conducted using the AGREE II instrument, which contains 23 appraisal items (scored on a 1-7 scale) in six domains and two overall assessments. Each policy was independently appraised by five trained appraisers. Primary outcomes were AGREE II ratings for each item, domain and "Overall Assessment," and scores were reported as standardised percentages from all five appraisers. Secondary analyses examined associations between AGREE II ratings and policy publication date, strength of underlying evidence and strength of recommendations. Additional analysis examined relationships between domain and "Overall Assessment" ratings.

RESULTS

Twenty guidelines published from April 2007 to November 2017 were included. Of the six domains, "Scope and Purpose" scored highest (mean 90%) and "Applicability" scored lowest (mean 35%). The four remaining domains ("Stakeholder Involvement," "Rigor of Development," "Clarity of Presentation" and "Editorial Independence") had mean scores of 53%-78%. The mean "Overall Assessment" rating was 69% and was not associated with policy publication date, strength of underlying evidence or strength of recommendations. We found positive associations between "Overall Assessment" ratings and two domains: "Rigor of Development" (r = 0.70) and "Clarity of Presentation" (r = 0.70).

CONCLUSIONS

Based on validated AGREE II criteria, ACEP clinical policies can be most improved by addressing their application in practice. ACEP clinical policies' overall quality did not improve over the assessed time period and is not explained by the quality of underlying evidence.