The structure of the quality of clinical practice guidelines with the items and overall assessment in AGREE II: a regression analysis

Evidence-based clinical practice guidelines for the management of acute ankle injuries according to: a PRISMA systematic review and quality appraisal with AGREE II

BACKGROUND

Acute ankle injuries are commonly seen in emergency rooms, with significant social impact and potentially devastating consequences. While several clinical practice guidelines (CPGs) related to ankle injuries have been developed by various organizations, there is a lack of critical appraisal of them. The purpose of this systematic review is to identify and critically appraise evidence-based clinical practice guidelines (EB-CPGs) related to acute ankle injuries in adults.

METHOD

We conducted searches in the Cochrane Library, MEDLINE, EMBASE databases, WHO, and reviewed 98 worldwide orthopedic association websites up until early 2023. Two authors independently applied the inclusion and exclusion criteria, and each evidence-based clinical practice guideline (EB-CPG) underwent independent critical appraisal of its content by all four authors using the Appraisal of Guidelines for REsearch and Evaluation (AGREE II) instrument. AGREE II scores for each domain were then calculated.

RESULTS

This review included five evidence-based clinical practice guidelines. The mean scores for all six domains were as follows: Scope and Purpose (87.8%), Stakeholder Involvement (69.2%), Rigour of Development (72.5%), Clarity of Presentation (86.9%), Applicability (45.6%), and Editorial Independence (53.3%).

CONCLUSION

The number of EB-CPGs related to ankle injuries are limited and the overall quality of the existing evidence-based clinical practice guidelines (EB-CPGs) for ankle injuries is not strong, with three of them being outdated. However, valuable guidance related to Ottawa rules, manual therapy, cryotherapy, functional supports, early ambulation, and rehabilitation has been highlighted. Challenges remain in areas such as monitoring and/or auditing criteria, consideration of the target population's views and preferences, and ensuring editorial independence. Future guidelines should prioritize improvements in these domains to enhance the quality and relevance of ankle injury management.

SYSTEMATIC REVIEW

Systematic review.

Clinical practice guideline recommendations for the management of challenging behaviours after traumatic brain injury in acute hospital and inpatient rehabilitation settings: a systematic review

PURPOSE

Clinical practice guideline (CPG) recommendations for the management of challenging behaviours after traumatic brain injury (TBI) in hospital and inpatient rehabilitation settings are sparse. This systematic review aims to identify and appraise CPGs, and report high-quality recommendations for challenging behaviours after TBI in hospital and rehabilitation settings.

MATERIALS AND METHODS

A three-step search strategy was conducted to identify CPGs that met inclusion criteria. Two reviewers independently scored the AGREE II domains. Guideline quality was assessed based on CPGs adequately addressing four out of the six AGREE II domains. Data extraction was performed with a compilation of high-quality CPG recommendations.

RESULTS

Seven CPGs out of 408 identified records met the inclusion criteria. Two CPGs were deemed high-quality. High-quality CPG recommendations with the strongest supporting evidence include behaviour management plans; beta-blockers for the treatment of aggression; selective serotonin reuptake inhibitors for moderate agitation; adamantanes for impaired arousal/attention in agitation; specialised, multi-disciplinary TBI behaviour management services.

CONCLUSIONS

This systematic review identified and appraised the quality of CPGs relating to the management of challenging behaviours after TBI in acute hospital and rehabilitation settings. Further research to rigorously evaluate TBI behaviour management programs, investigation of evidence-practice gaps, and implementation strategies for adopting CPG recommendations into practice is needed.Implications for rehabilitationTwo clinical practice guidelines appraised as high-quality outline recommendations for the management of challenging behaviours after traumatic brain injury in hospital and inpatient rehabilitation settings.High-quality guideline recommendations with the strongest supporting evidence for non-pharmacological treatment include behaviour management plans considering precipitating factors, antecedents, and reinforcing events.High-quality guideline recommendations with the strongest supporting evidence for pharmacological management include beta blockers for aggression in traumatic brain injury.Few guidelines provide comprehensive detail on the implementation of recommendations into clinical care which may limit adoption.

Advancing guideline quality through country-wide and regional quality assessment of CPGs using AGREE: a scoping review

BACKGROUND AND OBJECTIVE

Clinical practice guidelines (CPGs) are evaluated for quality with the Appraisal of Guidelines for Research and Evaluation (AGREE) tool, and this is increasingly done for different countries and regional groupings. This scoping review aimed to describe, map, and compare these geographical synthesis studies, that assessed CPG quality using the AGREE tool. This allowed a global interpretation of the current landscape of these country-wide or regional synthesis studies, and a closer look at its methodology and results.

STUDY DESIGN AND METHODS

A scoping review was conducted searching databases Medline, Embase, Epistemonikos, and grey literature on 5 October 2021 for synthesis studies using the later versions of AGREE (AGREE II, AGREE-REX and AGREE GRS) to evaluate country-wide or regional CPG quality. Country-wide or regional synthesis studies were the units of analysis, and simple descriptive statistics was used to conduct the analysis. AGREE scores were analysed across subgroups into one of the seven Sustainable Development Goal regions, to allow for meaningful interpretation.

RESULTS

Fifty-seven studies fulfilled our eligibility criteria, which had included a total of 2918 CPGs. Regions of the Global North, and Eastern and South-Eastern Asia were most represented. Studies were consistent in reporting and presenting their AGREE domain and overall results, but only 18% (n = 10) reported development methods, and 19% (n = 11) reported use of Grading of Recommendations Assessment, Development, and Evaluation (GRADE). Overall scores for domains Rigor of development and Editorial independence were low, notably in middle-income countries. Editorial Independence scores, especially, were low across all regions with a maximum domain score of 46%. There were no studies from low-income countries.

CONCLUSION

There is an increasing tendency to appraise country-wide and regionally grouped CPGs, using quality appraisal tools. The AGREE tool, evaluated in this scoping review, was used well and consistently across studies. Findings of low report rates of development of CPGs and of use of GRADE is concerning, as is low domain scores globally for Editorial Independence. Transparent reporting of funding and competing interests, as well as highlighting evidence-to-decision processes, should assist in further improving CPG quality as clinicians are in dire need of high-quality guidelines.

Involvement of methodological experts and the quality of clinical practice guidelines: a critical appraisal of clinical practice guidelines and a questionnaire survey of the development groups in Japan

OBJECTIVE

To evaluate whether the involvement of methodological experts improves the quality of clinical practice guidelines (CPGs) after adjusting for other factors.

SETTING

The quality of Japanese CPGs published in 2011-2019 was assessed using the Appraisal of Guidelines, Research, and Evaluation (AGREE) II instrument. A questionnaire survey targeting CPG development groups was conducted through postal mail.

PARTICIPANTS

405 CPGs were retrieved from a Japanese CPG clearinghouse. Questionnaires were distributed to the 405 CPG development groups. Of the 178 respondents, 22 were excluded because of missing values. Finally, 156 participants representing their CPG development groups were included in the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

CPG quality was assessed using the AGREE II tool. The characteristics of CPGs, including publication year, development organisation, versions, number of members in the development group and involvement of methodological experts, were corrected from the description in the CPGs and the questionnaire survey. We performed multiple logistic regressions using the quality of CPGs as the dependent variable and the involvement of experts as the independent variable, adjusting for other possible factors.

RESULTS

A total of 156 CPGs were included. Expert involvement was significantly associated with the AGREE II instrument scores in domains 1 (β=0.207), 2 (β=0.370), 3 (β=0.413), 4 (β=0.289), 5 (β=0.375), 6 (β=0.240) and overall (β=0.344).

CONCLUSION

This study revealed that the involvement of methodological experts in the CPG development process improves the quality of CPGs. The results suggest the importance of establishing a training and certification programme for experts and constructing expert referral systems that meet CPG developers' needs to improve the quality of CPGs.

Level of Evidence of Guidelines for Perioperative Management of Patients With Obstructive Sleep Apnea: An Evaluation Using the Appraisal of Guidelines for Research and Evaluation II Tool

Currently, the quality of guidelines for the perioperative management of patients with obstructive sleep apnea (OSA) is unknown, leaving anesthesiologists to make perioperative management decisions with some degree of uncertainty. This study evaluated the quality of clinical practice guidelines regarding the perioperative management of patients with OSA. This study was reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A systematic search of the MedlineALL (Ovid) database was conducted from inception to February 26, 2021, for clinical practice guidelines in the English language. Quality appraisal of guidelines was evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) framework. Descriptive statistical analysis of each of the 6 domains was expressed as a percentage using the formula: (obtained score - minimum possible score)/(maximum possible score - minimum possible score). Of 192 articles identified in the search, 41 full texts were assessed for eligibility, and 10 articles were included in this review. Intraclass correlation coefficients of the AGREE II scores across the 7 evaluators for each guideline were each >0.9, suggesting that the consistency of the scores among evaluators was high. Sixty percent of recommendations were based on evidence using validated methods to grade medical literature, while the remainder were consensus based. The median and range scores of each domain were: (1) scope and purpose, 88% (60%-95%); (2) stakeholder involvement, 52% (30%-82%); (3) rigor of development, 67% (40%-90%); (4) clarity of presentation, 74% (57%-88%); (5) applicability, 46% (20%-73%); and (6) editorial independence, 67% (19%-83%). Only 4 guidelines achieved an overall score of >70%. This critical appraisal showed that many clinical practice guidelines for perioperative management of patients with OSA used validated methods to grade medical literature, such as Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) and Oxford classification, with lower scores for stakeholder involvement due to lack of engagement of patient partners and applicability domain due to lack of focus on the complete perioperative period such as postdischarge counseling. Future efforts should be directed toward establishing higher focus on the quality of evidence, stakeholder involvement, and applicability to the wider perioperative patient experience.

Quality appraisal of clinical nutrition practice guidelines for critically ill adult patients: a systematic review using the Advancing Guideline Development, Reporting and Evaluation in Health Care instrument II (AGREE-II) and AGREE-Recommendation Excellence (AGREE-REX)

Nutritional therapy should follow evidence-based practice, thus several societies regarding nutrition and critical care have developed specific Clinical Practice Guidelines (CPG). However, to be regarded as trustworthy, the quality of the CPG for critically ill patients and its recommendations need to be high. This systematic review aimed to appraise the methodology and recommendations of nutrition CPG for critically ill patients. We performed a systematic review (protocol number CRD42020184199) with literature search conducted on PubMed, Embase, Cochrane Library and other four specific databases of guidelines up to October 2021. Two reviewers, independently, assessed titles and abstracts and potentially eligible full-text reports to determine eligibility and subsequently four reviewers appraised the guidelines quality using the Advancing Guideline Development, Reporting and Evaluation in Health Care instrument II (AGREE-II) and AGREE-Recommendation Excellence (AGREE-REX). Ten CPG for nutrition in critically ill patients were identified. Only Academy of Nutrition and Dietetics and European Society of Intensive Care Medicine had a total acceptable quality and were recommended for daily practice according AGREE-II. None of the CPG recommendations had an overall quality score above 70 %, thus being classified as moderate quality according AGREE-REX. The methodological evaluation of the critically ill adult patient CPG revealed significant discrepancies and showed a need for improvement in its development and/or reporting. In addition, recommendations about nutrition care process presented a moderate quality.

Review of clinical practice guidelines relating to cognitive assessment in stroke

PURPOSE

To assess the content, quality, and supporting evidence base of clinical practice guidelines (CPGs) with reference to cognitive assessment in stroke.

MATERIALS AND METHODS

We performed a systematic review to identify eligible CPGs pertaining to cognitive assessment in adult stroke survivors. We compared content and strength of recommendations. We used the AGREE-II (appraisal of guidelines for research and evaluation) tool to appraise the quality of the guideline.

RESULTS

Eight eligible guidelines were identified and seven were rated as high quality (i.e., appropriately addressing at least four domains of the AGREE-II tool including "rigor of development"). There was heterogeneity in the recommendations offered and limited guidance on fundamental topics such as which cognitive test to use or when to perform testing. Generally, the lowest quality of evidence (expert opinion) was used to inform these recommendations.

CONCLUSIONS

Although assessment of cognition is a key aspect of stroke care, there is a lack of guidance for clinicians. The limited evidence base, in part, reflects the limited research in the area. A prescriptive approach to cognitive assessment may not be suitable, but more primary research may help inform practice.Implications for rehabilitationCognitive assessment in stroke exhibits substantial variation in practice, clinical practice guidelines rarely give prescriptive recommendations on which approach to take.Where guideline recommendations on cognitive assessment in stroke were made these were based on expert opinion.Our summary of the guideline content found certain areas of consensus, for example, routine assessment using validated tools.

What Determines the Quality of Rehabilitation Clinical Practice Guidelines?: An Overview Study

OBJECTIVE

The aim of the study was to determine what factors determine the quality of rehabilitation clinical practice guidelines.

DESIGN

Six databases were searched for articles that had applied the Appraisal of Guidelines for Research & Evaluation II quality assessment tool to rehabilitation clinical practice guidelines. The 573 deduplicated abstracts were independently screened by two authors, resulting in 81 articles, the full texts of which were independently screened by two authors for Appraisal of Guidelines for Research & Evaluation II application to rehabilitation clinical practice guidelines, resulting in a final selection of 40 reviews appraising 504 clinical practice guidelines. Data were extracted from these by one author and checked by a second. Data on each clinical practice guideline included the six Appraisal of Guidelines for Research & Evaluation II domain scores, as well as the two Appraisal of Guidelines for Research & Evaluation II global evaluations.

RESULTS

All six Appraisal of Guidelines for Research & Evaluation II domain scores were statistically significant predictors of overall clinical practice guideline quality rating; D3 (rigor of development) was the strongest and D1 (scope and purpose) the weakest (overall model P < 0.001, R2 = 0.53). Five of the six domain scores were significant predictors of the clinical practice guideline use recommendation, with D3 the strongest predictor and D5 (applicability) the weakest (overall model P < 0.001, pseudo R2 = 0.53).

CONCLUSIONS

Quality of rehabilitation clinical practice guidelines may be improved by addressing key domains such as rigor of development.