Quality assessment of clinical practice guidelines using the AGREE instrument in Japan: A time trend analysis

Involvement of methodological experts and the quality of clinical practice guidelines: a critical appraisal of clinical practice guidelines and a questionnaire survey of the development groups in Japan

OBJECTIVE

To evaluate whether the involvement of methodological experts improves the quality of clinical practice guidelines (CPGs) after adjusting for other factors.

SETTING

The quality of Japanese CPGs published in 2011-2019 was assessed using the Appraisal of Guidelines, Research, and Evaluation (AGREE) II instrument. A questionnaire survey targeting CPG development groups was conducted through postal mail.

PARTICIPANTS

405 CPGs were retrieved from a Japanese CPG clearinghouse. Questionnaires were distributed to the 405 CPG development groups. Of the 178 respondents, 22 were excluded because of missing values. Finally, 156 participants representing their CPG development groups were included in the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

CPG quality was assessed using the AGREE II tool. The characteristics of CPGs, including publication year, development organisation, versions, number of members in the development group and involvement of methodological experts, were corrected from the description in the CPGs and the questionnaire survey. We performed multiple logistic regressions using the quality of CPGs as the dependent variable and the involvement of experts as the independent variable, adjusting for other possible factors.

RESULTS

A total of 156 CPGs were included. Expert involvement was significantly associated with the AGREE II instrument scores in domains 1 (β=0.207), 2 (β=0.370), 3 (β=0.413), 4 (β=0.289), 5 (β=0.375), 6 (β=0.240) and overall (β=0.344).

CONCLUSION

This study revealed that the involvement of methodological experts in the CPG development process improves the quality of CPGs. The results suggest the importance of establishing a training and certification programme for experts and constructing expert referral systems that meet CPG developers' needs to improve the quality of CPGs.

Quality assessment of Japanese clinical practice guidelines including recommendations for acupuncture

Background

The quality of clinical practice guidelines (CPGs) should be extensively evaluated. This study aimed to evaluate Japanese CPGs that include recommendations for acupuncture.

Methods

In a literature search, CPGs including recommendations for acupuncture published in Japan until October 2021 were sought. We assessed (1) whether the CPGs were developed in accordance with the Grading Recommendations Assessment, Development and Evaluation (GRADE) system, (2) the quality of the CPGs using the Appraisal of Guidelines for Research and Evaluation (AGREE) II, and (3) whether the strength of the recommendations for acupuncture was consistent with each CPG's predefined procedure.

Results

Seventeen CPGs including 23 recommendations in total were identified and assessed. (1) Three CPGs were in accordance with the GRADE system. (2) The mean score of overall assessment using AGREE II was 4.5 on a 7-point Likert scale. The mean domain scores were 77% for domain 1 (scope and purpose), 54% for domain 2 (stakeholder involvement), 48% for domain 3 (rigor of development), 78% for domain 4 (clarity of presentation), 20% for domain 5 (applicability), and 51% for domain 6 (editorial independence). (3) The strength of the recommendations for acupuncture in two CPGs was judged to be underestimated. Some of the CPGs contained elementary problems that were not considered in AGREE II.

Conclusion

The methodological quality of Japanese CPGs including recommendations for acupuncture was not necessarily high. Since technical issues exist in each field of therapy, the respective experts should be involved in developing and reviewing CPGs to disseminate accurate health information.

Factors Associated with the Quality and Transparency of National Guidelines: A Mixed-Methods Study

We assessed the methodological quality and transparency of all the national clinical practice guidelines that were published in Croatia up until 2017 and explored the factors associated with their quality rating. An in-depth quantitative and qualitative analysis was performed using rigorous methodology. We evaluated the guidelines using a validated AGREE II instrument with four raters; we used multiple linear regressions to identify the predictors of quality; and two focus groups, including guideline developers, to further explore the guideline development process. The majority of the guidelines (N = 74) were developed by medical societies. The guidelines’ quality was rated low: the median standardized AGREE II score was low, 36% (IQR 28–42), and so were the overall-assessments. The aspects of the guidelines that were rated best were the “clarity of presentation” and the “scope and purpose” (median ≥ 59%); however, the other four domains received very low scores (15–33%). Overall, the guideline quality did not improve over time. The guidelines that were developed by medical societies scored significantly worse than those developed by governmental, or unofficial working groups (12–43% per domain). In focus group discussions, inadequate methodology, a lack of implementation systems in place, a lack of awareness about editorial independence, and broader expertise/perspectives in working groups were identified as factors behind the low scores. The factors identified as affecting the quality of the national guidelines may help stakeholders who are developing interventions and education programs aimed at improving guideline quality worldwide.

Clinical practice guidelines adaptation for internists - An EFIM methodology

The rising number of clinical guidelines poses a new challenge to the internists. The main problems are: 1) available documents suffer from heterogeneous methodological quality, and 2) most of clinical guidelines target an 'ideal' patient affected by a single condition, while in real practice internists must face with comorbid patients typically undergoing a polypharmacy. To help address this challenge, EFIM Clinical Practice Working Group started a project aimed to answer a series of relevant clinical questions, by selecting the best available guidance containing recommendations applicable to complex patients under polypharmacy. The project started with the creation of a research protocol containing details about all the steps needed to write the Clinical Practice Guideline Summary. In particular, this methodological document specifies the rules: 1) to select topics and clinical questions; 2) to build up a panel of experts, carefully managing eventual conflict of interests; 3) to critically appraise clinical guidelines (using a validated tool as AGREE II), selecting the most valid and applicable to the common clinical practice (using ADAPTE; 4) to address and solve potential disagreements among the selected documents.