Evaluation of the quality of guidelines for myasthenia gravis with the AGREE II instrument

A mixed methods analysis of existing assessment and evaluation tools (AETs) for mental health applications

Introduction

Mental health Applications (MH Apps) can potentially improve access to high-quality mental health care. However, the recent rapid expansion of MH Apps has created growing concern regarding their safety and effectiveness, leading to the development of AETs (Assessment and Evaluation Tools) to help guide users. This article provides a critical, mixed methods analysis of existing AETs for MH Apps by reviewing the criteria used to evaluate MH Apps and assessing their effectiveness as evaluation tools.

Methods

To identify relevant AETs, gray and scholarly literature were located through stakeholder consultation, Internet searching via Google and a literature search of bibliographic databases Medline, APA PsycInfo, and LISTA. Materials in English that provided a tool or method to evaluate MH Apps and were published from January 1, 2000, to January 26, 2021 were considered for inclusion.

Results

Thirteen relevant AETs targeted for MH Apps met the inclusion criteria. The qualitative analysis of AETs and their evaluation criteria revealed that despite purporting to focus on MH Apps, the included AETs did not contain criteria that made them more specific to MH Apps than general health applications. There appeared to be very little agreed-upon terminology in this field, and the focus of selection criteria in AETs is often IT-related, with a lesser focus on clinical issues, equity, and scientific evidence. The quality of AETs was quantitatively assessed using the AGREE II, a standardized tool for evaluating assessment guidelines. Three out of 13 AETs were deemed 'recommended' using the AGREE II.

Discussion

There is a need for further improvements to existing AETs. To realize the full potential of MH Apps and reduce stakeholders' concerns, AETs must be developed within the current laws and governmental health policies, be specific to mental health, be feasible to implement and be supported by rigorous research methodology, medical education, and public awareness.

Clinical guidelines for herpes labialis: recommendations and quality evaluation according to AGREE II

Cold sores require Healthcare professionals to employ specific approaches for prevention and management, with the need for effective therapeutic guidelines and ongoing improvement in patient care.

AIM

To evalúate the methodological quality of Clinical Guidelines (CG), clinical guides and manuals for care of the population affected by herpes labialis, to verify their compliance with evidence-based health standards.

MATERIALS AND METHOD

A search was conducted for CG on labial herpes in the MedicalLiteratureAnalysis andRetrieval System Online (Medline) database, Google Scholar, Brazilian Virtual Health Library (BVS), and sites of institutions/professional categories, using the descriptors "herpes labialis" or "oral herpes". Document quality was assessed using the Appraisal of Guidelines for Research & Evaluation Instrument (AGREE II). The Kappa test was used to avoid randomness or poor agreement between results.

RESULTS

Analysis of the 12 selected publications on the management of labial herpes revealed flaws in quality, as the publications did not follow a quality standard. The main quality flaws identified were in "rigor in development" and "applicability. ".

CONCLUSIONS

Priorities need to be redefined in the development of CG for clinical practice related to fever blisters to reduce the variability of the quality standard, and generate reliable, applicable recommendations.

Quality appraisal of clinical guidelines for recurrent urinary tract infections using AGREE II: a systematic review

INTRODUCTION AND HYPOTHESIS

Recommendations for preventing and diagnosing recurrent urinary tract infection (UTI) tend to vary between clinical practice guidelines (CPGs) because of low-quality scientific evidence, potentially leading to practice variation and suboptimal care. We assessed the quality of existing CPGs for recurrent UTI.

METHODS

A systematic search was performed from January 2000 to June 2021 in PubMed and EMBASE for CPGs on recurrent UTI prevention or hospital diagnostics in Dutch, English, and Spanish. Each CPG was assessed by four appraisers in a multidisciplinary review team, using the Appraisal of Guidelines, Research, and Evaluation II (AGREE II) instrument.

RESULTS

We identified and assessed eight CPGs published between 2013 and 2021. The scope and purpose (mean and standard deviation: 67.3 ± 21.8) and clarity of presentation (74.8 ± 17.6) domains scored highly. However, issues with methods, patient participation, conflict of interests, and facilitators and barriers were common and resulted in lower scores for the rigour of development (56.9 ± 25.9), applicability (19.6 ± 23.4), stakeholder involvement (50.4 ± 24.6), and editorial independence (62.1 ± 23.1) domains. Overall, two CPGs were recommended, three were recommended with modifications, and three were not recommended.

CONCLUSIONS

Significant room for improvement exists in the quality of CPGs for recurrent UTI, with most displaying serious limitations in the stakeholder involvement, rigour of development, and applicability domains. These aspects must be improved to decrease diagnostic and therapeutic uncertainty. Developers could benefit from using checklists and following guidelines when developing de novo CPGs.

A systematic review of infant feeding food allergy prevention guidelines - can we AGREE?

Food allergy is a significant issue worldwide, particularly in Westernised countries. There is no clear explanation why food allergy appears to have increased so rapidly in recent years, particularly in young children, hence ongoing research to identify effective primary prevention strategies. Food allergy prevention guidelines for health professionals have been developed based on existing clinical trial evidence for effective translation and implementation. As these guidelines underpin clinical practice, it is important to ensure robust processes of development. We conducted a systematic review to identify food allergy prevention guidelines for health professional use; to compare the recommendations made by the identified guideline documents; and to assess the quality of the identified guideline documents. We searched Medline, EMBASE, CINAHL, Scopus, Global Health and Guidelines International Network for the period 1990 to 13 August 2019, to identify articles referring to English-language food allergy prevention guidelines or the guidelines themselves. A grey literature search of Google Scholar and reference checking was also undertaken. The guidelines were compared for recommendation similarities and differences. An Appraisal Guidelines for Research and Evaluation (AGREE II) appraisal was undertaken to assess guideline quality. The electronic database search yielded 1121 publications and reference checking identified an additional 16 publications. After title, abstract and full text screening, data extraction was undertaken on 156 publications and with additional reference checking, 28 food allergy prevention guidelines and advice documents were identified. Comparison of the recommendations within the guidelines and advice documents indicated the greatest variation in recommendations related to exclusive breastfeeding and timing of solid food introduction. Eight of the 10 guidelines and none of the 18 advice documents met the quality threshold set by the reviewers. Overall, documents specifically termed "guidelines" scored better than advice documents when assessed using the AGREE II tool. Variation in recommendations may create confusion for health professionals and result in inconsistent advice being provided to parents, and less translation of the evidence into actual food allergy reduction in the population. Appraisal using the AGREE II tool identified that there is considerable room for improvement in the development of guidelines and advice documents for food allergy prevention. The AGREE II appraisal identified common areas of poorer quality development and/or documentation of processes to inform future guideline development. Based on this study, we recommend the use of validated guideline development tools, to direct food allergy prevention guideline review or development. Use of the AGREE II tool, to direct the review and development of guidelines, is very likely to improve guideline quality.

Osteoporosis guidelines from a rehabilitation perspective: systematic analysis and quality appraisal using AGREE II

INTRODUCTION

People affected by osteoporosis and fragility fractures often report disability and poor health-related quality of life. Albeit rehabilitation has a crucial role in older people, post-menopausal women and other subjects with high risk of fragility fractures, the rehabilitation perspective has been poorly investigated in the available guidelines for osteoporosis. The aim of this systematic review was to systematically evaluate the quality of guidelines for osteoporosis from a rehabilitation perspective.

EVIDENCE ACQUISITION

On May 2020, we performed a systematic search on medical literature of all guidelines published in the last 10 years on PubMed, Pedro, and international guideline databases. The study selection was based on key terms "exercise," "physical activity" or "rehabilitation." All authors independently assessed the methodological quality through the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, consisting of six domains (scope, stakeholder involvement, rigor and development, clarity of presentation, applicability, editorial independence).

EVIDENCE SYNTHESIS

Out of 331 documents retrieved, a total of 34 guidelines were selected after the screening phases. Twenty (58.8%) high quality guidelines were reported. According to AGREE II instrument, a mean score of 78.1±21.8% was reported for "scope and purpose" domain; for stakeholder involvement, the mean score was 58.1±22.1%; the rigor of development was good (mean score of 61.3±27.3%); for clarity of presentation the mean score was 79.4±20.3%; the applicability was poor (mean score of 30.9±25.2%); for editorial independence the mean score was 75.1±24.6%. Rehabilitation recommendations for osteoporotic patients were reported in 21 (61.8%) of the selected guidelines.

CONCLUSIONS

This is the first systematic analysis evaluating quality of the guidelines for osteoporosis using AGREE II instrument. Starting from a state of the art of the currently available evidence, we could conclude that therapeutic exercise at moderate to high intensity is encouraged by several guidelines for the management of people with osteoporosis and fragility fractures. More than half of guidelines were of high-quality. However, most guidelines are lacking specific indications about exercise features. This study might support the implementation of a rehabilitation perspective in the guidelines for osteoporotic patients.

The structure of the quality of clinical practice guidelines with the items and overall assessment in AGREE II: a regression analysis

Background

The Appraisal of Guidelines for Research & Evaluation (AGREE) II has been widely used to evaluate the quality of clinical practice guidelines (CPGs). While the relationship between the overall assessment of CPGs and scores of six domains were reported in previous studies, the relationship between items constituting these domains and the overall assessment has not been analyzed. This study aims to investigate the relationship between the score of each item and the overall assessment and identify items that could influence the overall assessment.

Methods

All Japanese CPGs developed using the evidence-based medicine method and published from 2011 to 2015 were used. They were independently evaluated by three appraisers using AGREE II. The evaluation results were analyzed using regression analysis to evaluate the influence of 6 domains and 23 items on the overall assessment.

Results

A total of 206 CPGs were obtained. All domains and all items except one were significantly correlated to the overall assessment. Regression analysis revealed that Domain 3 (Rigour of Development), Domain 4 (Clarity of Presentation), Domain 5 (Applicability), and Domain 6 (Editorial Independence) had influence on the overall assessment. Additionally, four items of AGREE II, clear selection of evidence (Item 8), specific/unambiguous recommendations (Item 15), advice/tools for implementing recommendations (Item 19), and conflicts of interest (Item 22), significantly influenced the overall assessment and explained 72.1% of the variance.

Conclusions

These four items may highlight the areas for improvement in developing CPGs.

Examining the implementation of clinical practice guidelines for the management of adult cancers: A mixed methods study

RATIONALE, AIMS, AND OBJECTIVES

Focusing on the implementation of clinical practice guidelines (CPGs) for the management of adult cancers, the objectives of this study were to (a) describe the intrinsic elements known to influence CPG use; (b) identify the ways in which CPGs are implemented; and (c) explore how CPG characteristics and contextual factors influence implementation and use.

METHODS

We conducted a sequential mixed methods study. First, we performed a content analysis of all CPGs developed and approved for the management of adult cancers in Nova Scotia, Canada, from 2005 to 2015. CPGs were examined for the presence of 22 elements known to influence CPG use. Next, we conducted semistructured interviews with CPG developers and end users. Participants were purposively sampled, based on the findings of the content analysis. All interviews were audiotaped and transcribed verbatim. Data were analysed by two researchers using the Framework Method.

RESULTS

CPGs (n = 20) demonstrated large variation with respect to elements shown to influence CPG use. For example, 85% included content related to individualization and objectives. Yet no CPGs (0%) had journal or patient versions; discussed the education, training, or competencies needed to deliver recommendations; contained an explicit statement on anticipated work changes, or on potential direct or productivity costs; or identified barriers or facilitators that might influence CPG adoption. Interview data from CPG developers (n = 4) and users (n = 6) revealed five themes related to CPG implementation and use: (a) lack of consistency in CPG development; (b) timing and nature of stakeholder engagement; (c) credibility of the CPG development process and final CPGs; (d) limited understanding of implementation as an active process; and (e) factors at organizational and system levels influence CPG implementation and use.

CONCLUSIONS

This mixed methods study provides complementary data that may help inform more effective CPG implementation efforts and optimize their use in practice.

Quality assessment of clinical practice guidelines using the AGREE instrument in Japan: A time trend analysis

Background

Clinical practice guidelines (CPGs) are representative methods for promoting the standardization of healthcare and improvement of its quality. Few studies have investigated changes in the quality of CPGs published in a country over time. Our aim was to investigate changes in the quality of CPGs over time in the context of the available infrastructure for CPG development, public interest in healthcare quality, and healthcare providers’ responses to this interest.

Methods

All CPGs pertaining to evidence-based medicine (EBM) issued between 2000 and 2014 in Japan (n = 373) were evaluated using the Japanese version of the Appraisal of Guidelines for Research and Evaluation (AGREE) I. Additionally, time trends in quality were analyzed. Using a cut-off point based on the publication year of CPG development literature, the evaluated CPGs were classified into those published until 2008 (pre-2008) and those published since 2009 (post-2008). Subsequently, we compared these groups in terms of 1) first edition CPGs and its second editions, and 2) patients’ version of CPGs.

Results

Scores on all six domains of AGREE I improved each year. A comparison of the first- and second-edition of CPGs (n = 64) showed that scores on all domains improved significantly after revision. Significant improvement was observed in three domains (#2 stakeholder involvement, #3 rigor of development, and #4 clarity of presentation) in the pre-2008 group and in all domains in the post-2008 group. The comparison between the pre- and post-2008 groups in terms of CPGs for patients showed that the score increased in only one domain (#1 scope and purpose).

Conclusions

The number of published CPGs has been increasing and the quality of CPGs, as assessed using the AGREE I instrument, has been improving. These changes seem to be influenced by improvements in social infrastructure, such as the publication of CPG development procedures, availability of CPG preparation methodology training, and increase in CPG-related skills.

Appraising of the Clinical Practice Guidelines Quality in the Non-Pharmacological Management of Chemotherapy-Induced Febrile Neutropenia; A Review

Background:

Febrile neutropenia is a common and serious chemotherapy side effect, is associated with increased mortality, morbidity, and treatment expenditures. Several CPGs (Clinical practice guidelines) have been released for managing chemotherapy-induced febrile neutropenia. The aim of this study is Appraisal of the clinical practice Guidelines quality in the management of chemotherapy-induced febrile neutropenia.

Methods:

A review study with a systematic search of the present CPGs for the management of chemotherapy-induced febrile neutropenia. After screening the CPGs based on eligibility criteria, three CPGs were selected and 5 independent reviewers appraised them for methodological quality by using the AGREE II Instrument.

Results:

Three CPGs were included; all of them were evidence-based guidelines. The clarity of presentation domain scored the highest and the applicability domain has the lowest score among all domains of AGREE instrument and the rest of domains scored as descending respectively; Scope and purpose, stakeholder involvement, editorial independence, rigor of development. In general, the intra-class correlation coefficient (ICC) scores of all domains were very good according to the Landis and Koch’s scale, except the Applicability domain scored as substantial.

Conclusions:

This study showed the quality of appraised CPGs. Three domains of these CPGs based on the AGREE instrument scored less than other domains and were in relatively unfavorable status: applicability, rigor of development, editorial independence. Given the importance of these domains in guideline implementation, it is necessary to take actions for reducing these defects.

The different clinical guideline standards in Brazil: High cost treatment diseases versus poverty-related diseases

Each year, evidence-based clinical guidelines gain more space in the health professionals' practice and in services organization. Due to the scarcity of scientific publications focused on diseases of poverty, the development of well-founded clinical guidelines becomes more and more important. In view of that, this paper aims to evaluate the quality of Brazilian guidelines for those diseases. The AGREE II method was used to evaluate 16 guidelines for poverty-related diseases (PRD) and 16 guidelines for global diseases whose treatment require high-cost technologies (HCD), with the ultimate aim of comparing the results. It was found that, in general, the guideline development quality standard is higher for the HCD guidelines than for the PRD guidelines, with emphasis on the "rigour of development" (48% and 7%) and "editorial independence" (43% and 1%) domains, respectively, which had the greatest discrepancies. The HCD guidelines showed results close to or above international averages, whereas the PRD guidelines showed lower results in the 6 domains evaluated. It can be concluded that clinical protocol development priorities need some redirecting in order to qualify the guidelines that define the healthcare organization and the care of vulnerable populations.

Management of physical pain during induced second-trimester medical abortions: a cross-sectional study of methodological quality and recommendations in local clinical practice guidelines at Swedish hospitals

AIM

The aim was to assess the methodological quality and describe recommendations for pain management in local clinical practice guidelines about induced second-trimester medical abortions at Swedish university and county hospitals.

METHODS

In 2017, Swedish university and county hospitals that provided abortion care in the second trimester of pregnancy were contacted (n = 29), and guidelines from 25 were received (university: n = 6, county: n = 19). Guideline quality was assessed according to two systematic instruments. Recommendations were systematically assessed regarding frequency and tools for pain measurement, prophylactic pharmacologic treatment, as needed pharmacologic treatment and nonpharmacologic treatment.

RESULTS

Overall methodological quality was poor across both instruments, as the majority of the guidelines did not fulfil the investigated quality criteria. For pain measurements, no guideline recommended measurement frequency and four recommended specific measurement tools. Prophylactic pharmacologic treatment, described in 23 (92%) guidelines, included paracetamol (n = 23, 92%), anti-inflammatory drugs (n = 23, 92%) and opioids (n = 18, 72%). As needed pharmacologic treatment, described in 23 (92%) guidelines, included anaesthetics (n = 21, 84%), opioids (n = 21, 84%) and paracetamol (n = 1, 4%). Recommendations for as needed anaesthetics included paracervical block (n = 21, 84%), epidural analgesia (n = 16, 64%) and inhalation of nitrous oxide (n = 5, 20%). Nonpharmacologic treatments were recommended in nine (36%) guidelines.

CONCLUSIONS

The findings indicate that local clinical practice guidelines about induced second-trimester medical abortions are of inadequate methodological quality and that a large majority lack recommendations concerning systematic pain measurements. Although most recommend prophylactic and as needed pharmacologic management, national inconsistencies exist in Sweden with regard to recommendations of epidural analgesia, nitrous oxide and nonpharmacologic methods. In Sweden, there is room for improvement in the development of these guidelines.

Quality assessment of clinical practice guidelines for infectious diseases in China

AIM

We aimed to appraise the quality of Chinese Society of Infectious Diseases of China Medical Association clinical practice guidelines (CPGs) for infectious diseases in China.

METHODS

We performed the electronic search and hand search on the above-mentioned CPGs for infectious diseases published from January 1997 to October 2017. Each publication was evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument.

RESULTS

Thirteen guidelines were identified and enrolled for the further evaluation. The median scores of the AGREE II domains across all 13 guidelines were as follows: scope and purpose 73.61% (36.11% to 84.72%), stakeholder involvement 12.50% (8.33% to 20.83%), rigor of development 19.27% (12.50% to 26.04%), clarity of presentation 86.11% (23.61% to 94.44%), applicability 33.33% (26.04% to 50.00%), and editorial independence 8.33% (4.17% to 16.67%). The clarity of presentation domain got the highest score among the six domains. Moreover, when compared the CPGs by the year of publication (2002 to 2006 vs. 2007 to 2017), the quality scores were higher in the CPGs published in the last 10 years than those published before, but with no significant difference.

CONCLUSIONS

The quality score of CPGs for infectious diseases in China from January 1997 to October 2017 was moderate, but had significant shortcomings, especially in the stakeholder involvement and editorial independence domains. However, the AGREE II instrument is useful and feasible for appraising the CPGs and should be applied to develop, implement, and promote the CPGs in China.

A critical appraisal of the quality of adult musculoskeletal ultrasound guidelines using the AGREE II tool: an EuroAIM initiative

OBJECTIVES

Our aim was to evaluate the quality of published guidelines on musculoskeletal ultrasound (MSK-US) for adults.

METHODS

Between June and July 2016, we conducted an online search for MSK-US guidelines, which were evaluated by four independent readers blinded to each other using the AGREE II tool. A fifth independent reviewer calculated scores per each domain and agreement between reviewers' scores using the intraclass correlation coefficient (ICC).

RESULTS

Five guidelines were included in this appraisal. They were published between 2001 and 2014. Our appraisal showed intermediate results, with four out of five guidelines scoring "average" as overall quality. Domain 1 (scope and purpose) achieved the highest result (total score = 71.1% ± 18.7%). Domain 6 (editorial independence) had the lowest score (total score = 26.3% ± 19.3%). Interobserver agreement was very good for all the evaluated guidelines (ICC ranged between 0.932 and 0.956).

CONCLUSIONS

Overall, quality of MSK-US guidelines ranges from low to average when evaluated using the AGREE II tool. The editorial independence domain was the most critical, thus deserving more attention when developing future guidelines.

MAIN MESSAGES

• Four of five guidelines on MSK-US had an average quality level. • Scope/purpose had the highest score (71.1% ± 18.7%). • Editorial independence had the lowest score (26.3% ± 19.3%). • Interobserver agreement was very good (ranges: 0.932-0.956).

A critical appraisal of the quality of adult dual-energy X-ray absorptiometry guidelines in osteoporosis using the AGREE II tool: An EuroAIM initiative

Objectives

Dual energy X-ray absorptiometry (DXA) is the most widely used technique to measure bone mineral density (BMD). Appropriate and accurate use of DXA is of great importance, and several guidelines have been developed in the last years. Our aim was to evaluate the quality of published guidelines on DXA for adults.

Methods

Between June and July 2016 we conducted an online search for DXA guidelines, which were evaluated by four independent readers blinded to each other using the AGREE II instrument. A fifth independent reviewer calculated scores per each domain and agreement between reviewers’ scores.

Results

Four out of 59 guidelines met inclusion criteria and were included. They were published between 2005 and 2014. Three out of four guidelines reached a high level of quality, having at least five domain scores higher than 60%. Domain 1 (Scope and Purpose) achieved the highest result (total score = 86.8 ± 3.7%). Domain 6 (Editorial Independence) had the lowest score (total score = 54.7 ± 12.5%). Interobserver agreement ranged from fair (0.230) to good (0.702).

Conclusions

Overall, the quality of DXA guidelines is satisfactory when evaluated using the AGREE II instrument. The Editorial Independence domain was the most critical, thus deserving more attention when developing future guidelines.

Main messages

• Three of four guidelines on DXA had a high quality level (>60%).

• Scope/purpose had the highest score (86.8 ± 3.7%).

• Editorial Independence had the lowest score (54.7 ± 12.5%).

• Interobserver agreement ranged from fair (0.230) to good (0.702).

Electronic supplementary material

The online version of this article (doi:10.1007/s13244-017-0553-6) contains supplementary material, which is available to authorized users.

Quality assessment of clinical practice guidelines for eosinophilic esophagitis using the AGREE II instrument

BACKGROUND

High-quality evidence-based clinical practice guidelines can guide diagnosis and treatment to optimize outcomes. We aimed to systematically review the quality of international guidelines on eosinophilic esophagitis (EoE).

METHODS

MEDLINE and Scopus databases were searched for appropriate guidelines up to 2016. Two gastroenterologists and two methodologists independently evaluated the documents using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument.

RESULTS

Amongst the 25 records initially retrieved, four guidelines developed by recognized scientific organizations met inclusion criteria. AGREE II results varied widely across domains, but none achieved an overall assessment score of over 60%. Scope and purpose (61.82 ± 19.24%), clarity of presentation (57.13 ± 40.56%) and editorial independence (93.75 ± 1.69%) showed the highest mean rating, whereas stakeholder involvement (28.82 ± 11.19%), rigor of development (32.29 ± 12.02%) and applicability (21.62 ± 7.14%) did not reach quality thresholds. Intraclass correlation coefficients for agreement was excellent among appraisers (0.903), between gastroenterologists and methodologists (0.878) and for each individual guideline (0.838 to 0.955).

CONCLUSION

Clinical practice guidelines for EoE vary significantly in quality, are invariably limited and currently, none can be 'strongly recommended'.

Clinical Practice Guidelines for Hypertension: Evaluation of Quality Using the AGREE II Instrument

INTRODUCTION

Hypertension is an important public health challenge. The purpose of clinical practice guidelines (CPGs) is to provide explicit recommendations for clinical practice, reduce inadequate variations, optimize results, minimize risks, and promote cost-effective practice. Therefore, a highly methodological quality development process for CPGs is more likely to yield a CPG that contains relevant and appropriate recommendations.

METHOD

To assess the quality of CPGs for the prevention and treatment of hypertension, a systematic search was performed using the following literature databases: PubMed, Excerpta Medica Database (EMBASE), Web of Science, the National Guideline Clearinghouse (NGC), Chinese National Knowledge Infrastructure (CNKI), Wan fang Data, VIP, and Chinese Biomedical Literature Database (CBM). Then the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument was used to assess the quality of the CPGs.

RESULTS

Forty-one CPGs were identified (ten CPGs in Chinese and 31 CPGs in English). The results for the overall recommendation were as follows: strongly recommended (15 %), recommended (29 %), weakly recommended (51 %), and not recommended (5 %). The scores for all domains (mean ± standard deviations) were "scope and purpose" (58.65 ± 12.67 %), "stakeholder involvement" (48.07 ± 11.41 %), "rigor of development" (27.31 ± 12.29 %), "clarity of presentation" (53.89 ± 11.09 %), "applicability" (40.10 ± 13.33 %), and "editorial independence" (38.75 ± 16.43 %). All differences were statistically insignificant for all domains (P > 0.05) according to publication time. CPGs using an evidence-based (EB) method were of a higher quality than non-EBs for all domains, but the differences were significant for the following domains: "Scope and Purpose," "Rigor of Development," "Applicability," and "Editorial Independence" (P < 0.05). The scores for the CPGs developed based on associations and society appeared slightly higher than those developed by individuals. However, the differences were insignificant for all domains (P > 0.05).

CONCLUSION

A more systematic approach for the development and report of these guidelines is recommended. The AGREE II instrument can be a useful tool to improve the quality of guidelines.

The quality of clinical practice guidelines for chronic respiratory diseases and the reliability of the AGREE II: an observational study

OBJECTIVES

To survey the quality of evidence-based clinical practice guidelines for chronic respiratory diseases relevant to physiotherapy practice using the Appraisal of Guidelines for Research and Evaluation version II instrument (AGREE II) and to evaluate the inter-rater reliability of AGREE II.

DESIGN

Observational survey.

PROCEDURES

Guidelines indexed in the Physiotherapy Evidence Database (PEDro) on chronic respiratory diseases were evaluated by four assessors using AGREE II.

MAIN OUTCOME MEASURES

The six domains and two global items of AGREE II.

RESULTS

Thirty-three guidelines were evaluated (58% were published in the last 5 years and 36% were for chronic obstructive pulmonary disease). The domains with the highest scores were scope and purpose (79%, SD 10%) and clarity of presentation (79%, SD 10%). The domain with the lowest score was applicability (37%, SD 23%). Mean overall quality was five out of seven (SD 1). Intraclass correlation coefficients ranged from 0.66 to 0.93 for the six domains and first global item, suggesting good to excellent reliability. The second global item had very poor reliability (Kappa 0.097).

CONCLUSION

The quality of evidence-based clinical practice guidelines for chronic respiratory diseases relevant to physiotherapy could be improved, particularly in consideration with applicability. The number of assessors for AGREE II could be reduced because of the good inter-rater reliability.

Non-Communicable Disease Clinical Practice Guidelines in Brazil: A Systematic Assessment of Methodological Quality and Transparency

BACKGROUND

Annually, non-communicable diseases (NCDs) kill 38 million people worldwide, with low and middle-income countries accounting for three-quarters of these deaths. High-quality clinical practice guidelines (CPGs) are fundamental to improving NCD management. The present study evaluated the methodological rigor and transparency of Brazilian CPGs that recommend pharmacological treatment for the most prevalent NCDs.

METHODS

We conducted a systematic search for CPGs of the following NCDs: asthma, atrial fibrillation, benign prostatic hyperplasia, chronic obstructive pulmonary disease, congestive heart failure, coronary artery disease and/or stable angina, dementia, depression, diabetes, gastroesophageal reflux disease, hypercholesterolemia, hypertension, osteoarthritis, and osteoporosis. CPGs comprising pharmacological treatment recommendations were included. No language or year restrictions were applied. CPGs were excluded if they were merely for local use and referred to NCDs not listed above. CPG quality was independently assessed by two reviewers using the Appraisal of Guidelines Research and Evaluation instrument, version II (AGREE II).

MAIN FINDINGS

"Scope and purpose" and "clarity and presentation" domains received the highest scores. Sixteen of 26 CPGs were classified as low quality, and none were classified as high overall quality. No CPG was recommended without modification (77% were not recommended at all). After 2009, 2 domain scores ("rigor of development" and "clarity and presentation") increased (61% and 73%, respectively). However, "rigor of development" was still rated < 30%.

CONCLUSION

Brazilian healthcare professionals should be concerned with CPG quality for the treatment of selected NCDs. Features that undermined AGREE II scores included the lack of a multidisciplinary team for the development group, no consideration of patients' preferences, insufficient information regarding literature searches, lack of selection criteria, formulating recommendations, authors' conflict of interest disclosures, and funding body influence.

Nutrition in critically ill adults: A systematic quality assessment of clinical practice guidelines

BACKGROUND & AIMS

Nutritional support in the acutely ill is a complex topic. Clinical practice guidelines (CPGs) have been developed to assist healthcare professionals working in this field. However, the quality of these clinical guidelines has not yet been systematically assessed. The objective of our study was to identify and assess the quality of CPGs on nutrition in critically ill adult patients.

METHODS

We performed a systematic search to identify CPGs on nutrition in critically ill adult patients. Three independent appraisers assessed six domains (scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, applicability and editorial independence) of the eligible CPGs using the Appraisal of Guidelines, Research, and Evaluation II (AGREE II) instrument.

RESULTS

Nine CPGs were selected. Overall agreement among appraisers was very good (ICC: 0.853; 95% CI: 0.820-0.881). The mean scores for each AGREE domain were the following: "scope and purpose" 76.2% ± 13.7%; "stakeholder involvement" 42.8% ± 16.5%; "rigour of development" 57.9% ± 18.1%; "clarity of presentation" 76.9% ± 13.7%; "applicability" 30.1% ± 22.8%; and 42.1% ± 23.9% for "editorial independence". Four CPGs were deemed "Recommended"; three "Recommended with modifications"; and two "Not recommended". We did not observe improvement over time in the overall quality of the CPGs.

CONCLUSIONS

The overall quality of CPGs on nutrition in critically ill adults is suboptimal, with only four CPGs being recommended for clinical use. Our results highlight the need to revise and improve CPG development processes in this field.

Healthcare Quality Indicators for Physiotherapy Management in Hip and Knee Osteoarthritis and Rheumatoid Arthritis: A Delphi Study

OBJECTIVES

The aim of the present study was to develop healthcare quality indicators (HCQIs) for the physiotherapy (PT) management of patients with hip or knee osteoarthritis (HKOA) or rheumatoid arthritis (RA) in the Netherlands.

METHODS

Two multidisciplinary expert panels, including patients, were instituted. A draft HCQI set was derived from recommendations included in two existing Dutch PT guidelines for HKOA and RA. The panels suggested additional topics, after which a Delphi procedure was performed. All propositions were scored for their potential to represent good-quality PT care (score range 0-9). Based on predefined rules, the Delphi panel HCQIs were discussed and selected. Lastly, every indicator was rephrased, resulting in its output consisting of a numerator and denominator, to facilitate comparisons within and among practices.

RESULTS

After two Delphi rounds, two final sets of 17 HCQI - one for HKOA and one for RA - were composed, both containing 16 process indicators (regarding initial assessment, treatment and evaluation) and one outcome indicator.

CONCLUSIONS

Two sets of HCQIs for PT management in HKOA and RA were developed for measuring the quality of PT care in daily clinical practice. Each indicator was formulated in a measurable way. Future research should focus on the feasibility of both indicator sets for daily clinical practice.

Quality assessment of osteoporosis clinical practice guidelines for physical activity and safe movement: an AGREE II appraisal

Many osteoporosis clinical practice guidelines are published, and the extent to which physical activity and safe movement is addressed varies. To better inform clinical decision-making, a quality assessment and structured analysis of recommendations was undertaken. Guideline quality varied substantially, and improvement is necessary in physical activity and safe movement recommendations.

PURPOSE

The purpose of the present study is to survey available osteoporosis clinical practice guidelines (CPGs) containing physical activity and safe movement recommendations in order to assess the methodological quality with which they were developed. An analysis of the various physical activity and safe movement recommendations was conducted to determine variability between CPGs.

METHODS

An online literature search revealed 19 CPGs meeting our inclusion criteria. Three independent scorers evaluated CPG quality using the Appraisal of Guidelines for Research and Evaluation version II (AGREE II) instrument. Two separate individuals used a standard table to extract relevant recommendations.

RESULTS

Intra-reviewer AGREE II score agreement ranged from fair to good (intra-class correlation coefficient (ICC) = 0.34 to 0.65). The quality of the 19 included CPGs was variable (AGREE sub-scores: 14 to 100%). CPGs scored higher in the "scope and purpose" and "clarity of presentation" domains. They scored the lowest in "applicability" and "editorial independence." Four CPGs were classified as high quality, ten average quality, and five low quality. Most CPGs recommended weight-bearing, muscle-strengthening, and resistance exercises. Information on exercise dosage, progression, and contraindications was often absent. Immobility and movements involving spinal flexion and/or torsion were discouraged.

CONCLUSIONS

There were several high-quality CPGs; however, variability in quality and lack of specific parameters for implementation necessitates caution and critical examination by readers. CPG development groups should pay special attention to the clinical applicability of their CPGs as well as fully disclosing conflicts of interest. CPGs were in general an agreement regarding safe physical activity and safe movement recommendations. However, recommendations were often vague and the more specific recommendations were inconsistent between CPGs.