Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners

A critical appraisal of acute sore throat guidelines using the AGREE II instrument: a scoping review

BACKGROUND

Conflicting international guidelines exist on the management of sore throat by antibiotics.

OBJECTIVES

To assess with the Appraisal of Guidelines for Research and Evaluation II (AGREE) instrument the quality of guidelines for uncomplicated acute group A beta-haemolytic streptococcal (GABHS) sore-throat. To make a sensitivity analysis restricted to guidelines with a rigour of development score higher than 60% and to describe their recommendations on scores, tests, and antibiotic therapy, including their justification.

METHODS

A guideline literature review of acute GABHS sore throat, published between January 2000 and December 2019 in primary care and secondary care. The PubMed database, the Canadian Medical Association Infobase on Clinical Practice Guidelines and the International Network Guidelines were used. The quality of guidelines was assessed using the AGREE II instrument. The guidelines were classified into 2 categories: high-quality guidelines had to rate >60% for the rigour of development score, the others were classified as low-quality guidelines.

RESULTS

Significant heterogeneity between the 15 guidelines concerned the scores of the 6 assessment domains. Among them, 6 guidelines presented a score above 60% with regards to the rigor of development domain and used a systematic literature search method, citing meta-analyses of recent randomised clinical trials. Most of the 6 high-quality guidelines no longer recommended the systematic use of diagnostic scores and tests, nor antibiotic therapy to prevent acute rheumatic fever or loco-regional complications, except for high-risk patients.

CONCLUSION

Major discrepancies emphasise the need for only high-quality guidelines, based on adequately assessed evidence. Restricted antibiotic prescriptions to severe cases or high-risk patients would avoid antibiotic resistance.

Management of Inpatient Elevated Blood Pressures : A Systematic Review of Clinical Practice Guidelines

BACKGROUND

Management of elevated blood pressure (BP) during hospitalization varies widely, with many hospitalized adults experiencing BPs higher than those recommended for the outpatient setting.

PURPOSE

To systematically identify guidelines on elevated BP management in the hospital.

DATA SOURCES

MEDLINE, Guidelines International Network, and specialty society websites from 1 January 2010 to 29 January 2024.

STUDY SELECTION

Clinical practice guidelines pertaining to BP management for the adult and older adult populations in ambulatory, emergency department, and inpatient settings.

DATA EXTRACTION

Two authors independently screened articles, assessed quality, and extracted data. Disagreements were resolved via consensus. Recommendations on treatment targets, preferred antihypertensive classes, and follow-up were collected for ambulatory and inpatient settings.

DATA SYNTHESIS

Fourteen clinical practice guidelines met inclusion criteria (11 were assessed as high-quality per the AGREE II [Appraisal of Guidelines for Research & Evaluation II] instrument), 11 provided broad BP management recommendations, and 1 each was specific to the emergency department setting, older adults, and hypertensive crises. No guidelines provided goals for inpatient BP or recommendations for managing asymptomatic moderately elevated BP in the hospital. Six guidelines defined hypertensive urgency as BP above 180/120 mm Hg, with hypertensive emergencies requiring the addition of target organ damage. Hypertensive emergency recommendations consistently included use of intravenous antihypertensives in intensive care settings. Recommendations for managing hypertensive urgencies were inconsistent, from expert consensus, and focused on the emergency department. Outpatient treatment with oral medications and follow-up in days to weeks were most often advised. In contrast, outpatient BP goals were clearly defined, varying between 130/80 and 140/90 mm Hg.

LIMITATION

Exclusion of non-English-language guidelines and guidelines specific to subpopulations.

CONCLUSION

Despite general consensus on outpatient BP management, guidance on inpatient management of elevated BP without symptoms is lacking, which may contribute to variable practice patterns.

PRIMARY FUNDING SOURCE

National Institute on Aging. (PROSPERO: CRD42023449250).

Moving toward the Development and Effective Implementation of High-Quality Guidelines in Pediatric Surgery: A Review of the Literature

Applying evidence-based guidelines can enhance the quality of patient care. While robust guideline development methodology ensures credibility and validity, methodological variations can impact guideline quality. Besides methodological rigor, effective implementation is crucial for achieving improved health outcomes. This review provides an overview of recent literature pertaining to the development and implementation of guidelines in pediatric surgery. Literature was reviewed to provide an overview of sound guideline development methodologies and approaches to promote effective guideline implementation. Challenges specific to pediatric surgery were highlighted. A search was performed to identify published guidelines relevant to pediatric surgery from 2018 to June 2023, and their quality was collectively appraised using the AGREE II instrument. High-quality guideline development can be promoted by using methodologically sound tools such as the Guidelines 2.0 checklist, the GRADE system, and the AGREE II instrument. While implementation can be promoted during guideline development and post-publication, its effectiveness may be influenced by various factors. Challenges pertinent to pediatric surgery, such as limited evidence and difficulties with outcome selection and heterogeneity, may impact guideline quality and effective implementation. Fifteen guidelines were identified and collectively appraised as suboptimal, with a mean overall AGREE II score of 58%, with applicability being the lowest scoring domain. There are identified challenges and barriers to the development and effective implementation of high-quality guidelines in pediatric surgery. It is valuable to prioritize the identification of adapted, innovative methodological strategies and the use of implementation science to understand and achieve effective guideline implementation.

Clinical practice guidelines on physical activity and exercise for pregnant women with gestational diabetes mellitus: A systematic review

AIM

This review aimed to appraise clinical guidelines about exercise for women with gestational diabetes mellitus and summarize consensus and inconsistent recommendations.

BACKGROUND

Exercise is an effective non-pharmacological therapeutic for gestational diabetes mellitus, but the variety of relevant clinical practice guidelines is confusing for healthcare professionals.

DESIGN

This is a systematic review of clinical practice guidelines.

DATA SOURCES

Websites of guideline development institutions, eight literature databases and organizations of obstetricians, gynaecologists, midwives, and medical sports associations were searched for guidelines published from January 2011 to October 2021.

REVIEW METHODS

Two reviewers independently extracted recommendations. Four reviewers assessed guideline quality using the AGREE II instrument independently.

RESULTS

Fifteen guidelines were included. All women with diabetes are recommended to exercise during pregnancy. The consistent recommendations were for pre-exercise screening, for 30 min per exercise session on 5 days of the week or every day after meals, exercise at moderate intensity, using aerobic and resistance exercise, and walking. The main non-consistent recommendations included warning signs for women on insulin during exercise, minimum duration per session, intensity assessment, duration and frequency of sessions for strengthening and flexibility exercise and detailed physical activity giving birth.

CONCLUSIONS

Guidelines strongly support pregnant women with diabetes to exercise regularly. Research is needed to make non-consistent recommendations clear.

A critical appraisal and recommendation synthesis of delirium clinical practice guidelines relevant to the care of older adults in the emergency department: An umbrella review

RATIONALE

Older adults are at high risk of developing delirium in the emergency department (ED); however, it is often missed or undertreated. Improving ED delirium care is challenging in part due to a lack of standards to guide best practice. Clinical practice guidelines (CPGs) translate evidence into recommendations to improve practice.

AIM

To critically appraise and synthesize CPG recommendations for delirium care relevant to older ED patients.

METHODS

We conducted an umbrella review to retrieve relevant CPGs. Quality of the CPGs and their recommendations were critically appraised using the Appraisal of Guidelines, Research, and Evaluation (AGREE)-II; and Appraisal of Guidelines Research and Evaluation-Recommendations Excellence (AGREE-REX) instruments. A threshold of 70% or greater in the AGREE-II Rigour of Development domain was used to define high-quality CPGs. Delirium recommendations from CPGs meeting this threshold were included in the synthesis and narrative analysis.

RESULTS

AGREE-II Rigour of Development scores ranged from 37% to 83%, with 5 of 10 CPGs meeting the predefined threshold. AGREE-REX overall calculated scores ranged from 44% to 80%. Recommendations were grouped into screening, diagnosis, risk reduction, and management. Although none of the included CPGs were ED-specific, many recommendations incorporated evidence from this setting. There was agreement that screening for nonmodifiable risk factors is important to define high-risk populations, and those at risk should be screened for delirium. The '4A's Test' was the recommended tool to use in the ED specifically. Multicomponent strategies were recommended for delirium risk reduction, and for its management if it occurs. The only area of disagreement was for the short-term use of antipsychotic medication in urgent situations.

CONCLUSION

This is the first known review of delirium CPGs including a critical appraisal and synthesis of recommendations. Researchers and policymakers can use this synthesis to inform future improvement efforts and research in the ED.

REGISTRATION

This study has been registered in the Open Science Framework registries: https://doi.org/10.17605/OSF.IO/TG7S6OSF.IO/TG7S6.

A systematic review and evaluation of post-stroke depression clinical practice guidelines

OBJECTIVE

Post-stroke depression is a depressive mood disorder that follows a cerebrovascular accident and is a burden on stroke patients. Its management is included in clinical practice guidelines focused on stroke, and the recommended treatment is selective serotonin reuptake inhibitors in conjunction with psychotherapy. Clinical practice guidelines are recommendations used to standardize best medical practice, but there is no current evaluation of guidelines containing post stroke depression recommendations. Thus, the objective is to appraise the selected guidelines manner of development and quality.

MATERIALS AND METHODS

A systematic literature review across three databases and a manual google search was performed to collect guidelines that included recommendations on the management of post-stroke depression. 1236 guidelines were screened, and 27 were considered for inclusion. Considered guidelines were manually reviewed by the authors, and ultimately, 7 met inclusion criteria. The appraisal of guidelines for research and evaluation was used to evaluate these guidelines' recommendations around post-stroke depression.

RESULTS

Three guidelines met the threshold considered "High", with all of them having five or more quality domains eclipse the cutoff score of 70%. Across all guidelines, the highest scoring domains were "Scope and Purpose", "Clarity of Presentation", and "Editorial Independence" with scores of 76.98%, 73.81%, and 91.36% respectively. The lowest scoring domains were "Applicability", "Rigor of Development", and "Stakeholder Involvement" with respective scores of 58.73%, 54.02%, and 43.90%.

CONCLUSIONS

The domains "Applicability", "Rigor of Development," and "Stakeholder Involvement" were the lowest scoring domains. These specific domains represent areas in which future guidelines could be more developed.

Editor's Choice - Quality Assessment of European Society for Vascular Surgery Clinical Practice Guidelines

OBJECTIVE

An increasing number of clinical practice guidelines (CPGs) have emerged over recent years. To have clinical utility, they need to be rigorously developed and scientifically robust. Instruments have been developed to assess the quality of clinical guideline development and reporting. The aim of this study was to evaluate CPGs from the European Society for Vascular Surgery (ESVS) using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument.

METHODS

CPGs published by the ESVS during the period January 2011 to January 2023 were included. Two independent reviewers assessed the guidelines after receiving training in the use and application of the AGREE II instrument. Inter-reviewer reliability was assessed with the intraclass correlation coefficient. Maximum scaled scores were 100. Statistical analysis was performed using SPSS Statistics v.26.

RESULTS

Sixteen guidelines were included in the study. Good inter-reviewer score reliability was found on statistical analysis (> 0.9). The mean ± standard deviation domain scores were 68.1 ± 20.3% for scope and purpose, 57.1 ± 21.1% for stakeholder involvement, 67.8 ± 19.5% for rigour of development, 78.1 ± 20.6% for clarity of presentation, 50.3 ± 15.4% for applicability, 77.6 ± 17.6% for editorial independence, and 69.8 ± 20.1% for overall quality. Stakeholder involvement and applicability have improved in quality over time but are still the lowest scoring domains.

CONCLUSION

Most ESVS clinical guidelines are of high quality and reporting. There is scope for improvement, specifically by addressing the domains of stakeholder involvement and clinical applicability.

A critical appraisal of delirium clinical practice guidelines relevant to the care of older adults in the emergency department with a synthesis of recommendations: an umbrella review protocol

BACKGROUND

Up to 35% of older adults present to the emergency department (ED) with delirium or develop the condition during their ED stay. Delirium associated with an ED visit is independently linked to poorer outcomes such as loss of independence, increased length of hospital stay, and mortality. Improving the quality of delirium care for older ED patients is hindered by a lack of knowledge and standards to guide best practice. High-quality clinical practice guidelines (CPGs) have the power to translate the complexity of scientific evidence into recommendations to improve and standardize practice. This study will identify and synthesize recommendations from high-quality delirium CPGs relevant to the care of older ED patients.

METHODS

We will conduct a multi-phase umbrella review to retrieve relevant CPGs. Quality of the CPGs and their recommendations will be critically appraised using the Appraisal of Guidelines, Research, and Evaluation (AGREE)-II; and Appraisal of Guidelines Research and Evaluation - Recommendations Excellence (AGREE-REX) instruments, respectively. We will also synthesize and conduct a narrative analysis of high-quality CPG recommendations.

DISCUSSION

This review will be the first known evidence synthesis of delirium CPGs including a critical appraisal and synthesis of recommendations. Recommendations will be categorized according to target population and setting as a means to define the bredth of knowledge in this area. Future research will use consensus building methods to identify which are most relevant to older ED patients.

TRIAL REGISTRATION

This study has been registered in the Open Science Framework registries: https://doi.org/10.17605/OSF.IO/TG7S6 .

Quality assessment of clinical practice guidelines on psychological distress of cancer patients using the AGREE II instrument

Objective

This study aimed to assess the quality of the clinical practice guidelines on psychological distress among cancer patients and provide users with recommendations for coping with psychological distress.

Methods

A systematic search of relevant clinical practice guidelines was undertaken to identify and select the clinical practice guidelines related to psychological distress among cancer patients. Literature databases were searched in PubMed, Web of Science, Excerpta Medica Database, the Cumulative Index to Nursing & Allied Health Literature, China Biology Medicine, China National Knowledge Infrastructure, WanFang and Weipu Journal Database. The guideline databases include Yimaitong Guidelines Network, National Guideline Clearinghouse, National Institute for Health and Clinical Excellence, American Society of Clinical Oncology (ASCO), New Zealand Guidelines Group, Scottish Intercollegiate GuidelinesNetwork, American Psychological Association, Registered Nurses’ Association of Ontario and Cancer Care Ontario (CCO). Four independent reviewers assessed the eligible guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument.

Results

Six clinical practice guidelines were included and assessed for critical evaluation. The median score for the scope and purpose domain was 71.5% (IQR 64%-77.25%), the stakeholder involvement domain was 65% (IQR 47.5%-74.5%), the rigour of the development domain was 61.5% (IQR 45.5%-85.25%), the clarity of the presentation domain was 91% (IQR 72.25%-94.5%), the applicability domain was 70% (IQR 33%-78.75%), and the editorial independence domain was 48.84% (IQR 61.75%-95%). Four guidelines (ASCO, 2014; Canadian Association of Psychosocial Oncology, 2015; NCCN, 2020, and CCO, 2016) were classified as “recommended,” and the remaining (European Palliative Care Research Collaborative and Chinese Psychosocial Oncology Society) were “recommended with modifications,” especially in the domains of Stakeholder involvement, rigour of development, and applicability. The inter-rater consistency of each domain showed moderate level (0.52–0.90) analyzing by intraclass correlation.

Conclusions

The clinical practice guidelines on psychological distress among cancer patients varied in quality, and there were discrepancies in terms of the recommendations and recommendation grades. These findings could contribute to improving the quality of clinical practice guidelines on psychological distress, and enable the development and implementation of evidence-based guidelines for cancer patients.

Systematic Review Registration

https://www.crd.york.ac.uk/PROSPERO, identifier CRD42020209204.

Quality Assessment of the Clinical Practice Guidelines of Ostomy Care Based on the AGREE II Instrument

Objectives

To assess the quality of clinical practice guidelines (CPGs) of ostomy care, and to analyze the status quo and challenges of guideline development.

Methods

CPGs of ostomy care were systematically searched in relevant guideline websites and electronic databases, including PubMed, ProQuest, Web of Science, CNKI, VIP, WANFANG, and SinoMed, from January 1, 2012, to November 24, 2021. Two appraisers used the Appraisal of Guidelines for Research and Evaluation, 2nd edition (AGREE II) instrument to assess the quality of the included CPGs independently and objectively. The consistency of assessment was calculated using intraclass correlation coefficients (ICC).

Results

A total of 5 CPGs relevant to ostomy care were assessed by AGREE II and the general quality of them was good. There were two CPGs of grade A and three CPGs of grade B. The domain scope and purpose (87.78%) had the highest scores, followed by the clarity of presentation (87.22%), the rigor of development (69.17%), stakeholder involvement (68.33%), and editorial independence (65.00%), and the lowest was applicability (55.42%). The overall assessment score was 5.40. All the ICCs for the AGREE II appraisal conducted by the two appraisers were >0.75.

Conclusions

The five CPGs of ostomy care have the potential to be adopted in clinical practice. However, they still have some room for improvement, especially in the applicability domain. The development of ostomy care CPGs should follow the evidence-based progress and methodology of guideline formulation specifications while considering the effects of the CPGs and the practical issues.

Recommendations for the pharmacological treatment of treatment-resistant depression: A systematic review protocol

INTRODUCTION

Depression is a serious and widespread mental health disorder. Although effective treatment does exist, a significant proportion of patients with depression fail to respond to antidepressant treatment trials, a condition named treatment-resistant depression. Efficient approach should be given this condition in order to revert the burden caused by depression. Clinical practice guidelines (CPGs) are evidence-based health promotion instruments to improve diagnosis and treatment. CPGs recommendations for treatment-resistant depression must be trustworthy. The objective of the proposed study is to systematically identify, appraise the quality of CPGs for the treatment of depression and elaborate a synthesis of recommendations for treatment-resistant depression of CPGs considered to be of high quality and with high quality recommendations.

METHODS AND ANALYSIS

We will search the databases of organizations, such as PubMed, Embase, Cochrane Library, PsycInfo, and the Virtual Health Library, and organizations that develop CPGs. Three independent researchers will assess the quality of the CPGs and their recommendations using the AGREE II and AGREE-REX instruments, respectively. Given the identification of divergences and convergences as well as weak and strong points among high quality CPGs, our work may help developers, clinicians and eventually patients.

ETHICS AND DISSEMINATION

No ethical approval is required for a systematic review, as no patient data will be used. The research results will be disseminated in conferences and submitted to a peer reviewed journal.

Lifestyle and complementary therapies in multiple sclerosis guidelines: Systematic review

Management of multiple sclerosis (MS) may comprise clinical interventions and self-management strategies, including complementary therapies and modifiable lifestyle factors such as exercise and smoking cessation. Lifestyle modifications and complementary therapies with proven safety and efficacy are essential as part of best-practice MS management, especially when faced with limited access to healthcare services. However, it is unclear to what extent MS clinical practice guidelines and consensus statements address these strategies. A systematic review was conducted, wherein MEDLINE, EMBASE, PsycINFO, CINAHL, Scopus, Web of Science, guideline databases and developer sites were searched for guidelines and consensus statements that addressed lifestyle modifications and complementary therapies of interest. Two researchers independently screened articles, extracted data and assessed guideline quality using the Appraisal of Guidelines for Research and Evaluation version II. Thirty-one guidelines and consensus statements were included. Quality was high for 'clarity of presentation' (77%) and 'scope and purpose' (73%), moderate for 'stakeholder development' (56%), 'rigour of development' (48%) and 'editorial independence' (47%), and low for 'applicability' (29%). Two guidelines, related to physical activity and exercise, mindfulness, smoking cessation, and vitamin D and polyunsaturated fatty acid supplementation, scored high in all domains. These guidelines were two of only four guidelines intended for use by people with MS. High-quality guidelines and consensus statements to guide lifestyle modifications and complementary therapies in MS management are limited. Our findings indicate the need for more guidelines intended for use by people with MS, and a further focus on implementation resources.

Clinical practice guidelines and consensus statements for antenatal oral healthcare: An assessment of their methodological quality and content of recommendations

OBJECTIVES

To review the content of recommendations within antenatal oral healthcare guidance documents and appraise the quality of their methodology to inform areas of development, clinical practice, and research focus.

METHOD

A systematic search of five electronic databases, Google search engine, and databases from relevant professional and guideline development groups published in English, developed countries, and between 2010 and 2020 was undertaken to identify guidance documents related to antenatal oral healthcare. Quality of documents was appraised using the Appraisal of Guidelines Research and Evaluation II tool, and a 3-step quality cut-off value was used. Inductive thematic analysis was employed to categories discreet recommendations into themes.

RESULTS

Six guidelines and one consensus statement were analysed. Two documents developed within Australia scored ≥60% across five of the six domains of the quality appraisal tool and were recommended for use. Four documents (developed in the United States and Canada) were recommended for use with modifications, whilst one document (developed in Europe) was not recommended. A total of 98 discreet recommendations were identified and demonstrated considerable unanimity but differed in scope and level of information. The main content and number of recommendations were inductively categorised within the following clinical practice points: risk factor assessments (n = 2), screening and assessment (n = 10), pre-pregnancy care (referral, n = 1), antenatal care (health education and advice, n = 14; management of nausea and vomiting, n = 7; referral, n = 2), postnatal care (health education and advice, n = 1; anticipatory guidance, n = 6), documentation (n = 4), coordinated care (n = 4), capacity building (n = 6), and community engagement (n = 1).

CONCLUSION

The methodological rigour of included guidance documents revealed areas of strengths and limitations and posit areas for improvement. Further research could centre on adapting antenatal oral healthcare guidelines and consensus statements to local contexts. More high-quality studies examining interventions within antenatal oral healthcare are needed to support the development of recommendations.

Clinical Practice Guidelines That Address Physical Activity and Exercise During Pregnancy: A Systematic Review

INTRODUCTION

Physical activity is recommended for healthy pregnant women. Clinical practice guidelines vary with regard to initial time, duration, or type of physical activity that is recommended, which is confusing for health care professionals and pregnant women alike. This study aimed to appraise clinical guidelines relevant to physical activity for pregnant women and summarize consensus and discrepant recommendations.

METHODS

The websites of organizations that develop guidelines including those of obstetricians, gynecologists, midwives, and medical sports associations were searched in addition to 9 literature databases for the period from January 2010 through November 2020. Guidelines were included if they were published in a journal or on a website in English or Chinese; were labeled as a recommendation guideline, position paper, practice parameter, or consensus statement; and addressed physical activity for pregnant women. Two authors independently extracted recommendations. Four reviewers independently assessed guideline quality using the AGREE II instrument.

RESULTS

Thirteen guidelines met the inclusion criteria. The guidelines developed by the World Health Organization and the National Institute for Health and Clinical Excellence were deemed to have the highest methodological quality. The recommendations were consistent with regard to recommended exercise duration, frequency, intensity, and type of physical activity. The main discrepant recommendations included best tools for conducting pre-exercise screening and assessing intensity of exertion. Guidelines also differ on optimal heart rate during exercise, when in pregnancy to initiate an exercise regimen, and duration or frequency of strengthening exercises.

DISCUSSION

Of the 13 guidelines, 2 were found to have good methodological quality. The recommendations that were consistent across the differing guidelines can support health care providers in counseling women about physical activity during pregnancy. The recommendations that are discrepant among these guidelines may contribute to confusion and a reluctance to recommend exercise during pregnancy. Research is needed to clarify discrepant recommendations.

Systematic review of diabetic eye disease practice guidelines: more applicability, transparency and development rigor are needed

OBJECTIVES

To assess the quality of diabetic eye disease clinical practice guidelines.

STUDY DESIGN AND SETTING

A systematic search of diabetic eye disease guidelines was conducted on six online databases and guideline repositories. Four reviewers independently rated quality using the Appraisal of Guidelines, Research, and Evaluation (AGREE II) instrument. Aggregate scores (%) for six domains and overall quality assessment were calculated. A "good quality" guideline was one with ≥60% score for "rigor of development" and in at least two other domains.

RESULTS

Eighteen guidelines met the inclusion criteria, of which 13 were evidence-based guidelines (involved systematic search and grading of evidence). The median scores (interquartile range (IQR)) for "scope and purpose," "stakeholder involvement," "rigor of development," "clarity of presentation," "applicability" and "editorial independence" were 73.6% (54.2%-80.6%), 48.6% (29.2%-71.5%), 60.2% (30.9%-78.1%), 86.6% (76.7%-94.4%), 28.6% (18.0%-37.8%) and 60.2% (30.9%-78.1%), respectively. The median overall score (out of 7) of all guidelines was 5.1 (IQR: 3.7-5.8). Evidence-based guidelines scored significantly higher compared to expert-consensus guidelines. Half (n = 9) of the guidelines (all evidence-based) were of "good quality."

CONCLUSION

A wide variation in methodological quality exists among diabetic eyecare guidelines, with nine demonstrating "good quality." Future iterations of guidelines could improve by appropriately engaging stakeholders, following a rigorous development process, including support for application in clinical practice and ensuring editorial transparency.

What Determines the Quality of Rehabilitation Clinical Practice Guidelines?: An Overview Study

OBJECTIVE

The aim of the study was to determine what factors determine the quality of rehabilitation clinical practice guidelines.

DESIGN

Six databases were searched for articles that had applied the Appraisal of Guidelines for Research & Evaluation II quality assessment tool to rehabilitation clinical practice guidelines. The 573 deduplicated abstracts were independently screened by two authors, resulting in 81 articles, the full texts of which were independently screened by two authors for Appraisal of Guidelines for Research & Evaluation II application to rehabilitation clinical practice guidelines, resulting in a final selection of 40 reviews appraising 504 clinical practice guidelines. Data were extracted from these by one author and checked by a second. Data on each clinical practice guideline included the six Appraisal of Guidelines for Research & Evaluation II domain scores, as well as the two Appraisal of Guidelines for Research & Evaluation II global evaluations.

RESULTS

All six Appraisal of Guidelines for Research & Evaluation II domain scores were statistically significant predictors of overall clinical practice guideline quality rating; D3 (rigor of development) was the strongest and D1 (scope and purpose) the weakest (overall model P < 0.001, R2 = 0.53). Five of the six domain scores were significant predictors of the clinical practice guideline use recommendation, with D3 the strongest predictor and D5 (applicability) the weakest (overall model P < 0.001, pseudo R2 = 0.53).

CONCLUSIONS

Quality of rehabilitation clinical practice guidelines may be improved by addressing key domains such as rigor of development.

Critical Review of the Scientific Evidence and Recommendations in COVID-19 Management Guidelines

BACKGROUND

Little is known about the quality and potential impacts of the guidelines for coronavirus disease 2019 (COVID-19) management.

METHODS

We systematically searched PubMed, Web of Science, Cochrane Library, guideline databases, and specialty society websites to evaluate the quality of the retrieved guidelines using the Appraisal of Guidelines for Research and Evaluation II.

RESULTS

A total of 66 guidelines were identified. Only 24% were categorized as "recommended" for clinical practice. The 211 identified recommendations for COVID-19 management were classified into 4 topics: respiratory support (27), acute respiratory distress syndrome management (31), antiviral or immunomodulatory therapy (95), or other medicines (58). Only 63% and 56% of recommendations were supported by, respectively, assessment of the strength of the recommendations or level of evidence. There were notable discrepancies between the different guidelines regarding the recommendations on COVID-19 management.

CONCLUSIONS

The quality of the guidelines for COVID-19 management is heterogeneous, and the recommendations are rarely supported by evidence.

A Systematic Review of Clinical Practice Guidelines for Acute Procedural Pain on Neonates

OBJECTIVES

During hospitalization in neonatal intensive care units, neonates are exposed to many painful procedures within a stressful environment. To date, many evidence-based guidelines are available. However, the quality of these guidelines and their clinical application remain unclear. This systematic review aimed to determinie the quality of existing guidelines on the management of procedural pain in neonates and to summarize the recommendations provided by these guidelines.

MATERIALS AND METHODS

A structured search was conducted in Embase, PubMed, CINAHL, JBI database, and gray literature resources in November 2018 to identify relevant guidelines published from 2007 onward. Published guidelines and guidelines from complementary searches were included in the treating assessment or management of procedural pain in neonates. The methodological quality was analyzed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument.

RESULTS

A total of 1154 records were identified. After screening for eligibility, 17 guidelines were included in this review. Among these, 11 were identified to be high-quality guidelines. Besides the usual recommendations for pharmacological and nonpharmacological treatments, the inclusion of parents, improving interprofessional collaboration, and considering the setting were identified as important elements.

DISCUSSION

The results of this review show that there is a need to improve the methodological quality of guidelines for procedural pain in newborns. The set of recommendations for procedural pain prevention needs to involve not only pharmacological and nonpharmacological pain treatment but also parents and interprofessional collaboration. It is also essential to take into account facilitators, barriers, and the context to improve pain management.

Quality of pediatric clinical practice guidelines

BACKGROUND

There is a lack of a comprehensive evaluation for pediatric clinical practice guidelines (CPGs) published in recent years. Here, we assessed the quality of pediatric CPGs, considering factors that might affect their quality. The aim of the study is to promote a more coherent development and application of CPGs.

METHODS

Pediatric CPGs published in PubMed, MedLive, Guidelines International Network, National Institute for Health and Care Excellence, and World Health Organization between 2017 and 2019 were searched and collected. Paired researchers conducted screening, data extraction, and quality assessment using the Appraisal of Guidelines for Research and Evaluation II (AGREE II). Linear regression analysis determined the factors affecting CPGs' quality.

RESULTS

The study included a total of 216 CPGs, which achieved a mean score of 4.26 out of 7 points (60.86%) in the AGREE II assessment. Only 6.48% of the CPGs reached the "recommend" level. The remaining 69.91% should have been modified before recommendation, while the other 23.61% did not reach the recommended level at all. The overall quality of recent pediatric CPGs was higher than previously, and the proportion of CPGs with low-quality decreased over time. However, there were still too few CPGs that reached a high-quality level. The "applicability" and "rigor of development" domains had generally low scores. CPGs formulated by developing countries or regions, those that are not under an organizations or groups responsibility, and those that used non-evidence-based methods were found to be associated with poorer quality in different domains as independent or combinational factors.

CONCLUSIONS

The quality of pediatric CPGs still needs to be improved. Specifically, a quality control before applying new CPGs should be essential to ensure their quality and applicability.

Appraisal of the diagnostic procedures of acute pancreatitis in the guidelines

OBJECTIVES

The purpose of this study was to comprehensively assess the heterogeneity of procedures in the diagnostic guidelines for acute pancreatitis and to identify gaps limiting knowledge in diagnosing this disease.

METHODS

A systematic search of a number of databases was performed to determine the guidelines for the diagnosis of acute pancreatitis in patients with severe pancreatitis. The guidelines for the diagnosis of severe pancreatitis were evaluated by AGREE II. The Measurement Scale of Rate of Agreement (MSRA) was used to assess the guidelines (2015-2020) and extract evidence supporting these recommendations for analysis.

RESULTS

Seven diagnostic guidelines for acute pancreatitis were included. Only the 2019 WSES Guidelines for the Management of Severe Acute Pancreatitis and the Japanese Guidelines for the Management of Acute Pancreatitis: Japanese Guidelines 2015 had a total score of more than 60%, which is worthy of clinical recommendation. The average scores of the Scope and Purpose domain and the Clarity and Expression domain were the highest at 71.62% and 75.59%, respectively, while the average score of the Applicability area was the lowest at 16.67%. The included guidelines were further analyzed to determine the heterogeneity of the diagnosis of acute pancreatitis. The main reasons for the heterogeneity were the citation of low-quality evidence, the presence of far too many indicators for the classification of acute pancreatitis, unclear depictions of the standard, and poorly comprehensive recommendations for the diagnosis of the aetiology in the primary diagnosis of acute pancreatitis, the severity classification, the aetiological diagnosis, and the diagnosis of comorbidities.

CONCLUSIONS

The quality of different diagnostic guidelines for severe pancreatitis is uneven. The recommendations are largely based on low-quality evidence, and the guidelines still have much room for improvement to reach a high level of quality. The diagnostic procedures for acute pancreatitis vary widely in different guidelines. There are large differences between them, and resolving the abovementioned reasons would be a very wise choice for guideline developers to revise and upgrade the guidelines in the future.

Quality of 2019 American optometric association clinical practice guideline for diabetic eye care

PURPOSE

The 2019 American Optometric Association (AOA) clinical practice guideline intends to assist optometrists in providing evidence-based eye care for people with diabetes. This technical report evaluated the methodological and reporting quality of the guideline.

METHODS

Four independent reviewers appraised the 2014 and 2019 versions of the AOA's guideline using the AGREE II instrument. Average scaled scores across the six domains of the AGREE II and an overall independent score were calculated based on the formula provided.

RESULTS

The 2019 guideline scored high (range: 75-93%) in all domains except for the domain of applicability (34%). In the domain of rigour of development, significant improvements were noted in the 2019 guideline (median score: 7.0, interquartile range (IQR): 6.0-7.0) compared to the 2014 guideline (median: 5.0, IQR: 4.0-6.0) (p < 0.0001). The appraisal of the guideline also identified room for further improvements, especially in relation to implementing the guideline.

CONCLUSION

The overall and domain specific quality of the AOA 2019 guideline was high, however, improvement in its applicability domain is required. The findings of this study will aid uptake of the guideline and inform improvement efforts for other international optometric guidelines.

Assessment of the quality of recommendations from 161 clinical practice guidelines using the Appraisal of Guidelines for Research and Evaluation-Recommendations Excellence (AGREE-REX) instrument shows there is room for improvement

Objective

To assess the quality of recommendations from 161 clinical practice guidelines (CPGs) using AGREE-REX-D (Appraisal of Guidelines REsearch and Evaluation-Recommendations Excellence Draft).

Design

Cross-sectional study

Setting

International CPG community.

Participants

Three hundred twenty-two international CPG developers, users, and researchers.

Intervention

Participants were assigned to appraise one of 161 CPGs selected for the study using the AGREE-REX-D tool

Main outcome measures

AGREE-REX-D scores of 161 CPGs (7-point scale, maximum 7).

Results

Recommendations from 161 CPGs were appraised by 322 participants using the AGREE-REX-D. CPGs were developed by 67 different organizations. The total overall average score of the CPG recommendations was 4.23 (standard deviation (SD) = 1.14). AGREE-REX-D items that scored the highest were (mean; SD): evidence (5.51; 1.14), clinical relevance (5.95; SD 0.8), and patients/population relevance (4.87; SD 1.33), while the lowest scores were observed for the policy values (3.44; SD 1.53), local applicability (3,56; SD 1.47), and resources, tools, and capacity (3.49; SD 1.44) items. CPGs developed by government-supported organizations and developed in the UK and Canada had significantly higher recommendation quality scores with the AGREE-REX-D tool (p < 0.05) than their comparators.

Conclusions

We found that there is significant room for improvement of some CPGs such as the considerations of patient/population values, policy values, local applicability and resources, tools, and capacity. These findings may be considered a baseline upon which to measure future improvements in the quality of CPGs.

Methodological Quality of Physical Therapy Guidelines and Their Suitability for Adaptation: A Scoping Review

OBJECTIVE

Clinical practice guidelines (CPGs) can be characterized to the extent that they specifically address physical therapists and mainly contain recommendations for physical therapist interventions. The primary aim of this study was to identify existing physical therapy CPGs regardless of medical condition, with a secondary aim of assessing their methodological quality to determine whether they are potentially suitable for adaptation.

METHODS

Systematic searches of the Medline and Physiotherapy Evidence Database were performed (August 2019), and the websites of World Confederation for Physical Therapy members were screened (September 2019). Only CPGs published in German or English were included. Two independent reviewers screened records according to previously defined inclusion criteria. Information was extracted regarding country of origin, year of publication, and clinical subject area addressed. Four independent reviewers assessed the quality of physical therapy CPGs using the Appraisal of Guidelines Research and Evaluation instrument. A descriptive data analysis was performed.

RESULTS

Thirty-five CPGs met the inclusion criteria; 46% (16/35) of the included CPGs were from the United States, and 31% (11/35) were from the Netherlands. Assessment using the Appraisal of Guidelines Research and Evaluation tool resulted in the following domain scores, presented as median percentage (interquartile range): domain 1 (scope and purpose), 76 (63-92); domain 2 (stakeholder involvement), 63 (55-76); domain 3 (rigor of development), 67 (53-75); domain 4 (clarity of presentation), 74 (67-77); domain 5 (applicability), 44 (30-57); and domain 6 (editorial independence), 52 (35-66).

CONCLUSIONS

In general, the methodological quality of the included CPGs was moderate to good. Possibilities of adapting recommendations from existing CPGs should be considered with the development of new physical therapy CPGs.

IMPACT STATEMENT

This study can raise awareness of existing physical therapy CPGs and can support their application by physical therapists. Further, the study can support decisions on adapting existing CPGs with the planning of new physical therapy CPGs.

Development and Validation of a Tool to Assess the Quality of Clinical Practice Guideline Recommendations

Importance

Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined.

Objective

To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations.

Design, Setting, and Participants

A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users.

Exposures

Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey.

Main Outcomes and Measures

Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach α, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters.

Results

A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach α = 0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r = 0.81) and the clinical credibility score (r = 0.76).

Conclusions and Relevance

This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.

Quality assessment of kidney cancer clinical practice guidelines using AGREE II instrument: A critical review

BACKGROUND

Evidence-based guidelines are expected to provide clinicians with explicit recommendations on how to manage health conditions and bridge the gap between research and clinical practice. However, the existing practice guidelines(CPGs) vary in quality. This study aimed to evaluate the quality of CPGs of kidney cancer.

METHODS

We systematically searched PubMed, Embase, China Biology Medicine disc, and relevant guideline websites from their inception to April, 2018. We identified CGPs that provided recommendations on kidney cancer; 4 independent reviewers assessed the eligible CGPs using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. The consistency of evaluations was calculated using intraclass correlation coefficients (ICC).

RESULTS

A total of 13 kidney cancer CGPs were included. The mean scores for each AGREEII domain were as follows: scope and purpose-76.9%; clarity and presentation-76.4%; stakeholder involvement-62.8%; rigor of development-58.7%; editorial independence-53.7%; and applicability-49.4%. Two CPGs were rated as "recommended"; 8 as "recommended with modifications"; and 3 as "not recommended." Seven grading systems were used by kidney cancer CGPs to rate the level of evidence and the strength of recommendation.

CONCLUSIONS

Overall, the quality of CPGs of kidney cancer is suboptimal. AGREE II assessment results highlight the need to improve CPG development processes, editorial independence, and applicability in this field. It is necessary to develop a standardized grading system to provide clear information about the level of evidence and the strength of recommendation for future kidney cancer CGPs.

Diagnosis and treatment for hyperuricemia and gout: a systematic review of clinical practice guidelines and consensus statements

OBJECTIVES

Despite the publication of hundreds of trials on gout and hyperuricemia, management of these conditions remains suboptimal. We aimed to assess the quality and consistency of guidance documents for gout and hyperuricemia.

DESIGN

Systematic review and quality assessment using the appraisal of guidelines for research and evaluation (AGREE) II methodology.

DATA SOURCES

PubMed and EMBASE (27 October 2016), two Chinese academic databases, eight guideline databases, and Google and Google scholar (July 2017).

ELIGIBILITY CRITERIA

We included the latest version of international and national/regional clinical practice guidelines and consensus statements for diagnosis and/or treatment of hyperuricemia and gout, published in English or Chinese.

DATA EXTRACTION AND SYNTHESIS

Two reviewers independently screened searched items and extracted data. Four reviewers independently scored documents using AGREE II. Recommendations from all documents were tabulated and visualised in a coloured grid.

RESULTS

Twenty-four guidance documents (16 clinical practice guidelines and 8 consensus statements) published between 2003 and 2017 were included. Included documents performed well in the domains of scope and purpose (median 85.4%, range 66.7%-100.0%) and clarity of presentation (median 79.2%, range 48.6%-98.6%), but unsatisfactory in applicability (median 10.9%, range 0.0%-66.7%) and editorial independence (median 28.1%, range 0.0%-83.3%). The 2017 British Society of Rheumatology guideline received the highest scores. Recommendations were concordant on the target serum uric acid level for long-term control, on some indications for urate-lowering therapy (ULT), and on the first-line drugs for ULT and for acute attack. Substantially inconsistent recommendations were provided for many items, especially for the timing of initiation of ULT and for treatment for asymptomatic hyperuricemia.

CONCLUSIONS

Methodological quality needs improvement in guidance documents on gout and hyperuricemia. Evidence for certain clinical questions is lacking, despite numerous trials in this field. Promoting standard guidance development methods and synthesising high-quality clinical evidence are potential approaches to reduce recommendation inconsistencies.

PROSPERO REGISTRATION NUMBER

CRD42016046104.

Current guidelines on the management of gestational diabetes mellitus: a content analysis and appraisal

BACKGROUND

Despite many guidelines for the management of gestational diabetes available internationally, little work has been done to summarize and assess the content of existing guidelines. A paucity of analysis guidelines within in a unified system may be one explanatory factor. So this study aims to analyze and evaluate the contents of all available guidelines for the management of gestational diabetes.

METHOD

Relevant clinical guidelines were collected through a search of relevant guideline websites and databases (PubMed, Web of Science, Embase, etc.). Fourteen guidelines were identified, and each guideline was assessed for quality using the Appraisal of Guidelines for Research & Evaluation (AGREE) II instrument. Two independent reviewers extracted guideline recommendations using a "recommendation matrix" through which basic guideline information and consistency between search strategy and selection of evidence, between selected evidence and interpretation, and between interpretation and resulting recommendations were analyzed.

RESULTS

Fourteen documents were analyzed, and a total of 361 original recommendations for gestational diabetes mellitus (GDM) management were assessed. In all guidelines included, the recommendations were developed in five domains, namely, diagnosis of GDM, prenatal care, intrapartum care, neonatal care and postpartum care. Different guidelines appeared to have significant discrepancy in consistency of guideline content, but overall, there was consistency between search strategy and selection of evidence, between selected evidence and interpretation, and between interpretation and resulting recommendations (scilicet 49.31, 57.20 and 58.17%, respectively).

CONCLUSION

Although commonality in most recommendations existed, there were still some discrepancies between guidelines. Consistency of guidelines on the management of GDM in pregnancy is highly variable and needs to be improved.

Factors Associated With High-Quality Guidelines for the Pharmacologic Management of Chronic Diseases in Primary Care: A Systematic Review

IMPORTANCE

As the rate of publication of new and sometimes conflicting medical research increases, clinicians rely heavily on clinical practice guidelines (CPGs) to inform practice. However, CPGs are of widely variable quality, and there are no existing objective measures to rate the quality of CPGs.

OBJECTIVE

To systematically assess 421 CPGs for the management of common noncommunicable diseases in primary care using the validated Appraisal of Guidelines for Research and Evaluation Instrument, version II (AGREE-II) tool and elucidate the factors associated with quality of CPGs.

EVIDENCE REVIEW

MEDLINE, Embase, the Cochrane Library, and 12 websites for CPGs were searched for CPGs for the management of common noncommunicable diseases in primary care published between January 1, 2011, and August 30, 2017. The assessment of the quality of CPGs was performed by 3 appraisers using the 6 domains of the AGREE-II instrument. A multiple logistic regression was performed to identify factors associated with quality of CPGs.

FINDINGS

Of the 421 CPGs reviewed, 23.5% (99) were classified as high quality. Among included guidelines, clarity of presentation (70%) and scope and purpose (61%) had the highest median AGREE-II scores. The domains with the lowest median scores were applicability (22%) and rigor of development (33%). Factors associated with high-quality CPGs included having more than 20 authors (odds ratio, 9.08; 95% CI, 3.35-24.62), development at governmental institutions (odds ratio, 10.38; 95% CI, 2.72-39.60), and reporting funding (odds ratio, 10.34; 95% CI, 4.77-22.39). Year of publication, region, guideline version, and scope were not associated with quality among included CPGs.

CONCLUSIONS AND RELEVANCE

Primary care professionals and policymakers should be aware that CPGs in primary care are of widely variable quality, with less than 25% of included CPGs rated as high quality. High-quality CPGs were associated with a higher number of authors, governmental institutions, and the report of funding. Region of origin was not associated with quality of CPGs, which suggests that the improvement of the quality of CPGs should be an international concern.

A critical review to traumatic brain injury clinical practice guidelines

The aim of this study was to assess the quality of clinical practice guidelines of traumatic brain injury (TBI) and investigate the evidence grading systems.A systematic search of relevant guideline websites and literature databases (including PubMed, NGC, SIGN, NICE, GIN, and Google) was undertaken from inception to May 2018 to identify and select TBI guidelines. Four independent reviewers assessed the eligible guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. The degree of agreement was evaluated with intraclass correlation coefficient (ICC).From 1802 records retrieved, 12 TBI guidelines were included. The mean scores for each AGREE II domain were as follows: scope and purpose (mean ± SD= 74.2 ± 9.09); stakeholder involvement (mean± SD= 54.6 ± 11.6); rigor of development (mean ± SD=70.1 ± 13.6); clarity and presentation (mean ± SD=78.4 ± 11.5); applicability (mean ± SD= 60.5 ± 13.6); and editorial independence (mean ± SD=61.7 ± 14.8). Ten guidelines were rated as "recommended." The ICC values ranged from 0.73 to 0.95. Seven grading systems were used by TBI guidelines to rate the level of evidence and the strength of recommendation.Most TBI guidelines got a high-quality rating, whereas a standardized grading system should be adopted to provide clear information about the level of evidence and strength of recommendation in TBI guidelines.

Systematic reviews of clinical practice guidelines: a methodological guide

OBJECTIVES

Systematic reviews (SRs) of clinical practice guidelines (CPGs) are unique knowledge syntheses that require tailored approaches to, and greater subjectivity in, design and execution compared with other SRs in clinical epidemiology. We provide review authors structured direction on how to design and conduct methodologically rigorous SRs of CPGs.

STUDY DESIGN AND SETTING

A guidance paper outlining suggested methodology for conducting all stages of an SR of CPGs. We present concrete examples of approaches used by published reviews, including a case exemplar demonstrating how this methodology was applied to our own SR of CPGs.

RESULTS

Review context and the unique characteristics of CPGs as research syntheses or clinical guidance statements must be considered in all aspects of review design and conduct. Researchers should develop a "PICAR" statement to help form and focus on the research question(s) and eligibility criteria, assess CPG quality using a validated appraisal tool, and extract, analyze, and summarize data in a way that is cogent and transparent.

CONCLUSION

SRs of CPGs can be used to systematically identify, assess, and summarize the current state of guidance on a clinical topic. These types of reviews often require methodological tailoring to ensure that their objectives and timelines are effectively and efficiently addressed; however, they should all meet the criteria for an SR, follow a rigorous methodological approach, and adhere to transparent reporting practices.

A systematic review of clinical practice guidelines on the use of low molecular weight heparin and fondaparinux for the treatment and prevention of venous thromboembolism: Implications for research and policy decision-making

Background

Venous thromboembolism (VTE) is a major global cause of morbidity and mortality. Low molecular weight heparin (LMWH) and fondaparinux (FDP) are frequently used to treat and prevent VTE and have a variety of safety and practical advantages over other anticoagulants, including use in outpatient settings. These medications are commonly listed on drug formularies, which act as a gateway for health plan prescription coverage by outlining the circumstances under which patients will be covered for specific drugs and drug products. Because patient access to medications is impacted by the nature of their listing on formularies, they must be rigorously reviewed and modernized as new evidence emerges.

Methods

As part of a broader drug class review team, we completed a systematic review of clinical practice guidelines to determine whether the recommendations they reported aligned with the indications listed for the coverage of LMWH and FDP in an outpatient drug formulary. Guideline quality was assessed using the Appraisal of Guidelines for Research & Evaluation (AGREE) II tool. Recommendation matrices were used to systematically compare, categorize, and summarize included recommendations.

Results

Twenty-seven guidelines were included from which 168 eligible recommendations were identified. Generally, AGREE II domains were adequately addressed; however, domain five (applicability) was poorly addressed. Most recommendations were based on moderate- to low-quality/limited evidence and reported on the use of LMWHs generally; few reported on specific agents.

Conclusions

Our findings contributed to the recommendation that the formulary listing for LMWH and FDP be streamlined to include coverage for specific outpatient indications. The paucity of available evidence on the comparative efficacy of specific LMWH agents against each other and FDP limited agent-specific listing recommendations, highlighting the need for high-quality comparative studies on this topic.

Quality assessment of cancer cachexia clinical practice guidelines

OBJECTIVES

The aim of this study was to investigate the quality of clinical practice guidelines of cancer cachexia and identify gaps limiting knowledge.

METHODS

A systematic search of relevant guideline websites and literature databases (including PubMed, NCCN, NGC, SIGN, NICE, and google) was undertaken from inception to March 2017 to identify and select clinical guidelines related to cancer cachexia. Four independent reviewers assessed the eligible guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. Agreement among reviewers of the guidelines was measured by using intra-class correlation coefficient (ICC). The number of recommendations, strength of recommendation, and levels of evidence were determined.

RESULTS

Nine cancer cachexia guidelines published from 2006 to 2017 were identified. An overall high degree of agreement among reviewers to each domain was observed (ICC ranged from 0.75 to 0.91). The median scores and range for each AGREE II domain were as follows: (i) scope and purpose (median = 61.1%, range: 13.9% to 80.7%); (ii) stakeholder involvement (median = 26.4%, range: 8.3% to 81.9%); (iii) rigour of development (median = 35.9%, range: 3.6% to 84.4%); (iv) clarity and presentation (median = 56.9%, range: 30.6% to 76.4%); (v) applicability (median = 19.8%, range: 0% to 77.1%) and (vi) editorial independence (median = 27.1%, range: 0% to 85.4%). Two cancer cachexia guidelines (ESPEN, 2017 and University of Queensland, 2013) scored higher on all domains and were classified as recommended for clinical practice, among which, one was developed by European Society for Parenteral and Enteral Nutrition and European Partnership for Action Against Cancer, and the other was developed by University of Queensland. In addition, more than a half recommendations were based on nonrandomized studies (Level C, 50.0%) and expert opinion (Level D, 8.2%).

CONCLUSIONS

The quality of cancer cachexia guidelines was highly heterogeneous among different domains even within the same guideline. There is significant room for improvement to develop high quality cancer cachexia guidelines, which urgently warrants first-class research to minimize the vital gaps in the evidence for formulation of cancer cachexia guidelines.

A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal

Background

This study aimed to develop the clinical practice guidelines applicability evaluation (CPGAE-V1.0) scale and to evaluate its validity and reliability.

Methods

One hundred fifty assessors were invited to rate two rounds of importance scoring of the applicability indicators by using the 5-point Likert scale. Approved indicators formed the CPGAE-V1.0 scale, consisting of 19 items, arranged into 4 domains. We enrolled eligible clinicians from 8 institutions to evaluate 9 clinical practice guidelines using the CPGAE-V1.0 scale. Content validity, construct validity, internal reliability, intra-rater reliability, and responsiveness were analyzed.

Results

A total of 220 clinicians participated, and the response rate was 98.6% (217/220). The CPGAE-V1.0 scale had favorable content validity. The four-factor model produced acceptable fit indices. The scale had an excellent internal consistency and item discrimination. It could identify the degree of applicability of the different dimensions between different guidelines. In all domains, 77.8% (7/9) of CPGs in the minimum-scoring domain were concentrated in the “coordination of support” domain.

Conclusions

The CPGAE-V1.0 scale is a valid and reliable instrument for measuring the applicability of CPG.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0746-5) contains supplementary material, which is available to authorized users.

Evaluation of the current guidelines for resection of hepatocellular carcinoma using the Appraisal of Guidelines for Research and Evaluation II instrument

BACKGROUND & AIMS

Numerous guidelines for the management of hepatocellular carcinoma (HCC) have been developed. The Appraisal of Guidelines for Research & Evaluation (AGREE II) is the only validated instrument to assess the methodological quality of guidelines. We aim to appraise the methodological quality of existing guidelines for the resection of HCC using the AGREE II instrument.

METHODS

Cochrane, Medline, Google Scholar and Embase were searched using both PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) criteria and free text. The assessment of the included clinical practice guidelines and consensuses were performed using the AGREE II instrument, version 2013. Guidelines with a score ⩾80% for the overall appraisal item were considered as applicable without modifications.

RESULTS

Literature searches identified 22 clinical practice guidelines. Five out of 22 guidelines passed the 70% mark on overall assessment, 11 out of 22 had shortcomings on indications, contraindications, side effects, key recommendations, technical aspects, transparency and health economics. Ten of 22 scored below the 50% mark showing that the guideline had low methodological and overall quality. Only 3/22 clinical practice guidelines were considered applicable without modifications.

CONCLUSIONS

The methodological quality of guidelines for the surgical management of HCC is generally poor. Future guideline development should be informed by the use of the AGREE II instrument. Guidelines based upon high quality evidence could improve stratification of patients and individualized treatment strategies. Lay summary: The methodology of clinical practice guidelines for resection for hepatocellular carcinoma (HCC) evaluated with the Appraisal of Guidelines for Research & Evaluation (AGREE II) instrument is generally poor. However, there are some clinical practice guidelines that are based upon higher quality evidence and can form the framework within which patients with HCC can be selected for surgical resection. Future guideline development should be informed by the use of the AGREE II instrument.

Could accreditation bodies facilitate the implementation of medical guidelines in laboratories?

Several studies have shown that recommendations related to how laboratory testing should be performed and results interpreted are limited in medical guidelines and that the uptake and implementation of the recommendations that are available need improvement. The EFLM/UEMS Working Group on Guidelines conducted a survey amongst the national societies for clinical chemistry in Europe regarding development of laboratory-related guidelines. The results showed that most countries have guidelines that are specifically related to laboratory testing; however, not all countries have a formal procedure for accepting such guidelines and few countries have guideline committees. Based on this, the EFLM/UEMS Working Group on Guidelines conclude that there is still room for improvement regarding these processes in Europe and raise the question if the accreditation bodies could be a facilitator for an improvement.

International variations in clinical practice guidelines for palliative sedation: a systematic review

OBJECTIVES

Palliative sedation is a highly debated medical practice, particularly regarding its proper use in end-of-life care. Worldwide, guidelines are used to standardise care and regulate this practice. In this review, we identify and compare national/regional clinical practice guidelines on palliative sedation against the European Association for Palliative Care (EAPC) palliative sedation Framework and assess the developmental quality of these guidelines using the Appraisal Guideline Research and Evaluation (AGREE II) instrument.

METHODS

Using the PRISMA criteria, we searched multiple databases (PubMed, CancerLit, CINAHL, Cochrane Library, NHS Evidence and Google Scholar) for relevant guidelines, and selected those written in English, Dutch and Italian; published between January 2000 and March 2016.

RESULTS

Of 264 hits, 13 guidelines-Belgium, Canada (3), Ireland, Italy, Japan, the Netherlands, Norway, Spain, Europe, and USA (2) were selected. 8 contained at least 9/10 recommendations published in the EAPC Framework; 9 recommended 'pre-emptive discussion of the potential role of sedation in end-of-life care'; 9 recommended 'nutrition/hydration while performing sedation' and 8 acknowledged the need to 'care for the medical team'. There were striking differences in terminologies used and in life expectancy preceding the practice. Selected guidelines were conceptually similar, comparing closely to the EAPC Framework recommendations, albeit with notable variations.

CONCLUSIONS

Based on AGREE II, 3 guidelines achieved top scores and could therefore be recommended for use in this context. Also, domains 'scope and purpose' and 'editorial independence' ranked highest and lowest, respectively-underscoring the importance of good reportage at the developmental stage.

AAA (2010) CAPD clinical practice guidelines: need for an update

OBJECTIVE

Review and critique of the clinical value of the AAA CAPD guidance document in light of criteria for credible and useful guidance documents, as discussed by Field and Lohr.

DESIGN

A qualitative review of the of the AAA CAPD guidelines using a framework by Field and Lohr to assess their relative value in supporting the assessment and management of CAPD referrals.

STUDY SAMPLE

Relevant literature available through electronic search tools and published texts were used along with the AAA CAPD guidance document and the chapter by Field and Lohr.

RESULTS

The AAA document does not meet many of the key requirements discussed by Field and Lohr. It does not reflect the current literature, fails to help clinicians understand for whom auditory processing testing and intervention would be most useful, includes contradictory suggestions which reduce clarity and appears to avoid conclusions that might cast the CAPD construct in a negative light. It also does not include input from diverse affected groups. All of these reduce the document's credibility.

CONCLUSIONS

The AAA CAPD guidance document will need to be updated and re-conceptualised in order to provide meaningful guidance for clinicians.

A critical appraisal of chronic kidney disease mineral and bone disorders clinical practice guidelines using the AGREE II instrument

BACKGROUND

Patients with chronic kidney disease mineral and bone disorders (CKD-MBD) suffer high rates of morbidity and mortality, in particular related to bone and cardiovascular outcomes. The management of CKD-MBD remains challenging. The objective of this systematic survey is to critically appraise clinical practice guidelines (CPGs) addressing CKD-MBD.

METHODS/DESIGN

Data sources included MEDLINE, EMBASE, the National Guideline Clearinghouse, Guideline International Network and Turning Research into Practice up to May 2016. Teams of two reviewers, independently and in duplicate, screened titles and abstracts and potentially eligible full text reports to determine eligibility and subsequently appraised the guidelines using the Advancing Guideline Development, Reporting and Evaluation in Health Care instrument II (AGREE).

RESULTS

Sixteen CPGs published from 2003 to 2015 addressing the diagnosis and management of CKD-MBD in adult patients (11 English, two Spanish, one Italian, one Portuguese and one Slovak) proved eligible. The National Institute for Health and Care Excellence guideline performed best with respect to AGREE II criteria; only three other CPGs warranted high scores on all domains. All other guidelines received scores of under 60% on one or more domains. Major discrepancies in recommendations were not, however, present, and we found no association between quality of CPGs which was not associated with resulting recommendations.

CONCLUSIONS

Most guidelines assessing CKD-MBD suffer from serious shortcomings using AGREE criteria although limitations with respect to AGREE criteria do not necessarily lead to inappropriate recommendations.

Contemporaneous International Asthma Guidelines Present Differing Recommendations: An Analysis

Background. Several international groups develop asthma guidelines. Conflicting recommendations across guidelines have been described in several disease areas and may contribute to practice variability. Accordingly, we compared the latest Canadian Thoracic Society (CTS) asthma guideline with contemporaneous international asthma guidelines to evaluate conflicting recommendations and their causes. Methods. We identified the latest CTS asthma guideline update (2012) and the following societies which also updated their guidelines in 2012: the British Thoracic Society and Scottish Intercollegiate Guidelines Network and the Global Initiative for Asthma. We compared these three guidelines on (1) key methodological factors and (2) adult pharmacotherapy recommendations. Results. Methods used and documentation provided for literature search strategy and dates, evidence synthesis, outcomes considered, evidence appraisal, and recommendation formulation varied between guidelines. Criteria used to define suboptimal asthma control varied widely between guidelines. Inhaled corticosteroid dosing recommendations diverged, as did recommendations surrounding use of budesonide/formoterol as a reliever and controller and recommendations in the subsequent step. Conclusions. There are important differences between recommendations provided in contemporaneous asthma guidelines. Causes include differences in methods used for interpreting evidence and formulating recommendations. Adopting a common set of valid and explicit methods across international societies could harmonize recommendations and facilitate guideline implementation.

Validation and reliability of a guideline appraisal mini-checklist for daily practice use

BACKGROUND

The use of comprehensive instruments for guideline appraisal is time-consuming and requires highly qualified personnel. Since practicing physicians are generally busy, the rapid-assessment Mini-Checklist (MiChe) tool was developed to help them evaluate the quality and utility of guidelines quickly. The aim of this study was to validate the MiChe in comparison to the AGREE II instrument and to determine its reliability as a tool for guideline appraisal.

METHODS

Ten guidelines that are relevant to general practice and had been evaluated by 2 independent reviewers using AGREE II were assessed by 12 GPs using the MiChe. The strength of the correlation between average MiChe ratings and AGREE II total scores was estimated using Pearson's correlation coefficient. Inter-rater reliability for MiChe overall quality ratings and endorsements was determined using intra-class correlations (ICC) and Kendall's W for ordinal recommendations. To determine the GPs' satisfaction with the MiChe, mean scores for the ratings on five questions were computed using a six-point Likert scale.

RESULTS

The study showed a high level of agreement between MiChe and AGREE II in the quality rating of guidelines (Pearson's r = 0.872; P < 0.001). Inter-rater-reliability for overall MiChe ratings (ICC = 0.755; P < 0.001) and endorsements (Kendall's W = 0.73; P < 0.001) were high. The mean time required for guideline assessment was less than 15 min und user satisfaction was generally high.

CONCLUSIONS

The MiChe performed well in comparison to AGREE II and is suitable for the rapid evaluation of guideline quality and utility in practice.

TRIAL REGISTRATION

German Clinical Trials Register: DRKS00007480.

Quality Assessment of Clinical Practice Guidelines for Respiratory Diseases in China: A Systematic Appraisal

BACKGROUND

There has been a significant increase in the publication of clinical practice guidelines (CPGs) for respiratory diseases in China. However, little is known about the quality and potential impacts of these CPGs. Our objective was to critically evaluate the quality of Chinese CPGs for respiratory diseases that were published in peer-reviewed medical journals.

METHODS

A systematic search of scientific literature published between 1979 and 2013 was undertaken to identify and select CPGs that were related to respiratory diseases. Four Chinese databases (the Chinese Biomedical Literature database [CBM], the China National Knowledge Infrastructure [CNKI], the VIP database, and the WANFANG database) were used. The quality of eligible guidelines was assessed independently by four reviewers using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument. The overall agreement among reviewers was evaluated using an intraclass correlation coefficient.

RESULTS

A total of 109 guidelines published in 27 medical journals from 1979 to 2013 were evaluated. The overall agreement among reviewers was considered good (intraclass correlation coefficient, 0.838; 95% CI, 0.812-0.862). The scores of the six AGREE domains were low: 57.3% for scope and purpose (range, 4.2%-80.5%), 23.8% for stakeholder involvement (range, 2.8%-54.2%), 7.7% for rigor of development (range, 0%-27.1%), 59.8% for clarity and presentation (range, 22.2%-80.6%), 10.9% for applicability (range, 0%-22.9%), and 0.6% for editorial independence (range, 0%-16.7%). Scores for all guidelines were below 60%, and only three guidelines (2.8%) were recommended for clinical practice with modifications.

CONCLUSIONS

The quality of the guidelines was low, and stakeholder involvement, rigor of development, applicability, and editorial independence should be considered in the future development of CPGs for respiratory diseases in China.

[Cancer pain management: Systematic review and critical appraisal of clinical practice guidelines]

OBJECTIVE

Although several clinical practice guidelines have been developed in the last decades, cancer pain management is still deficient. The purpose of this work was to carry out a comprehensive and systematic literature review of current clinical practice guidelines on cancer pain management, and critically appraise their methodology and content in order to evaluate their quality and validity to cope with this public health issue.

MATERIALS AND METHODS

A systematic review was performed in the main databases, using English, French and Spanish as languages, from 2008 to 2013. Reporting and methodological quality was rated with the Appraisal of Guidelines, Research and Evaluation II (AGREE-II) tool, including an inter-rater reliability analysis. Guideline recommendations were extracted and classified into several categories and levels of evidence, aiming to analyse guidelines variability and evidence-based content comprehensiveness.

RESULTS

Six guidelines were included. A wide variability was found in both reporting and methodological quality of guidelines, as well as in the content and the level of evidence of their recommendations. The Scottish Intercollegiate Guidelines Network guideline was the best rated using AGREE-II, while the Sociedad Española de Oncología Médica guideline was the worst rated. The Ministry of Health Malaysia guideline was the most comprehensive, and the Scottish Intercollegiate Guidelines Network guideline was the second one.

CONCLUSIONS

The current guidelines on cancer pain management have limited quality and content. We recommend Ministry of Health Malaysia and Scottish Intercollegiate Guidelines Network guidelines, whilst Sociedad Española de Oncología Médica guideline still needs to improve.

Hypertensive disorders of pregnancy: a systematic review of international clinical practice guidelines

BACKGROUND

Clinical practice guidelines (CPGs) are developed to assist health care providers in decision-making. We systematically reviewed existing CPGs on the HDPs (hypertensive disorders of pregnancy) to inform clinical practice.

METHODOLOGY & PRINCIPAL FINDINGS

MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Health Technology Assessments, and Database of Abstracts of Reviews of Effects (Ovid interface), Grey Matters, Google Scholar, and personal records were searched for CPGs on the HDPs (Jan/03 to Nov/13) in English, French, Dutch, or German. Of 13 CPGs identified, three were multinational and three developed for community/midwifery use. Length varied from 3-1188 pages and three guidelines did not formulate recommendations. Eight different grading systems were identified for assessing evidence quality and recommendation strength. No guideline scored ≧80% on every domain of the AGREE II, a tool for assessing guideline methodological quality; two CPGs did so for 5/6 domains. Consistency was seen for (i) definitions of hypertension, proteinuria, chronic and gestational hypertension; (ii) pre-eclampsia prevention for women at increased risk: calcium when intake is low and low-dose aspirin, but not vitamins C and E or diuretics; (iii) antihypertensive treatment of severe hypertension; (iv) MgSO4 for eclampsia and severe pre-eclampsia; (v) antenatal corticosteroids at <34 wks when delivery is probable within 7 days; (vi) delivery for women with severe pre-eclampsia pre-viability or pre-eclampsia at term; and (vii) active management of the third stage of labour with oxytocin. Notable inconsistencies were in: (i) definitions of pre-eclampsia and severe pre-eclampsia; (ii) target BP for non-severe hypertension; (iii) timing of delivery for women with pre-eclampsia and severe pre-eclampsia; (iv) MgSO4 for non-severe pre-eclampsia, and (v) postpartum maternal monitoring.

CONCLUSIONS

Existing international HDP CPGs have areas of consistency with which clinicians and researchers can work to develop auditable standards, and areas of inconsistency that should be addressed by future research.