A critical appraisal of the quality of critical care pharmacotherapy clinical practice guidelines and their strength of recommendations

Quality of critical care clinical practice guidelines involving pharmacotherapy recommendations

DISCLAIMER

In an effort to expedite the publication of articles related to the COVID-19 pandemic, AJHP is posting these manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

To assess the quality of critical care clinical practice guidelines (CPGs) involving pharmacotherapy recommendations.

METHODS

A systematic electronic search was performed using PubMed, MEDLINE, and Embase for critical care CPGs published between 2012 and 2022 and involving pharmacotherapy recommendations. The Appraisal of Guidelines for Research & Evaluation II (AGREE II) instrument was employed to appraise CPG quality through independent assessment by 2 appraisers.

RESULTS

Twenty-one CPGs were evaluated. The number of recommendations in each guideline ranged from 2 to 250, with a total of 1,604 recommendations. The number of strong (vs weak) recommendations in each guideline ranged from 0 to 31, with a total of 116 strong recommendations, or 7.23% of the total number of recommendations. There was at least 1 pharmacist author for 9 (43%) of the guidelines. The AGREE II domains for which mean quality scores of evaluated guidelines were highest were scope and purpose (0.88; 95% CI, 0.85-0.92), rigor of development (0.80; 95% CI, 0.77-0.83), clarity of presentation (0.84; 95% CI, 0.81-0.87), and editorial independence (0.86; 95% CI, 0.79-0.94), while those for which mean scores were lowest were stakeholder involvement (0.69; 95% CI, 0.63-0.75) and applicability (0.49; 95% CI, 0.43-0.55). Involvement of a pharmacist in CPG development was associated with significantly higher scoring for stakeholder involvement (P = 0.0356).

CONCLUSION

Strong recommendations accounted for less than 10% of the recommendations in the evaluated CPGs. Moreover, there are concerns related to guideline applicability (ie, advice or tools for putting recommendations into practice) and stakeholder involvement (ie, inclusion of individuals from all relevant groups). It is important to involve pharmacists in CPGs with pharmacotherapy recommendations.

Prevention and screening recommendations in type 2 diabetes: Review and critical appraisal of clinical practice guidelines

The aim of this review was to identify and appraise guidelines reporting recommendations for the screening and prevention of type 2 diabetes. Five guidelines were included for analysis and all were endorsed by national or international organizations. All guidelines were recommended for practice with or without modifications for both prevention and screening. The overall appraisal scores ranged from 62.5 to 91.7 for prevention and 62.5-83.3 for screening. The highest scored domain was 'clarity of presentation' and the lowest was 'rigor of development'. Findings call for greater attention to rigor when formulating recommendations for prevention and screening of diabetes.

Critical appraisal of clinical practice guidelines for adult cancer patients with febrile neutropenia

OBJECTIVES

To critically appraise published international clinical practice guidelines (CPGs) for management of febrile neutropenia in adult patients with cancer and to determine opportunities for improved development and reporting.

METHODS

A literature search identified CPGs for adult cancer patients with febrile neutropenia. Four independent assessors evaluated each included CPG according to the Appraisal of Guidelines for Research and Evaluation II instrument. Standardized scores were calculated for each guideline and polled collectively. Reliability of assessment was determined using a two-way random model intraclass correlation coefficients.

KEY FINDINGS

Eight CPGs were independently evaluated by four assessors. Collectively, the highest scoring domain was editorial independence (83.3), followed by clarity of presentation (55.4), scope and purpose (53.4), stakeholder involvement (53.1), rigour of development (52.7) and applicability (47.8). Overall assessments ranged from 28.6 to 96.4 of 100 possible points. Three (37.5%) guidelines were recommended for use without alterations, two (25%) guidelines were recommended with alterations, and three (37.5%) guidelines were not recommended for implementation into practice. Reliability varied between guidelines with intraclass correlation coefficients ranging from 0.41 to 0.82.

CONCLUSIONS

Clinical practice guidelines for febrile neutropenia in adult patients with cancer were moderately rated with a 37.5% of guidelines being recommended for use in practice. Guideline developers should focus on improving CPG applicability and rigour in the development and reporting processes. Critical appraisal of guidelines should become a standard practice prior to implementation into clinical settings.

Clinical Practice Guidelines: A Primer on Development and Dissemination

Trustworthy clinical practice guidelines should be based on a systematic review of the literature, provide ratings of the quality of evidence and the strength of recommendations, consider patient values, and be developed by a multidisciplinary panel of experts. The quality of evidence reflects our certainty that the evidence warrants a particular action. Transforming evidence into a decision requires consideration of the quality of evidence, balance of benefits and harms, patients' values, available resources, feasibility of the intervention, acceptability by stakeholders, and effect on health equity. Empirical evidence shows that adherence to guidelines improves patient outcomes; however, adherence to guidelines is variable. Therefore, guidelines require active dissemination and innovative implementation strategies.

Critical appraisal of clinical practice guidelines in pediatric infectious diseases

BACKGROUND

There is a need to critically appraise clinical practice guidelines in order to ensure safe and effective practices are being implemented to optimize patient care. Appraising guidelines within one therapeutic area enable recommendations for improvement during guideline creation and dissemination.

OBJECTIVES

Study objectives were to systematically appraise selected published guidelines used in the treatment of pediatric infectious diseases and to make recommendations for improvement throughout the development and dissemination processes.

SETTING

The study occurred between collaborative academic and practice-based institutions located in Canada and Qatar.

METHODS

A literature search identified guidelines for management of pediatric infectious diseases from 1997 to 2013. Each guideline was appraised by four independent assessors, according to the appraisal of guidelines for research and evaluation II (AGREE II) instrument. Standardized domain scores were calculated for each guideline and pooled. Final endorsements for use in clinical practice were also determined. Inter-rater reliability was assessed using intraclass correlation coefficients.

MAIN OUTCOME MEASURE

Standardized domain scores according to the AGREE II instrument.

RESULTS

Twenty guidelines met inclusion criteria and were appraised. Pooled domain scores were: scope and purpose (69.9), stakeholder involvement (40.1), rigour of development (47.1), clarity of presentation (73.4), applicability (23.7), editorial independence (46.7), and overall assessment (55.8). Two (10%) guidelines were recommended for use without revision, 13 (65%) guidelines were recommended with modifications, and 5 (25%) guidelines were not recommended for implementation into practice. Inter-rater reliability was moderate to good with intra-class correlations of 0.65-0.93 per guideline.

CONCLUSION

The majority of appraised guidelines were moderately rated, with a 25% of guidelines not recommended for use. Strategies for improvement require the involvement of all key stakeholders (caregivers, patients, and allied health professionals), and consideration of facilitators, barriers and resource implications during implementation. Additionally, critical appraisal of guidelines should become standard practice prior to adoption into clinical settings.

Emergency department antimicrobial considerations in severe sepsis

Severe sepsis and septic shock are common problems in the emergency department patient population and require expert clinical skill by members of the emergency department team to maximize optimal patient outcomes. Although various guidelines have been developed for the management of these patients, issues around antimicrobial-related considerations in critically ill patients require further evidence-based attention. In this review article, important factors related to patient illness, microorganism, timing of antimicrobial administration, and source control are discussed.

Appraisal of clinical practice guidelines for ischemic stroke management in Chinese medicine with appraisal of guidelines for research and evaluation instrument: A systematic review

OBJECTIVE

To systematically review the clinical practice guidelines (CPGs) for ischemic stroke in Chinese medicine (CM) with the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument.

METHODS

CM CPGs for ischemic stroke were searched in 5 online databases and hand-searches in CPGrelated handbooks published from January 1990 to December 2012. The CPGs were categorized into evidence based (EB) guideline, consensus based with no explicit consideration of evidence based (CB-EB) guideline and consensus based (CB) guideline according to the development method. Three reviewers independently appraised the CPGs based on AGREE II instrument, and compared the CPGs' recommendations on CM pattern classification and treatment.

RESULTS

Five CM CPGs for ischemic stroke were identified and included. Among them, one CPG was EB guideline, two were CB guidelines and two were CB-EB guidelines. The quality score of the EB guideline was higher than those of the CB-EB and CB guidelines. Five CM patterns in the CPGs were recommended in the EB CPG. The comprehensive protocol of integrative Chinese and Western medicine recommended in the EB CPG was mostly recommended for ischemic stroke in the CPGs. The recommendations varied based on the CM patterns.

CONCLUSION

The quality of EB CPG was higher than those of CB and CB-EB CPGs in CM for ischemic stroke and integrative approaches were included in CPGs as major interventions.

Methodological quality of English-language genetic guidelines on hereditary breast-cancer screening and management: an evaluation using the AGREE instrument

BACKGROUND

We examined the methodological quality of guidelines on syndromes conferring genetic susceptibility to breast cancer.

METHODS

PubMed, EMBASE, and Google were searched for guidelines published up to October 2010. All guidelines in English were included. The Appraisal of Guidelines, Research and Evaluation (AGREE) instrument was used to assess the quality of the guidelines, and their reported evidence base was evaluated.

RESULTS

Thirteen guidelines were deemed eligible: seven had been developed by independent associations, and the other six had national/state endorsements. Four guidelines performed satisfactorily, achieving a score of greater than 50% in all six AGREE domains. Mean ± SD standardized scores for the six AGREE domains were: 90 ± 9% for 'scope and purpose', 51 ± 18% for 'stakeholder involvement', 55 ± 27% for 'rigour of development', 80 ± 11% for 'clarity and presentation', 37 ± 32% for 'applicability', and 47 ± 38% for 'editorial independence'. Ten of the thirteen guidelines were found to be based on research evidence.

CONCLUSIONS

Given the ethical implications and the high costs of genetic testing for hereditary breast cancer, guidelines on this topic should provide clear and evidence-based recommendations. Our analysis shows that there is scope for improving many aspects of the methodological quality of current guidelines. The AGREE instrument is a useful tool, and could be used profitably by guidelines developers to improve the quality of recommendations.

Self-monitoring of blood glucose in patients with diabetes who do not use insulin--are guidelines evidence-based?

AIMS

To evaluate if clinical practice guideline recommendations regarding self-monitoring of blood glucose in patients with diabetes not using insulin follow the principles of evidence-based medicine.

METHODS

After a search from 1999 to 2011, 18 clinical practice guidelines were included. Recommendations regarding self-monitoring of blood glucose were graded on a scale from one (strongly against self-monitoring) to four (strongly in favour of self-monitoring) and compared with the similarly graded conclusions of systematic reviews that were cited by the clinical practice guidelines. We also investigated how clinical practice guideline characteristics, for example funding sources, and quality of references cited could be related to the guideline recommendations.

RESULTS

The clinical practice guidelines cited in total 15 systematic reviews, 14 randomized controlled trials, 33 non-randomized controlled trials papers and 18 clinical practice guidelines or position statements. The clinical practice guideline recommendations had an average grade of 3.4 (range 2.0-4.0). Higher grades were seen for clinical practice guidelines that acknowledged industry funding (mean value 4.0) or were issued by organizations depending on private funding (mean value 3.6 vs. 3.0 for governmental funding). The conclusions of the 15 systematic reviews had a mean grade of 2.2 (range 1.0-3.8). Systematic reviews with low grades were less cited. In total, 21 randomized controlled trials were included in the systematic reviews. Approximately half of these evaluated an educational intervention where the effect of self-monitoring of blood glucose could not be clearly isolated.

CONCLUSIONS

Clinical practice guidelines were more in favour of self-monitoring use than the systematic reviews that were cited. The citation practice was non-systematic and industry funding seemingly led to a more positive attitude towards use of self-monitoring of blood glucose.

Endocrine clinical practice guidelines in North America. A systematic assessment of quality

OBJECTIVE

To assess the quality of endocrine guidelines developed in North America.

STUDY DESIGN AND SETTING

A systematic review of the literature was conducted to identify all endocrine clinical practice guidelines developed in North America and published between January 1, 2007 and January 13, 2010. Two independent reviewers used the Appraisal of Guidelines, Research and Evaluation instrument to evaluate the quality of the guidelines in six domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence.

RESULTS

One hundred eligible endocrine guidelines had high scores in the scope-and-purpose (mean pooled standardized score [MPSD] of 82±14) and clarity domains (MPSD=64±17) and low scores in the stakeholder-involvement (MPSD of 36±12) and editorial independence domains (MPSD=36±36). Only 29% of guidelines scored above 60% for more than three domains. Rigor-of-development domain score was significantly higher in guidelines using the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) approach, nondiabetes guidelines, and in published in-print vs. online publications.

CONCLUSIONS

The quality of endocrine guidelines published in 2007-2009 is moderate and can be improved by (1) using methodologically sound development frameworks, (2) increasing stakeholder involvement, and (3) paying more attention to resource implications of guideline implementation.

Quality evaluation of guidelines on genetic screening, surveillance and management of hereditary colorectal cancer

BACKGROUND

We examined the methodological quality of guidelines on colorectal cancer genetic susceptibility syndromes.

METHODS

PubMed, EMBASE, and Google were searched up to July 2010. Adjourned guidelines in English were included. The Appraisal of Guidelines, Research and Evaluation (AGREE) instrument was used to assess their quality, and the reported evidence base of the guidelines was evaluated.

RESULTS

The search yielded 17 eligible guidelines: 11 were developed by independent associations, while 6 had national\state endorsement. Only three guidelines performed satisfactorily, achieving a score >50% in all 6 AGREE domains. Mean standardized scores for the 6 AGREE domains were: 'scope and purpose', 83.9 ± 22.5%; 'stakeholder involvement', 35.6 ± 24.9%; 'rigour of development', 48.6 ± 25.3%; 'clarity and presentation', 71.6 ± 19.3%; 'applicability', 33.8 ± 30.1%; 'editorial independence', 42.2 ± 39.7%. Guidelines with national endorsement performed better in all the domains, with a statistically significant difference in three domains. Fifteen guidelines out of 17 were found to be based on research evidence.

CONCLUSIONS

There is scope, in many areas, for improving the guidelines analysed, among which are the involvement of various professional figures and patients' representatives, and policies for their application. The AGREE instrument is a useful tool and could also be used profitably by guideline developers to improve the quality of recommendations.

Clarity and applicability of drug-drug interaction management guidelines: a systematic appraisal by general practitioners and community pharmacists in the Netherlands

BACKGROUND

Despite the availability and daily use of computerized drug-drug interaction surveillance systems, exposure to potentially relevant drug-drug interactions (DDIs) continues. DDI management guidelines are often inadequate and clear management options are lacking, which attributes to overriding of DDI signals. Although general criteria for the development and reporting of high-quality clinical practice guidelines have been identified, it appears these have not yet been applied to DDI management guidelines.

OBJECTIVES

The aim of the study was to assess the clarity and applicability of guidelines for the management of potentially harmful DDIs.

METHODS

We selected 13 DDIs that are potentially harmful for patients and frequently occur in community pharmacy practice in the Netherlands. The clarity and applicability of the management guidelines of these DDIs were appraised using the appropriate two domains - 'Clarity and presentation' and 'Applicability', of the validated Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument. The appraisal was performed by 12 community pharmacists and 12 general practitioners. The standardized domain scores and mean item scores for 'Clarity and presentation' and 'Applicability' were compared.

RESULTS

All DDI management guidelines were generally found to score well on 'Clarity and presentation', but poorly with respect to 'Applicability' (standardized domain scores 68.0 vs 26.1%). Within the domain 'Clarity and presentation', the item 'tools for application' received the lowest scores. Within the domain 'Applicability', cost implications, organizational barriers and key review criteria were all poorly documented. All guidelines presented non-directive advice using words such as 'consider' and 'regularly'.

CONCLUSIONS

Developers of DDI management guidelines should take the appropriate domains of the AGREE Instrument into consideration in their development processes. The applicability of DDI management guidelines should be pretested before publishing. To improve guideline quality, more attention should particularly be paid to the available tools for applications and cost implications.