Appraisal tools for clinical practice guidelines: a systematic review

Managing vestibular schwannomas with radiosurgery and radiotherapy: AGREE II appraisal of clinical practice guidelines

INTRODUCTION

Vestibular schwannomas (VSs) are rare, benign intracranial tumours that have prompted clinical practice guideline (CPG) creation given their complex management. Our aim was to utilize the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument to assess if such CPGs on the management of VSs with radiosurgery and radiotherapy are of acceptable quality.

METHODS

Relevant CPGs were identified following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocols. Experienced reviewers then extracted general CPG properties and rated their quality via the AGREE II instrument. Intraclass correlation coefficients (ICCs) were quantified to assess interrater reliability.

RESULTS

Nine CPGs on the management of VSs with radiosurgery and radiotherapy were identified. All CPGs were created in the past six years and developed recommendations based on literature review and expert consensus. One guideline was deemed as high quality with seven others being moderate and one being low in quality. The clarity of the presentation domain had the highest mean scaled domain score of 96.0%. The domains of stakeholder involvement and applicability had the lowest means of 49.2% and 47.2%, respectively. ICCs were either good or excellent across all domains.

CONCLUSION

Current CPGs on the management of VSs with radiosurgery and radiotherapy are of acceptable quality but would greatly benefit from improvements in applicability, stakeholder involvement, editorial independence and rigour of development. We recommend CPG authors reference the European Association of Neuro-Oncology (EANO) guideline as a developmental framework with the Congress of Neurological Surgeons/American Association of Neurological Surgeons (CNS/AANS) CPG being a valid alternative.

An evaluation of the quality, suitability and impact on equity of clinical practice guidelines relevant to preterm birth for use in Aotearoa New Zealand

BACKGROUND

Preterm birth is a leading cause of perinatal morbidity and mortality and a defining event for pregnant people, infants, and whānau (extended families). Recommendations have been made for a national preterm birth prevention initiative focusing on equity in Aotearoa New Zealand, including the development of a national best practice guide. An understanding of the number and quality of guidelines, and consideration of their suitability and impact on equity is required.

METHODS

Guidelines were identified through a systematic literature search, search of professional bodies websites, and invitation to regional health services in Aotearoa New Zealand. Obstetric and midwifery clinical directors were invited to report on guideline use. Identified guidelines were appraised by a 23-member trans-disciplinary Review Panel; quantitatively using the AGREE-II instrument and qualitatively using modified ADAPTE questions. The quality of guidelines available but not in use was compared against those in current use, and by health services by level of maternity and neonatal care. Major themes affecting implementation and impact on equity were identified using Braun and Clarke methodology.

RESULTS

A total of 235 guidelines were included for appraisal. Guidelines available but not in use by regional health services scored higher in quality than guidelines in current use (median domain score Rigour and Development 47.5 versus 18.8, p < 0.001, median domain score Overall Assessment 62.5 versus 44.4, p < 0.001). Guidelines in use by regional health services with tertiary maternity and neonatal services had higher median AGREE II scores in several domains, than those with secondary level services (median domain score Overall Assessment 50.0 versus 37.5, p < 0.001). Groups identified by the Review Panel as experiencing the greatest constraints and limitations to guideline implementation were rural, provincial, low socioeconomic, Māori, and Pacific populations. Identified themes to improve equity included a targeted approach to groups experiencing the least advantage; a culturally considered approach; nationally consistent guidance; and improved funding to support implementation of guideline recommendations.

CONCLUSIONS

We have systematically identified and assessed guidelines on preterm birth. High-quality guidelines will inform a national best practice guide for use in Taonga Tuku Iho, a knowledge translation project for equity in preterm birth care and outcomes in Aotearoa.

Assessment of the methodological, recommendation and reporting quality of global guidelines for neck pain and synthesis of evidence and recommendations: a systematic review protocol

INTRODUCTION

Neck pain is a global health problem that can cause severe disability and a huge medical burden. Clinical practice guideline (CPG) is an important basis for clinical diagnosis and treatment. A high-quality CPG plays a significant role in clinical practice. However, the quality of the CPGs for neck pain lacks comprehensive assessment. This protocol aims to evaluate the methodological, recommendation, reporting quality of global CPGs for neck pain and identify key recommendations and gaps that limit evidence-based practice.

METHOD

CPGs from January 2013 to November 2023 will be identified through a systematic search on 13 scientific databases (PubMed, Cochrane Library, Embase, etc) and 7 online guideline repositories. Six reviewers will independently evaluate the quality of CPGs for neck pain by using the Appraisal of Guidelines for Research and Evaluation, the Appraisal of Guidelines Research and Evaluation-Recommendations Excellence and the Reporting Items for Practice Guidelines in Healthcare tools. Intraclass correlation coefficient will be used to test the consistency of the assessment. We will identify the distribution of evidence and recommendations in each evidence-based CPGs for neck pain and regrade the level of evidence and strength of recommendations by adopting the commonly used Grading of Recommendations, Assessment, Development and Evaluations system. The key recommendations based on high-quality evidence will be summarised. In addition, we will categorise CPGs by different characteristics and conduct a subgroup analysis of the results of assessment.

ETHICS AND DISSEMINATION

No subjects will be involved in this systematic review, so there is no need for ethical approval. The finding of this review will be summarised as a paper for publication in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42023417717.

Measuring evidence-based clinical guideline compliance in the paediatric intensive care unit

BACKGROUND

Evidence-based clinical care guidelines improve medical treatment by reducing error, improving outcomes and possibly lowering healthcare costs. While some data exist on individual guideline compliance, no data exist on overall compliance to multiple nuanced guidelines in a paediatric intensive care setting.

METHODS

Guideline compliance was observed and measured with a prospective cohort at a tertiary academic paediatric medical-surgical intensive care unit. Adherence to 19 evidence-based clinical care guidelines was evaluated in 814 patients, and reasons for non-compliance were noted along with other associated outcomes.

MEASUREMENTS AND MAIN RESULTS

Overall facility compliance was unexpectedly high at 77.8% over 4512 compliance events, involving 826 admissions. Compliance varied widely between guidelines. Guidelines with the highest compliance were stress ulcer prophylaxis (97.1%) and transfusion administration such as fresh frozen plasma (97.4%) and platelets (94.8%); guidelines with the lowest compliance were ventilator-associated pneumonia prevention (28.7%) and vitamin K administration (34.8%). There was no significant change in compliance over time with observation. Guidelines with binary decision branch points or single-page decision flow diagrams had a higher average compliance of 90.6%. Poor compliance was more often observed with poor perception of guideline trustworthiness and time limitations.

CONCLUSIONS

Measuring guideline compliance, though onerous, allowed for evaluation of current clinical practices and identified actionable areas for institutional improvement.

A systematic review and evaluation of post-stroke depression clinical practice guidelines

OBJECTIVE

Post-stroke depression is a depressive mood disorder that follows a cerebrovascular accident and is a burden on stroke patients. Its management is included in clinical practice guidelines focused on stroke, and the recommended treatment is selective serotonin reuptake inhibitors in conjunction with psychotherapy. Clinical practice guidelines are recommendations used to standardize best medical practice, but there is no current evaluation of guidelines containing post stroke depression recommendations. Thus, the objective is to appraise the selected guidelines manner of development and quality.

MATERIALS AND METHODS

A systematic literature review across three databases and a manual google search was performed to collect guidelines that included recommendations on the management of post-stroke depression. 1236 guidelines were screened, and 27 were considered for inclusion. Considered guidelines were manually reviewed by the authors, and ultimately, 7 met inclusion criteria. The appraisal of guidelines for research and evaluation was used to evaluate these guidelines' recommendations around post-stroke depression.

RESULTS

Three guidelines met the threshold considered "High", with all of them having five or more quality domains eclipse the cutoff score of 70%. Across all guidelines, the highest scoring domains were "Scope and Purpose", "Clarity of Presentation", and "Editorial Independence" with scores of 76.98%, 73.81%, and 91.36% respectively. The lowest scoring domains were "Applicability", "Rigor of Development", and "Stakeholder Involvement" with respective scores of 58.73%, 54.02%, and 43.90%.

CONCLUSIONS

The domains "Applicability", "Rigor of Development," and "Stakeholder Involvement" were the lowest scoring domains. These specific domains represent areas in which future guidelines could be more developed.

Clinical Practice Guidelines in an Era of Accountability, Saudi Arabia: A Call for Action

INTRODUCTION

Clinical Practice Guidelines (CPGs) development and implementation in the Kingdom of Saudi Arabia are suboptimal. The Kingdom's Vision 2030 envisages a transformational change to achieve an effective, integrated, value-based ecosystem focused on patient health.

OBJECTIVES

This study aimed to develop a CPG appraisal tool that will support the realization of the Kingdom's Vision 2030 through the development of high-quality and highly implementable CPGs. To maximize its impact, all vital healthcare paradigms, such as systems thinking, value-based healthcare, and information technology, will robustly be incorporated in the tool.

METHODS

The Saudi Health Council through its National Center of Evidence-Based Medicine (NCEBM) embarked on a program to develop this appraisal tool. A taskforce of experts was selected based on their experience in evidence-based practice and training. The task force, through a methodology of extensive literature review, deliberation, outside experts' feedback, and Delphi and consensus voting, developed a prototype appraisal tool that was named the Holistic Appraisal Tool for CPGs (HAT-CPG).

RESULTS

The HAT-CPG was developed comprising three sections: an initial basic information section, an internal validity section, and an external validity section with a total of 13 section items and 73 reporting elements.

CONCLUSION

It is envisaged that the Holistic Appraisal Tool will support CPG developers and users in Saudi Arabia in realizing the objectives for which it was developed.

Quality appraisal of clinical practice guidelines for motor neuron diseases or related disorders using the AGREE II instrument

Objectives

This study aimed to systematically assess the quality of CPGs for motor neuron diseases (MNDs) or related disorders and identify the gaps that limit evidence-based practice.

Methods

Four scientific databases and six guideline repositories were searched for eligible CPGs. Three researchers assessed the eligible CPGs using the Appraisal of Guidelines Research and Evaluation II instrument. The distribution of the level of evidence and strength of recommendation of these CPGs were determined. The univariate regression analysis was used to explore the characteristic factors affecting the quality of CPGs.

Results

Fifteen CPGs met the eligibility criteria: 10 were for MND and 5 were for spinal muscular atrophy. The mean overall rating score was 44.5%, and only 3 of 15 CPGs were of high quality. The domains that achieved low mean scores were applicability (24.4%), rigor of development (39.9%), and stakeholder involvement (40.3%). Most recommendations were based on low-quality evidence and had a weak strength. The CPGs that were updated, meant for adults, and evidence based, and used a CPG quality tool and a grading system were associated with higher scores in certain specific domains and overall rating.

Conclusion

The overall quality of CPGs for MNDs or related disorders was poor and recommendations were largely based on low-quality evidence. Many areas still need improvement to develop high-quality CPGs, and the use of CPG quality tools should be emphasized. A great deal of research on MNDs or related disorders is still needed to fill the large evidence gap.

A Systematic Review of Clinical Practice Guidelines for Caries Prevention following the AGREE II Checklist

Untreated oral diseases are detrimental to overall well-being and quality of life and are in close relationship with social and economic consequences. The presence of strong evidence for caries primary and secondary prevention is a compulsory tool for the development of clinical practice guidelines (CPGs). This paper was aimed to assess systematically the importance of clinical practice guidelines in caries prevention management considering both the adult and pediatric populations and evaluate them using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Checklist. Records were extracted from EMBASE, SCOPUS, PubMed/Medline and seven other relevant guideline databases between 6 January and 14 February 2023. Two reviewers independently conducted the appraisal using the web-based platform My AGREE PLUS. Twenty-one guidelines/papers met the inclusion criteria and were reviewed. Eight CPGs included both primary and secondary prevention interventions, whereas thirteen presented a single preventive model. Overall, 12 guidelines were published in the USA. The mean AGREE II scores ranged from 35.4% to 84.3%. Of the total twenty-one included guidelines, twelve were classified as "Recommended", ranging from 56.3% to 84.3%, the others were described as "Recommended with modification", ranging from 35.4% to 68.9%. From the AGREE II analysis carried out, the CPGs included in this survey adopted a punctual methodological rigor but lacked applicative power. The present survey showed that the public, as the primary beneficiary, played a limited role in the development of the twenty-one CPGs. Hence, methodological improvement can better support high-quality CPG development in the future.

Development of the Scientific, Transparent and Applicable Rankings (STAR) tool for clinical practice guidelines

BACKGROUND

This study aimed to develop a comprehensive instrument for evaluating and ranking clinical practice guidelines, named Scientific, Transparent and Applicable Rankings tool (STAR), and test its reliability, validity, and usability.

METHODS

This study set up a multidisciplinary working group including guideline methodologists, statisticians, journal editors, clinicians, and other experts. Scoping review, Delphi methods, and hierarchical analysis were used to develop the STAR tool. We evaluated the instrument's intrinsic and interrater reliability, content and criterion validity, and usability.

RESULTS

STAR contained 39 items grouped into 11 domains. The mean intrinsic reliability of the domains, indicated by Cronbach's α coefficient, was 0.588 (95% confidence interval [CI]: 0.414, 0.762). Interrater reliability as assessed with Cohen's kappa coefficient was 0.774 (95% CI: 0.740, 0.807) for methodological evaluators and 0.618 (95% CI: 0.587, 0.648) for clinical evaluators. The overall content validity index was 0.905. Pearson's r correlation for criterion validity was 0.885 (95% CI: 0.804, 0.932). The mean usability score of the items was 4.6 and the median time spent to evaluate each guideline was 20 min.

CONCLUSION

The instrument performed well in terms of reliability, validity, and efficiency, and can be used for comprehensively evaluating and ranking guidelines.

Transparency of clinical practice guideline funding: a cross-sectional analysis of the German AWMF registry

BACKGROUND

While reporting of individual conflicts of interest is formalised, it is unclear to what extent the funding of clinical practice guidelines (CPGs) is formally reported. The aim of this study is to explore the accuracy and comprehensiveness of reporting on funding in German CPGs.

METHODS

We searched for CPGs in the registry of the Association of the Scientific Medical Societies in Germany in July 2020. Information on guideline funding was categorised by two reviewers independently and discrepancies were clarified by discussion with a third reviewer. Accuracy and comprehensiveness of reporting on funding was assessed using the German Instrument for Methodological Guideline Appraisal (DELBI).

RESULTS

We included 507 CPGs published between 2015 and 2020 in the main analysis. 23/507 (4.5%) of the CPGs achieved the highest DELBI score by including information on funding sources, expenses and the amount of funding provided, as well as a statement on the independence of the guideline authors from the funding institution(s). CPGs with more rigorous methodological requirements (systematic review of the literature and/or structured consensus-building) received higher DELBI scores.

CONCLUSION

German CPGs do not communicate their funding transparently. Transparency of CPG funding could be achieved by making it mandatory to publish information for all guidelines. For that purpose, a standardised form and guidance should be developed.

Effect of physical activity on coronary heart disease risk factors among firefighters: a scoping review protocol

INTRODUCTION

Physical activity interventions have been used for various health conditions, including cardiovascular disease. However, the literature is still limited regarding the effect of physical activity on coronary heart disease among firefighters.

METHODS AND ANALYSIS

The review will be conducted according to recommendations from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) and PRISMA Protocol guidelines. This scoping review will provide a synthesis of current evidence on the effects of physical activity on coronary heart disease among firefighters. Search strategies will be performed in the following databases: Cochrane database, PubMed, Medline, (EbscoHost), Web of Science, Academic Search Complete, CINAHL (EBSCO), SAGE journals, ScienceDirect and Scopus. We will include literature in the English language that are full-text peer-reviewed articles from inception to November 2021. Screening of (titles, abstracts and full text of potential articles) will be done by two independent authors using EndNote V.9 software tool. A standardised data extraction form will be designed for the extraction. Two authors will independently extract the data from the selected articles and all differences will be discussed by an invited third reviewer if a consensus cannot be reached. The primary outcomes will be the impact of physical fitness on firefighters experiencing coronary artery disease. This information can assist policy-makers in decision-making related to the use of physical activity in firefighters experiencing coronary heart disease.

ETHICS AND DISSEMINATION

Ethical clearance has been obtained from the University ethics committee and the City of Cape Town. The findings will be disseminated through publications and the physical activity guidelines will be submitted to the Fire Departments within the City of Cape Town. Data analysis will start on 1 April 2023.

Involvement of methodological experts and the quality of clinical practice guidelines: a critical appraisal of clinical practice guidelines and a questionnaire survey of the development groups in Japan

OBJECTIVE

To evaluate whether the involvement of methodological experts improves the quality of clinical practice guidelines (CPGs) after adjusting for other factors.

SETTING

The quality of Japanese CPGs published in 2011-2019 was assessed using the Appraisal of Guidelines, Research, and Evaluation (AGREE) II instrument. A questionnaire survey targeting CPG development groups was conducted through postal mail.

PARTICIPANTS

405 CPGs were retrieved from a Japanese CPG clearinghouse. Questionnaires were distributed to the 405 CPG development groups. Of the 178 respondents, 22 were excluded because of missing values. Finally, 156 participants representing their CPG development groups were included in the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

CPG quality was assessed using the AGREE II tool. The characteristics of CPGs, including publication year, development organisation, versions, number of members in the development group and involvement of methodological experts, were corrected from the description in the CPGs and the questionnaire survey. We performed multiple logistic regressions using the quality of CPGs as the dependent variable and the involvement of experts as the independent variable, adjusting for other possible factors.

RESULTS

A total of 156 CPGs were included. Expert involvement was significantly associated with the AGREE II instrument scores in domains 1 (β=0.207), 2 (β=0.370), 3 (β=0.413), 4 (β=0.289), 5 (β=0.375), 6 (β=0.240) and overall (β=0.344).

CONCLUSION

This study revealed that the involvement of methodological experts in the CPG development process improves the quality of CPGs. The results suggest the importance of establishing a training and certification programme for experts and constructing expert referral systems that meet CPG developers' needs to improve the quality of CPGs.

Self-management education for children with epilepsy and their caregivers. A scoping review

Self-management education programs have been highly successful in preparing people to manage medical conditions with recurring events. A detailed curriculum for epilepsy patients, and their caretakers, is lacking. Here we assess what is available for patients who have disorders with recurring events and offer an approach to developing a potential self-care curriculum for patients with seizures and their caregivers. Among the anticipated components are a baseline efficacy assessment and training tailored to increasing self-efficacy, medication compliance, and stress management. Those at risk of status epilepticus will also need guidance in preparing a personalized seizure action plan and training in how to decide when rescue medication is appropriate and how to administer the therapy. Peers, as well as professionals, could teach and provide support. To our knowledge, no such programs are currently available in English. We encourage their creation, dissemination, and widespread use.

A systematic review of quality and consistency of clinical practice guidelines on the primary prevention of food allergy and atopic dermatitis

Background and aims

With an increasing number of Clinical Practice Guidelines (CPGs) addressing primary prevention of food allergy and atopic dermatitis, it is timely to undertake a comprehensive assessment of the quality and consistency of recommendations and evaluation of their implementability in different geographical settings.

Methods

We systematically reviewed CPGs from 8 international databases and extensive website searches. Seven reviewers screened records in any language and then used the AGREE II and AGREE REX instruments to critically appraise CPGs published between January 2011 and April 2022.

Results

Our search identified 2138 relevant articles, of which 30 CPGs were eventually included. Eight (27%) CPGs were shortlisted based on our predefined quality criteria of achieving scores >70% in the "Scope and Purpose" and "Rigour of Development" domains of the AGREE II instrument. Among the shortlisted CPGs, scores on the "Applicability" domain were generally low, and only 3 CPGs rated highly in the "Implementability" domain of AGREE-REX, suggesting that the majority of CPGs fared poorly on global applicability. Recommendations on maternal diet and complementary feeding in infants were mostly consistent, but recommendations on use of hydrolysed formula and supplements varied considerably.

Conclusion

The overall quality of a CPG for Food Allergy and Atopic Dermatitis prevention did not correlate well with its global applicability. It is imperative that CPG developers consider stakeholders' preferences, local applicability, and adapt existing recommendations to each individual population and healthcare system to ensure successful implementation. There is a need for development of high-quality CPGs for allergy prevention outside of North America and Europe.

PROSPERO registration number

CRD42021265689.

Effective web-based clinical practice guidelines resources: recommendations from a mixed methods usability study

BACKGROUND

Clinical practice guidelines (CPG) are an important knowledge translation resource to help clinicians stay up to date about relevant clinical knowledge. Effective communication of guidelines, including format, facilitates its implementation. Despite the digitalization of healthcare, there is little literature to guide CPG website creation for effective dissemination and implementation. Our aim was to assess the effectiveness of the content and format of the Diabetes Canada CPG website, and use our results to inform recommendations for other CPG websites.

METHODS

Fourteen clinicians (family physicians, nurses, pharmacists, and dieticians) in diabetes care across Canada participated in this mixed-methods study (questionnaires, usability testing and interviews). Participants "thought-aloud" while completing eight usability tasks on the CPG website. Outcomes included task success rate, completion time, click per tasks, resource used, paths, search attempts and success rate, and error types. Participants were then interviewed.

RESULTS

The Diabetes Canada CPG website was found to be usable. Participants had a high task success rate of 79% for all tasks and used 144 (standard deviation (SD) = 152) seconds and 4.6 (SD = 3.9) clicks per task. Interactive tools were most frequently used compared to full guidelines and static tools. Misinterpretation accounted for 48% of usability errors. Participants overall found the website intuitive, with effective content and design elements.

CONCLUSION

Different versions of CPG information (e.g. interactive tools, quick reference guide, static tools) can help answer clinical questions more quickly. Effective web design should be assessed during CPG website creation for effective guideline dissemination and implementation.

The quality of guidelines for treatment of carotid artery disease: a critical appraisal using the AGREE II instrument

Clinical Practice Guidelines (CPG) are structured recommendations based on systematic reviews of the available evidence and are useful tools to support clinical decision-making. However, studies have raised concerns about the methodological and scientific quality of several CPG, which can affect their application in clinical practice. The objective of this study was to perform a systematic appraisal of the methodological quality of carotid atherosclerotic disease clinical guidelines, published from 2000 to 2019, using the AGREE II instrument (Appraisal of Guidelines for Research and Evaluation Instrument II). The appraisers independently assessed the quality of the CPG included in the study for each of the 6 domains of the AGREE II tool. The CPG were rated as high, moderate, or low quality using a points scale. A total of 9 CPGs were selected for appraisal. Except for domain 2 (kappa=0.715), excellent agreement was observed between the appraisers (kappa>0.75). Five of the CPGs were rated as high overall methodological quality rating, 5 were rated as moderate overall methodological quality, and 2 were rated low overall methodological quality. The authors conclude that: (1) appraisal of carotid atherosclerotic disease clinical guidelines using the AGREE II instrument is feasible, with a high degree of agreement among appraisers; and (2) that most CPGs on the management of atherosclerotic carotid disease have high methodological quality.

AGREEing on Clinical Practice Guidelines for Autism Spectrum Disorders in Children: A Systematic Review and Quality Assessment

Background: Autism spectrum disorder (ASD) is a multifaceted neurodevelopmental disorder requiring multimodal intervention and an army of multidisciplinary teams for a proper rehabilitation plan. Accordingly, multiple practice guidelines have been published for different disciplines. However, systematic evidence to detect and intervene must be updated regularly. Our main objective is to compare and summarize the recommendations made in the clinical practice guidelines (CPGs) for ASD in children released from November 2015 to March 2022. Methods: CPGs were subjected to a systematic review. We developed the inclusion and exclusion criteria and health-related questions, then searched and screened for CPGs utilizing bibliographic and CPG databases. Each of the CPGs used in the study were critically evaluated using the Appraisal of Guidelines for REsearch and Evaluation II (AGREE II) instrument. In a realistic comparison table, we summarized the recommendations. Results: Four eligible CPGs were appraised: Australian Autism CRC (ACRC); Ministry of Health New Zealand (NZ); National Institute for Health and Care Excellence (NICE); and Scottish Intercollegiate Guidelines Network, Healthcare Improvement Scotland (SIGN-HIS). The overall assessments of all four CPGs scored greater than 80%; these findings were consistent with the high scores in the six domains of AGREE II, including: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of development, (4) clarity of presentation, (5) applicability, and (6) editorial independence domains. Domain (3) scored 84%, 93%, 86%, and 85%; domain (5) 92%, 89%, 54%, and 85%; and domain (6) 92%, 96%, 88%, and 92% for ACRC, NICE, NZ, and SIGN-HIS, respectively. Overall, there were no serious conflicts between the clinical recommendations of the four CPGs, but some were more comprehensive and elaborative than others. Conclusions: All four assessed evidence-based CPGs demonstrated high methodological quality and relevance for use in practice.

A Systematic Review of the Guidelines on Venous Thromboembolism Prophylaxis in Gynecologic Oncology

Simple Summary

Cancer-associated venous thromboembolism (VTE) is the second leading cause of death in cancer patients. Gynecological cancer patients are considered at high risk of VTE and of bleeding due to the intrinsic nature of the tumor itself. Prevention of VTE in this special subgroup of cancer patients is a current topic of interest since there are no standardized protocols. This review aimed to summarize the protocols for VTE prevention in gynecological cancer patients given by the selected national and international guidelines in order to help the clinicians to identify patients who would benefit from VTE prophylaxis during daily practice.

Abstract

(1) Background: This review aimed to summarize the indications for venous thromboembolic (VTE) events’ prophylaxis in a gynecological cancer population, according to the most recent guidelines. (2) Methods: A systematic review of the guidelines in PubMed, SCOPUS, Web of Science, EMBASE, and CINHAL regarding VTE prevention in gynecological cancer patients was conducted according to PRISMA criteria. We compared the recommendations given by oncological and hematological societies regarding VTE prevention in gynecological cancer patients published from January 2010 through March 2021. We searched for the following keywords: “venous thromboembolism prevention”, “cancer”, and “guidelines”. The AGREE II checklist was used to critically analyze the guidelines’ quality. (3) Results: There were 1003 documents available; 14 met the inclusion criteria, 5 were excluded and, eventually, the guidelines of 10 societies were evaluated. (4) Conclusions: The guidelines agree that low-molecular-weight heparin (LMWH) and fondaparinux achieve better results in VTE prevention in gynecological cancer patients. Direct oral anticoagulants (DOACs) can be used to prevent VTE in outpatients and high-risk medical patients after discharge. VTE risk scores should be applied to all oncological patients to identify those who would benefit from a prevention program. More attention should be paid to mechanical prophylactic methods due to the high bleeding risk of gynecological cancer patients.

Developing guideline-based key performance indicators for recurrent miscarriage care: lessons from a multi-stage consensus process with a diverse stakeholder group

BACKGROUND

Standardised care pathways tailored to women/couples who experience recurrent miscarriage are needed; however, clinical practice is inconsistent and poorly organised. In this paper, we outline our processes and experiences of developing guideline-based key performance indicators (KPIs) for recurrent miscarriage care with a diverse stakeholder group which will be used to evaluate national services. To date, such exercises have generally only involved clinicians, with the need for greater stakeholder involvement highlighted.

METHODS

Our study involved six stages: (i) identification and synthesis of recommendations for recurrent miscarriage care through a systematic review of clinical practice guidelines; (ii) a two-round modified e-Delphi survey with stakeholders to develop consensus on recommendations and outcomes; (iii) four virtual meetings to develop this consensus further; (iv) development of a list of candidate KPIs; (v) survey to achieve consensus on the final suite of KPIs and a (vi) virtual meeting to agree on the final set of KPIs. Through participatory methods, participants provided feedback on the process of KPI development.

RESULTS

From an initial list of 373 recommendations and 14 outcomes, 110 indicators were prioritised for inclusion in the final suite of KPIs: (i) structure of care (n = 20); (ii) counselling and supportive care (n = 7); (iii) investigations (n = 30); treatment (n = 34); outcomes (n = 19). Participants' feedback on the process comprised three main themes: accessibility, richness in diversity, streamlining the development process.

CONCLUSIONS

It is important and feasible to develop guideline-based KPIs with a diverse stakeholder group. One hundred and ten KPIs were prioritised for inclusion in a suite of guideline-based KPIs for recurrent miscarriage care. Insights into our experiences may help others undertaking similar projects, particularly those undertaken in the absence of a clinical guideline and/or involving a range of stakeholders.

The International Guideline Evaluation Screening Tool (IGEST): development and validation

Background

Guideline adaptation provides an important alternative to de novo guideline development by making the process more efficient and reducing unnecessary duplication. The quality evaluation of international guidelines is an essential part of the adaptation process.

The study aims at describing the development and validation of a new tool to screen trustworthy Clinical Practice Guidelines (CPGs) for their adoption/adaption: the International Guideline Evaluation Screening Tool (IGEST).

Methods

The process of developing the IGEST involved two main phases: 1) tool development and 2) content validation. The tool development phase comprised three stages, where the scope of the IGEST was defined and the item pool was generated and refined. The content validation was performed through the computation of a content validity index (CVI) based on the opinions of an expert panel.

Results

All the items obtained a CVI >0.78, which resulted in the validation of the instrument. The final instrument comprised four preliminary conditions and 12 criteria organised into three dimensions: (i) the management of conflict of interest; (ii) the quality of evidence and the coherence between evidence and recommendations; and (iii) the panel composition.

Conclusion

The IGEST showed good content validity for assessing the quality of international guidelines. Using the new tool to select trustworthy guidelines might increase the likelihood that international clinical practice guidelines will be adopted/adapted to the local context by allowing a quick screening of existing guidelines trustworthiness and providing an acceptability threshold that supports the decision-making process.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01618-5.

Recommendations for the pharmacological treatment of treatment-resistant depression: A systematic review protocol

INTRODUCTION

Depression is a serious and widespread mental health disorder. Although effective treatment does exist, a significant proportion of patients with depression fail to respond to antidepressant treatment trials, a condition named treatment-resistant depression. Efficient approach should be given this condition in order to revert the burden caused by depression. Clinical practice guidelines (CPGs) are evidence-based health promotion instruments to improve diagnosis and treatment. CPGs recommendations for treatment-resistant depression must be trustworthy. The objective of the proposed study is to systematically identify, appraise the quality of CPGs for the treatment of depression and elaborate a synthesis of recommendations for treatment-resistant depression of CPGs considered to be of high quality and with high quality recommendations.

METHODS AND ANALYSIS

We will search the databases of organizations, such as PubMed, Embase, Cochrane Library, PsycInfo, and the Virtual Health Library, and organizations that develop CPGs. Three independent researchers will assess the quality of the CPGs and their recommendations using the AGREE II and AGREE-REX instruments, respectively. Given the identification of divergences and convergences as well as weak and strong points among high quality CPGs, our work may help developers, clinicians and eventually patients.

ETHICS AND DISSEMINATION

No ethical approval is required for a systematic review, as no patient data will be used. The research results will be disseminated in conferences and submitted to a peer reviewed journal.

Lifestyle and complementary therapies in multiple sclerosis guidelines: Systematic review

Management of multiple sclerosis (MS) may comprise clinical interventions and self-management strategies, including complementary therapies and modifiable lifestyle factors such as exercise and smoking cessation. Lifestyle modifications and complementary therapies with proven safety and efficacy are essential as part of best-practice MS management, especially when faced with limited access to healthcare services. However, it is unclear to what extent MS clinical practice guidelines and consensus statements address these strategies. A systematic review was conducted, wherein MEDLINE, EMBASE, PsycINFO, CINAHL, Scopus, Web of Science, guideline databases and developer sites were searched for guidelines and consensus statements that addressed lifestyle modifications and complementary therapies of interest. Two researchers independently screened articles, extracted data and assessed guideline quality using the Appraisal of Guidelines for Research and Evaluation version II. Thirty-one guidelines and consensus statements were included. Quality was high for 'clarity of presentation' (77%) and 'scope and purpose' (73%), moderate for 'stakeholder development' (56%), 'rigour of development' (48%) and 'editorial independence' (47%), and low for 'applicability' (29%). Two guidelines, related to physical activity and exercise, mindfulness, smoking cessation, and vitamin D and polyunsaturated fatty acid supplementation, scored high in all domains. These guidelines were two of only four guidelines intended for use by people with MS. High-quality guidelines and consensus statements to guide lifestyle modifications and complementary therapies in MS management are limited. Our findings indicate the need for more guidelines intended for use by people with MS, and a further focus on implementation resources.

Short communication: Do clinical guidelines for bipolar disorders adequately address sleep, circadian rhythm, activity and lifestyle problems?

Objectives Clinical practice guidelines (CPGs) for the treatment of bipolar disorder (BD) provide guidance to health care professionals and patients about the management of core aspects of BD. This study evaluated the overall quality of CPGs and examined the quantity, specificity, clarity and utility of recommendations about four key contemporary themes: sleep, circadian rhythms, activity and energy, and healthy lifestyles. Methods English language editions of CPGs for the treatment of BD were identified by a systematic search of the literature from 2007 onwards (i.e. 15 years). Blind independent ratings were combined to give consensus scores for the quality of each CPGs (using the 14-item International Center for Allied Health Evidence guideline checklist). Composite ratings of the quantity, specificity, clarity and utility of recommendations about the key themes were undertaken using a 0-3 scale. Results Twenty-five CPGs were eligible for review. Overall quality was high (median checklist score=10), but only 11 (44%) CPGs included even basic information about circadian rhythm disturbances. Combined scores for composite ratings about sleep and circadian rhythms were significantly correlated with overall quality of the CPG (sleep, r = 0.43; circadian rhythms, r = 0.42) but not with year of publication. Limitations No reliable scale exists for generating composite ratings of the four themes we examined. Conclusions Circadian rhythms and chronobiology represent neglected domains in CPGs. Incorporating this important theme into future editions of CPGs would aid health care professionals to identify, prevent, or intervene with these problems and improve outcomes for a significant proportion of individuals with BD.

Space Norms for Constructing Quality Reviews on Online Consumer Review Sites

As consumer-generated online reviews are recognized as an essential component of many e-commerce websites, practitioners and digital platforms strive to elicit high-quality reviews. Yet research so far mainly focuses on what drives consumers to submit reviews, whereas little has been done to help understand why review writers construct high-quality reviews versus post poor review content. This paper demonstrates that the behavior of writing quality reviews is largely directed by space norms, a distinct instantiation of social norms inferred from locally observing how others write reviews in the digital space (i.e., the review site). The perception of the space norm begets a sense that one should write reviews in accordance with the norm and induces the corresponding behavior on the given site. Our work lends insights to practitioners and digital platforms: managers looking to elicit quality reviewing behavior should devise the local website environment to foster the space norms for the desired behavior. We offer several cost-effective methods to foster space norms, including a machine learning approach.

Quality of clinical practice guidelines and recommendations for the management of pain, sedation, delirium and iatrogenic withdrawal in pediatric intensive care: a systematic review protocol

INTRODUCTION

Inadequate management of pain and sedation in critically ill children can cause unnecessary suffering and agitation, but also delirium and iatrogenic withdrawal. It is, therefore, important to address these four interrelated conditions together. Some clinical practice guidelines (CPGs) are available for the management of pain and sedation, and a few for delirium and iatrogenic withdrawal in the paediatric intensive care unit; none address the four conditions altogether. Critical appraisal of the quality of CPGs is necessary for their recommendations to be adopted into clinical practice. The aim of this systematic review is to identify and appraise the quality of CPGs and recommendations for management of either pain, sedation, delirium and iatrogenic withdrawal.

METHODS AND ANALYSIS

Researchers will conduct a systematic review in electronic databases (Medline ALL (Ovid), Embase.com, CINAHL with Full Text (EBSCO), JBI EBP Database (Ovid)), guideline repositories and websites of professional societies to identify CPGs published from 2010 to date. They will then combine index and free terms describing CPGs with pain, sedation, delirium and withdrawal. The researchers will include CPGs if they can be applied in the paediatric intensive care population (newborns to 18 years old) and include recommendation(s) for assessment of at least one of the four conditions. Two independent reviewers will screen for eligibility, complete data extraction and quality assessments using the Appraisal of Guidelines for Research and Evaluation (AGREE) II and the AGREE Recommendation Excellence instruments. Researchers will report characteristics, content and recommendations from CPGs in tabulated forms.

ETHICS AND DISSEMINATION

Ethical approval is not required for this systematic review. Results will be published in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42021274364.

Trends in the development process of clinical practice guidelines: a questionnaire survey for the guideline development groups in Japan

Background

Clinical practice guidelines (CPGs) are representative methods for promoting healthcare standardization and improving its quality. Previous studies on the CPG (published by 2006) development process in Japan reported that the involvement of experts and patients, efficient evidence collection and appraisal, and paucity of evidence on Japanese patients should be improved for the efficient CPG development. This study aimed to clarify the trends of CPG development process in Japan, focusing on the involvement of experts and patients, efficient evidence collection and appraisal, and paucity of Japanese evidence.

Methods

A cross-sectional questionnaire survey was conducted for CPG development groups to collect information on the development activities of the CPGs published from 2012 to 2019. These CPGs were identified from the Japanese guideline clearinghouse. The questionnaire included the questions on composing the group, securing funding sources, collecting and appraising the research evidence, and the difficulties in the CPG development process. The questionnaires were distributed to the chairpersons of the CPG development groups through postal mail from November 2020 to January 2021. Combining the data from the current survey with those of previous studies reporting the development process of CPGs published by 2011, we analyzed the trend in the CPG development process.

Results

Of the total 265 CPGs included in the analysis, 164 (response rate: 41.4%) were from the current survey and 101 (response rate: 44.5%) were from previous studies. Among these, 40 (15.1%) were published by 2005, 47 (17.7%) in 2006–2010, 77 (29.1%) in 2011–2015, and 101 (38.1%) in 2016–2019. The proportion of CPGs involving methodologists did not increase through the publication periods. The proportion of CPGs involving patients almost doubled from the first period (15.9%) to the fourth period (32.4%). The yield rates of the articles did not change through the publication periods. The difficulty in “Coping with the paucity of Japanese evidence” has been improving consistently (69.2% in the first period to 37.4% in the fourth period).

Conclusions

Our results suggest the need for methodological improvement in the efficient collection and appraisal of evidence and in the system assigning experts to the CPG development groups.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07492-7.

Quality assessment of clinical practice guidelines for neonatal sepsis using the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument: A systematic review of neonatal guidelines

BACKGROUND AND OBJECTIVE

Neonatal sepsis (NS) continues to be a critical healthcare priority for the coming decades worldwide. The aim of this study was to critically appraise the quality of recent clinical practice guidelines (CPGs) for neonatal sepsis and to summarize and compare their recommendations.

METHODS

This study involves a systematic review of CPGs. We identified clinical questions and eligibility criteria and searched and screened for CPGs using bibliographic and CPG databases and professional societies. Each included CPG was assessed by four independent appraisers using the Appraisal of Guidelines for REsearch & Evaluation II (AGREE II) instrument. We summarized the recommendations in a comparison practical table. The systematic review was drafted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. Its protocol was registered in the PROSPERO International Prospective Register of Systematic Reviews (ID: CRD42021258732).

RESULTS

Our search retrieved 4,432 citations; of which five CPGs were eligible and appraised: American Academy of Pediatrics (AAP 2018) (35 and 34 weeks); Canadian Pediatric Society (CPS 2017); National Institute for Health and Care Excellence (NICE 2021); and Queensland Maternity and Neonatal Services (QH 2020). Among these, the overall assessment of two evidence-based CPGs scored > 70% (NICE and QH), which was consistent with their higher scores in the six domains of the AGREE II instrument. In domain 3 (rigor of development), NICE and QH scored 99 and 60%, respectively. In domain 5 (applicability), they scored 96 and 74%, respectively, and in domain 6 (editorial independence), they scored 90 and 71%, respectively.

CONCLUSION

The methodological quality of the NICE CPG was superior followed by the QH CPG with relevant recommendations for use in practice.

SYSTEMATIC REVIEW REGISTRATION

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021258732, PROSPERO (CRD42021258732).

The quality of guidelines for treatment of carotid artery disease: a critical appraisal using the AGREE II instrument

Abstract Clinical Practice Guidelines (CPG) are structured recommendations based on systematic reviews of the available evidence and are useful tools to support clinical decision-making. However, studies have raised concerns about the methodological and scientific quality of several CPG, which can affect their application in clinical practice. The objective of this study was to perform a systematic appraisal of the methodological quality of carotid atherosclerotic disease clinical guidelines, published from 2000 to 2019, using the AGREE II instrument (Appraisal of Guidelines for Research and Evaluation Instrument II). The appraisers independently assessed the quality of the CPG included in the study for each of the 6 domains of the AGREE II tool. The CPG were rated as high, moderate, or low quality using a points scale. A total of 9 CPGs were selected for appraisal. Except for domain 2 (kappa=0.715), excellent agreement was observed between the appraisers (kappa>0.75). Five of the CPGs were rated as high overall methodological quality rating, 5 were rated as moderate overall methodological quality, and 2 were rated low overall methodological quality. The authors conclude that: (1) appraisal of carotid atherosclerotic disease clinical guidelines using the AGREE II instrument is feasible, with a high degree of agreement among appraisers; and (2) that most CPGs on the management of atherosclerotic carotid disease have high methodological quality.

Evaluation of Clinical Practice Guidelines on Fall Prevention and Management for Older Adults: A Systematic Review

Importance

With the global population aging, falls and fall-related injuries are ubiquitous, and several clinical practice guidelines for falls prevention and management for individuals 60 years or older have been developed. A systematic evaluation of the recommendations and agreement level is lacking.

Objectives

To perform a systematic review of clinical practice guidelines for falls prevention and management for adults 60 years or older in all settings (eg, community, acute care, and nursing homes), evaluate agreement in recommendations, and identify potential gaps.

Evidence Review

A systematic review following Preferred Reporting Items for Systematic Reviews and Meta-analyses statement methods for clinical practice guidelines on fall prevention and management for older adults was conducted (updated July 1, 2021) using MEDLINE, PubMed, PsycINFO, Embase, CINAHL, the Cochrane Library, PEDro, and Epistemonikos databases. Medical Subject Headings search terms were related to falls, clinical practice guidelines, management and prevention, and older adults, with no restrictions on date, language, or setting for inclusion. Three independent reviewers selected records for full-text examination if they followed evidence- and consensus-based processes and assessed the quality of the guidelines using Appraisal of Guidelines for Research & Evaluation II (AGREE-II) criteria. The strength of the recommendations was evaluated using Grades of Recommendation, Assessment, Development, and Evaluation scores, and agreement across topic areas was assessed using the Fleiss κ statistic.

Findings

Of 11 414 records identified, 159 were fully reviewed and assessed for eligibility, and 15 were included. All 15 selected guidelines had high-quality AGREE-II total scores (mean [SD], 80.1% [5.6%]), although individual quality domain scores for clinical applicability (mean [SD], 63.4% [11.4%]) and stakeholder (clinicians, patients, or caregivers) involvement (mean [SD], 76.3% [9.0%]) were lower. A total of 198 recommendations covering 16 topic areas in 15 guidelines were identified after screening 4767 abstracts that proceeded to 159 full texts. Most (≥11) guidelines strongly recommended performing risk stratification, assessment tests for gait and balance, fracture and osteoporosis management, multifactorial interventions, medication review, exercise promotion, environment modification, vision and footwear correction, referral to physiotherapy, and cardiovascular interventions. The strengths of the recommendations were inconsistent for vitamin D supplementation, addressing cognitive factors, and falls prevention education. Recommendations on use of hip protectors and digital technology or wearables were often missing. None of the examined guidelines included a patient or caregiver panel in their deliberations.

Conclusions and Relevance

This systematic review found that current clinical practice guidelines on fall prevention and management for older adults showed a high degree of agreement in several areas in which strong recommendations were made, whereas other topic areas did not achieve this level of consensus or coverage. Future guidelines should address clinical applicability of their recommendations and include perspectives of patients and other stakeholders.

Appraising clinical applicability of studies: mapping and synthesis of current frameworks, and proposal of the FrACAS framework and VICORT checklist

BACKGROUND

Not all research findings are translated to clinical practice. Reasons for lack of applicability are varied, and multiple frameworks and criteria exist to appraise the general applicability of epidemiological and clinical research. In this two-part study, we identify, map, and synthesize frameworks and criteria; we develop a framework to assist clinicians to appraise applicability specifically from a clinical perspective.

METHODS

We conducted a literature search in PubMed and Embase to identify frameworks appraising applicability of study results. Conceptual thematic analysis was used to synthesize frameworks and criteria. We carried out a framework development process integrating contemporary debates in epidemiology, findings from the literature search and synthesis, iterative pilot-testing, and brainstorming and consensus discussions to propose a concise framework to appraise clinical applicability.

RESULTS

Of the 4622 references retrieved, we identified 26 unique frameworks featuring 21 criteria. Frameworks and criteria varied by scope and level of aggregation of the evidence appraised, target user, and specific area of applicability (internal validity, clinical applicability, external validity, and system applicability). Our proposed Framework Appraising the Clinical Applicability of Studies (FrACAS) classifies studies in three domains (research, practice informing, and practice changing) by examining six criteria sequentially: Validity, Indication-informativeness, Clinical relevance, Originality, Risk-benefit comprehensiveness, and Transposability (VICORT checklist).

CONCLUSIONS

Existing frameworks to applicability vary by scope, target user, and area of applicability. We introduce FrACAS to specifically assess applicability from a clinical perspective. Our framework can be used as a tool for the design, appraisal, and interpretation of epidemiological and clinical studies.

Individualisation of glycaemic management in older people with type 2 diabetes: a systematic review of clinical practice guidelines recommendations

BACKGROUND

Recommendations for individualised glycaemic management in older people with type 2 diabetes (T2D) have recently been provided in clinical practice guidelines (CPGs) issued by major scientific societies. The aim of this systematic review is to compare the content of these recommendations concerning health assessment, targets for glycaemic control, lifestyle management and glucose-lowering therapy across CPGs.

METHODS

The CPGs on T2D management in people aged ≥65 years published in English after 2015 by major scientific societies were systematically reviewed in accordance with the PRISMA statement. The quality of the CPGs included was assessed using the AGREE-II tool. The recommendations for individualised glycaemic management were extracted, and their level of evidence (LOE) and strength of recommendation recorded.

RESULTS

Three CPGs of high methodological quality were included, namely those from the American Diabetes Association 2020, the Endocrine Society 2019 and the Diabetes Canada Expert Committee 2018. They made 27 recommendations addressing individualised glycaemic management, a minority of which (40%) had a high LOE. Comparison of the 27 recommendations identified some discrepancies between CPGs, e.g. the individualised values of HbA1c targets. The 13 strong recommendations addressed 10 clinical messages, five of which are recommended in all three CPGs, i.e. assess health status, screen for cognitive impairment, avoid hypoglycaemia, prioritise drugs with low hypoglycaemic effects and simplify complex drug regimens.

CONCLUSIONS

Although there is a consensus on avoiding hypoglycaemia in older patients with T2D, significant discrepancies regarding individualised HbA1c targets exist between CPGs.

Development and evaluation of guidelines for prevention of retraumatisation in torture survivors during surgical care: protocol for a multistage qualitative study

INTRODUCTION

Healthcare professionals working in somatic departments are not trained to recognise signs of torture or provide appropriate healthcare to torture survivors, which may result in retraumatisation during surgical treatment.

METHODS AND ANALYSIS

This protocol outlines a four-stage qualitative-method strategy for the development and evaluation of guidelines for prevention of retraumatisation of torture survivors during surgical care. The systematic search for literature review in stages 1 and 2 was conducted in August 2019 and March 2021, respectively, using nine databases. The search strategies employed in stage 1, without imposing any date limits, resulted in the inclusion of eight studies that addressed inadequate healthcare strategies associated with retraumatisation. The clinical guidelines review in stage 2 will include publications from 2000 onwards, which will be appraised using the Appraisal of Guidelines Research and Evaluation Version II instrument. Following multi-institutional recruitment in Norway, stage 3 will explore survivors' experiences of receiving surgical treatment using indepth interviews (n=8-12), which will be audio-recorded, transcribed verbatim and analysed using the interpretative phenomenological analysis approach. In stage 4a, based on the findings from stages 1, 2 and 3, a set of clinical guidelines for preventing retraumatisation during surgical treatment will be developed. Next, the feasibility and acceptability of the guidelines will be assessed in stage 4b in three interdisciplinary focus group interviews (n=5 per group) and text condensation analyses.

ETHICS AND DISSEMINATION

The Regional (South-East C) Committee for Medical and Health Research Ethics approved the study in May 2021 (#227624). In stages 3 and 4, an informational letter and an informed consent form will be distributed to the participants to sign before the interview. The study results will be disseminated through publications, conference presentations, and national and local public forums to healthcare professionals, service managers, policymakers and refugee-supporting agencies.

Systematic review of diabetic eye disease practice guidelines: more applicability, transparency and development rigor are needed

OBJECTIVES

To assess the quality of diabetic eye disease clinical practice guidelines.

STUDY DESIGN AND SETTING

A systematic search of diabetic eye disease guidelines was conducted on six online databases and guideline repositories. Four reviewers independently rated quality using the Appraisal of Guidelines, Research, and Evaluation (AGREE II) instrument. Aggregate scores (%) for six domains and overall quality assessment were calculated. A "good quality" guideline was one with ≥60% score for "rigor of development" and in at least two other domains.

RESULTS

Eighteen guidelines met the inclusion criteria, of which 13 were evidence-based guidelines (involved systematic search and grading of evidence). The median scores (interquartile range (IQR)) for "scope and purpose," "stakeholder involvement," "rigor of development," "clarity of presentation," "applicability" and "editorial independence" were 73.6% (54.2%-80.6%), 48.6% (29.2%-71.5%), 60.2% (30.9%-78.1%), 86.6% (76.7%-94.4%), 28.6% (18.0%-37.8%) and 60.2% (30.9%-78.1%), respectively. The median overall score (out of 7) of all guidelines was 5.1 (IQR: 3.7-5.8). Evidence-based guidelines scored significantly higher compared to expert-consensus guidelines. Half (n = 9) of the guidelines (all evidence-based) were of "good quality."

CONCLUSION

A wide variation in methodological quality exists among diabetic eyecare guidelines, with nine demonstrating "good quality." Future iterations of guidelines could improve by appropriately engaging stakeholders, following a rigorous development process, including support for application in clinical practice and ensuring editorial transparency.

Methodological quality, guidance, and tools in scoping reviews: a scoping review protocol

OBJECTIVE

The objective of this scoping review is to identify and report upon evidence (such as guidance) or tools regarding methodological quality or risk of bias of scoping reviews.

INTRODUCTION

Scoping reviews have gained popularity in recent years but have been criticized for variations in their approaches. This scoping review will examine evidence on the methodological quality of scoping reviews, as well as identify and describe potential tools to inform the development of a tool for appraising the quality of scoping reviews.

INCLUSION CRITERIA

The scoping review will include all documents reporting on the development, evaluation, or conduct of tools addressing the critical appraisal or risk of bias of scoping reviews. The literature search will seek evidence published from 2005 onwards, corresponding with the publication of Arksey and O'Malley's framework for scoping reviews.

METHODS

The search strategy will aim to locate both published and unpublished documents utilizing a three-step search strategy. An initial search of PubMed has identified keyword and MeSH terms. A second search of PubMed, Embase, and CINAHL will follow. Google and Google Scholar will be searched for difficult-to-locate and unpublished literature. Authors will endeavor to consult with relevant methodologists through the authors' professional networks, social media accounts, and professional newsletters to obtain materials that can be considered for inclusion. Documents will be independently screened, selected, and extracted by two researchers and the data will be presented in tables.

What Determines the Quality of Rehabilitation Clinical Practice Guidelines?: An Overview Study

OBJECTIVE

The aim of the study was to determine what factors determine the quality of rehabilitation clinical practice guidelines.

DESIGN

Six databases were searched for articles that had applied the Appraisal of Guidelines for Research & Evaluation II quality assessment tool to rehabilitation clinical practice guidelines. The 573 deduplicated abstracts were independently screened by two authors, resulting in 81 articles, the full texts of which were independently screened by two authors for Appraisal of Guidelines for Research & Evaluation II application to rehabilitation clinical practice guidelines, resulting in a final selection of 40 reviews appraising 504 clinical practice guidelines. Data were extracted from these by one author and checked by a second. Data on each clinical practice guideline included the six Appraisal of Guidelines for Research & Evaluation II domain scores, as well as the two Appraisal of Guidelines for Research & Evaluation II global evaluations.

RESULTS

All six Appraisal of Guidelines for Research & Evaluation II domain scores were statistically significant predictors of overall clinical practice guideline quality rating; D3 (rigor of development) was the strongest and D1 (scope and purpose) the weakest (overall model P < 0.001, R2 = 0.53). Five of the six domain scores were significant predictors of the clinical practice guideline use recommendation, with D3 the strongest predictor and D5 (applicability) the weakest (overall model P < 0.001, pseudo R2 = 0.53).

CONCLUSIONS

Quality of rehabilitation clinical practice guidelines may be improved by addressing key domains such as rigor of development.

Methodological quality of guidelines for the management of opioid use disorder: A systematic review

WHAT IS KNOWN AND OBJECTIVE

Opioid Use Disorder (OUD) has a high mortality rate and affects millions of people worldwide. Many organizations and societies develop Clinical Practice Guidelines (CPGs) to serve as a framework for healthcare providers to decide and support best practice to manage and treat OUD. However, not all CPGs sufficiently address all the important aspects of optimal care for managing OUD. This study aims to review current CPGs for management of OUD, evaluate their methodological quality and summarize their recommendations.

METHODS

We conducted this systematic review according to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA). Various databases were searched for CPGs and Appraisal of Guidelines for Research and Evaluation (AGREE-II) instrument was used to assess the methodological quality. We also summarized the treatments plans of CPGs across continuum of care (diagnosis and assessment, treatment initiation, pharmacotherapy and psychosocial).

RESULTS

This review included 28 CPGs of varying qualities. CPGs from high-income countries and international organizations rated high for their methodological quality. Most CPGs scored high for the scope and purpose domain and scored low for applicability domain. Recommendations for the continuum of care for OUD varied across CPGs. Buprenorphine was recommended in most of the CPGs, followed by methadone. Recommendations for psychosocial interventions also varied, with cognitive behaviour therapies and counselling or education being the common recommendations in many CPGs WHAT IS NEW AND CONCLUSION: We found most CPGs have scope and purpose and clarity of presentation. However, the methodological rigour and applicability scored low. CPGs need to frame health questions in a comprehensible manner and provide an update as evidence grows. It is important for CPG developers to consider methodological quality as a factor when developing CPG recommendations.

Quality changes of clinical practice guidelines for respiratory diseases in China: A systematic review

BACKGROUND

After the low quality of Chinese clinical practice guidelines (CPGs) for respiratory diseases published from 1979 to 2013 was reported, some handbooks were published to standardize guidelines' development recently. There was a great increase in the production and dissemination of CPGs annually in China, whose quality and potential impact were unknown.

METHODS

A systematic search of four literature databases was performed for the period January 2013 to December 2018 to identify Chinese CPGs for respiratory diseases. Eligible CPGs were evaluated using the appraisal of guidelines for research and evaluation II (AGREE II) instrument.

RESULTS

A total of 197 CPGs were identified for review. Compared with the result of previous study, the increased scores of the six AGREE II domains were screened: Scope and purpose (57.3% vs. 57.8%), Stakeholder involvement (17.6% vs. 25.0%), Rigor of development (10.2% vs. 13.2%), Clarity and presentation (55.2% vs. 58.4%), Applicability (9.3% vs. 25.9%), and Editorial independence (1.1% vs. 6.3%). The improved overall assessment for included CPGs were: Recommended (4, 2.0% vs. 0, 0%) and Recommended with modifications (26, 13.2% vs. 3, 2.8%). The improved level of evidence used to make recommendations were 59, 11.9% versus 168, 22.4% and 88, 17.7% versus 195, 26.0%, A and B, respectively.

CONCLUSIONS

The overall quality of CPGs for respiratory diseases published from 2013 to 2018 in China was slightly improved, but had a big gap with the optimum level, especially in Rigor of development and Editorial independence. Increased efforts are required to enable the development of high-quality evidence-based CPGs for respiratory diseases.

Long-term treatment of bipolar disorder type I: A systematic and critical review of clinical guidelines with derived practice algorithms

OBJECTIVES

This systematic review aimed at providing a critical, comprehensive synthesis of international guidelines' recommendations on the long-term treatment of bipolar disorder type I (BD-I).

METHODS

MEDLINE/PubMed and EMBASE databases were searched from inception to January 15th, 2019 following PRISMA and PICAR rules. International guidelines providing recommendations for the long-term treatment of BD-I were included. A methodological quality assessment was conducted with the Appraisal of Guidelines for Research and Evaluation-AGREE II.

RESULTS

The final selection yielded five international guidelines, with overall good quality. The evaluation of applicability was the weakest aspect across the guidelines. Differences in their updating strategies and the rating of the evidence, particularly for meta-analyses, randomized clinical trials (RCTs) and observational studies, could be responsible of some level of heterogeneity among recommendations. Nonetheless, the guidelines recommended lithium as the 'gold standard' in the long-term treatment of BD-I. Quetiapine was another possible first-line option as well as aripiprazole (for the prevention of mania). Long-term treatment should contemplate monotherapy, at least initially. Clinicians should check regularly for efficacy and side effects and if necessary, switch to first-line alternatives (i.e. Valproate), combine first-line compounds with different mechanisms of action or switch to second-line options or combinations.

CONCLUSIONS

The possibility to monitor improvements in long-term outcomes, namely relapse prevention and inter-episode subthreshold depressive symptoms, based on the application of their recommendations is an unmet need of clinical guidelines. In terms of evidence of clinical guidelines, there is a need for more efficacious treatment strategies for the prevention of bipolar depression.

The Feedback Session of an Autism Assessment: A Scoping Review of Clinical Practice Guideline Recommendations

This review aimed to assess the quality and content of recommendations for delivering an autism diagnosis, published internationally within clinical practice guidelines. Seventeen relevant guidelines were identified. When methodological information was provided, recommendations for feedback were predominantly formed through consensus. Recommendations consistently included who should attend feedback, the timing and mode of delivery, the clinician's manner, and what should be discussed and/or included in an accompanying report. Specific recommendations were not consistent however, and a number of gaps were identified, such as the inclusion of educators and educational specific recommendations. Although individual variation is necessary for autism diagnosis disclosure, agreement on minimum standards of practice is warranted. Further investigation is required to establish best practice.

AGREEing on clinical practice guidelines for idiopathic steroid-sensitive nephrotic syndrome in children

BACKGROUND

Nephrotic syndrome is the most common kidney disease in children worldwide. Our aim was to critically appraise the quality of recent Clinical Practice Guidelines (CPGs) for idiopathic steroid-sensitive nephrotic syndrome (SSNS) in children in addition to summarize and compare their recommendations.

METHODS

Systematic review of CPGs. We identified clinical questions and eligibility criteria and searched and screened for CPGs using bibliographic and CPG databases. Each included CPG was assessed by four independent appraisers using the Appraisal of Guidelines for REsearch & Evaluation II (AGREE-II) instrument. We summarized the recommendations in a comparison practical table.

RESULTS

Our search retrieved 282 citations, of which three CPGs were eligible and appraised: Kidney Disease: Improving Global Outcomes (KDIGO) 2012, Japan Society for Pediatric Nephrology (JSPN) 2014, and American Academy of Pediatrics (AAP) 2009. Among these, the overall assessment of two evidence-based CPGs scored > 70% (KDIGO and JSPN), which was consistent with their higher scores in the six domains of the AGREE II Instrument. In domain 3 (rigor of development), KDIGO, JSPN, and AAP scored 84%, 74%, and 41%, respectively. In domain 5 (applicability), they scored 22%, 16%, and 19%, respectively, and in domain 6 (editorial independence), they scored 94%, 65%, and 88%, respectively.

CONCLUSIONS

The methodological quality of the KDIGO CPG was superior, followed by JSPN and AAP CPGs with the relevant recommendations for use in practice.

SYSTEMATIC REVIEW REGISTRATION

The protocol was registered in the Center for Open Science (OSF) DOI: 10.17605/OSF.IO/6QTMD and in the International prospective register of systematic reviews PROSPERO 2020 CRD42020197511 .

Quality of pediatric clinical practice guidelines

BACKGROUND

There is a lack of a comprehensive evaluation for pediatric clinical practice guidelines (CPGs) published in recent years. Here, we assessed the quality of pediatric CPGs, considering factors that might affect their quality. The aim of the study is to promote a more coherent development and application of CPGs.

METHODS

Pediatric CPGs published in PubMed, MedLive, Guidelines International Network, National Institute for Health and Care Excellence, and World Health Organization between 2017 and 2019 were searched and collected. Paired researchers conducted screening, data extraction, and quality assessment using the Appraisal of Guidelines for Research and Evaluation II (AGREE II). Linear regression analysis determined the factors affecting CPGs' quality.

RESULTS

The study included a total of 216 CPGs, which achieved a mean score of 4.26 out of 7 points (60.86%) in the AGREE II assessment. Only 6.48% of the CPGs reached the "recommend" level. The remaining 69.91% should have been modified before recommendation, while the other 23.61% did not reach the recommended level at all. The overall quality of recent pediatric CPGs was higher than previously, and the proportion of CPGs with low-quality decreased over time. However, there were still too few CPGs that reached a high-quality level. The "applicability" and "rigor of development" domains had generally low scores. CPGs formulated by developing countries or regions, those that are not under an organizations or groups responsibility, and those that used non-evidence-based methods were found to be associated with poorer quality in different domains as independent or combinational factors.

CONCLUSIONS

The quality of pediatric CPGs still needs to be improved. Specifically, a quality control before applying new CPGs should be essential to ensure their quality and applicability.

International clinical practice guidelines for gender minority/trans people: systematic review and quality assessment

OBJECTIVES

To identify and critically appraise published clinical practice guidelines (CPGs) regarding healthcare of gender minority/trans people.

DESIGN

Systematic review and quality appraisal using AGREE II (Appraisal of Guidelines for Research and Evaluation tool), including stakeholder domain prioritisation.

SETTING

Six databases and six CPG websites were searched, and international key opinion leaders approached.

PARTICIPANTS

CPGs relating to adults and/or children who are gender minority/trans with no exclusions due to comorbidities, except differences in sex development.

INTERVENTION

Any health-related intervention connected to the care of gender minority/trans people.

MAIN OUTCOME MEASURES

Number and quality of international CPGs addressing the health of gender minority/trans people, information on estimated changes in mortality or quality of life (QoL), consistency of recommended interventions across CPGs, and appraisal of key messages for patients.

RESULTS

Twelve international CPGs address gender minority/trans people's healthcare as complete (n=5), partial (n=4) or marginal (n=3) focus of guidance. The quality scores have a wide range and heterogeneity whichever AGREE II domain is prioritised. Five higher-quality CPGs focus on HIV and other blood-borne infections (overall assessment scores 69%-94%). Six lower-quality CPGs concern transition-specific interventions (overall assessment scores 11%-56%). None deal with primary care, mental health or longer-term medical issues. Sparse information on estimated changes in mortality and QoL is conflicting. Consistency between CPGs could not be examined due to unclear recommendations within the World Professional Association for Transgender Health Standards of Care Version 7 and a lack of overlap between other CPGs. None provide key messages for patients.

CONCLUSIONS

A paucity of high-quality guidance for gender minority/trans people exists, largely limited to HIV and transition, but not wider aspects of healthcare, mortality or QoL. Reference to AGREE II, use of systematic reviews, independent external review, stakeholder participation and patient facing material might improve future CPG quality.

PROSPERO REGISTRATION NUMBER

CRD42019154361.

Quality of the Australian National Health and Medical Research Council's clinical practice guidelines for the management of diabetic retinopathy

Clinical relevance: Understanding the quality of the commonly used clinical practice guidelines can help busy clinicians in selecting appropriate guidelines for evidence-based eye care for people with diabetes.Background: The National Health and Medical Research Council's (NHMRC) clinical practice guideline on diabetic retinopathy management has been widely used locally and internationally for over 10 years. However, the quality of this guideline has never been formally assessed. This study aimed to systematically evaluate the quality of the NHMRC guideline and compare it against other international guidelines.Methods: The 2008 NHMRC and another five established diabetic retinopathy management international guidelines (Scottish Intercollegiate Guidelines Network, 2017; American Academy of Ophthalmology, 2019; American Optometric Association, 2019; Royal College of Ophthalmologists, UK, 2013, and Canadian Ophthalmologic Society, 2012) were examined using the Appraisal of Guidelines, Research and Evaluation (AGREE II) instrument. Scoring by four independent reviewers was aggregated into six domain and overall rating scores. Consistency among the reviewers was assessed using intraclass correlation coefficient (ICC).Results: The AGREE II domain scores for the NHMRC guideline were: scope and purpose 72%, stakeholder involvement 64%, rigour of development 77%, clarity of presentation 96%, applicability 35%, and editorial independence 15%. The NHMRC guideline's overall score (5.3 of 7) was lower than that of most other guidelines. Compared to others, the NHMRC guideline scored well in clarity of presentation and rigour of development, but less well for editorial independence. The NHMRC guideline was the least current and a need to update it was recognised by all reviewers who identified key areas for improvement.Conclusion: The quality of the NHMRC guideline was comparable to most other established international guidelines. Several areas of strengths and weaknesses in this guideline were identified. Future updates should aim to improve transparency in development and applicability in clinical practice.

The use of systematic review evidence to support the development of guidelines for positron emission tomography: a cross-sectional survey

OBJECTIVES

To examine to what degree guidelines for PET and PET/CT used systematic review evidence.

METHODS

The latest version of guidelines for PET, PET/CT or PET/MRI published in English in PubMed until December 2019 was analysed in two categories: (1) for indications, if mainly discussing the appropriate use of PET in diverse conditions; (2) for procedures, if providing step-by-step instructions for imaging. We surveyed the general characteristics and the use of systematic review evidence for developing recommendations across all guidelines, and surveyed the citation of evidence for five recommendation topics in guidelines for procedures.

RESULTS

Forty-seven guidelines, published between 2004 and 2020, were included. Guidelines for indications were developed mainly on systematic reviews (13 of 19, 68.4%). Among those, 12 (63.2%) reported the level of evidence, 4 (21.1%) reported the strength of recommendations, 3 (15.8%) described external review and 7 (36.8%) involved methodologists. Guidelines for procedures were seldom developed on systematic reviews (1 of 27, 3.7%). Among those, 1 (3.7%) reported the level of evidence, 1 (3.7%) reported the strength of recommendations, 3 (11.1%) described external review and 1 (3.7%) involved methodologists. Systematic review evidence was cited by 2 (7.4%) procedure guidelines per recommendation topic in median.

CONCLUSION

The use of systematic review evidence for developing recommendations among PET or PET/CT guidelines was suboptimal. While our survey is an icebreaking attempt to explore a key element (i.e. use of systematic review evidence) for developing nuclear medicine guidelines, assessments of other domains of guideline quality may help capture the entire picture.

KEY POINTS

• The use of systematic review evidence for developing recommendations among guidelines for PET or PET/CT was suboptimal. • Only 13 (68.4%) guidelines for indications and 1 (3.7%) guideline for procedures systematically reviewed the literature during guideline development. • For each recommendation topic we examined, only a median of 2 (7.4%) procedure guidelines cited systematic review evidence.

Adaptation of clinical practice guidelines for osteoporosis in a Mexican context. Experience using methodologies ADAPTE, GRADE-ADOLOPMENT, and RAND/UCLA

High quality guidelines are required to enhance clinical practice, but its development is time consuming and a complex process. Adaptation might shorten development time and prevent double effort adjusting recommendations for a local context.

BACKGROUND AND OBJECTIVE

The aim of this article is to present our experience in a process in which we combined two methodologies for the adaptation of high quality osteoporosis CPGs for a primary health care context, with the inclusion of a formal consensus.

METHODS

We began an adaptation process with ADAPTE, and required to migrate to GRADE-ADOLOPMENT methodology, based on GIN-McMaster Guideline Development Checklist. To identify high quality clinical practice guidelines, we performed a systematic review as per the PRISMA-statement methodology (PROSPERO: CRD42019138548, August 19th, 2019); methodological quality was assessed using the Appraisal of Guidelines for Research & Evaluation version II system. We developed a RAND/UCLA consensus to support the inclusion of good practice statements and feasibility of selected recommendations.

RESULTS

Thirteen clinical questions were integrated, and fracture risk was selected as the main outcome for intervention recommendations analysis. Six high quality guidelines were selected. We prepared final recommendations from selected guides in an evidence synthesis framework. After the consensus, we integrated 50 recommendations.

CONCLUSION

By starting the adaptation process with ADAPTE, we experienced a time consuming process, which we could overcome when migrating to GRADE-Adolopment in combination with a consensus panel.

Clinical practice guidelines for recurrent miscarriage in high-income countries: a systematic review

Recurrent miscarriage affects 1-2% of women of reproductive age, depending on the definition used. A systematic review was conducted to identify, appraise and describe clinical practice guidelines (CPG) published since 2000 for the investigation, management, and/or follow-up of recurrent miscarriage within high-income countries. Six major databases, eight guideline repositories and the websites of 11 professional organizations were searched to identify potentially eligible studies. The quality of eligible CPG was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Tool. A narrative synthesis was conducted to describe, compare and contrast the CPG and recommendations therein. Thirty-two CPG were included, from which 373 recommendations concerning first-trimester recurrent miscarriage were identified across four sub-categories: structure of care (42 recommendations, nine CPG), investigations (134 recommendations, 23 CPG), treatment (153 recommendations, 24 CPG), and counselling and supportive care (46 recommendations, nine CPG). Most CPG scored 'poor' on applicability (84%) and editorial independence (69%); and to a lesser extent stakeholder involvement (38%) and rigour of development (31%). Varying levels of consensus were found across CPG, with some conflicting recommendations. Greater efforts are required to improve the quality of evidence underpinning CPG, the rigour of their development and the inclusion of multi-disciplinary perspectives, including those with lived experience of recurrent miscarriage.