Evidence-basing for quality improvement; bringing clinical practice guidelines closer to their promise of improving care practices

Challenges and Stimulating Factors for the Incorporation of Economic Considerations in Clinical Practice Guidelines: A Scoping Review

RATIONALE, AIMS AND OBJECTIVES

The incorporation of economic considerations in clinical practice guidelines (CPGs) could help promote cost-conscious decision-making in healthcare. Though healthcare expenditures increase, and resources are becoming scarcer, the extent to which economic considerations are incorporated into CPGs remains limited. This scoping review aims to identify the challenges and potential stimulating factors to incorporate economic considerations in CPGs.

METHOD

This scoping review was conducted following the Joanna Briggs Institute Methodology and findings were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines. A systematic search was conducted in eight databases considering literature published from July 2017 and onwards. Data extraction was conducted via an iterative and inductive approach to identify challenges and potential stimulating factors from the included reports. Included documents focused on the (para)medical field and reported on CPG development and economic considerations.

RESULTS

The search identified 2445 documents from which 33 documents were included for analysis. The analysis identified five challenges: discourse surrounding economic considerations in CPGs, methodological ambiguities, scarcity of (high-quality) economic evidence, transferability of evidence, and resource constraints. Additionally, three potential stimulating factors were identified: acceptance, economic evidence knowledge, and guidance on incorporating economic considerations.

CONCLUSION

These findings reflect the complexity of incorporating economic considerations in CPGs. The identified challenges highlight the need for clearer guidance (i.e. by training) and standardised methodologies for incorporating economic considerations in CPGs. The potential stimulating factors provide a roadmap for future efforts to enhance the integration of economic evidence in CPGs. Collaborative initiatives between health economists, CPG developers, and other stakeholders are essential to drive progress in this area and promote cost-conscious decision-making in healthcare.

Nonpharmacological Spine Pain Management in Clinical Practice Guidelines: A Systematic Review Using AGREE II and AGREE-REX Tools

OBJECTIVE: To summarize the content and critically appraise the quality and applicability of recent clinical practice guidelines (CPGs) for nonpharmacological, nonsurgical management of spine pain. DESIGN: Systematic review of CPGs. LITERATURE SEARCH: Six databases and seven guideline clearinghouses. STUDY SELECTION CRITERIA: Included de novo CPGs for nonpharmacological, nonsurgical management of spine pain designed for any primary health care providers, published in English, Arabic, French, or traditional Chinese within the past 12 years. DATA SYNTHESIS: Five reviewers independently appraised the guidelines using AGREE II and AGREE-REX. Interrater agreements were calculated for each domain and the total score of these tools using the intraclass correlation coefficient (2, 1) with absolute agreement. RESULTS: We included 30 CPGs, primarily (90%) developed in Western countries, which contained 404 recommendations. High-quality CPGs consistently recommended exercise therapy and multimodal care, encompassing a combination of exercises, mobilization/manipulation, education, alternative medicine, and cognitive-behavioral treatments. Generally, CPGs did not recommend assistive (eg, corsets and orthosis) devices or electro/thermotherapies (eg, therapeutic ultrasound and transcutaneous electrical nerve stimulation). Approximately half of the CPGs demonstrated good methodological quality according to AGREE II, whereas the rest were of poor quality. On the AGREE-REX assessment, one third of the recommendations were of excellent quality. CONCLUSION: Although recent guidelines frequently recommended exercise therapy and multimodal care for the management of spine pain, their recommendations often overlooked demographics and comorbidities. Despite methodological improvements, most CPGs lacked simple clinical applicability and considerations of knowledge users' values. J Orthop Sports Phys Ther 2025;55(1):1-14. Epub 4 November 2024. doi:10.2519/jospt.2024.12729.

Contrasting Approaches in the Implementation of GRADE Methodology in Guidelines for Haemophilia and Von Willebrand Disease

INTRODUCTION

The 2024 ISTH clinical practice guideline (CPG) for treatment of congenital haemophilia, the NBDF-McMaster Guideline on Care Models for Haemophilia Management, and ASH ISTH NBDF WFH guidelines on the diagnosis and management of VWD all utilised GRADE methodology.

AIM

Discuss missed opportunities and the methodological approach of the ISTH Guideline in contrast to how GRADE was previously applied in rare diseases.

METHODS

Critically analyse the methodology of each guideline along with best practices in the use of GRADE. Where applicable, the WFH Guidelines for the Management of Haemophilia were analysed.

RESULTS

Important differentiating features in applying GRADE were identified. Where a strong evidence base is lacking, data other than those from randomized controlled trials, which may not always be justified, need to be considered, including incorporation of outcomes important to people living with the disease. Justification and stakeholder input to prioritize questions requiring a new guideline, panel composition with necessary patient participation and content expertise were also found to be significant differentiating features.

CONCLUSION

The puristic approach taken in the ISTH Guideline development process, without consideration of accepted adaptations to GRADE implementation, created a missed opportunity for progressing haemophilia care, leading to guideline recommendations that have been widely deemed invalid and obsolete by expert healthcare professionals and by those living with the condition, the very people who are expected to implement or bear the impact of the recommendations. Lessons learnt from this comparative analysis should guide future guideline development and encourage collaboration to further advance haemophilia.

Development and validity testing of a matrix to evaluate maturity of clinical pathways: a case study in Saskatchewan, Canada

BACKGROUND

Healthcare systems are transforming into learning health systems that use data-driven and research-informed approaches to achieve continuous improvement. One of these approaches is the use of clinical pathways, which are tools to standardize care for a specific population and improve healthcare quality. Evaluating the maturity of clinical pathways is necessary to inform pathway development teams and health system decision makers about required pathway revisions or implementation supports. In an effort to improve the development, implementation, and sustainability of provincial clinical pathways, we developed a clinical pathways maturity evaluation matrix. To explore the initial content and face validity of the matrix, we used it to evaluate a case pathway within a provincial health authority in Saskatchewan, Canada.

METHODS

By using iterative consensus-based processes, we gathered feedback from stakeholders including patient and family partners, policy makers, clinicians, and quality improvement specialists, to rank, retain, or remove enablers and sub-enablers of the draft matrix. We tested the matrix on the Chronic Pain Pathway (CPP) for primary care in a local pilot area and revised the matrix based on feedback from the CPP development team leader.

RESULTS

The final matrix contains five enablers (i.e., Design, Ownership and Performer, Infrastructure, Performance Management, and Culture), 20 sub-enablers, and three trajectory definitions for each sub-enabler. Supplemental documents were created for six sub-enablers. The CPP scored 15 out of 40 possible points of maturity. Although the pathway scored highest in the Design enabler (10/12), it requires more attention in several areas, specifically the Ownership and Performer and the Performance Management enablers, each of which scored zero. Additionally, the Infrastructure and Culture enablers scored 2/4 and 3/8 points, respectively. These areas of the CPP are in need of improvement in order to enhance the overall maturity of the CPP.

CONCLUSIONS

We developed a clinical pathways maturity matrix to evaluate the various dimensions of clinical pathways' development and implementation. The goals of this initial work were to develop and validate a tool to assess the maturity and readiness of new or existing pathways and to track pathways' revisions and improvements.

A Guideline for Guidelines: A Novel Method to Assess the Helpfulness of Medical Guidelines

Background/Objectives: The recommendations included in medical guidelines (GLs) provide important help to medical professionals for making clinical decisions regarding the diagnosis and treatment of various diseases. However, there are no systematic methods to measure the helpfulness of GLs. Thus, we developed an objective assessment of GLs which indicates their helpfulness and quality. We hypothesized that a simple mathematical analysis of 'Recommendations' and 'Evidence' would suffice. Methods: As a proof of concept, a mathematical analysis was conducted on the '2020 European Society of Cardiology Guidelines on Sports Cardiology and Exercise in Patients with Cardiovascular Disease Guideline' (SCE-guideline). First, the frequencies of Classes of Recommendations (CLASS) and the Levels of Evidence (LEVEL) (n = 159) were analysed. Then, LEVEL areas under CLASS were calculated to form a certainty index (CI: -1 to +1). Results: The frequency of CLASS I ('to do') and CLASS III ('not to do') was relatively high in the SCE-guideline (52.2%). Yet, the most frequent LEVEL was C (41.2-83.8%), indicating only a relatively low quality of scientific evidence in the SCE-guideline. The SCE-guideline showed a relatively high CI (+0.57): 78.4% certainty and 21.6% uncertainty. Conclusions: The SCE-guideline provides substantial help in decision making through the recommendations (CLASS), while the supporting evidence (LEVEL) in most cases is of lower quality. This is what the newly introduced certainty index showed: a tool for 'quality control' which can identify specific areas within GLs, and can promote the future improvement of GLs. The newly developed mathematical analysis can be used as a Guideline for the Guidelines, facilitating the assessment and comparison of the helpfulness and quality of GLs.

Appraisal of quality and analysis of the similarities and differences between osteoarthritis Clinical Practice Guideline recommendations: A systematic review

OBJECTIVE

Clinical Practice Guidelines (CPGs) aim to support management of hip and knee osteoarthritis (OA), but recommendations are often conflicting and implementation is poor, contributing to evidence-to-practice gaps. This systematic review investigated the contextual and methodological factors contributing to conflicting recommendations for hip and knee OA.

METHOD

Our systematic review appraised CPGs for managing hip and knee OA in adults ≥18 years (PROSPERO CRD42021276635). We used AGREE-II and AGREE-REX to assess quality and extracted data on treatment gaps, conflicts, biases, and consensus. Heterogeneity of recommendations was determined using Weighted Fleiss Kappa (K). The relationship between (K) and AGREE-II/AGREE-REX scores was explored.

RESULTS

We identified 25 CPGs across eight countries and four international organisations. The ACR, EULAR, NICE, OARSI and RACGP guidelines scored highest for overall AGREE-II quality (83%). The highest overall AGREE-REX scores were for BMJ Arthroscopy (80%), RACGP (78%) and NICE (76%). CPGs with the least agreement for pharmacological recommendations were ESCEO and NICE (-0.14), ACR (-0.08), and RACGP (-0.01). The highest agreements were between RACGP and NICE (0.53), RACGP and ACR (0.61), and NICE and ACR (0.91). Decreased internal validity determined by low-quality AGREE scores(<60%) in editorial independence were associated with less agreement for pharmacological recommendations.

CONCLUSION

There were associations between guideline quality and agreement scores. Future guideline development should be informed by robust evidence, editorial independence and methodological rigour to ensure a harmonisation of recommendations. End-users of CPGs must recognise the contextual factors associated with the development of OA CPGs and balance these factors with available evidence.

Comparative assessment of scientific reach and utilization of the International Association of Dental Traumatology 2020 guidelines: An altmetric and citation analysis

BACKGROUND/AIM

The updated version of the International Association of Dental Traumatology (IADT) guidelines was published online in May 2020 in the form of four papers. They are extensively used in dental trauma education in textbooks, undergraduate and postgraduate programs, and clinical workshops. Hence, this study aimed to assess the impact, global scientific reach, and utilization of these guidelines through altmetric and citation analysis.

MATERIAL AND METHODS

The protocol was prepared after an expert group discussion. Dimensions database was used to derive the altmetric and citation data on April 10th, 2023. The data was extracted by using a self-designed pre-piloted form by two authors independently. The citing articles were further categorized into the type of study and the domains and subdomains of dental traumatology. The VOSviewer program was utilized to explore the correlation between the subdomains and the overlap of citations was assessed by creating a citation matrix.

RESULTS

A total of 552 citations had been collectively received by the four papers. The highest was seen for the guidelines for fracture and luxation in permanent teeth (FL), avulsion in permanent teeth (AV), general introduction (GI), and primary teeth (PT). Dental Traumatology had the maximum number of citing papers for each guideline. They were from 52 countries with English being the commonest language. Overall, the maximum number of citations had been received in narrative reviews (n = 104) and the majority of cited papers were categorized in the therapeutic domains. The most common subdomain for the papers that cited GI, FL, and AV was "treatment protocols in permanent teeth", while it was "awareness of prevention and emergency management of traumatic dental injuries" for the papers that had cited PT.

CONCLUSIONS

This analysis highlighted that the IADT 2020 guidelines are globally popular resources, which are widely used by the researchers across specialities and are intended towards clinical application.

Kaleidoscopic integration: Advancing the integration of incommensurable knowledge in healthcare guidelines

The integration of different types of knowledge in epistemically hierarchical settings remains one of the greatest challenges when developing standards for healthcare practices. In this article, we open up the notion of knowledge integration and empirically examine the various ways in which different types of knowledge interact and can be integrated. To allow us to focus on the diverse forms of knowledge as well as their interaction and integration, we combine Moreira's work on repertoires of evaluation with that of Dewulf and Bouwen on frame interactions. We examine the quest for knowledge integration by studying interactions in the case of the development of the COVID-19 vaccination guideline in the Netherlands, a prime example of the encounter of a wide range and diversity of knowledge that needs to be appraised and integrated into guideline recommendations. Drawing on ethnographic observations of more than 70 guideline development meetings between 2021 and 2022, we first map the different types of knowledge and reasonings used by the guideline developers and subsequently analyze their interactions. We identified eight knowledge interaction patterns, being disconnection, polarization, accommodation, incorporation, reconnection, reconciliation, passive juxtaposition, and kaleidoscopic integration. We hereby draw attention to the various possible knowledge interactions encompassed in the concept of "knowledge integration", especially to those in which integration is achieved while differences and incompatibilities are maintained. Finally, we discuss potential ways to facilitate fruitful knowledge interactions during collaborative work which include the ability to accept and sustain tensions between different types of knowledge and making more explicit use of frame or rather repertoire reflection.

Creating an (ethical) epistemic space for the normalization of clinical and "real food" oral immunotherapy for food allergy

Researchers and sociologists have argued the consequences of standardization vis-à-vis clinical practice guidelines are diverse and argue they should be explored empirically. Sociologists have also argued that "best evidence" for the development of clinical practice guidelines is not restricted to randomized controlled trials and that other forms of knowledge should be embedded in and inform CPGs. There is little research concerning how other types of knowledge are mobilized and taken up in CPGs. This article presents the results of an ethnographic investigation in Canada between 2015 and 2020 of the development of a clinical practice guideline for immunotherapy for food allergy. My research shows that immunotherapy has become the source of controversy regarding whether immunotherapy should be offered in the clinic or remain experimental and whether it should be offered using food or commercial products. I argue that the clinical practice guideline for oral immunotherapy reaffirms what has been previously noted by sociologists; guidelines can serve normative purposes and are not merely technical documents. This case study is unique as it demonstrates how guidelines can serve as "community-making devices" to consolidate "epistemic communities" through the explicit and formal mobilization of ethical principles alongside other forms of "traditional" evidence. The mobilization of a multi-criteria approach that included ethical principles was mobilized in part to counter the de-legitimization and peripheralization of clinical and real food oral immunotherapy.

Capturing Emerging Experiential Knowledge for Vaccination Guidelines Through Natural Language Processing: Proof-of-Concept Study

BACKGROUND

Experience-based knowledge and value considerations of health professionals, citizens, and patients are essential to formulate public health and clinical guidelines that are relevant and applicable to medical practice. Conventional methods for incorporating such knowledge into guideline development often involve a limited number of representatives and are considered to be time-consuming. Including experiential knowledge can be crucial during rapid guidance production in response to a pandemic but it is difficult to accomplish.

OBJECTIVE

This proof-of-concept study explored the potential of artificial intelligence (AI)-based methods to capture experiential knowledge and value considerations from existing data channels to make these insights available for public health guideline development.

METHODS

We developed and examined AI-based methods in relation to the COVID-19 vaccination guideline development in the Netherlands. We analyzed Dutch messages shared between December 2020 and June 2021 on social media and on 2 databases from the Dutch National Institute for Public Health and the Environment (RIVM), where experiences and questions regarding COVID-19 vaccination are reported. First, natural language processing (NLP) filtering techniques and an initial supervised machine learning model were developed to identify this type of knowledge in a large data set. Subsequently, structural topic modeling was performed to discern thematic patterns related to experiences with COVID-19 vaccination.

RESULTS

NLP methods proved to be able to identify and analyze experience-based knowledge and value considerations in large data sets. They provide insights into a variety of experiential knowledge that is difficult to obtain otherwise for rapid guideline development. Some topics addressed by citizens, patients, and professionals can serve as direct feedback to recommendations in the guideline. For example, a topic pointed out that although travel was not considered as a reason warranting prioritization for vaccination in the national vaccination campaign, there was a considerable need for vaccines for indispensable travel, such as cross-border informal caregiving, work or study, or accessing specialized care abroad. Another example is the ambiguity regarding the definition of medical risk groups prioritized for vaccination, with many citizens not meeting the formal priority criteria while being equally at risk. Such experiential knowledge may help the early identification of problems with the guideline's application and point to frequently occurring exceptions that might initiate a revision of the guideline text.

CONCLUSIONS

This proof-of-concept study presents NLP methods as viable tools to access and use experience-based knowledge and value considerations, possibly contributing to robust, equitable, and applicable guidelines. They offer a way for guideline developers to gain insights into health professionals, citizens, and patients' experience-based knowledge, especially when conventional methods are difficult to implement. AI-based methods can thus broaden the evidence and knowledge base available for rapid guideline development and may therefore be considered as an important addition to the toolbox of pandemic preparedness.