Continuous surveillance of a pregnancy clinical guideline: an early experience

An International Needs Assessment Survey of Guideline Developers Demonstrates Variability in Resources and Challenges to Collaboration between Organizations

BACKGROUND

The development of rigorous, high-quality clinical guidelines increases the need for resources and skilled personnel within guideline-producing organizations. While collaboration between organizations provides a unique opportunity to pool resources and save time and effort, the collaboration presents its own unique challenges.

OBJECTIVE

To assess the perceived needs and current challenges of guideline producers worldwide related to guideline development and collaboration efforts.

DESIGN

Survey questions were developed by the Guidelines International Network and the US GRADE Network, pilot-tested among attendees of a guideline development workshop, and disseminated electronically using convenience and snowball sampling methods.

PARTICIPANTS

A total of 171 respondents representing 30 countries and more than 112 unique organizations were included in this analysis.

MAIN MEASURES

The survey included free-response, multiple-choice, and seven-point Likert-scale questions. Questions assessed respondents' perceived value of guidelines, resource availability and needs, guideline development processes, and collaboration efforts of their organization.

KEY RESULTS

Time required to develop high-quality systematic reviews and guidelines was the most relevant need (median=7; IQR=5.5-7). In-house resources to conduct literature searches (median=4; IQR=3-6) and the resources to develop rigorous guidelines rapidly (median=4; IQR=2-5) were perceived as the least available resources. Difficulties reconciling differences in guideline methodology (median=6; IQR=4-7) and the time required to establish collaborative agreements (median=6; IQR=5-6) were the most relevant barriers to collaboration between organizations. Results also indicated a general need for improvement in conflict of interest (COI) disclosure policies.

CONCLUSION

The survey identified organizational challenges in supporting rigorous guideline development, including the time, resources, and personnel required. Connecting guideline developers to existing databases of high-quality systematic reviews and the use of freely available online platforms may facilitate guideline development. Guideline-producing organizations may also consider allocating resources to hiring or training personnel with expertise in systematic review methodologies or utilizing resources more effectively by establishing collaborations with other organizations.

An adapted 'Ottawa' method allowed assessing the need to update topic areas within clinical practice guidelines

OBJECTIVE

To adapt and evaluate a method for assessing the need to update guideline topic areas involving multiple recommendations.

STUDY DESIGN AND SETTING

The 'Ottawa method' uses literature signals to determine changes in evidence that trigger a need to update individual guideline questions. We adapted the Ottawa method to include a process for aggregating updating signals by topic area (e.g., resuscitation) and tested this method using the German guideline on the treatment of patients with severe/multiple injuries. This involved a focused systematic evaluation of current evidence to identify updating signals, and classifying the need to update for each topic area. Then, we surveyed the guideline group online about the modified method.

RESULTS

We conducted focused literature searches for 37 topic areas and screened a mean of 97 abstracts per topic area in 2021. The need to update was high for eight (21.6%), intermediate for eight (21.6%), and low for 21 topic areas (56.8%) based on updating signals. The survey response rate was 56% (24/43). Most guideline group members (94%, 16/17 responders) would use the Ottawa method again, but their comments identified some weaknesses.

CONCLUSION

The modified Ottawa method is a suitable, efficient tool to generate evidence-based updating signals for guideline topic areas involving multiple recommendations. Further fine-tuning is recommended.

Feasibility of national living guideline methods: the Australian Stroke Guidelines

OBJECTIVE

Maintaining clinical guideline currency has been one challenge to traditional guideline development. This paper describes the methods used to maintain a large national guideline for stroke management.

STUDY DESIGN AND SETTING

The Australian Stroke Clinical Guidelines are developed to meet Australian National Health and Medical Research Council (NHMRC) standards. Monthly surveillance is conducted for new systematic reviews and randomised controlled studies. Included studies undergo data extraction followed by preparation of updated evidence-to-decision frameworks which are used to inform updates, or development of new recommendations. Small writing groups made up of clinical experts and those with lived experience review and agree on changes, which are finally reviewed by a multidisciplinary Guidelines Steering Group. Draft changes are developed and published using the online MAGICapp platform, with dissemination and promotion via traditional methods as well as social media.

RESULTS

Each month approximately 350 abstracts are considered, covering 96 clinical topics and taking on average 16 hours to review. There have been four major guideline updates covering 34 new and updated recommendations.

CONCLUSION

It is feasible to use 'living' methods to maintain the Australian Clinical Guidelines for Stroke Management. Further work is now needed to understand the impact of living guidelines.

The UpPriority tool supported prioritization processes for updating clinical guideline questions

OBJECTIVE

We aim to 1) use the UpPriority tool to identify which clinical questions (CQs) within the clinical guidelines (CGs) need to be prioritized for updating and 2) assess the implementation of the tool in a real-world set of CGs.

STUDY DESIGN AND SETTING

We systematically assessed CQs from a sample of CGs developed in the Spanish National Health System CG program. We applied the UpPriority tool to each CG using a step-by-step process that included: 1) establishment of the UpPriority Implementation Working Group, 2) mapping of the original CG questions and recommendations, 3) development of a survey to prioritize CQs, 4) assessment of CQ's priority according to six items, 5) calculation and ranking of priority scores, 6) decision of prioritized CQs for updating, and 7) development of the priority report. We assessed the tool implementation process (appraisers' experience when using the tool) and the inter-observer reliability of the tool, and we provided suggestions for improvement.

RESULTS

We included four CGs with a total of 107 CQs on the following topics: chronic heart failure (10 CQs), inherited retinal dystrophies (39 CQs), menopause (20 CQs), and open-angle glaucoma (38 CQs). We included a total of 30 participants, most of them clinicians that were members of the original CG development groups. CQs were classified in three groups: 1) high priority (CQs prioritized for updating [16/107; 15.0%]), 2) medium priority (CQs that could be prioritized for updating [47/107; 43.9%]), and low priority (CQs that were not prioritized for updating [44/107; 41.1%]). The mean time each appraiser needed to assess the CQs with the tool was 3.8 hours (range 0.5 to 10). Agreement among the appraisers varied among the CGs. Appraisers considered that the tool was useful. We suggest some areas for consideration when using the tool including: 1) identification of key appraisers, 2) customization of training materials, 3) establishment of priority thresholds, and 4) provision of methodological support.

CONCLUSION

The UpPriority is a useful tool to identify which CQs within a CG need to be prioritized for update in a real-world scenario. Recruitment and training of topic experts are the main challenges when using the tool.

GRADE-Based Recommendations for Surgical Repair of Nonruptured Abdominal Aortic Aneurysm

The objective of this study was to provide evidence-based recommendations for endovascular aneurysm repair (EVAR) versus open surgical repair (OSR) for patients with a nonruptured abdominal aortic aneurysm (AAA). We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement and adhered to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. Both low- and high surgical risk patients treated with EVAR showed decreased 30-day mortality, but the low-risk group had no differences in 4-year mortality. Compared with friendly anatomy, patients with hostile anatomy had an increased risk of type I endoleak. Young patients may prefer OSR. Endovascular aneurysm repair was not cost-effective in Europe. Four conditional recommendations were formulated: (1) OSR for low-risk patients up to 80 years old, (2) EVAR for low-risk patients older than 80 years, (3) EVAR for high-risk patients as long as is anatomically feasible, and (4) OSR in patients in whom it is not anatomically feasible to perform EVAR. Based on GRADE criteria, either OSR or EVAR can be suggested to patients with nonruptured AAA taking into account their surgical risk, hostile anatomy, and age. Given the weakness of the recommendations, personal preferences are determinant.

Updated clinical guidelines experience major reporting limitations

BACKGROUND

The Checklist for the Reporting of Updated Guidelines (CheckUp) was recently developed. However, so far, no systematic assessment of the reporting of updated clinical guidelines (CGs) exists. We aimed to examine (1) the completeness of reporting the updating process in CGs and (2) the inter-observer reliability of CheckUp.

METHODS

We conducted a systematic assessment of the reporting of the updating process in a sample of updated CGs using CheckUp. We performed a systematic search to identify updated CGs published in 2015, developed by a professional society, reporting a systematic review of the evidence, and containing at least one recommendation. Three reviewers independently assessed the CGs with CheckUp (16 items). We calculated the median score per item, per domain, and overall, converting scores to a 10-point scale. Multiple linear regression analyses were used to identify differences according to country, type of organisation, scope, and health topic of updated CGs. We calculated the intraclass coefficient (ICC) and 95% confidence interval (95% CI) for domains and overall score.

RESULTS

We included in total 60 updated CGs. The median domain score on a 10-point scale for presentation was 5.8 (range 1.7 to 10), for editorial independence 8.3 (range 3.3 to 10), and for methodology 5.7 (range 0 to 10). The median overall score on a 10-point scale was 6.3 (range 3.1 to 10). Presentation and justification items at recommendation level (respectively reported by 27 and 38% of the CGs) and the methods used for the external review and implementing changes in practice were particularly poorly reported (both reported by 38% of the CGs). CGs developed by a European or international institution obtained a statistically significant higher overall score compared to North American or Asian institutions (p = 0.014). Finally, the agreement among the reviewers on the overall score was excellent (ICC 0.88, 95% CI 0.75 to 0.95).

CONCLUSIONS

The reporting of updated CGs varies considerably with significant room for improvement. We recommend using CheckUp to assess the updating process in updated CGs and as a blueprint to inform methods and reporting strategies in updating.

Living systematic reviews: 4. Living guideline recommendations

While it is important for the evidence supporting practice guidelines to be current, that is often not the case. The advent of living systematic reviews has made the concept of "living guidelines" realistic, with the promise to provide timely, up-to-date and high-quality guidance to target users. We define living guidelines as an optimization of the guideline development process to allow updating individual recommendations as soon as new relevant evidence becomes available. A major implication of that definition is that the unit of update is the individual recommendation and not the whole guideline. We then discuss when living guidelines are appropriate, the workflows required to support them, the collaboration between living systematic reviews and living guideline teams, the thresholds for changing recommendations, and potential approaches to publication and dissemination. The success and sustainability of the concept of living guideline will depend on those of its major pillar, the living systematic review. We conclude that guideline developers should both experiment with and research the process of living guidelines.

Development of a prioritisation tool for the updating of clinical guideline questions: the UpPriority Tool protocol

INTRODUCTION

Due to a continuous emergence of new evidence, clinical guidelines (CGs) require regular surveillance of evidence to maintain their trustworthiness. The updating of CGs is resource intensive and time consuming; therefore, updating may include a prioritisation process to efficiently ensure recommendations remain up to date. The objective of our project is to develop a pragmatic tool to prioritise clinical questions for updating within a CG.

METHODS AND ANALYSIS

To develop the tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating and will adopt a methodological approach we have successfully implemented in a previous experience.We will perform a multistep process including (1) generation of an initial version of the tool, (2) optimisation of the tool (feasibility test of the tool, semistructured interviews, Delphi consensus survey, external review by CG methodologists and users and pilot test of the tool) and (3) approval of the final version of the tool.At each step of the process, we will (1) calculate absolute frequencies and proportions (quantitative data), (2) use content analysis to summarise and draw conclusions (qualitative data) and (3) draft a final report, discuss results and refine the previous versions of the tool. Finally, we will calculate intraclass coefficients with 95% CIs for each item and overall as indicators of agreement among reviewers.

ETHICS AND DISSEMINATION

We have obtained a waiver of approval from the Clinical Research Ethics Committee at the Hospital de la Santa Creu i Sant Pau (Barcelona). The results of the study will be published in peer-reviewed journal and communicated to interested stakeholders.The tool could support the standardisation of prioritisation processes for updating CGs and therefore have important implications for a more efficient use of resources in the CG field.

Methodological systematic review identifies major limitations in prioritization processes for updating

OBJECTIVES

The aim of the study was to identify and describe strategies to prioritize the updating of systematic reviews (SRs), health technology assessments (HTAs), or clinical guidelines (CGs).

STUDY DESIGN AND SETTING

We conducted an SR of studies describing one or more methods to prioritize SRs, HTAs, or CGs for updating. We searched MEDLINE (PubMed, from 1966 to August 2016) and The Cochrane Methodology Register (The Cochrane Library, Issue 8 2016). We hand searched abstract books, reviewed reference lists, and contacted experts. Two reviewers independently screened the references and extracted data.

RESULTS

We included 14 studies. Six studies were classified as descriptive (6 of 14, 42.9%) and eight as implementation studies (8 of 14, 57.1%). Six studies reported an updating strategy (6 of 14, 42.9%), six a prioritization process (6 of 14, 42.9%), and two a prioritization criterion (2 of 14, 14.2%). Eight studies focused on SRs (8 of 14, 57.1%), six studies focused on CGs (6 of 14, 42.9%), and none were about HTAs. We identified 76 prioritization criteria that can be applied when prioritizing documents for updating. The most frequently cited criteria were as follows: available evidence (19 of 76, 25.0%), clinical relevance (10 of 76; 13.2%), and users' interest (10 of 76; 13.2%).

CONCLUSION

There is wide variability and suboptimal reporting of the methods used to develop and implement processes to prioritize updating of SRs, HTAs, and CGs.