Updating clinical recommendations for breast, colorectal and lung cancer treatments: an opportunity to improve methodology and clinical relevance

A systematic decision-making process on the need for updating clinical practice guidelines proved to be feasible in a pilot study

OBJECTIVES

The objective of this study was to test and evaluate a new decision-making process on the need for updating within the update of a German clinical practice guideline (CPG).

STUDY DESIGN AND SETTING

The pilot study comprised (1) limited searches in Pubmed to identify new potentially relevant evidence, (2) an online survey among the members of the CPG group to assess the need for update, and (3) a consensus conference for determination and prioritization of guideline sections with a high need for update. Subsequently, we conducted a second online survey to evaluate the procedure.

RESULTS

The searches resulted in 902 abstracts that were graded as new potentially relevant evidence. Twenty five of 39 members of the CPG group (64%) participated in the online survey. Seventy six percent of those took part in the second online survey. The evaluation study found on average a grade of support of the procedure regarding the determination of the need for update of 3.65 (standard deviation: 0.76) on a likert scale with 1 = "no support" to 5 = "very strong support."

CONCLUSION

The conducted procedure presents a systematic approach for assessing whether and to what extent a CPG requires updating and enables setting priorities for which particular guideline section to update within a CPG.

Updated clinical guidelines experience major reporting limitations

BACKGROUND

The Checklist for the Reporting of Updated Guidelines (CheckUp) was recently developed. However, so far, no systematic assessment of the reporting of updated clinical guidelines (CGs) exists. We aimed to examine (1) the completeness of reporting the updating process in CGs and (2) the inter-observer reliability of CheckUp.

METHODS

We conducted a systematic assessment of the reporting of the updating process in a sample of updated CGs using CheckUp. We performed a systematic search to identify updated CGs published in 2015, developed by a professional society, reporting a systematic review of the evidence, and containing at least one recommendation. Three reviewers independently assessed the CGs with CheckUp (16 items). We calculated the median score per item, per domain, and overall, converting scores to a 10-point scale. Multiple linear regression analyses were used to identify differences according to country, type of organisation, scope, and health topic of updated CGs. We calculated the intraclass coefficient (ICC) and 95% confidence interval (95% CI) for domains and overall score.

RESULTS

We included in total 60 updated CGs. The median domain score on a 10-point scale for presentation was 5.8 (range 1.7 to 10), for editorial independence 8.3 (range 3.3 to 10), and for methodology 5.7 (range 0 to 10). The median overall score on a 10-point scale was 6.3 (range 3.1 to 10). Presentation and justification items at recommendation level (respectively reported by 27 and 38% of the CGs) and the methods used for the external review and implementing changes in practice were particularly poorly reported (both reported by 38% of the CGs). CGs developed by a European or international institution obtained a statistically significant higher overall score compared to North American or Asian institutions (p = 0.014). Finally, the agreement among the reviewers on the overall score was excellent (ICC 0.88, 95% CI 0.75 to 0.95).

CONCLUSIONS

The reporting of updated CGs varies considerably with significant room for improvement. We recommend using CheckUp to assess the updating process in updated CGs and as a blueprint to inform methods and reporting strategies in updating.

Reporting Items for Updated Clinical Guidelines: Checklist for the Reporting of Updated Guidelines (CheckUp)

BACKGROUND

Scientific knowledge is in constant development. Consequently, regular review to assure the trustworthiness of clinical guidelines is required. However, there is still a lack of preferred reporting items of the updating process in updated clinical guidelines. The present article describes the development process of the Checklist for the Reporting of Updated Guidelines (CheckUp).

METHODS AND FINDINGS

We developed an initial list of items based on an overview of research evidence on clinical guideline updating, the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument, and the advice of the CheckUp panel (n = 33 professionals). A multistep process was used to refine this list, including an assessment of ten existing updated clinical guidelines, interviews with key informants (response rate: 54.2%; 13/24), a three-round Delphi consensus survey with the CheckUp panel (33 participants), and an external review with clinical guideline methodologists (response rate: 90%; 53/59) and users (response rate: 55.6%; 10/18). CheckUp includes 16 items that address (1) the presentation of an updated guideline, (2) editorial independence, and (3) the methodology of the updating process. In this article, we present the methodology to develop CheckUp and include as a supplementary file an explanation and elaboration document.

CONCLUSIONS

CheckUp can be used to evaluate the completeness of reporting in updated guidelines and as a tool to inform guideline developers about reporting requirements. Editors may request its completion from guideline authors when submitting updated guidelines for publication. Adherence to CheckUp will likely enhance the comprehensiveness and transparency of clinical guideline updating for the benefit of patients and the public, health care professionals, and other relevant stakeholders.

Point of care information services: a platform for self-directed continuing medical education for front line decision makers

The structure and aim of continuing medical education (CME) is shifting from the passive transmission of knowledge to a competency-based model focused on professional development. Self-directed learning is emerging as the foremost educational method for advancing competency-based CME. In a field marked by the constant expansion of knowledge, self-directed learning allows physicians to tailor their learning strategy to meet the information needs of practice. Point of care information services are innovative tools that provide health professionals with digested evidence at the front line to guide decision making. By mobilising self-directing learning to meet the information needs of clinicians at the bedside, point of care information services represent a promising platform for competency-based CME. Several points, however, must be considered to enhance the accessibility and development of these tools to improve competency-based CME and the quality of care.

Updating contextualized clinical practice guidelines on stroke rehabilitation and low back pain management using a novel assessment framework that standardizes decisions

BACKGROUND

Clinical practice guidelines need to be regularly updated with current literature in order to remain relevant. This paper reports on the approach taken by the Philippine Academy of Rehabilitation Medicine (PARM). This dovetails with its writing guide, which underpinned its foundational work in contextualizing guidelines for stroke and low back pain (LBP) in 2011.

METHODS

Working groups of Filipino rehabilitation physicians and allied health practitioners met to reconsider and modify, where indicated, the 'typical' Filipino patient care pathways established in the foundation guidelines. New clinical guidelines on stroke and low back pain which had been published internationally in the last 3 years were identified using a search of electronic databases. The methodological quality of each guideline was assessed using the iCAHE Guideline Quality Checklist, and only those guidelines which provided full text references, evidence hierarchy and quality appraisal of the included literature, were included in the PARM update. Each of the PARM-endorsed recommendations was then reviewed, in light of new literature presented in the included clinical guidelines. A novel standard updating approach was developed based on the criteria reported by Johnston et al. (Int J Technol Assess Health Care 19(4):646-655, 2003) and then modified to incorporate wording from the foundational PARM writing guide. The new updating tool was debated, pilot-tested and agreed upon by the PARM working groups, before being applied to the guideline updating process.

RESULTS

Ten new guidelines on stroke and eleven for low back pain were identified. Guideline quality scores were moderate to good, however not all guidelines comprehensively linked the evidence body underpinning recommendations with the literature. Consequently only five stroke and four low back pain guidelines were included. The modified PARM updating guide was applied by all working groups to ensure standardization of the wording of updated recommendations and the underpinning evidence bases.

CONCLUSIONS

The updating tool provides a simple, standard and novel approach that incorporates evidence hierarchy and quality, and wordings of recommendations. It could be used efficiently by other guideline updaters particularly in developing countries, where resources for guideline development and updates are limited. When many people are involved in guideline writing, there is always the possibility of 'slippage' in use of wording and interpretation of evidence. The PARM updating tool provides a mechanism for maintaining a standard process for guideline updating processes that can be followed by clinicians with basic training in evidence-based practice principles.

Partial updating of clinical practice guidelines often makes more sense than full updating: a systematic review on methods and the development of an updating procedure

OBJECTIVES

To conduct a systematic review of the methods used to determine when and how to update clinical practice guidelines (CPGs) and develop a procedure for updating CPGs.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, and the Cochrane Methodology Register for methodological publications on updating CPGs. Guideline development manuals were obtained from the Web sites of guideline-developing organizations. Using the information obtained from these records, a procedure for updating CPGs was developed.

RESULTS

A total of 5,116 journal articles were screened, and seven articles met the criteria for inclusion. Forty-seven manuals were included; of these, eight included details about the methods used to update the guidelines. Most of the included publications focused on assessing whether the CPGs needed updating and not on how to update them. The developed procedure includes a systematic monitoring system and a scheduled process for updating the CPGs, which includes guidance on how to determine the type and scope of an update.

CONCLUSION

Partial updating often makes more sense than updating the whole CPG because topics and recommendations differ in terms of the need for updating. Guideline developers should implement a systematic updating procedure that includes an ongoing monitoring system that is appropriate for the nature of the guideline topics and the capabilities of the developers.

Strategies to assess the validity of recommendations: a study protocol

Background

Clinical practice guidelines (CPGs) become quickly outdated and require a periodic reassessment of evidence research to maintain their validity. However, there is little research about this topic. Our project will provide evidence for some of the most pressing questions in this field: 1) what is the average time for recommendations to become out of date?; 2) what is the comparative performance of two restricted search strategies to evaluate the need to update recommendations?; and 3) what is the feasibility of a more regular monitoring and updating strategy compared to usual practice?. In this protocol we will focus on questions one and two.

Methods

The CPG Development Programme of the Spanish Ministry of Health developed 14 CPGs between 2008 and 2009. We will stratify guidelines by topic and by publication year, and include one CPG by strata.

We will develop a strategy to assess the validity of CPG recommendations, which includes a baseline survey of clinical experts, an update of the original exhaustive literature searches, the identification of key references (reference that trigger a potential recommendation update), and the assessment of the potential changes in each recommendation.

We will run two alternative search strategies to efficiently identify important new evidence: 1) PLUS search based in McMaster Premium LiteratUre Service (PLUS) database; and 2) a Restrictive Search (ReSe) based on the least number of MeSH terms and free text words needed to locate all the references of each original recommendation.

We will perform a survival analysis of recommendations using the Kaplan-Meier method and we will use the log-rank test to analyse differences between survival curves according to the topic, the purpose, the strength of recommendations and the turnover. We will retrieve key references from the exhaustive search and evaluate their presence in the PLUS and ReSe search results.

Discussion

Our project, using a highly structured and transparent methodology, will provide guidance of when recommendations are likely to be at risk of being out of date. We will also assess two novel restrictive search strategies which could reduce the workload without compromising rigour when CPGs developers check for the need of updating.

How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks

BACKGROUND

The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care.

OBJECTIVE

To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence.

METHODS

Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks--UpToDate, Physicians' Information Education Resource (PIER), DynaMed, and Best Practice--were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated.

RESULTS

478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks' treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ(2) (3)=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks).

CONCLUSIONS

In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.

Strategies for monitoring and updating clinical practice guidelines: a systematic review

Background

Scientific knowledge is in constant change. The flow of new information requires a frequent re-evaluation of the available research results. Clinical practice guidelines (CPGs) are not exempted from this phenomenon and need to be kept updated to maintain the validity of their recommendations. The objective of our review is to systematically identify, describe and assess strategies for monitoring and updating CPGs.

Study design and setting

We conducted a systematic review of studies evaluating one or more methods of updating (with or without monitoring) CPGs or recommendations. We searched MEDLINE (PubMed) and The Cochrane Methodology Register (The Cochrane Library) from 1966 to June 2012. Additionally, we hand-searched reference lists of the included studies and the Guidelines International Network book of abstracts. If necessary, we contacted study authors to obtain additional information.

Results

We included a total of eight studies. Four evaluated if CPGs were out of date, three updated CPGs, and one continuously monitored and updated CPGs. The most detailed reported phase of the process was the identification of new evidence. As opposed to studies updating guidelines, studies evaluating if CPGs were out of date applied restricted searches. Only one study compared a restricted versus an exhaustive search suggesting that a restricted search is sufficient to assess recommendations’ Validity. One study analyzed the survival time of CPGs and suggested that these should be reassessed every three years.

Conclusions

There is limited evidence about the optimal strategies for monitoring and updating clinical practice guidelines. A restricted search is likely to be sufficient to monitor new evidence and assess the need to update, however, more information is needed about the timing and type of search. Only the exhaustive search strategy has been assessed for the update of CPGs. The development and evaluation of more efficient strategies is needed to improve the timeliness and reduce the burden of maintaining the validity of CPGs.

Speed of updating online evidence based point of care summaries: prospective cohort analysis

OBJECTIVE

To evaluate the ability of international point of care information summaries to update evidence relevant to medical practice.

DESIGN

Prospective cohort bibliometric analysis.

SETTING

Top five point of care information summaries (Clinical Evidence, EBMGuidelines, eMedicine, Dynamed, UpToDate) ranked for coverage of medical conditions, editorial quality, and evidence based methodology.

MAIN OUTCOME MEASURES

From June 2009 to May 2010 we measured the incidence of research findings relating to potentially eligible newsworthy evidence. As samples, we chose systematic reviews rated as relevant by international research networks (such as, Evidence-Based Medicine, ACP Journal Club, and the Cochrane Collaboration). Every month we assessed whether each sampled review was cited in at least one chapter of the five summaries. The cumulative updating rate was analysed with Kaplan-Meier curves. Results From April to December 2009, 128 reviews were retrieved; 53% (68) from the literature surveillance journals and 47% (60) from the Cochrane Library. At nine months, Dynamed had cited 87% of the sampled reviews, while the other summaries had cited less than 50%. The updating speed of Dynamed clearly led the others. For instance, the hazard ratios for citations in EBM Guidelines and Clinical Evidence versus the top performer were 0.22 (95% confidence interval 0.17 to 0.29) and 0.03 (0.01 to 0.05). Conclusions Point of care information summaries include evidence relevant to practice at different speeds. A qualitative analysis of updating mechanisms is needed to determine whether greater speed corresponds to more appropriate incorporation of new information.