Physicians found an interactive tool displaying structured evidence summaries for multiple comparisons understandable and useful: a qualitative user testing study

OBJECTIVES

To evaluate and improve "Making Alternative Treatment Choices Intuitive and Trustworthy" (MATCH-IT)-a digital, interactive decision support tool displaying structured evidence summaries for multiple comparisons-to help physicians interpret and apply evidence from network meta-analysis (NMA) for their clinical decision-making.

STUDY DESIGN AND SETTING

We conducted a qualitative user testing study, applying principles from user-centered design in an iterative development process. We recruited a convenience sample of practicing physicians in Norway, Belgium, and Canada, and asked them to interpret structured evidence summaries for multiple comparisons-linked to clinical guideline recommendations-displayed in MATCH-IT. User testing included (a) introduction of a clinical scenario, (b) a think-aloud session with participant-tool interaction, and (c) a semistructured interview. We video recorded, transcribed, and analyzed user tests using directed content analysis. The results informed new updates in MATCH-IT.

RESULTS

Distributed across 5 development cycles we tested MATCH-IT with 26 physicians. Of these, 24 (94%) reported either no or sparse prior experience with interpretation of NMA. Physicians perceived MATCH-IT as easy to interpret and navigate, and appreciated its ability to provide an overview of the evidence. Visualization of effects in pictograms and inclusion of information on burden of treatment ("practical issues") were highlighted as potentially useful features in interacting with patients. We also identified problems, including undiscovered functionalities (drag and drop), suboptimal tutorial, and cumbersome navigation of the tool. In addition, physicians wanted definition/explanation of key terms (eg, outcomes and "certainty"), and there were concerns that overwhelming evidence from a large NMA would complicate applicability to clinical practice. This led to several updates with development of a new start page, tutorial, updated user interface for more efficient maneuvering, solutions to display definition of key terms and a "frequently asked questions" section. To facilitate interpretation of large networks, we improved categorization of results using color coding and added filtering functionality. These modifications allowed physicians to focus on interventions of interest and reduce information overload.

CONCLUSION

This study provides proof of concept that physicians can use MATCH-IT to understand NMA evidence. Key features of MATCH-IT in a clinical context include providing an overview of the evidence, visualization of effects, and the display of information on burden of treatments. However, unfamiliarity with the Grading of Recommendations Assessment, Development and Evaluation concepts, time constraints, and accessibility at the point of care may be challenges for use. To what extent our results are transferable to real-world clinical contexts remains to be explored.

Effective web-based clinical practice guidelines resources: recommendations from a mixed methods usability study

BACKGROUND

Clinical practice guidelines (CPG) are an important knowledge translation resource to help clinicians stay up to date about relevant clinical knowledge. Effective communication of guidelines, including format, facilitates its implementation. Despite the digitalization of healthcare, there is little literature to guide CPG website creation for effective dissemination and implementation. Our aim was to assess the effectiveness of the content and format of the Diabetes Canada CPG website, and use our results to inform recommendations for other CPG websites.

METHODS

Fourteen clinicians (family physicians, nurses, pharmacists, and dieticians) in diabetes care across Canada participated in this mixed-methods study (questionnaires, usability testing and interviews). Participants "thought-aloud" while completing eight usability tasks on the CPG website. Outcomes included task success rate, completion time, click per tasks, resource used, paths, search attempts and success rate, and error types. Participants were then interviewed.

RESULTS

The Diabetes Canada CPG website was found to be usable. Participants had a high task success rate of 79% for all tasks and used 144 (standard deviation (SD) = 152) seconds and 4.6 (SD = 3.9) clicks per task. Interactive tools were most frequently used compared to full guidelines and static tools. Misinterpretation accounted for 48% of usability errors. Participants overall found the website intuitive, with effective content and design elements.

CONCLUSION

Different versions of CPG information (e.g. interactive tools, quick reference guide, static tools) can help answer clinical questions more quickly. Effective web design should be assessed during CPG website creation for effective guideline dissemination and implementation.

Decision aids linked to the recommendations in clinical practice guidelines: results of the acceptability of a decision aid for patients with generalized anxiety disorder

Background

Generalized anxiety disorder (GAD) is one of the most prevalent mental health problems. Patients with GAD have unmet needs related to the information received about their disorder, its treatments and their participation in the decision-making process. The aim of this study is to develop and assess the acceptability of a patient decision aid (PtDA) for patients with GAD.

Method

The PtDA was developed following the International Patient Decision Aid Standards. The recommendations of the Spanish clinical practice guideline (CPG) for patients with GAD were used as the basis. The first prototype was developed by an expert committee, further improvements were made with patients (n = 2), clinical experts (n = 13) and the project management group (n = 7). The acceptability of this second draft was assessed by patients non-involved in the previous phases (n = 11).

Results

The final PtDA version included a brief description of GAD and its treatments. Most participants agreed that the PtDA was easy to use, visually appealing and useful. At least half of the participants learned new things about treatments and adverse effects.

Conclusions

A PtDA was developed for patients with GAD based on recommendations from the Spanish CPG. It was improved and accepted by patients and clinical experts involved. An evaluation of its effectiveness on the shared decision-making process during the clinical encounter is planned.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01899-2.

User testing of a Scottish Intercollegiate Guideline Network public guideline for the parents of children with autism

Background

The Scottish Intercollegiate Guidelines Network (SIGN) is the leading national clinical guideline producer in Scotland. Improved design and dissemination of guidelines produced for the public can empower people to take an active role in self-management and shared decision-making. The public version of the guideline examined covered getting assessed and diagnosed with autism, and approaches that can help. The aim of this study was to test a public version of a guideline for the parents of children and young people with autism, implement improvements, and identify what works in making it usable and accessible.

Methods

We recruited mothers from across Scotland. User testing involved formal ‘think aloud’ semi-structured interviews that guided users through the booklet. Interviews took place individually and were recorded and transcribed. Key findings were identified and themed using the honeycomb user experience model.

Results

Fourteen user-testing interviews were conducted. Facilitators for usability and desirability of the guideline included the chunking of text, consistent use of colour and boxes to highlight important information. Simple language, written in a tone of partnership, helped to engage mothers. Value arose from the guidelines ability to explain the process of diagnosis and make mothers feel empowered in their relationships with healthcare professionals. There was a lack of consensus on the usefulness of rating the strength of evidence and recommendations.

Conclusion

There was a marked similarity between what was important to the mothers and what has been found to be important to other groups. The involvement of service users and carers in the guidelines development was key to its credibility. One size does not fit all in presenting evidence-based recommendations to the public and it is a challenge to provide sufficient information while avoiding information overload. Recommendations and evidence levels are suitable for use in public versions, but these should be kept as simple as possible.

A scoping review of knowledge authoring tools used for developing computerized clinical decision support systems

Objective

Clinical Knowledge Authoring Tools (CKATs) are integral to the computerized Clinical Decision Support (CDS) development life cycle. CKATs enable authors to generate accurate, complete, and reliable digital knowledge artifacts in a relatively efficient and affordable manner. This scoping review aims to compare knowledge authoring tools and derive the common features of CKATs.

Materials and Methods

We performed a keyword-based literature search, followed by a snowball search, to identify peer-reviewed publications describing the development or use of CKATs. We used PubMed and Embase search engines to perform the initial search (n = 1579). After removing duplicate articles, nonrelevant manuscripts, and not peer-reviewed publication, we identified 47 eligible studies describing 33 unique CKATs. The reviewed CKATs were further assessed, and salient characteristics were extracted and grouped as common CKAT features.

Results

Among the identified CKATs, 55% use an open source platform, 70% provide an application programming interface for CDS system integration, and 79% provide features to validate/test the knowledge. The majority of the reviewed CKATs describe the flow of information, offer a graphical user interface for knowledge authors, and provide intellisense coding features (94%, 97%, and 97%, respectively). The composed list of criteria for CKAT included topics such as simulating the clinical setting, validating the knowledge, standardized clinical models and vocabulary, and domain independence. None of the reviewed CKATs met all common criteria.

Conclusion

Our scoping review highlights the key specifications for a CKAT. The CKAT specification proposed in this review can guide CDS authors in developing more targeted CKATs.

Towards User-Oriented Recommendations for Local Therapy of Leg and Foot Ulcers-An Update of a S3-German Guideline

Background: The German S3- guideline on local therapy of leg ulcers and diabetic foot ulcers is in the process of being updated. Major goals are to improve the guidelines’ applicability and to take steps towards a living guideline according to current methodological standards. The aim of this article is to describe the main measures to achieve these goals. Methods: The context of the guideline in the field of local wound care and the stakeholder requirements are briefly described. Based on a derived framework, the project team adjusted the methods for the guideline. Results: Main adjustments are more specific inclusion criteria, online consensus meetings and the use of an authoring and publication platform to provide information in a multi-layered format. A new set of practice-oriented key questions were defined by the guideline panel to foster the formulation of action-oriented recommendations. Conclusions: The set of new key questions addressing practical problems and patients’ preferences as well as the adjustments made to improve not only the guidelines’ applicability, but also the feasibility of the further dynamic updating processes in the sense of a living guideline, should be steps in the right direction.

Decision aids linked to evidence summaries and clinical practice guidelines: results from user-testing in clinical encounters

Background

Tools for shared decision-making (e.g. decision aids) are intended to support health care professionals and patients engaged in clinical encounters involving shared decision-making. However, decision aids are hard to produce, and onerous to update. Consequently, they often do not reflect best current evidence, and show limited uptake in practice. In response, we initiated the Sharing Evidence to Inform Treatment decisions (SHARE-IT) project. Our goal was to develop and refine a new generation of decision aids that are generically produced along digitally structured guidelines and evidence summaries.

Methods

Applying principles of human-centred design and following the International Patient Decision Aid Standards (IPDAS) and GRADE methods for trustworthy evidence summaries we developed a decision aid prototype in collaboration with the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence project (DECIDE). We iteratively user-tested the prototype in clinical consultations between clinicians and patients. Semi-structured interviews of participating clinicians and patients were conducted. Qualitative content analysis of both user-testing sessions and interviews was performed and results categorized according to a revised Morville’s framework of user-experience. We made it possible to produce, publish and use these decision aids in an electronic guideline authoring and publication platform (MAGICapp).

Results

Direct observations and analysis of user-testing of 28 clinical consultations between physicians and patients informed four major iterations that addressed readability, understandability, usability and ways to cope with information overload. Participants reported that the tool supported natural flow of the conversation and induced a positive shift in consultation habits towards shared decision-making. We integrated the functionality of SHARE-IT decision aids in MAGICapp, which has since generated numerous decision aids.

Conclusion

Our study provides a proof of concept that encounter decision aids can be generically produced from GRADE evidence summaries and clinical guidelines. Online authoring and publication platforms can help scale up production including continuous updating of electronic encounter decision aids, fully integrated with evidence summaries and clinical practice guidelines.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01541-7.

Developing Evidence to Decision Frameworks and an Interactive Evidence to Decision Tool for Making and Using Decisions and Recommendations in Health Care

Evidence-informed health care decisions and recommendations need to be made systematically and transparently. Mediating technology can help manage boundaries between groups making decisions and target audiences, enhancing salience, credibility, and legitimacy for all. This article describes the development of the Evidence to Decision (EtD) framework and an interactive tool to create and use frameworks (iEtD) to support communication in decision making. Methods: Using a human-centered design approach, we created prototypes employing a broad range of methods to iteratively develop EtD framework content and iEtD tool functionality. Results: We developed tailored EtD frameworks for making evidence-informed decisions and recommendations about clinical practice interventions, diagnostic and screening tests, coverage, and health system and public health options. The iEtD tool provides functionality for preparing frameworks, using them in group discussions, and publishing output for implementation or adaption. EtD and iEtD are intuitive and useful for producers and users of frameworks, and flexible for use across different types of topics, decisions, and organizations. They bring valued structure to panel discussions and transparency to published output. Conclusion: EtD and iEtD can resolve some of the challenges inherent in multicriteria, multistakeholder decision systems. They are freely available online for all to use at https://ietd.epistemonikos.org/ and https://gradepro.org.

UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey

INTRODUCTION

UpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified.

OBJECTIVES

Our objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations.

DESIGN

Analytical survey of all recommendations in UpToDate.

METHODS

We identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate.

RESULTS

UpToDate included 9451 GRADE recommendations, of which 6501 (68.8%) were formulated as weak recommendations and 2950 (31.2%) as strong. Among the strong, 844 (28.6%) were based on high certainty in effect estimates, 1740 (59.0%) on moderate certainty and 366 (12.4%) on low certainty. Of the 349 discordant recommendations 204 (58.5%) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5%) as good practice statements; 38 (10.9%) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2%) warranted a weak rather than a strong recommendation.

CONCLUSION

The proportion of discordant recommendations in UpToDate is small (3.7% of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6%). Clinicians should nevertheless be cautious and look for clear explanations-in UpToDate and elsewhere-when guidelines offer strong recommendations based on low certainty evidence.

Reviewing clinical guideline development tools: features and characteristics

BACKGROUND

To improve consistency and streamline development and publication of clinical guidelines (GL), there is a need for appropriate software support. We have found few specific tools for the actual authoring and maintaining of GLs, and correspondingly few analyses or reviews of GL development tool functionality. In order to assist GL developers in selecting and evaluating tools, this study tries to address the perceived gap by pursuing four goals: 1) identifying available tools, 2) reviewing a representative group of tools and their supported functionalities, 3) uncovering themes of features that the studied tools support, and 4) compare the selected tools with respect to the themes.

METHODS

We conducted a literature search using PubMed and Google Scholar in order to find GL development tools (GDT). We also explored tools and Content Management Systems (CMS) used in representative organisations and international communities that develop and maintain GLs. By reading a selected representative group of five GL tool manuals, exploring tools hands-on, we uncovered 8 themes of features. All found tools were compared according to these themes in order to identify the level of functionality they offer to support the GL development and publishing process. In order to limit the scope, tools for designing computer-interpretable/executable GL are excluded.

RESULTS

After finding 1552 published papers, contacting 7 organizations and international communities, we identified a total of 19 unique tools, of which 5 tools were selected as representative in this paper. We uncovered a total of 8 themes of features according to the identified functionalities that each tool provides. Four features were common among tools: Collaborative authoring process support, user access control, GL repository management, electronic publishing. We found that the GRADE methodology was supported by three of the reviewed tools, while only two tools support annotating GL with MeSH terms. We also identified that monitoring progress, reference management, Managing versions (version control), and Change control (tracking) were often the missing features.

CONCLUSION

The results can promote sector discussion and eventual agreement on important tool functionality. It may aid tool and GL developers towards more efficient, and effective, GL authoring.

[GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction]

Clinicians, guideline developers, and policymakers sometimes neglect important criteria, give undue weight to criteria, and do not use the best available evidence to inform their judgments. Explicit and transparent systems for decision making can help to ensure that all important criteria are considered and that decisions are informed by the best available research evidence. The GRADE Working Group has developed Evidence to Decision (EtD) frameworks for the different type of recommendations or decisions. The purpose of EtD frameworks is to help people use evidence in a structured and transparent way to inform decisions in the context of clinical recommendations, coverage decisions, and health system or public health recommendations and decisions. EtD frameworks have a common structure that includes formulation of the question, an assessment of the evidence, and drawing conclusions, though there are some differences between frameworks for each type of decision. EtD frameworks inform users about the judgments that were made and the evidence supporting those judgments by making the basis for decisions transparent to target audiences. EtD frameworks also facilitate dissemination of recommendations and enable decision makers in other jurisdictions to adopt recommendations or decisions, or adapt them to their context. This article is a translation of the original article published in British Medical Journal. The EtD frameworks are currently used in the Clinical Practice Guideline Programme of the Spanish National Health System, co-ordinated by GuíaSalud.

Clinical Practice Guidelines: A Primer on Development and Dissemination

Trustworthy clinical practice guidelines should be based on a systematic review of the literature, provide ratings of the quality of evidence and the strength of recommendations, consider patient values, and be developed by a multidisciplinary panel of experts. The quality of evidence reflects our certainty that the evidence warrants a particular action. Transforming evidence into a decision requires consideration of the quality of evidence, balance of benefits and harms, patients' values, available resources, feasibility of the intervention, acceptability by stakeholders, and effect on health equity. Empirical evidence shows that adherence to guidelines improves patient outcomes; however, adherence to guidelines is variable. Therefore, guidelines require active dissemination and innovative implementation strategies.

Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial

OBJECTIVES

To investigate practicing physicians' preferences, perceived usefulness and understanding of a new multilayered guideline presentation format-compared to a standard format-as well as conceptual understanding of trustworthy guideline concepts.

DESIGN

Participants attended a standardised lecture in which they were presented with a clinical scenario and randomised to view a guideline recommendation in a multilayered format or standard format after which they answered multiple-choice questions using clickers. Both groups were also presented and asked about guideline concepts.

SETTING

Mandatory educational lectures in 7 non-academic and academic hospitals, and 2 settings involving primary care in Lebanon, Norway, Spain and the UK.

PARTICIPANTS

181 practicing physicians in internal medicine (156) and general practice (25).

INTERVENTIONS

A new digitally structured, multilayered guideline presentation format and a standard narrative presentation format currently in widespread use.

PRIMARY AND SECONDARY OUTCOME MEASURES

Our primary outcome was preference for presentation format. Understanding, perceived usefulness and perception of absolute effects were secondary outcomes.

RESULTS

72% (95% CI 65 to 79) of participants preferred the multilayered format and 16% (95% CI 10 to 22) preferred the standard format. A majority agreed that recommendations (multilayered 86% vs standard 91%, p value=0.31) and evidence summaries (79% vs 77%, p value=0.76) were useful in the context of the clinical scenario. 72% of participants randomised to the multilayered format vs 58% for standard formats reported correct understanding of the recommendations (p value=0.06). Most participants elected an appropriate clinical action after viewing the recommendations (98% vs 92%, p value=0.10). 82% of the participants considered absolute effect estimates in evidence summaries helpful or crucial.

CONCLUSIONS

Clinicians clearly preferred a novel multilayered presentation format to the standard format. Whether the preferred format improves decision-making and has an impact on patient important outcomes merits further investigation.

Living cumulative network meta-analysis to reduce waste in research: A paradigmatic shift for systematic reviews?

In a recent research article in BMC Medicine, Créquit and colleagues demonstrate how published systematic reviews in lung cancer provide a fragmented, out-of-date picture of the evidence for all treatments. The results and conclusions drawn from this study, based on cumulative network meta-analyses (NMA) of evidence from randomized clinical trials over time, are quite compelling. The inherent waste of research resulting from incomplete evidence synthesis has wide-reaching implications for a range of target groups including developers of systematic reviews and guidelines and their end-users, health care professionals and patients at the point of care. Building on emerging concepts for living systematic reviews and NMA, the authors propose "living cumulative NMA" as a potential solution and paradigmatic shift. Here we describe how recent innovations within authoring, dissemination, and updating of systematic reviews and trustworthy guidelines may greatly facilitate the production of living NMA. Some additional challenges need to be solved for NMA in general, and for living cumulative NMA in particular, before a paradigmatic shift for systematic reviews can become reality.Please see related research article: https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-016-0555-0.

What do patients and the public know about clinical practice guidelines and what do they want from them? A qualitative study

BACKGROUND

Guideline producers are increasingly producing versions of guidelines for the public. The aim of this study was to explore what patients and the public understand about the purpose and production of clinical guidelines, and what they want from clinical guidelines to support their healthcare decisions.

METHODS

Participants were purposively selected to represent a range of the likely users of patient versions of guidelines, including individuals with health conditions (diabetes and depression), general members of the public, health communication professionals and a group of young people. Participants were asked about their awareness and understanding of clinical guidelines and presented with scenario recommendations, or draft materials from patient guidelines to prompt discussion. Each discussion was facilitated by one or two researchers. All focus groups were recorded and transcribed prior to analysis. Data were analysed using framework analysis.

RESULTS

We ran nine focus groups involving 62 individuals, supplemented by four interviews with people experiencing homelessness. Eight groups were held in Scotland, one in England. The four interviews were held in Scotland. The framework analysis yielded five themes: access and awareness; what patients want to know; properties of guidelines; presenting evidence; and format. Awareness of guidelines was low. Participants emphasised the need for information that enables them to choose between treatment options, including harms. They would like help with this from healthcare professionals, especially general practitioners. Participants differed in their support for the inclusion of numerical information and graphs.

CONCLUSIONS

Members of the public want information to help them choose between treatments, including information on harms, particularly to support shared decisions with health professionals. Presenting numerical information is a challenge and layered approaches that present information in stages may be helpful. Ignoring the themes identified in this study is likely to lead to materials that fail to support public and patient healthcare decision making.

Improving the user experience of patient versions of clinical guidelines: user testing of a Scottish Intercollegiate Guideline Network (SIGN) patient version

Background

Guideline producers are increasingly producing versions of guidelines for the public, and knowledge of what the public want from them is also increasing. The aim of this study was to user test a patient version of a SIGN clinical guideline that was designed based on preliminary work for the DECIDE project. SIGN is the leading national guideline producer in Scotland.

Methods

People with a diagnosis of glaucoma and non-professional carers were recruited from across Scotland. User testing was conducted using a think-aloud protocol method. Each session was conducted by one interviewer and one observer. All sessions were recorded and transcribed. The data was analysed, problems with the guideline were identified and resolved and key findings were themed using a user experience model.

Results

Thirteen user testing sessions with people with glaucoma and one with a carer were conducted. Key facilitators of desirability and usability identified include clear branding as a patient version and a clearly described purpose, audience and contents page. Other facilitators include a “friendly” tone which is achieved by the use of colour, quotes, icons, simple language and charts, and brief chunked text.

The value and usefulness of the patient guideline was influenced by its ability to: inform the public, link information to actions, and empower people in their interaction with healthcare professionals. Participants were disappointed by the lack of information on treatment in the patient version, which was outside its scope.

Information on the evidence based guideline production process and the involvement of appropriately skilled professionals was key to the credibility of the guideline. Lack of awareness of guidelines and guideline producing bodies, is a potentially serious threat to findability/accessibility.

Conclusions

It is important for guideline producers to maximise the user experience of the public when they access patient versions of guidelines, particularly given the current low level of access and awareness. One size does not fit all and guideline producers need to strike a balance between keeping the patient version simple and providing sufficient information to facilitate shared decision making and empower the public. Guideline producers may find the results of this study useful in designing their own patient versions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1287-8) contains supplementary material, which is available to authorized users.

Decision aids that really promote shared decision making: the pace quickens

Decision aids can help shared decision making, but most have been hard to produce, onerous to update, and are not being used widely. Thomas Agoritsas and colleagues explore why and describe a new electronic model that holds promise of being more useful for clinicians and patients to use together at the point of care

Adaptation of trustworthy guidelines developed using the GRADE methodology: a novel five-step process

BACKGROUND

Adaptation of guidelines for use at the national or local level can facilitate their implementation. We developed and evaluated an adaptation process in adherence with standards for trustworthy guidelines and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, aiming for efficiency and transparency. This article is the first in a series describing our adaptation of Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines for a Norwegian setting.

METHODS

Informed by the ADAPTE framework, we developed a five-step adaptation process customized to guidelines developed using GRADE: (1) planning, (2) initial assessment of the recommendations, (3) modification, (4) publication, and (5) evaluation. We developed a taxonomy for describing how and why recommendations from the parent guideline were modified and applied a mixed-methods case study design for evaluation of the process.

RESULTS

We published the adapted guideline in November 2013 in a novel multilayered format. The taxonomy for adaptation facilitated transparency of the modification process for both the guideline developers and the end users. We excluded 30 and modified 131 of the 333 original recommendations according to the taxonomy and developed eight new recommendations. Unforeseen obstacles related to acquiring a licensing agreement and procuring a publisher resulted in a 9-month delay. We propose modifications of the adaptation process to overcome these obstacles in the future.

CONCLUSIONS

This case study demonstrates the feasibility of a novel guideline adaptation process. Replication is needed to further validate the usefulness of the process in increasing the organizational and methodologic efficiency of guideline adaptation.

Applying new strategies for the national adaptation, updating, and dissemination of trustworthy guidelines: results from the Norwegian adaptation of the Antithrombotic Therapy and the Prevention of Thrombosis, 9th Ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines

BACKGROUND

The Antithrombotic Therapy and the Prevention of Thrombosis, 9th Edition: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (AT9) represent trustworthy international guidelines for antithrombotic treatment and thromboprophylaxis. We describe major changes to the format and content resulting from applying new strategies for guideline adaptation and dissemination.

METHODS

A Norwegian guideline panel of 46 experts completed a structured and systematic adaptation process, updated the recommendations based on new evidence, and rewrote the recommendations in an electronic multilayered presentation format. We published the adapted guideline using the web-based Making GRADE the Irresistible Choice Guideline Authoring and Publication Platform.

RESULTS

We applied a novel presentation format to 333 recommendations from 11 of the 15 management chapters in AT9 and condensed and restructured them into 249 recommendations in a multilayered format. We added additional relevant information, such as 29 best-practice statements about new oral anticoagulants and practical information sections for 121 recommendations. Common reasons for modifications included feasibility of the recommendations in a national context, disagreement with applied baseline risk estimates, and reevaluation of the balance between the benefits and harms of interventions in relation to assumed typical patient preferences and values. The adapted guideline was published and disseminated online in November 2013.

CONCLUSIONS

New strategies for adapting, updating, and disseminating trustworthy guidelines proved feasible and will provide Norwegian health-care professionals and patients with up-to-date guidance tailored to national circumstances.