Automating complex guidelines for chronic disease: lessons learned

Improving providers' survival estimates and selection of prognosis- and guidelines-appropriate treatment for patients with symptomatic bone metastases: Development of the Bone Metastases Ensemble Trees for Survival Decision Support Platform

RATIONALE, AIMS AND OBJECTIVES

In the management of symptomatic bone metastases, current practice guidelines do not provide clear methodology for selecting palliative radiotherapy (RT) regimens based on specific patient and disease features. Decision support aids may offer an effective means for translating the complex data needed to render individualised treatment decisions, yet no such tools are available for use in this setting. Thus, we describe the development of the Bone Metastases Ensemble Trees for Survival-Decision Support Platform (BMETS-DSP), which aims to optimise selection of evidence-based, individualised palliative RT regimens.

METHOD

The Ottawa Decision Support Framework was used as the theoretical basis for development of BMETS-DSP. First, we utilised stakeholder input and review of the literature to assess determinants underlying the provider decision. Based on this assessment and iterative stakeholder feedback, we developed the web-based, provider-facing BMETS-DSP. Consistent with the underlying theoretical framework, our design also included assessment of decision quality using the International Patient Decision Aids Standards (IPDAS) certification checklist.

RESULTS

Stakeholder input and review of 54 evidence-based publications identified the following determinants of the provider decision: estimated prognosis, characteristics of the target symptomatic lesion and the primary cancer type, consideration of alternative interventions, access to patient-specific recommendations, and patient preferences. Based on these determinants, we developed the BMETS-DSP that (1) collects patient-specific data, (2) displays an individualised predicted survival curve, and (3) provides case-specific, evidence-based recommendations regarding RT, open surgery, systemic therapy, and hospice referral to aid in the decision-making process. The finalised tool met IPDAS quality requirements. Preliminary results of a pilot assessment suggest impact of clinical outcomes.

CONCLUSIONS

We describe the successful development of a provider-facing decision support platform to aid in the provision of palliative RT in better alignment with patient and disease features. Impact of the BMETS-DSP on decision outcomes will be further assessed in a randomised, controlled study.

Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care

OBJECTIVES

 The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design.

METHODS

 We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts.

RESULTS

 We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality.

CONCLUSION

 Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Preferences regarding the way of use and design of a work ability prognosis support tool: a focus group study among professionals

PURPOSE

To explore the preferable way of use and design of a work ability prognosis support tool for insurance physicians (IPs) and labour experts (LEs), based on a prediction model for future changes in work ability among individuals applying for a work disability benefit.

METHODS

We conducted three focus groups with professionals of the Dutch Social Security Institute (17 IPs and 7 LEs). Data were audio recorded and qualitatively analysed according to the main principles of thematic analysis.

RESULTS

Clarity and ease of use were mentioned as important features of the tool. Most professionals preferred to make their own judgement during the work disability assessment interview with the claimant and afterwards verify their evaluation with the tool. Concerning preferences on the design of the tool, dividing work disability claimants into categories based on the outcome of the prediction model was experienced as the most straightforward and clear way of presenting the results. Professionals expected that this encourages them to use the tool and act accordingly.

CONCLUSIONS

The tool should be easy to access and interpret, to increase the chance that professionals will use it. This way it can optimally help professionals making accurate prognoses of future changes in work ability.Implications for rehabilitationA work ability prognosis support tool based on a prediction model for changes in work ability at one-year follow-up can help occupational health professionals in making accurate prognosis of individuals applying for a work disability benefit.To be used in occupational health practice, these tools should have a simple and easy-to-use design.Graphical risk presentation can be used to provide intuitive meaning to numerical information and support users' understanding.Taking professionals' preferences into account when developing these tools encourages professionals to use the tools and act accordingly.

Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support

Objective

Provider acceptance and associated patient outcomes are widely discussed in the evaluation of clinical decision support systems (CDSSs), but critical design criteria for tools have generally been overlooked. The objective of this work is to inform electronic health record alert optimization and clinical practice workflow by identifying, compiling, and reporting design recommendations for CDSS to support the efficient, effective, and timely delivery of high-quality care.

Material and Methods

A narrative review was conducted from 2000 to 2016 in PubMed and The Journal of Human Factors and Ergonomics Society to identify papers that discussed/recommended design features of CDSSs that are associated with the success of these systems.

Results

Fourteen papers were included as meeting the criteria and were found to have a total of 42 unique recommendations; 11 were classified as interface features, 10 as information features, and 21 as interaction features.

Discussion

Features are defined and described, providing actionable guidance that can be applied to CDSS development and policy. To our knowledge, no reviews have been completed that discuss/recommend design features of CDSS at this scale, and thus we found that this was important for the body of literature. The recommendations identified in this narrative review will help to optimize design, organization, management, presentation, and utilization of information through presentation, content, and function. The designation of 3 categories (interface, information, and interaction) should be further evaluated to determine the critical importance of the categories. Future work will determine how to prioritize them with limited resources for designers and developers in order to maximize the clinical utility of CDSS.

Conclusion

This review will expand the field of knowledge and provide a novel organization structure to identify key recommendations for CDSS.

Informatics and interaction: Applying human factors principles to optimize the design of clinical decision support for sepsis

In caring for patients with sepsis, the current structure of electronic health record systems allows clinical providers access to raw patient data without imputation of its significance. There are a wide range of sepsis alerts in clinical care that act as clinical decision support tools to assist in early recognition of sepsis; however, there are serious shortcomings in existing health information technology for alerting providers in a meaningful way. Little work has been done to evaluate and assess existing alerts using implementation and process outcomes associated with health information technology displays, specifically evaluating clinician preference and performance. We developed graphical model displays of two popular sepsis scoring systems, quick Sepsis Related Organ Failure Assessment and Predisposition, Infection, Response, Organ Failure, using human factors principles grounded in user-centered and interaction design. Models will be evaluated in a larger research effort to optimize alert design to improve the collective awareness of high-risk populations and develop a relevant point-of-care clinical decision support system for sepsis.

Decision-centred design in healthcare: The process of identifying a decision support tool for airway management

Current decision support interventions for airway management in anaesthesia lack the application of Human Factors Engineering; leading to interventions that can be disruptive, inefficient and error-inducing. This study followed a decision-centred design process to identify decision support that can assist anaesthesia teams with challenging airway management situations. Field observations, Critical Decision Method interviews and focus groups were conducted to identify the most difficult decisions and their requirements. Data triangulation narrowed the focus to key decisions related to preparation and planning, and the transitioning between airway techniques during difficulties. Five decision-support interventions were identified and positively rated by anaesthesia team members in relation to their perceived effectiveness. An organized airway equipment trolley was chosen as the most beneficial decision support intervention. This study reiterated the key importance of both Human Factors Engineering and data triangulation when designing for healthcare.

The design of decisions: Matching clinical decision support recommendations to Nielsen's design heuristics

OBJECTIVE

While general design heuristics exist for graphic user interfaces, it remains a challenge to facilitate the implementation of these heuristics for the design of clinical decision support. Our goals were to map a set of recommendations for clinical decision support design found in current literature to Jakob Nielsen's traditional usability heuristics and to suggest usability areas that need more investigation.

MATERIALS AND METHODS

Using a modified nominal group process, the research team discussed, classified, and mapped recommendations, organized as interface, information, and interaction, to design heuristics. A previous narrative review identified 42 recommendations from the literature to define the design and functional characteristics that impact the performance of CDS in terms of provider preference, process of care, and patient outcomes.

MAIN FINDINGS

We matched 20 out of 42 recommendations to heuristics. The mapping reveals gaps in both heuristics and recommendations, identifying a set of Nielsen's heuristics that are underrepresented in the literature and subsets of recommendations important to design not covered in Nielsen's heuristics. We attributed this, in part, to the evolution of technology since the inception of Nielsen's heuristics. The team created a new interaction heuristic: Integration into real-time workflow to consider the needs of the end-user in the clinical space.

DISCUSSION

Clinical decision support has enabled clinicians to better address arising information needs; however there remains a lack of evidence-based guidelines in terms of functional and design requirements.

CONCLUSION

Results from this review suggest that interaction design principles were not fully satisfied by the current literature of clinical decision support.

Impact of e-alert systems on the care of patients with acute kidney injury

With the recent advancement in electronic health record systems and meaningful use of information technology incentive programs (i.e., the American Recovery and Reinvestment Act, the Health Information Technology for Economic and Clinical Health Act, and the Centers for Medicare & Medicaid Services), interest in clinical decision support systems has risen. These systems have been used to examine a variety of different syndromes with variable reported effects. In recent years, electronic alerts (e-alerts) have been implemented at various institutions to decrease the morbidity associated with acute kidney injury (AKI). AKI is common, accounting for 1 in 7 hospital admissions, and is associated with increased length of hospital stay and mortality. AKI is often underrecognized, causing delayed intervention. The use of e-alerts may result in earlier recognition and intervention, as well as decreased morbidity and mortality. This must be balanced with the possibility of increased resource utilization that e-alerts may cause. Before widespread implementation, the ethical and legal consequences of not following e-alert recommendations must be established, and the optimal algorithm for AKI e-alert detection must be determined.

Effect of a Novel Clinical Decision Support Tool on the Efficiency and Accuracy of Treatment Recommendations for Cholesterol Management

BACKGROUND

The 2013 American College of Cardiology / American Heart Association Guidelines for the Treatment of Blood Cholesterol emphasize treatment based on cardiovascular risk. But finding time in a primary care visit to manually calculate cardiovascular risk and prescribe treatment based on risk is challenging. We developed an informatics-based clinical decision support tool, MayoExpertAdvisor, to deliver automated cardiovascular risk scores and guideline-based treatment recommendations based on patient-specific data in the electronic heath record.

OBJECTIVE

To assess the impact of our clinical decision support tool on the efficiency and accuracy of clinician calculation of cardiovascular risk and its effect on the delivery of guideline-consistent treatment recommendations.

METHODS

Clinicians were asked to review the EHR records of selected patients. We evaluated the amount of time and the number of clicks and keystrokes needed to calculate cardiovascular risk and provide a treatment recommendation with and without our clinical decision support tool. We also compared the treatment recommendation arrived at by clinicians with and without the use of our tool to those recommended by the guidelines.

RESULTS

Clinicians saved 3 minutes and 38 seconds in completing both tasks with MayoExpertAdvisor, used 94 fewer clicks and 23 fewer key strokes, and improved accuracy from the baseline of 60.61% to 100% for both the risk score calculation and guideline-consistent treatment recommendation.

CONCLUSION

Informatics solution can greatly improve the efficiency and accuracy of individualized treatment recommendations and have the potential to increase guideline compliance.

Prescriber preferences for behavioural economics interventions to improve treatment of acute respiratory infections: a discrete choice experiment

OBJECTIVE

To elicit prescribers' preferences for behavioural economics interventions designed to reduce inappropriate antibiotic prescribing, and compare these to actual behaviour.

DESIGN

Discrete choice experiment (DCE).

SETTING

47 primary care centres in Boston and Los Angeles.

PARTICIPANTS

234 primary care providers, with an average 20 years of practice.

MAIN OUTCOMES AND MEASURES

Results of a behavioural economic intervention trial were compared to prescribers' stated preferences for the same interventions relative to monetary and time rewards for improved prescribing outcomes. In the randomised controlled trial (RCT) component, the 3 computerised prescription order entry-triggered interventions studied included: Suggested Alternatives (SA), an alert that populated non-antibiotic treatment options if an inappropriate antibiotic was prescribed; Accountable Justifications (JA), which prompted the prescriber to enter a justification for an inappropriately prescribed antibiotic that would then be documented in the patient's chart; and Peer Comparison (PC), an email periodically sent to each prescriber comparing his/her antibiotic prescribing rate with those who had the lowest rates of inappropriate antibiotic prescribing. A DCE study component was administered to determine whether prescribers felt SA, JA, PC, pay-for-performance or additional clinic time would most effectively reduce their inappropriate antibiotic prescribing. Willingness-to-pay (WTP) was calculated for each intervention.

RESULTS

In the RCT, PC and JA were found to be the most effective interventions to reduce inappropriate antibiotic prescribing, whereas SA was not significantly different from controls. In the DCE however, regardless of treatment intervention received during the RCT, prescribers overwhelmingly preferred SA, followed by PC, then JA. WTP estimates indicated that each intervention would be significantly cheaper to implement than pay-for-performance incentives of $200/month.

CONCLUSIONS

Prescribing behaviour and stated preferences are not concordant, suggesting that relying on stated preferences alone to inform intervention design may eliminate effective interventions.

TRIAL REGISTRATION NUMBER

NCT01454947; Results.

Augmenting GEM-encoded clinical practice guidelines with relevant best evidence autonomously retrieved from MEDLINE

Clinical practice guidelines (CPG) are instrumental in standardizing the quality and delivery of care across different practitioners, departments and institutions. Health practitioners will use current best evidence to validate or supplement their understanding of CPG. This study investigates the potential of supplementing computerized CPG with relevant best evidence sourced from reliable medical literature repositories. A web-enabled Best-evidence Retrieval and Delivery (BiRD) system facilitates autonomous retrieval of pertinent medical literature with respect to user-specified content from a GEM-encoded CPG. A multilevel literature search strategy categorizes the search towards predefined clinical query intentions, and subsequently filters insignificant medical terms. The resultant is a highly focused medical literature search query that is objectively derived from CPG content. The technical architecture comprises existing medical language processing tools and vocabularies, together with newly developed tools to automatically generate optimum search queries, retrieve medical articles from MEDLINE, and embed the articles within XML-based CPG.

The development of an automated ward independent delirium risk prediction model

Background A delirium is common in hospital settings resulting in increased mortality and costs. Prevention of a delirium is clearly preferred over treatment. A delirium risk prediction model can be helpful to identify patients at risk of a delirium, allowing the start of preventive treatment. Current risk prediction models rely on manual calculation of the individual patient risk. Objective The aim of this study was to develop an automated ward independent delirium riskprediction model. To show that such a model can be constructed exclusively from electronically available risk factors and thereby implemented into a clinical decision support system (CDSS) to optimally support the physician to initiate preventive treatment. Setting A Dutch teaching hospital. Methods A retrospective cohort study in which patients, 60 years or older, were selected when admitted to the hospital, with no delirium diagnosis when presenting, or during the first day of admission. We used logistic regression analysis to develop a delirium predictive model out of the electronically available predictive variables. Main outcome measure A delirium risk prediction model. Results A delirium risk prediction model was developed using predictive variables that were significant in the univariable regression analyses. The area under the receiver operating characteristics curve of the "medication model" model was 0.76 after internal validation. Conclusions CDSSs can be used to automatically predict the risk of a delirium in individual hospitalised patients' by exclusively using electronically available predictive variables. To increase the use and improve the quality of predictive models, clinical risk factors should be documented ready for automated use.

A Framework for Usable and Effective Clinical Decision Support: Experience from the iCPR Randomized Clinical Trial

INTRODUCTION

The promise of Clinical Decision Support (CDS) has always been to transform patient care and improve patient outcomes through the delivery of timely and appropriate recommendations that are patient specific and, more often than not, are appropriately actionable. However, the users of CDS-providers-are frequently bombarded with inappropriate and inapplicable CDS that often are not informational, not integrated into the workflow, not patient specific, and that may present out of date and irrelevant recommendations.

METHODS

The integrated clinical prediction rule (iCPR) project was a randomized clinical trial (RCT) conducted to determine if a novel form of CDS, i.e., clinical prediction rules (CPRs), could be efficiently integrated into workflow and result in changes in outcomes (e.g., antibiotic ordering) when embedded within a commercial electronic health record (EHR). We use the lessons learned from the iCPR project to illustrate a framework for constructing usable, useful, and effective actionable CDS while employing off-the-shelf functionality in a production system. Innovations that make up the framework combine the following: (1) active and actionable decision support, (2) multiple rounds of usability testing with iterative development for user acceptance, (3) numerous context sensitive triggers, (4) dedicated training and support for users of the CDS tool for user adoption, and (5) support from clinical and administrative leadership. We define "context sensitive triggers" as being workflow events (i.e., context) that result in a CDS intervention.

DISCUSSION

Success of the framework can be measured by CDS adoption (i.e., intervention is being used), acceptance (compliance with recommendations), and clinical outcomes (where appropriate). This framework may have broader implications for the deployment of Health Information Technology (HIT).

RESULTS AND CONCLUSION

iCPR was well adopted(57.4% of users) and accepted (42.7% of users). Usability testing identified and fixed many issues before the iCPR RCT. The level of leadership support and clinical guidance for iCPR was key in establishing a culture of acceptance for both the tool and its recommendations contributing to adoption and acceptance. The dedicated training and support lead to the majority of the residents reporting a high level of comfort with both iCPR tools strep pharyngitis (64.4 percent) and pneumonia (62.7 percent) as well as a high likelihood of using the tools in the future. A surprising framework addition resulted from usability testing: context sensitive triggers.

Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics

CONTEXT

It is important to consider the way in which information is presented by the interfaces of clinical decision support systems, to favor the adoption of these systems by physicians. Interface design can focus on decision processes (guided navigation) or usability principles.

OBJECTIVE

The aim of this study was to compare these two approaches in terms of perceived usability, accuracy rate, and confidence in the system.

MATERIALS AND METHODS

We displayed clinical practice guidelines for antibiotic treatment via two types of interface, which we compared in a crossover design. General practitioners were asked to provide responses for 10 clinical cases and the System Usability Scale (SUS) for each interface. We assessed SUS scores, the number of correct responses, and the confidence level for each interface.

RESULTS

SUS score and percentage confidence were significantly higher for the interface designed according to usability principles (81 vs 51, p=0.00004, and 88.8% vs 80.7%, p=0.004). The percentage of correct responses was similar for the two interfaces.

DISCUSSION/CONCLUSION

The interface designed according to usability principles was perceived to be more usable and inspired greater confidence among physicians than the guided navigation interface. Consideration of usability principles in the construction of an interface--in particular 'effective information presentation', 'consistency', 'efficient interactions', 'effective use of language', and 'minimizing cognitive load'--seemed to improve perceived usability and confidence in the system.

A highly scalable, interoperable clinical decision support service

OBJECTIVE

To create a clinical decision support (CDS) system that is shareable across healthcare delivery systems and settings over large geographic regions.

MATERIALS AND METHODS

The enterprise clinical rules service (ECRS) realizes nine design principles through a series of enterprise java beans and leverages off-the-shelf rules management systems in order to provide consistent, maintainable, and scalable decision support in a variety of settings.

RESULTS

The ECRS is deployed at Partners HealthCare System (PHS) and is in use for a series of trials by members of the CDS consortium, including internally developed systems at PHS, the Regenstrief Institute, and vendor-based systems deployed at locations in Oregon and New Jersey. Performance measures indicate that the ECRS provides sub-second response time when measured apart from services required to retrieve data and assemble the continuity of care document used as input.

DISCUSSION

We consider related work, design decisions, comparisons with emerging national standards, and discuss uses and limitations of the ECRS.

CONCLUSIONS

ECRS design, implementation, and use in CDS consortium trials indicate that it provides the flexibility and modularity needed for broad use and performs adequately. Future work will investigate additional CDS patterns, alternative methods of data passing, and further optimizations in ECRS performance.

Using a service oriented architecture approach to clinical decision support: performance results from two CDS Consortium demonstrations

The Clinical Decision Support Consortium has completed two demonstration trials involving a web service for the execution of clinical decision support (CDS) rules in one or more electronic health record (EHR) systems. The initial trial ran in a local EHR at Partners HealthCare. A second EHR site, associated with Wishard Memorial Hospital, Indianapolis, IN, was added in the second trial. Data were gathered during each 6 month period and analyzed to assess performance, reliability, and response time in the form of means and standard deviations for all technical components of the service, including assembling and preparation of input data. The mean service call time for each period was just over 2 seconds. In this paper we report on the findings and analysis to date while describing the areas for further analysis and optimization as we continue to expand our use of a Services Oriented Architecture approach for CDS across multiple institutions.

A conceptual framework and protocol for defining clinical decision support objectives applicable to medical specialties

Background

The U.S. Centers for Medicare and Medicaid Services established the Electronic Health Record (EHR) Incentive Program in 2009 to stimulate the adoption of EHRs. One component of the program requires eligible providers to implement clinical decision support (CDS) interventions that can improve performance on one or more quality measures pre-selected for each specialty. Because the unique decision-making challenges and existing HIT capabilities vary widely across specialties, the development of meaningful objectives for CDS within such programs must be supported by deliberative analysis.

Design

We developed a conceptual framework and protocol that combines evidence review with expert opinion to elicit clinically meaningful objectives for CDS directly from specialists. The framework links objectives for CDS to specialty-specific performance gaps while ensuring that a workable set of CDS opportunities are available to providers to address each performance gap. Performance gaps may include those with well-established quality measures but also priorities identified by specialists based on their clinical experience. Moreover, objectives are not constrained to performance gaps with existing CDS technologies, but rather may include those for which CDS tools might reasonably be expected to be developed in the near term, for example, by the beginning of Stage 3 of the EHR Incentive program. The protocol uses a modified Delphi expert panel process to elicit and prioritize CDS meaningful use objectives. Experts first rate the importance of performance gaps, beginning with a candidate list generated through an environmental scan and supplemented through nominations by panelists. For the highest priority performance gaps, panelists then rate the extent to which existing or future CDS interventions, characterized jointly as “CDS opportunities,” might impact each performance gap and the extent to which each CDS opportunity is compatible with specialists’ clinical workflows. The protocol was tested by expert panels representing four clinical specialties: oncology, orthopedic surgery, interventional cardiology, and pediatrics.

Computerization of workflows, guidelines, and care pathways: a review of implementation challenges for process-oriented health information systems

OBJECTIVE

There is a need to integrate the various theoretical frameworks and formalisms for modeling clinical guidelines, workflows, and pathways, in order to move beyond providing support for individual clinical decisions and toward the provision of process-oriented, patient-centered, health information systems (HIS). In this review, we analyze the challenges in developing process-oriented HIS that formally model guidelines, workflows, and care pathways.

METHODS

A qualitative meta-synthesis was performed on studies published in English between 1995 and 2010 that addressed the modeling process and reported the exposition of a new methodology, model, system implementation, or system architecture. Thematic analysis, principal component analysis (PCA) and data visualisation techniques were used to identify and cluster the underlying implementation 'challenge' themes.

RESULTS

One hundred and eight relevant studies were selected for review. Twenty-five underlying 'challenge' themes were identified. These were clustered into 10 distinct groups, from which a conceptual model of the implementation process was developed.

DISCUSSION AND CONCLUSION

We found that the development of systems supporting individual clinical decisions is evolving toward the implementation of adaptable care pathways on the semantic web, incorporating formal, clinical, and organizational ontologies, and the use of workflow management systems. These architectures now need to be implemented and evaluated on a wider scale within clinical settings.

Development of a workflow integration survey (WIS) for implementing computerized clinical decision support

Interventions that focus on improving computerized clinical decision support (CDS) demonstrate that successful workflow integration can increase the adoption and use of CDS. However, metrics for assessing workflow integration in clinical settings are not well established. The goal of this study was to develop and validate a survey to assess the extent to which CDS is integrated into workflow. Qualitative data on CDS design, usability, and integration from four sites was collected by direct observation, interviews, and focus groups. Thematic analysis based on the sociotechnical systems theory revealed consistent themes across sites. Themes related to workflow integration included navigation, functionality, usability, and workload. Based on these themes, a brief 12-item scale to assess workflow integration was developed, refined, and validated with providers in a simulation study. To our knowledge, this is one of the first tools developed to specifically measure workflow integration of CDS.

Building better guidelines with BRIDGE-Wiz: development and evaluation of a software assistant to promote clarity, transparency, and implementability

OBJECTIVE

To demonstrate the feasibility of capturing the knowledge required to create guideline recommendations in a systematic, structured, manner using a software assistant. Practice guidelines constitute an important modality that can reduce the delivery of inappropriate care and support the introduction of new knowledge into clinical practice. However, many guideline recommendations are vague and underspecified, lack any linkage to supporting evidence or documentation of how they were developed, and prove to be difficult to transform into systems that influence the behavior of care providers.

METHODS

The BRIDGE-Wiz application (Building Recommendations In a Developer's Guideline Editor) uses a wizard approach to address the questions: (1) under what circumstances? (2) who? (3) ought (with what level of obligation?) (4) to do what? (5) to whom? (6) how and why? Controlled natural language was applied to create and populate a template for recommendation statements.

RESULTS

The application was used by five national panels to develop guidelines. In general, panelists agreed that the software helped to formalize a process for authoring guideline recommendations and deemed the application usable and useful.

DISCUSSION

Use of BRIDGE-Wiz promotes clarity of recommendations by limiting verb choices, building active voice recommendations, incorporating decidability and executability checks, and limiting Boolean connectors. It enhances transparency by incorporating systematic appraisal of evidence quality, benefits, and harms. BRIDGE-Wiz promotes implementability by providing a pseudocode rule, suggesting deontic modals, and limiting the use of 'consider'.

CONCLUSION

Users found that BRIDGE-Wiz facilitates the development of clear, transparent, and implementable guideline recommendations.

Computerized decision support systems: improving patient safety in nephrology

Incorrect prescription and administration of medications account for a substantial proportion of medical errors in the USA, causing adverse drug events (ADEs) that result in considerable patient morbidity and enormous costs to the health-care system. Patients with chronic kidney disease or acute kidney injury often have impaired drug clearance as well as polypharmacy, and are therefore at increased risk of experiencing ADEs. Studies have demonstrated that recognition of these conditions is not uniform among treating physicians, and prescribed drug doses are often incorrect. Early interventions that ensure appropriate drug dosing in this group of patients have shown encouraging results. Both computerized physician order entry and clinical decision support systems have been shown to reduce the rate of ADEs. Nevertheless, these systems have been implemented at surprisingly few institutions. Economic stimulus and health-care reform legislation present a rare opportunity to refine these systems and understand how they could be implemented more widely. Failure to explore this technology could mean that the opportunity to reduce the morbidity associated with ADEs is missed.

Actionable reminders did not improve performance over passive reminders for overdue tests in the primary care setting

Actionable reminders (electronic reminders linked to computerized order entry) might improve care by facilitating direct ordering of recommended tests. The authors implemented four enhanced actionable reminders targeting performance of annual mammography, one-time bone-density screening, and diabetic testing. There was no difference in rates of appropriate testing between the four intervention and four matched, control primary care clinics for screening mammography (OR 0.81, 95% CI 0.64 to 1.02), bone-density exams (OR 1.29, 95% CI 0.82 to 2.02), HbA1c monitoring (OR 0.91, 95% CI 0.58 to 1.42) and LDL cholesterol monitoring (OR 1.40, 95% CI 0.76 to 2.59). Of the survey respondents, 79% almost never used the system or were unaware of the functionality. In the 9/228 (3.9%) cases with indirect evidence of mammography reminder use, there was a significantly lower proportion with test performance. Our actionable reminders did not improve receipt of overdue testing, potentially due to limitations of workflow integration.

Physicians, guidelines, and cognitive tasks

Using a case study analysis of the Expert Panel Report 3 (EPR-3): Guidelines for the Diagnosis and Treatment of Asthma, this article compares the workflows and knowledge requirements of primary care practice to the structure and content of a well-respected set of clinical guidelines. The authors show that there are discrepancies between physician workflow and the structure of the EPR-3, as well as between physicians' knowledge requirements and the content of the EPR-3. The analysis suggests that closing the gap between medical knowledge and practice will require alternative ways to represent guidelines' knowledge and recommendations.

Designing an automated clinical decision support system to match clinical practice guidelines for opioid therapy for chronic pain

BACKGROUND

Opioid prescribing for chronic pain is common and controversial, but recommended clinical practices are followed inconsistently in many clinical settings. Strategies for increasing adherence to clinical practice guideline recommendations are needed to increase effectiveness and reduce negative consequences of opioid prescribing in chronic pain patients.

METHODS

Here we describe the process and outcomes of a project to operationalize the 2003 VA/DOD Clinical Practice Guideline for Opioid Therapy for Chronic Non-Cancer Pain into a computerized decision support system (DSS) to encourage good opioid prescribing practices during primary care visits. We based the DSS on the existing ATHENA-DSS. We used an iterative process of design, testing, and revision of the DSS by a diverse team including guideline authors, medical informatics experts, clinical content experts, and end-users to convert the written clinical practice guideline into a computable algorithm to generate patient-specific recommendations for care based upon existing information in the electronic medical record (EMR), and a set of clinical tools.

RESULTS

The iterative revision process identified numerous and varied problems with the initially designed system despite diverse expert participation in the design process. The process of operationalizing the guideline identified areas in which the guideline was vague, left decisions to clinical judgment, or required clarification of detail to insure safe clinical implementation. The revisions led to workable solutions to problems, defined the limits of the DSS and its utility in clinical practice, improved integration into clinical workflow, and improved the clarity and accuracy of system recommendations and tools.

CONCLUSIONS

Use of this iterative process led to development of a multifunctional DSS that met the approval of the clinical practice guideline authors, content experts, and clinicians involved in testing. The process and experiences described provide a model for development of other DSSs that translate written guidelines into actionable, real-time clinical recommendations.

Mammography FastTrack: an intervention to facilitate reminders for breast cancer screening across a heterogeneous multi-clinic primary care network

Health care information technology can be a means to improve quality and efficiency in the primary care setting. However, merely applying technology without addressing how it fits into provider workflow and existing systems is unlikely to achieve improvement goals. Improving quality of primary care, such as cancer screening rates, requires addressing barriers at system, provider, and patient levels. The authors report the development, implementation, and preliminary use of a new breast cancer screening outreach program in a large multicenter primary care network. This installation paired population-based surveillance with customized information delivery based on a validated model linking patients to providers and practices. In the first six months, 86% of physicians and all case managers voluntarily participated in the program. Providers intervened in 83% of the mammogram-overdue population by initiating mailed reminders or deferring contact. Overall, 63% of patients were successfully contacted. Systematic population-based efforts are promising tools to improve preventative care.

Selecting data elements to build a patient-centric electronic health record that will support adherence to therapeutic lifestyle change

Cardiovascular disease (CVD) remains the leading cause of death in the US. Therapeutic lifestyle change (TLC) is an effective intervention to reduce the risk of CVD. In developing a patient-centric electronic health record (PC-EHR), our project aims to build an evidence-based support system to facilitate patient-provider interaction, foster cooperative chronic disease management, and promote adherence to TLC guidelines by both providers and patients. In this paper we report on the findings of a modified Delphi method (MDM) study to determine the behavioral and psychosocial elements needed to extend the clinical data model and support the TLC decision support algorithm. An eight-member expert advisory committee reviewed a list of 83 proposed data elements including those from the National Cholesterol Education Program Adult Treatment Panel III guidelines and finalized a set of 30 data elements for inclusion. The MDM proved to be an effective approach for prioritizing data elements.

Diabetes information technology: designing informatics systems to catalyze change in clinical care

Current computerized reminder and decision support systems intended to improve diabetes care have had a limited effect on clinical outcomes. Increasing pressures on health care networks to meet standards of diabetes care have created an environment where information technology systems for diabetes management are often created under duress, appended to existing clinical systems, and poorly integrated into the existing workflow. After defining the components of diabetes disease management, the authors present an eight-step conceptual framework to guide the development of more effective diabetes information technology systems for translating clinical information into clinical action.

IT support for clinical pathways—Lessons learned

Clinical pathways are an effective instrument to decrease undesired practice variability and improve clinician performance. IT-applications embedded into clinical routine work can help to increase pathway compliance. Successfully implementing such applications requires both a responsive IT infrastructure and a participatory and iterative design process aimed at achieving user acceptance and usability. Experiences from the implementation and iterative improvement of an online surgical pathway at Marburg University Medical Centre have shown that pathway conformance actually could be improved by the use of IT. An analysis of the iterative design process has shown that future pathway projects can benefit from the lessons learned during this project. Based on these lessons recommendations for developing well adapted interaction mechanisms are presented, aimed at improving process alignment. Our goal is to build up a library of tested reusable components to reduce the number of iterations for pathway implementation.

A description and functional taxonomy of rule-based decision support content at a large integrated delivery network

OBJECTIVE

This study sought to develop a functional taxonomy of rule-based clinical decision support.

DESIGN

The rule-based clinical decision support content of a large integrated delivery network with a long history of computer-based point-of-care decision support was reviewed and analyzed along four functional dimensions: trigger, input data elements, interventions, and offered choices.

RESULTS

A total of 181 rule types were reviewed, comprising 7,120 different instances of rule usage. A total of 42 taxa were identified across the four categories. Many rules fell into multiple taxa in a given category. Entered order and stored laboratory result were the most common triggers; laboratory result, drug list, and hospital unit were the most frequent data elements used. Notify and log were the most common interventions, and write order, defer warning, and override rule were the most common offered choices.

CONCLUSION

A relatively small number of taxa successfully described a large body of clinical knowledge. These taxa can be directly mapped to functions of clinical systems and decision support systems, providing feature guidance for developers, implementers, and certifiers of clinical information systems.

Adaptation of clinical guidelines: literature review and proposition for a framework and procedure

PURPOSE

The development and updating of high-quality clinical practice guidelines require substantial resources. Many guideline programmes throughout the world are using similar strategies to achieve similar goals, resulting in many guidelines on the same topic. One method of using resources more efficiently and avoiding unnecessary duplication of effort would be to adapt existing guidelines. The aim was to review the literature on adaptation of guidelines and to propose a systematic approach for adaptation of guidelines.

DATA SOURCES

We selected and reviewed reports describing the methods and results of adaptation of guidelines from those found by searching Medline, Internet, and reference lists of relevant papers. On the basis of this review and our experience in guideline development, we proposed a conceptual framework and procedure for adaptation of guidelines.

RESULTS

Adaptation of guidelines is performed either as an alternative to de novo guideline development or to improve guideline implementation through local tailoring of an international or national guideline. However, no validated process for the adaptation of guidelines produced in one cultural and organizational setting for use in another (i.e. trans-contextual adaptation) was found in the literature. The proposed procedure is a stepwise approach to trans-contextual adaptation, including searching for existing guidelines, quality appraisal, detailed analysis of the coherence between the evidence and the recommendations, and adaptation of the recommendations to the target context of use, taking into account the organization of the health care system and cultural context.

CONCLUSIONS

Trans-contextual adaptation of guidelines is increasingly being considered as an alternative to de novo guideline development. The proposed approach should be validated and evaluated to determine if it can reduce duplication of effort and inefficient use of resources, although guaranteeing a high-quality product, compared with de novo development.

The story behind the story: physician skepticism about relying on clinical information technologies to reduce medical errors

PURPOSE

In order to better understand physicians' perspectives about the use of clinical information technology (CIT) to reduce medical errors, we asked physicians about opportunities and issues around clinical use of computerized physician order entry (CPOE) systems, order sets within CPOE, and handheld computers (HHCs).

METHODS

We conducted 10 focus groups including 71 physicians involved in technology implementation efforts across the US between April 2002 and February 2005.

RESULTS

Two major themes emerged across focus groups around reliance on CIT to reduce errors: (1) can it work? and (2) at what cost to the medical profession? Within the first theme, physicians expressed concern about the appropriateness of physician-directed CIT as a solution for medical errors, concerns regarding the current technical capabilities and level of technical support for CIT solutions, and concern about the introduction of new errors. Within the second theme, physicians were particularly concerned about time efficiency and workload redistribution associated with the introduction of CIT. Across focus groups, physicians tended to generalize about the role of all IT in their lives, potentially biasing opinions about specific technologies.

CONCLUSIONS

Health care organizations attempting to promote physician use of CIT are advised to deepen consideration of physicians' perspectives about technology adoption and use in order to address their concerns, reduce skepticism, and increase the likelihood of implementation success.

Improving pathway compliance and clinician performance by using information technology

To deliver patient-specific advice at the time and place of a consultation is an important contribution to improving clinician performance. Using computer-based decision support on the basis of clinical pathways is a promising strategy to achieve this goal. Thereby integration of IT applications into the clinical workflow is a core precondition for success. User acceptance and usability play a critical role: additional effort has to be balanced with enough benefit for the users and interaction design and evaluation should be handled as an intertwined, continuous process. Experiences from routine use of an online surgical pathway at Marburg University Medical Center show that it is possible to successfully address this issue by seamlessly integrating patient-specific pathway recommendations with documentation tasks which have to be done anyway, by substantially reusing entered data to accelerate routine tasks (e.g. by automatically generating orders and reports), and by continuously and systematically monitoring pathway conformance and documentation quality.

Embedded guideline information without patient specificity in a commercial emergency department computerized order-entry system

BACKGROUND

Clinical practice guidelines and computerized provider order entry (CPOE) have potential for improving clinical care. Questions remain about feasibility and effectiveness of CPOE in the emergency department (ED). However, successful implementations in other settings typically incorporate decision support functions that are lacking in many commercially available ED information systems.

OBJECTIVES

To compare acute coronary syndrome (ACS) guideline compliance before and after implementation of a locally implemented ACS guideline, first on paper and then in a commercially available ED information system without patient-specific clinical decision support.

METHODS

Clinical data were abstracted retrospectively on patients seen before and after introduction of paper and, subsequently, CPOE versions of ACS guideline-based order-sets. Order-set use was determined. Risk category assignments were made retrospectively using guideline criteria and compliance with the guideline regarding beta-blockers, heparin, and aspirin was determined. Association between order-set use and compliance was determined.

RESULTS

The authors found increasing use of order-sets over the period of study. However, there was poor association between the order-sets used and risk stratification category. Some association between ED beta-blocker use and use of CPOE order-sets was found, but there was no improvement in overall compliance with any of the guideline recommendations.

CONCLUSIONS

Adherence to an ACS guideline did not improve with implementation of a commercial ED information system without provision for patient-specific decision support. This suggests that the lack of patient-specific decision-support functionality in most current ED information system products may hamper progress in the development of effective decision support.

Electronic implementation of guidelines in the EsPeR system: a knowledge specification method

Despite initiatives to standardize methods for the development of clinical guidelines, several barriers hinder their integration in daily clinical practice: failure to fulfil quality criteria, poor effectiveness of their dissemination. Computerization of guidelines can favor their dissemination. The initial step of computerization is the knowledge specification from the text of the guideline. We describe the method of knowledge specification, which is used in EsPeR (Personalized Estimate of Risks), a web-based decision support system in preventive medicine, which allows, for a given person, to estimate risks and access recommendations, based on clinical profile. This method is based on a structured and systematic analysis of text allowing detailed specification of a decision tree. We use decision tables to validate the decision algorithm and decision trees to specify this algorithm, along with elementary messages of recommendation. Editing tools are used to facilitate the process of validation and the workflow between expert physicians and computer scientists. Applied to eleven different guidelines, the method allows a quick and valid computerization and integration in the EsPeR system. The method used for computerization could help to define a framework usable at the initial step of guideline development in order to produce guidelines ready for electronic implementation.

The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt

The authors describe a pragmatic approach to the introduction of clinical decision support at the point of care, based on a decade of experience in developing and evolving Vanderbilt's inpatient "WizOrder" care provider order entry (CPOE) system. The inpatient care setting provides a unique opportunity to interject CPOE-based decision support features that restructure clinical workflows, deliver focused relevant educational materials, and influence how care is delivered to patients. From their empirical observations, the authors have developed a generic model for decision support within inpatient CPOE systems. They believe that the model's utility extends beyond Vanderbilt, because it is based on characteristics of end-user workflows and on decision support considerations that are common to a variety of inpatient settings and CPOE systems. The specific approach to implementing a given clinical decision support feature within a CPOE system should involve evaluation along three axes: what type of intervention to create (for which the authors describe 4 general categories); when to introduce the intervention into the user's workflow (for which the authors present 7 categories), and how disruptive, during use of the system, the intervention might be to end-users' workflows (for which the authors describe 6 categories). Framing decision support in this manner may help both developers and clinical end-users plan future alterations to their systems when needs for new decision support features arise.

Utilization of evidence-based computerized order sets in pediatrics

Little is known about utilization of different evidence-based order sets within computerized physician order entry (CPOE) systems. We designed a retrospective study of resident and attending physician order set utilization to evaluate the use of three evidence-based computerized order sets (asthma, post-appendectomy care, and community-acquired pneumonia (CAP)), and examine patient and admission characteristics associated with order set utilization in pediatrics. We studied all 529 asthma patients, 277 appendectomy patients, and 210 CAP patients admitted between 1 November 2001 and 30 November 2003 during implementation of standardized order sets at a large, independent, not-for-profit pediatric institution. We analyzed order set utilization for the three order sets and tested the relationship between order set use and potential factors associated with utilization. Order set utilization varied by condition (X(2)=339.2, p<0.001), with the asthma order set use rate highest (88.1%), followed by appendectomy order set utilization (79.4%), and substantially lower CAP order set use (21.1%). We found that trends in order set utilization also varied by condition. Only the asthma order set showed a trend of increasing use after implementation (z= -3.02, p=0.002). In addition, factors associated with order set utilization varied. Uses of the asthma and post-appendectomy order sets were associated with factors such as admission unit and case complexity. CAP order set utilization was associated with case complexity but not admission source. We conclude that health services organizations looking to implement computerized order sets to reduce unnecessary practice variation while promoting best practices must consider the different factors that may influence the use of each order set rather than relying on a one-size-fits-all implementation strategy. Further, issues such as the level of physician involvement in order set development and consensus around order set content may be particularly important factors influencing order set utilization.

A framework for a distributed, hybrid, multiple-ontology clinical-guideline library, and automated guideline-support tools

Clinical guidelines are a major tool in improving the quality of medical care. However, most guidelines are in free text, not in a formal, executable format, and are not easily accessible to clinicians at the point of care. We introduce a Web-based, modular, distributed architecture, the Digital Electronic Guideline Library (DeGeL), which facilitates gradual conversion of clinical guidelines from text to a formal representation in chosen target guideline ontology. The architecture supports guideline classification, semantic markup, context-sensitive search, browsing, run-time application, and retrospective quality assessment. The DeGeL hybrid meta-ontology includes elements common to all guideline ontologies, such as semantic classification and domain knowledge; it also includes four content-representation formats: free text, semi-structured text, semi-formal representation, and a formal representation. These formats support increasingly sophisticated computational tasks. The DeGeL tools for support of guideline-based care operate, at some level, on all guideline ontologies. We have demonstrated the feasibility of the architecture and the tools for several guideline ontologies, including Asbru and GEM.

Design and implementation of the GLIF3 guideline execution engine

We have developed the GLIF3 Guideline Execution Engine (GLEE) as a tool for executing guidelines encoded in the GLIF3 format. In addition to serving as an interface to the GLIF3 guideline representation model to support the specified functions, GLEE provides defined interfaces to electronic medical records (EMRs) and other clinical applications to facilitate its integration with the clinical information system at a local institution. The execution model of GLEE takes the "system suggests, user controls" approach. A tracing system is used to record an individual patient's state when a guideline is applied to that patient. GLEE can also support an event-driven execution model once it is linked to the clinical event monitor in a local environment. Evaluation has shown that GLEE can be used effectively for proper execution of guidelines encoded in the GLIF3 format. When using it to execute each guideline in the evaluation, GLEE's performance duplicated that of the reference systems implementing the same guideline but taking different approaches. The execution flexibility and generality provided by GLEE, and its integration with a local environment, need to be further evaluated in clinical settings. Integration of GLEE with a specific event-monitoring and order-entry environment is the next step of our work to demonstrate its use for clinical decision support. Potential uses of GLEE also include quality assurance, guideline development, and medical education.

Guideline-based modeling of therapeutic strategies in the special case of chronic diseases

Despite the availability of evidence-based clinical practice guidelines in most countries, patients with chronic diseases are still generally inadequately managed. One difficulty lies in the optimal synchronization of a patient with the guideline therapeutic strategy, especially when the history of her past treatments does not follow the recommended sequence of therapies. We propose a formal model to represent guideline-based therapeutic strategies as a two-level decision tree. The clinical level is used to identify a patient-specific clinical situation, on the basis of key elements of clinical examination (complication of hypertension, associated diseases). The therapeutic level is derived from the formalization of guideline-based strategies first represented as bidimensional matrices structured in lines of therapy and levels of therapeutic intention. A revised version based on the ordering of levels of therapeutic combination is then developed. The aim is to dynamically provide the best next step of treatment from the patient's therapeutic-history-based customization of the general therapeutic sequence established in the guideline for the corresponding clinical situation. A preliminary in vitro evaluation of the system on actual clinical cases showed a positive impact on physician compliance with a significant increase from 16 to 57%.

Use of a computerized guideline for glucose regulation in the intensive care unit improved both guideline adherence and glucose regulation

OBJECTIVE

To measure the impact of a computerized guideline for glucose regulation in an ICU.

DESIGN

A randomized, controlled trial with an off-on-off design.

METHODS

We implemented a glucose regulation guideline in an intensive care unit in paper form during the first study period. During the second period, the guideline was randomly applied in either paper or computerized form. In the third period, the guideline was available only in paper form.

MEASUREMENTS AND RESULTS

We analyzed data for 484 patients. During the intervention period, the control group included 54 patients and the computerized intervention group included 66 patients. The two guideline-related outcome measures consisted of compliance with: (a) glucose measurement timing recommendations and (b) insulin dose advice. We measured clinical impact as the proportion of time that glucose levels fell within target range. In the first (paper-based) study period, 29.0% of samples occurred with optimal timing; during the second period, this increased to 35.5% for paper-based and to 40.2% for computerized protocols. The third study period timeliness scores reverted to the first period rates. Late (suboptimal) sampling occurred for 66% of glucose measurements in the first study period, for 42% of paper-based and 28% of computer-based protocol samples in the second period, and for 50.0% of samples in the third study period. In the first study period, insulin-dosing guideline compliance was 56.3%; in the second period, it was 64.2% for paper-based and 77.3% for computer-based protocols, and it fell to 42.4% in the third period. For the second study period, the time that a patient's glucose values fell within target range improved for both the control (52.9%) and the computerized groups (54.2%) compared with the first study period (44.3%) and the third period (42.3%).

CONCLUSION

Implementing a computerized version of a guideline significantly improved timeliness of measurements and glucose level regulation for critically ill patients compared with implementing a paper-based version of the guideline.

Clinical informatics in critical care

Health care information systems have the potential to enable better care of patients in much the same manner as the widespread use of the automobile and telephone did in the early 20th century. The car and phone were rapidly accepted and embraced throughout the world when these breakthroughs occurred. However, the automation of health care with use of computerized information systems has not been as widely accepted and implemented as computer technology use in all other sectors of the global economy. In this article, the authors examine the need, risks, and rewards of clinical informatics in health care as well as its specific relationship to critical care medicine.