Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: Results of a Delphi study

Expanding access to veterinary clinical decision support in resource-limited settings: a scoping review of clinical decision support tools in medicine and antimicrobial stewardship

Introduction

Digital clinical decision support (CDS) tools are of growing importance in supporting healthcare professionals in understanding complex clinical problems and arriving at decisions that improve patient outcomes. CDS tools are also increasingly used to improve antimicrobial stewardship (AMS) practices in healthcare settings. However, far fewer CDS tools are available in lowerand middle-income countries (LMICs) and in animal health settings, where their use in improving diagnostic and treatment decision-making is likely to have the greatest impact. The aim of this study was to evaluate digital CDS tools designed as a direct aid to support diagnosis and/or treatment decisionmaking, by reviewing their scope, functions, methodologies, and quality. Recommendations for the development of veterinary CDS tools in LMICs are then provided.

Methods

The review considered studies and reports published between January 2017 and October 2023 in the English language in peer-reviewed and gray literature.

Results

A total of 41 studies and reports detailing CDS tools were included in the final review, with 35 CDS tools designed for human healthcare settings and six tools for animal healthcare settings. Of the tools reviewed, the majority were deployed in high-income countries (80.5%). Support for AMS programs was a feature in 12 (29.3%) of the tools, with 10 tools in human healthcare settings. The capabilities of the CDS tools varied when reviewed against the GUIDES checklist.

Discussion

We recommend a methodological approach for the development of veterinary CDS tools in LMICs predicated on securing sufficient and sustainable funding. Employing a multidisciplinary development team is an important first step. Developing standalone CDS tools using Bayesian algorithms based on local expert knowledge will provide users with rapid and reliable access to quality guidance on diagnoses and treatments. Such tools are likely to contribute to improved disease management on farms and reduce inappropriate antimicrobial use, thus supporting AMS practices in areas of high need.

Detecting atypical alert behavior through statistical process control: Clinical decision support alert frequency visualizations

Clinical decision support (CDS) alerts are designed to work according to a set of clearly defined criteria and have the potential to improve clinical care. To efficiently and proactively review abnormally functioning CDS alerts, we postulate that the introduction of a dashboard with statistical process control (SPC) charting will lead to effective detection of erratic alert behavior. We identified custom CDS alerts from an academic medical center that were recorded and monitored in a longitudinal fashion and the data warehouses where this information was stored. We created a dashboard of alert frequency using SPC charts, applied SPC rules for classification of variation, and validated dashboard data. From June-August 2022, the dashboard effectively pulled in data to visually depict alert behavior. SPC-defined parameters for standard deviation from the mean were applied to visualizations and allowed for rapid review of alerts with greatest variation. These alerts were subsequently investigated, and it was determined that they were functioning correctly. The most profound abnormalities detected during implementation reflected changes in practice and not system errors, though further investigation into thresholds for statistical significance will benefit this field. We conclude that SPC visualizations are a time-efficient and effective method of identifying CDS malfunctions.

Impact of a Clinical Decision Support Alert on Informed Consent Documentation in the Neonatal Intensive Care Unit

Background

Informed consent is necessary to preserve patient autonomy and shared decision-making, yet compliant consent documentation is suboptimal in the intensive care unit (ICU). We aimed to increase compliance with bundled consent documentation, which provides consent for a predefined set of common procedures in the neonatal ICU from 0% to 50% over 1 year.

Methods

We used the Plan-Do-Study-Act model for quality improvement. Interventions included education and performance awareness, delineation of the preferred consenting process, consent form revision, overlay tool creation, and clinical decision support (CDS) alert use within the electronic health record. Monthly audits categorized consent forms as missing, present but noncompliant, or compliant. We analyzed consent compliance on a run chart using standard run chart interpretation rules and obtained feedback on the CDS as a countermeasure.

Results

We conducted 564 audits over 37 months. Overall, median consent compliance increased from 0% to 86.6%. Upon initiating the CDS alert, we observed the highest monthly compliance of 93.3%, followed by a decrease to 33.3% with an inadvertent discontinuation of the CDS. Compliance subsequently increased to 73.3% after the restoration of the alert. We created a consultant opt-out selection to address negative feedback associated with CDS. There were no missing consent forms within the last 7 months of monitoring.

Conclusions

A multi-faceted approach led to sustained improvement in bundled consent documentation compliance in our neonatal intensive care unit, with the direct contribution of the CDS observed. A CDS intervention directed at the informed consenting process may similarly benefit other ICUs.

Ontologies Applied in Clinical Decision Support System Rules: Systematic Review

BACKGROUND

Clinical decision support systems (CDSSs) are important for the quality and safety of health care delivery. Although CDSS rules guide CDSS behavior, they are not routinely shared and reused.

OBJECTIVE

Ontologies have the potential to promote the reuse of CDSS rules. Therefore, we systematically screened the literature to elaborate on the current status of ontologies applied in CDSS rules, such as rule management, which uses captured CDSS rule usage data and user feedback data to tailor CDSS services to be more accurate, and maintenance, which updates CDSS rules. Through this systematic literature review, we aim to identify the frontiers of ontologies used in CDSS rules.

METHODS

The literature search was focused on the intersection of ontologies; clinical decision support; and rules in PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database. Grounded theory and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines were followed. One author initiated the screening and literature review, while 2 authors validated the processes and results independently. The inclusion and exclusion criteria were developed and refined iteratively.

RESULTS

CDSSs were primarily used to manage chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations among 81 included publications. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite the fact that ontologies have been used to provide medical knowledge, CDSS rules, and terminologies, they have not been used in CDSS rule management or to facilitate the reuse of CDSS rules.

CONCLUSIONS

Ontologies have been used to organize and represent medical knowledge, controlled vocabularies, and the content of CDSS rules. So far, there has been little reuse of CDSS rules. More work is needed to improve the reusability and interoperability of CDSS rules. This review identified and described the ontologies that, despite their limitations, enable Semantic Web technologies and their applications in CDSS rules.

Implementation of Machine Learning Pipelines for Clinical Practice: Development and Validation Study

BACKGROUND

Artificial intelligence (AI) technologies, such as machine learning and natural language processing, have the potential to provide new insights into complex health data. Although powerful, these algorithms rarely move from experimental studies to direct clinical care implementation.

OBJECTIVE

We aimed to describe the key components for successful development and integration of two AI technology-based research pipelines for clinical practice.

METHODS

We summarized the approach, results, and key learnings from the implementation of the following two systems implemented at a large, tertiary care children's hospital: (1) epilepsy surgical candidate identification (or epilepsy ID) in an ambulatory neurology clinic; and (2) an automated clinical trial eligibility screener (ACTES) for the real-time identification of patients for research studies in a pediatric emergency department.

RESULTS

The epilepsy ID system performed as well as board-certified neurologists in identifying surgical candidates (with a sensitivity of 71% and positive predictive value of 77%). The ACTES system decreased coordinator screening time by 12.9%. The success of each project was largely dependent upon the collaboration between machine learning experts, research and operational information technology professionals, longitudinal support from clinical providers, and institutional leadership.

CONCLUSIONS

These projects showcase novel interactions between machine learning recommendations and providers during clinical care. Our deployment provides seamless, real-time integration of AI technology to provide decision support and improve patient care.

Real-Time User Feedback to Support Clinical Decision Support System Improvement

OBJECTIVES

To improve clinical decision support (CDS) by allowing users to provide real-time feedback when they interact with CDS tools and by creating processes for responding to and acting on this feedback.

METHODS

Two organizations implemented similar real-time feedback tools and processes in their electronic health record and gathered data over a 30-month period. At both sites, users could provide feedback by using Likert feedback links embedded in all end-user facing alerts, with results stored outside the electronic health record, and provide feedback as a comment when they overrode an alert. Both systems are monitored daily by clinical informatics teams.

RESULTS

The two sites received 2,639 Likert feedback comments and 623,270 override comments over a 30-month period. Through four case studies, we describe our use of end-user feedback to rapidly respond to build errors, as well as identifying inaccurate knowledge management, user-interface issues, and unique workflows.

CONCLUSION

Feedback on CDS tools can be solicited in multiple ways, and it contains valuable and actionable suggestions to improve CDS alerts. Additionally, end users appreciate knowing their feedback is being received and may also make other suggestions to improve the electronic health record. Incorporation of end-user feedback into CDS monitoring, evaluation, and remediation is a way to improve CDS.

Clinical decision support malfunctions related to medication routes: a case series

Objective

To identify common medication route-related causes of clinical decision support (CDS) malfunctions and best practices for avoiding them.

Materials and Methods

Case series of medication route-related CDS malfunctions from diverse healthcare provider organizations.

Results

Nine cases were identified and described, including both false-positive and false-negative alert scenarios. A common cause was the inclusion of nonsystemically available medication routes in value sets (eg, eye drops, ear drops, or topical preparations) when only systemically available routes were appropriate.

Discussion

These value set errors are common, occur across healthcare provider organizations and electronic health record (EHR) systems, affect many different types of medications, and can impact the accuracy of CDS interventions. New knowledge management tools and processes for auditing existing value sets and supporting the creation of new value sets can mitigate many of these issues. Furthermore, value set issues can adversely affect other aspects of the EHR, such as quality reporting and population health management.

Conclusion

Value set issues related to medication routes are widespread and can lead to CDS malfunctions. Organizations should make appropriate investments in knowledge management tools and strategies, such as those outlined in our recommendations.

Recognizing Pediatric ARDS: Provider Use of the PALICC Recommendations in a Tertiary Pediatric ICU

BACKGROUND

For almost 50 years, pediatricians used adult guidelines to diagnose ARDS. In 2015, specific criteria for pediatric ARDS were defined. However, it remains unclear how frequently providers recognize pediatric ARDS and whether recognition affects adherence to consensus recommendations.

METHODS

This was a mixed-method, retrospective study of mechanically ventilated pediatric subjects after the release of the pediatric ARDS recommendation statement. Pediatric ARDS cases were identified according to the new criteria. Provider recognition was defined by documentation in the medical record. Pediatric ARDS subjects with and without provider recognition were compared quantitatively according to clinical characteristics, adherence to lung-protective ventilation (LPV), adjunctive therapies, and outcomes. A qualitative document analysis (QDA) was performed to evaluate knowledge and beliefs surrounding the Pediatric Acute Lung Injury Consensus Conference recommendations.

RESULTS

Of 1,983 subject encounters, pediatric ARDS was identified in 321 (16%). Provider recognition was present in 97 (30%) cases and occurred more often in subjects who were older, had worse oxygenation deficits, or were bone marrow transplant recipients. Recognition rates increased each studied year. LPV practices did not differ based on provider recognition; however, subjects who received it were more likely to experience permissive hypoxemia and adherence to extrapulmonary recommendations. Ultimately, there was no differences in outcomes between the provider recognition and non-provider recognition groups. Three themes emerged from the QDA: (1) pediatric ARDS presents within a complex, multidimensional context, with potentially competing organ system failures; (2) similar to historical conceptualizations, pediatric ARDS was often considered a visual diagnosis, with measures of oxygenation unreferenced; and (3) emphasis was placed on non-evidence-based interventions, such as pulmonary clearance techniques, rather than on consensus recommendations.

CONCLUSIONS

Among mechanically ventilated children, pediatric ARDS was common but recognized in a minority of cases. Potential opportunities, such as an opt-out approach to LPV, may exist for improved dissemination and implementation of recommended best practices.

Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts

Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.

Clinician collaboration to improve clinical decision support: the Clickbusters initiative

OBJECTIVE

We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program.

RESULTS

Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS.

CONCLUSIONS

At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

How to Develop and Implement a Computerized Decision Support System Integrated for Antimicrobial Stewardship? Experiences From Two Swiss Hospital Systems

Background: Computerized decision support systems (CDSS) provide new opportunities for automating antimicrobial stewardship (AMS) interventions and integrating them in routine healthcare. CDSS are recommended as part of AMS programs by international guidelines but few have been implemented so far. In the context of the publicly funded COMPuterized Antibiotic Stewardship Study (COMPASS), we developed and implemented two CDSSs for antimicrobial prescriptions integrated into the in-house electronic health records of two public hospitals in Switzerland. Developing and implementing such systems was a unique opportunity for learning during which we faced several challenges. In this narrative review we describe key lessons learned.

Recommendations: (1) During the initial planning and development stage, start by drafting the CDSS as an algorithm and use a standardized format to communicate clearly the desired functionalities of the tool to all stakeholders. (2) Set up a multidisciplinary team bringing together Information Technologies (IT) specialists with development expertise, clinicians familiar with “real-life” processes in the wards and if possible, involve collaborators having knowledge in both areas. (3) When designing the CDSS, make the underlying decision-making process transparent for physicians and start simple and make sure to find the right balance between force and persuasion to ensure adoption by end-users. (4) Correctly assess the clinical and economic impact of your tool, therefore try to use standardized terminologies and limit the use of free text for analysis purpose. (5) At the implementation stage, plan usability testing early, develop an appropriate training plan suitable to end users' skills and time-constraints and think ahead of additional challenges related to the study design that may occur (such as a cluster randomized trial). Stay also tuned to react quickly during the intervention phase. (6) Finally, during the assessment stage plan ahead maintenance, adaptation and related financial challenges and stay connected with institutional partners to leverage potential synergies with other informatics projects.

Recommendations for the safe, effective use of adaptive CDS in the US healthcare system: an AMIA position paper

The development and implementation of clinical decision support (CDS) that trains itself and adapts its algorithms based on new data-here referred to as Adaptive CDS-present unique challenges and considerations. Although Adaptive CDS represents an expected progression from earlier work, the activities needed to appropriately manage and support the establishment and evolution of Adaptive CDS require new, coordinated initiatives and oversight that do not currently exist. In this AMIA position paper, the authors describe current and emerging challenges to the safe use of Adaptive CDS and lay out recommendations for the effective management and monitoring of Adaptive CDS.

Algorithmic Detection of Boolean Logic Errors in Clinical Decision Support Statements

OBJECTIVE

Clinical decision support (CDS) can contribute to quality and safety. Prior work has shown that errors in CDS systems are common and can lead to unintended consequences. Many CDS systems use Boolean logic, which can be difficult for CDS analysts to specify accurately. We set out to determine the prevalence of certain types of Boolean logic errors in CDS statements.

METHODS

Nine health care organizations extracted Boolean logic statements from their Epic electronic health record (EHR). We developed an open-source software tool, which implemented the Espresso logic minimization algorithm, to identify three classes of logic errors.

RESULTS

Participating organizations submitted 260,698 logic statements, of which 44,890 were minimized by Espresso. We found errors in 209 of them. Every participating organization had at least two errors, and all organizations reported that they would act on the feedback.

DISCUSSION

An automated algorithm can readily detect specific categories of Boolean CDS logic errors. These errors represent a minority of CDS errors, but very likely require correction to avoid patient safety issues. This process found only a few errors at each site, but the problem appears to be widespread, affecting all participating organizations.

CONCLUSION

Both CDS implementers and EHR vendors should consider implementing similar algorithms as part of the CDS authoring process to reduce the number of errors in their CDS interventions.

Giving Your Electronic Health Record a Checkup After COVID-19: A Practical Framework for Reviewing Clinical Decision Support in Light of the Telemedicine Expansion

BACKGROUND

The transformation of health care during COVID-19, with the rapid expansion of telemedicine visits, presents new challenges to chronic care and preventive health providers. Clinical decision support (CDS) is critically important to chronic care providers, and CDS malfunction is common during times of change. It is essential to regularly reassess an organization's ambulatory CDS program to maintain care quality. This is especially true after an immense change, like the COVID-19 telemedicine expansion.

OBJECTIVE

Our objective is to reassess the ambulatory CDS program at a large academic medical center in light of telemedicine's expansion in response to the COVID-19 pandemic.

METHODS

Our clinical informatics team devised a practical framework for an intrapandemic ambulatory CDS assessment focused on the impact of the telemedicine expansion. This assessment began with a quantitative analysis comparing CDS alert performance in the context of in-person and telemedicine visits. Board-certified physician informaticists then completed a formal workflow review of alerts with inferior performance in telemedicine visits. Informaticists then reported on themes and optimization opportunities through the existing CDS governance structure.

RESULTS

Our assessment revealed that 10 of our top 40 alerts by volume were not firing as expected in telemedicine visits. In 3 of the top 5 alerts, providers were significantly less likely to take action in telemedicine when compared to office visits. Cumulatively, alerts in telemedicine encounters had an action taken rate of 5.3% (3257/64,938) compared to 8.3% (19,427/233,636) for office visits. Observations from a clinical informaticist workflow review included the following: (1) Telemedicine visits have different workflows than office visits. Some alerts developed for the office were not appearing at the optimal time in the telemedicine workflow. (2) Missing clinical data is a common reason for the decreased alert firing seen in telemedicine visits. (3) Remote patient monitoring and patient-reported clinical data entered through the portal could replace data collection usually completed in the office by a medical assistant or registered nurse.

CONCLUSIONS

In a large academic medical center at the pandemic epicenter, an intrapandemic ambulatory CDS assessment revealed clinically significant CDS malfunctions that highlight the importance of reassessing ambulatory CDS performance after the telemedicine expansion.

A qualitative study of prescribing errors among multi-professional prescribers within an e-prescribing system

Background Computerised Physician Order Entry (CPOE) is considered to enhance the safety of prescribing. However, it can have unintended consequences and new forms of prescribing error have been reported. Objective The aim of this study was to explore the causes and contributing factors associated with prescribing errors reported by multidisciplinary prescribers working within a CPOE system. Main Outcome Measure Multidisciplinary prescribers experience of prescribing errors in an CPOE system. Method This qualitative study was conducted in a hospital with a well-established CPOE system. Semi-structured qualitative interviews were conducted with prescribers from the professions of pharmacy, nursing, and medicine. Interviews analysed using a mixed inductive and deductive approach to develop a framework for the causes of error. Results Twenty-three prescribers were interviewed. Six main themes influencing prescribing were found: the system, the prescriber, the patient, the team, the task of prescribing and the work environment. Prominent issues related to CPOE included, incorrect drug name picking, default auto-population of dosages, alert fatigue and remote prescribing. These interacted within a complex prescribing environment. No substantial differences in the experience of CPOE were found between the professions. Conclusion Medical and non-medical prescribers have similar experiences of prescribing errors when using CPOE, aligned with existing published literature about medical prescribing. Causes of electronic prescribing errors are multifactorial in nature and prescribers describe how factors interact to create the conditions errors. While interventions should focus on direct CPOE issues, such as training and design, socio-technical, and environmental aspects of practice remain important.

Comparative Effectiveness of Best Practice Alerts with Active and Passive Presentations: A Retrospective Study

We assess the relationship of active or passive presentation of Best Practice Advisories (BPAs) for hospital clinicians with compliance rates of recommended actions. We identify the design characteristics of alerts that can be used to assess the effectiveness of design choices with superior usability. Alerts in Electronic Health Records (EHRs) are frequently overridden by healthcare providers. Identifying characteristics of effective alerts can increase the frequency that actions recommended in evidence-based care guidelines are done, reduce user frustration, and improve interface usability along with the willingness to use alerts. We conducted a retrospective analysis of data for 11 BPAs between June 2014 and May 2015. The outcome measure was the percent correspondence with recommended actions. A repeated measures regression model was used for the correlation of the BPA presentation type with the outcome measure. The BPA presentation type was significant such that the odds are 7.7 times greater that a recommended action would be taken by a provider with an active BPA presentation type after adjusting for whether an action was required. Active presentation alerts achieve higher compliance rates. CDS alerts that actively interrupted the provider’s workflow were associated with a higher compliance rate with recommended actions.

EHRtemporalVariability: delineating temporal data-set shifts in electronic health records

BACKGROUND

Temporal variability in health-care processes or protocols is intrinsic to medicine. Such variability can potentially introduce dataset shifts, a data quality issue when reusing electronic health records (EHRs) for secondary purposes. Temporal data-set shifts can present as trends, as well as abrupt or seasonal changes in the statistical distributions of data over time. The latter are particularly complicated to address in multimodal and highly coded data. These changes, if not delineated, can harm population and data-driven research, such as machine learning. Given that biomedical research repositories are increasingly being populated with large sets of historical data from EHRs, there is a need for specific software methods to help delineate temporal data-set shifts to ensure reliable data reuse.

RESULTS

EHRtemporalVariability is an open-source R package and Shiny app designed to explore and identify temporal data-set shifts. EHRtemporalVariability estimates the statistical distributions of coded and numerical data over time; projects their temporal evolution through non-parametric information geometric temporal plots; and enables the exploration of changes in variables through data temporal heat maps. We demonstrate the capability of EHRtemporalVariability to delineate data-set shifts in three impact case studies, one of which is available for reproducibility.

CONCLUSIONS

EHRtemporalVariability enables the exploration and identification of data-set shifts, contributing to the broad examination and repurposing of large, longitudinal data sets. Our goal is to help ensure reliable data reuse for a wide range of biomedical data users. EHRtemporalVariability is designed for technical users who are programmatically utilizing the R package, as well as users who are not familiar with programming via the Shiny user interface.Availability: https://github.com/hms-dbmi/EHRtemporalVariability/Reproducible vignette: https://cran.r-project.org/web/packages/EHRtemporalVariability/vignettes/EHRtemporalVariability.htmlOnline demo: http://ehrtemporalvariability.upv.es/.

Health information technology:Fallacies and Sober realities - Redux A homage to Bentzi Karsh and Robert Wears

Since the publication of "Health Information Technology: Fallacies and Sober Realities" in 2010, health information technology (HIT) has become nearly ubiquitous in US healthcare facilities. Yet, HIT has yet to achieve its putative benefits of higher quality, safer, and lower cost care. There has been variable but largely marginal progress at addressing the 12 HIT fallacies delineated in the original paper. Here, we revisit several of the original fallacies and add five new ones. These fallacies must be understood and addressed by all stakeholders for HIT to be a positive force in achieving the high value healthcare system the nation deserves. Foundational cognitive and human factors engineering research and development continue to be essential to HIT development, deployment, and use.

Reducing Alert Burden in Electronic Health Records: State of the Art Recommendations from Four Health Systems

BACKGROUND

Electronic health record (EHR) alert fatigue, while widely recognized as a concern nationally, lacks a corresponding comprehensive mitigation plan.

OBJECTIVES

The goal of this manuscript is to provide practical guidance to clinical informaticists and other health care leaders who are considering creating a program to manage EHR alerts.

METHODS

This manuscript synthesizes several approaches and recommendations for better alert management derived from four U.S. health care institutions that presented their experiences and recommendations at the American Medical Informatics Association 2019 Clinical Informatics Conference in Atlanta, Georgia, United States. The assembled health care institution leaders represent academic, pediatric, community, and specialized care domains. We describe governance and management, structural concepts and components, and human-computer interactions with alerts, and make recommendations regarding these domains based on our experience supplemented with literature review. This paper focuses on alerts that impact bedside clinicians.

RESULTS

The manuscript addresses the range of considerations relevant to alert management including a summary of the background literature about alerts, alert governance, alert metrics, starting an alert management program, approaches to evaluating alerts prior to deployment, and optimization of existing alerts. The manuscript includes examples of alert optimization successes at two of the represented institutions. In addition, we review limitations on the ability to evaluate alerts in the current state and identify opportunities for further scholarship.

CONCLUSION

Ultimately, alert management programs must strive to meet common goals of improving patient care, while at the same time decreasing the alert burden on clinicians. In so doing, organizations have an opportunity to promote the wellness of patients, clinicians, and EHRs themselves.

Structured override reasons for drug-drug interaction alerts in electronic health records

OBJECTIVE

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

MATERIALS AND METHODS

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

RESULTS

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: "will monitor or take precautions," "not clinically significant," and "benefit outweighs risk."

DISCUSSION

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

CONCLUSIONS

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.