Comparison of Overridden Medication-related Clinical Decision Support in the Intensive Care Unit between a Commercial System and a Legacy System

User Experiences of Transitioning From a Homegrown Electronic Health Record to a Vendor-Based Product in the Department of Veterans Affairs: Qualitative Findings From a Mixed Methods Evaluation

BACKGROUND

The Department of Veterans Affairs (VA), the largest nationally integrated health system in the United States, is transitioning from its homegrown electronic health record (EHR) to a new vendor-based EHR, Oracle Cerner. Experiences of the first VA site to transition have been widely discussed in the media, but in-depth accounts based on rigorous research are lacking.

OBJECTIVE

We sought to explore employee perspectives on the rationale for, and value of, transitioning from a VA-tailored EHR to a vendor-based product.

METHODS

As part of a larger mixed methods, multisite, formative evaluation of VA clinician and staff experiences with the EHR transition, we conducted semistructured interviews at the Mann-Grandstaff VA Medical Center before, during, and after going live in October 2020. In total, we completed 122 interviews with 26 participants across multiple departments.

RESULTS

Before the new vendor-based EHR went live, participants initially expressed cautious optimism about the transition. However, in subsequent interviews following the go-live, participants increasingly critiqued the vendor's understanding of VA's needs, values, and workflows, as well as what they perceived as an inadequate fit between the functionalities of the new vendor-based EHR system and VA's characteristic approach to care. As much as a year after going live, participants reiterated these concerns while also expressing a desire for substantive changes to the transition process, with some questioning the value of continuing with the transition.

CONCLUSIONS

VA's transition from a homegrown EHR to a vendor-based EHR system has presented substantial challenges, both practical and cultural in nature. Consequently, it is a valuable case study for understanding the sociotechnical dimension of EHR-to-EHR transitions. These findings have implications for both VA leadership and the broader community of policy makers, vendors, informaticists, and others involved in large-scale health information technology implementations.

A review of incidents related to health information technology in Swedish healthcare to characterise system issues as a basis for improvement in clinical practice

This study examined health information technology-related incidents to characterise system issues as a basis for improvement in Swedish clinical practice. Incident reports were collected through interviews together with retrospectively collected incidents from voluntary incident databases, which were analysed using deductive and inductive approaches. Most themes pertained to system issues, such as functionality, design, and integration. Identified system issues were dominated by technical factors (74%), while human factors accounted for 26%. Over half of the incidents (55%) impacted on staff or the organisation, and the rest on patients - patient inconvenience (25%) and patient harm (20%). The findings indicate that it is vital to choose and commission suitable systems, design out "error-prone" features, ensure contingency plans are in place, implement clinical decision-support systems, and respond to incidents on time. Such strategies would improve the health information technology systems and Swedish clinical practice.

Beyond the override: Using evidence of previous drug tolerance to suppress drug allergy alerts; a retrospective study of opioid alerts

OBJECTIVE

Despite the extensive literature exploring alert fatigue, most studies have focused on describing the phenomenon, but not on fixing it. The authors aimed to identify data useful to avert clinically irrelevant alerts to inform future research on clinical decision support (CDS) design.

METHODS

We conducted a retrospective observational study of opioid drug allergy alert (DAA) overrides for the calendar year of 2019 at a large academic medical center, to identify data elements useful to find irrelevant alerts to be averted.

RESULTS

Overall, 227,815 DAAs were fired in 2019, with an override rate of 91 % (n = 208196). Opioids represented nearly two-thirds of these overrides (n = 129063; 62 %) and were the drug class with the highest override rate (96 %). On average, 29 opioid DAAs were overridden per patient. While most opioid alerts (97.1 %) are fired for a possible match (the drug class of the allergen matches the drug class of the prescribed drug), they are overridden significantly less frequently for definite match (exact match between allergen and prescribed drug) (88 % vs. 95.9 %, p < 0.001). When comparing the triggering drug with previously administered drugs, override rates were equally high for both definite match (95.9 %), no match (95.5 %), and possible match (95.1 %). Likewise, when comparing to home medications, overrides were excessively high for possible match (96.3 %), no match (96 %), and definite match (94.4 %).

CONCLUSION

We estimate that 74.5% of opioid DAAs (46.4% of all DAAs) at our institution could be relatively safely averted, since they either have a definite match for previous inpatient administrations suggesting drug tolerance or are fired as possible match with low risk of cross-sensitivity. Future research should focus on identifying other relevant data elements ideally with automated methods and use of emerging standards to empower CDS systems to suppress false-positive alerts while avoiding safety hazards.

The Use of Electronic Health Records to Study Drug-Induced Hypersensitivity Reactions from 2000 to 2021: A Systematic Review

Electronic health records (EHRs) have revolutionized the field of drug hypersensitivity reaction (DHR) research. In this systematic review, we assessed 140 articles from 2000-2021, classifying them under six themes: observational studies (n=61), clinical documentation (n=27), case management (n=22), clinical decision support (CDS) (n=18), case identification (n=9), and genetic studies (n=3). EHRs provide convenient access to millions of medical records, facilitating epidemiological studies of DHRs. Though the goal of CDS is to promote safe drug prescribing, allergy alerts must be designed and used in a way that supports this effort. Ultimately, accurate allergy documentation is essential for DHR prevention.

Applying requisite imagination to safeguard electronic health record transitions

Over the next decade, many health care organizations (HCOs) will transition from one electronic health record (EHR) to another; some forced by hospital acquisition and others by choice in search of better EHRs. Herein, we apply principles of Requisite Imagination, or the ability to imagine key aspects of the future one is planning, to offer 6 recommendations on how to proactively safeguard these transitions. First, HCOs should implement a proactive leadership structure that values communication. Second, HCOs should implement proactive risk assessment and testing processes. Third, HCOs should anticipate and reduce unwarranted variation in their EHR and clinical processes. Fourth, HCOs should establish a culture of conscious inquiry with routine system monitoring. Fifth, HCOs should foresee and reduce information access problems. Sixth, HCOs should support their workforce through difficult EHR transitions. Proactive approaches using Requisite Imagination principles outlined here can help ensure safe, effective, and economically sound EHR transitions.

A systematic review of drug allergy alert systems

BACKGROUND AND OBJECTIVE

Drug allergy alert systems (DAAS), have been considered an effective strategy to reduce preventable adverse drug events (ADEs), improving patient's safety. To date, no review has been conducted analyzing characteristics of DAAS in the hospital setting. Therefore, the aim of this study is to identify, describe and summarize the DAAS used in hospitals. The secondary objectives are to analyse drug allergy alerts (DAA) characteristics, the override rate (OvR) and the clinical consequences of alert overrides.

METHODS

Searches were conducted in Medline and Cochrane Library to identify studies describing DAAS. Systems characteristics, generated alerts, DAA, OvR, and its clinical consequences were extracted and analyzed.

RESULTS

Twenty-eight articles were included in the review. Seventeen different electronic DAAS were identified, of which 53% were commercially available. Systems differed in drug allergy information and rules for generating alerts. DAA were generally interruptive, triggered by non-exact match at drug prescribing and when ignored, an override reason was mandatory. The OvR ranged from 43.7% to 97%. The main override reason given by providers was that 'patient had previously tolerated or had taken the drug without allergic reaction'. Clinical consequences of overriding DAA were only analyzed in four studies, with an ADE incidence between 0% and 6%.

CONCLUSIONS

Different DAAS are used in hospitals with some degree of heterogeneity. Accurate and updated drug allergy information is important to generate only high value alerts. A regular review of DAAS and a standardization of alert rules, alert information and override reasons are necessary to optimize systems. Future studies should evaluate the impact of the DAAS aspects on preventing ADEs.

Overall performance of a drug-drug interaction clinical decision support system: quantitative evaluation and end-user survey

BACKGROUND

Clinical decision support systems are implemented in many hospitals to prevent medication errors and associated harm. They are however associated with a high burden of false positive alerts and alert fatigue. The aim of this study was to evaluate a drug-drug interaction (DDI) clinical decision support system in terms of its performance, uptake and user satisfaction and to identify barriers and opportunities for improvement.

METHODS

A quantitative evaluation and end-user survey were performed in a large teaching hospital. First, very severe DDI alerts generated between 2019 and 2021 were evaluated retrospectively. Data collection comprised alert burden, override rates, the number of alert overrides reviewed by pharmacists and the resulting pharmacist recommendations as well as their acceptance rate. Second, an e-survey was carried out among prescribers to assess satisfaction, usefulness and relevance of DDI alerts as well as reasons for overriding.

RESULTS

A total of 38,409 very severe DDI alerts were generated, of which 88.2% were overridden by the prescriber. In 3.2% of reviewed overrides, a recommendation by the pharmacist was provided, of which 79.2% was accepted. False positive alerts were caused by a too broad screening interval and lack of incorporation of patient-specific characteristics, such as QTc values. Co-prescribing of a non-vitamin K oral anticoagulant and a low molecular weight heparin accounted for 49.8% of alerts, of which 92.2% were overridden. In 88 (1.1%) of these overridden alerts, concurrent therapy was still present. Despite the high override rate, the e-survey revealed that the DDI clinical decision support system was found useful by prescribers.

CONCLUSIONS

Identified barriers were the lack of DDI-specific screening intervals and inclusion of patient-specific characteristics, both leading to a high number of false positive alerts and risk for alert fatigue. Despite these barriers, the added value of the DDI clinical decision support system was recognized by prescribers. Hence, integration of DDI-specific screening intervals and patient-specific characteristics is warranted to improve the performance of the DDI software.

Reducing Inappropriate Outpatient Medication Prescribing in Older Adults across Electronic Health Record Systems

BACKGROUND

The American Geriatrics Society recommends against the use of certain potentially inappropriate medications (PIMs) in older adults. Prescribing of these medications correlates with higher rates of hospital readmissions, morbidity, and mortality. Vanderbilt University Medical Center previously deployed clinical decision support (CDS) to decrease PIM prescribing rates, but recently transitioned to a new electronic health record (EHR).

OBJECTIVE

The goal of this study was to evaluate PIM prescribing rates for older adults before and after migration to the new EHR system.

METHODS

We reviewed prescribing rates of PIMs in adults 65 years and older, normalized per 100 total prescriptions from the legacy and new EHR systems between July 1, 2014 and December 31, 2019. The PIM prescribing rates before and after EHR migration during November 2017 were compared using a U-chart and Poisson regression model. Secondary analysis descriptively evaluated the frequency of prescriber acceptance rates in the new EHR.

RESULTS

Prescribing rates of PIMs decreased 5.2% (13.5 per 100 prescriptions to 12.8 per 100 prescriptions; p < 0.0001) corresponding to the implementation of alternatives CDS in the legacy EHR. After migration of the alternative CDS from the legacy to the new EHR system, PIM prescribing rates dropped an additional 18.8% (10.4 per 100 prescriptions; p < 0.0001). Acceptance rates of the alternative recommendations for PIMs was low overall at 11.1%.

CONCLUSION

The prescribing rate of PIMs in adults aged 65 years and older was successfully decreased with the implementation of prescribing CDS. This decrease was not only maintained but strengthened by the transition to a new EHR system.

Clinical Decision Support System with Renal Dose Adjustment Did Not Improve Subsequent Renal and Hepatic Function among Inpatients: The Japan Adverse Drug Event Study

Background  Medication dose adjustment is crucial for patients with renal dysfunction (RD). The assessment of renal function is generally mandatory; however, the renal function may change during the hospital stay and the manual assessment is sometimes challenging.

Objective  We developed the clinical decision support system (CDSS) that provided a recommended dose based on automated calculated renal function.

Methods  We conducted a prospective cohort study in a single teaching hospital in Japan. All hospitalized patients were included except for obstetrics/gynecology and pediatric wards between September 2013 and February 2015. The CDSS was implemented on December 2013. Renal and hepatic dysfunction (HD) were defined as changes in the estimated glomerular filtration rate (eGFR) and alanine aminotransferase or alkaline phosphatase levels based on these measurements during hospital stay. These measurements were obtained before (phase I), after (phase II), and 1 year after (phase III) the CDSS implementation.

Results  We included 6,767 patients (phase I: 2,205; phase II: 2,279; phase III: 2,283). The patients' characteristics were similar among phases. Changes in eGFR were similar among phases, but the incidence of RD increased in phase III (phase I: 228 [10.3%]; phase II: 260 [11.4%]; phase III: 296 [13.0%], p  = 0.02). However, the differences in incidences of RD were not statistically significant after adjusting for eGFR at baseline and age. The incidences of HD were also similar among phases (phase I: 175 [13.2%]; phase II: 171 [12.9%]; phase III: 167 [12.2%], p  = 0.72).

Conclusion  The CDSS implementation did not affect the incidence of renal and HD and changes in renal and hepatic function among hospitalized patients. The effectiveness of the CDSS with renal-guided doses should be investigated with respect to other endpoints.

Impact of Patient Census and Admission Mortality on Pediatric Intensive Care Unit Attending Electronic Health Record Activity: A Preliminary Study

BACKGROUND

Physicians may spend a significant amount of time using the electronic health record (EHR), but this is understudied in the pediatric intensive care unit (PICU). The objective of this study is to quantify PICU attending physician EHR usage and determine its association with patient census and mortality scores.

METHODS

During the year 2016, total EHR, chart review, and documentation times of 7 PICU physicians were collected retrospectively utilizing an EHR-embedded time tracking software package. We examined associations between documentation times and patient census and maximum admission mortality scores. Odds ratios (ORs) are reported per 1-unit increase in patient census and mortality scores.

RESULTS

Overall, total daily EHR usage time (median time [hh:mm] [25th, 75th percentile]) was 2:10 (1:31, 3:08). For all hours (8 a.m.-8 a.m.), no strong association was noted between total EHR time, chart review, and documentation times and patient census, Pediatric Index of Mortality 2 (PIM2), or Pediatric Risk of Mortality 3 (PRISM3) scores. For regular hours (8 a.m.-7 p.m.), no strong association was noted between total EHR, chart review, and documentation times and patient census, PIM2, or PRISM3 scores. When patient census was higher, the odds of EHR after-hour usage (7 p.m.-8 a.m.) was higher (OR 1.262 [1.135, 1.403], p < 0.0001), but there were no increased odds with PIM2 (OR 1.090 [0.956, 1.242], p = 0.20) and PRISM3 (OR 1.010 [0.984, 1.036], p = 0.47) scores. A subset of physicians spent less time performing EHR-related tasks when patient census and admission mortality scores were elevated.

CONCLUSION

We performed a novel evaluation of physician EHR workflow in our PICU. Our pediatric critical care physicians spend approximately 2 hours (out of an expected 10-hour shift) each service day using the EHR, but there was no strong or consistent association between EHR usage and patient census or mortality scores. Future larger scale studies are needed to ensure validity of these results.

Differences, Opportunities, and Strategies in Drug Alert Optimization-Experiences of Two Different Integrated Health Care Systems

BACKGROUND

Concerns about the number of automated medication alerts issued within the electronic health record (EHR), and the subsequent potential for alarm fatigue, led us to examine strategies and methods to optimize the configuration of our drug alerts.

OBJECTIVES

This article reports on comprehensive drug alerting rates and develops strategies across two different health care systems to reduce the number of drug alerts.

METHODS

Standardized reports compared drug alert rates between the two systems, among 13 categories of drug alerts. Both health care systems made modifications to the out-of-box alerts available from their EHR and drug information vendors, focusing on system-wide approaches, when relevant, while performing more drug-specific changes when necessary.

RESULTS

Drug alerting rates even after initial optimization were 38 alerts and 51 alerts per 100 drug orders, respectively. Eight principles were identified and developed to reflect the themes in the implementation and optimization of drug alerting.

CONCLUSION

A team-based, systematic approach to optimizing drug-alerting strategies can reduce the number of drug alerts, but alert rates still remain high. In addition to strategic principles, additional tactical guidelines and recommendations need to be developed to enhance out-of-the-box clinical decision support for drug alerts.

Structured override reasons for drug-drug interaction alerts in electronic health records

OBJECTIVE

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

MATERIALS AND METHODS

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

RESULTS

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: "will monitor or take precautions," "not clinically significant," and "benefit outweighs risk."

DISCUSSION

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

CONCLUSIONS

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Development and Evaluation of a Clinical Decision Support System to Improve Medication Safety

BACKGROUND

Clinical decision support systems (CDSSs) are a good strategy for preventing medication errors and reducing the incidence and severity of adverse drug events (ADEs). However, these systems are not very effective and are subject to multiple limitations that prevent their implementation in clinical practice.

OBJECTIVES

The objective of this study was to evaluate the effectiveness of an advanced CDSS, HIGEA, which generates alerts based on predefined clinical rules to identify patients at risk of an ADE.

METHODS

A multidisciplinary team defined the system and the clinical rules focusing on medication errors commonly encountered in clinical practice. Four intervention programs were defined: (1) dose adjustment in renal impairment; (2) adjustment of anticoagulation/antiplatelet therapy; (3) detection of biochemical/hematologic toxicities; and (4) therapeutic drug monitoring. We performed a 6-month observational prospective study to analyze the effectiveness of these clinical rules by calculating the positive predictive value (PPV).

RESULTS

The team defined 211 clinical rules. During the study period, HIGEA generated 1,086 alerts (8.9 alerts per working day), which were reviewed by pharmacists. Fifty-one percent (554/1,086) of alerts generated an intervention to prevent a possible ADE; of these, 66% (368/554) required a documented modification to therapy owing to a real prescription error intercepted. The intervention program that induced the highest number of modifications to therapy was the dose adjustment in renal impairment program (PPV = 0.51), followed by the adjustment of anticoagulation/antiplatelet therapy program (PPV = 0.24). The percentage of accepted interventions was similar in surgical units (68%), medical units (67%), and critical care units (63%).

CONCLUSION

Our study offers evidence that HIGEA is highly effective in preventing potential ADEs at the prescription stage.

Understanding Health Information Technology Induced Medication Safety Events by Two Conceptual Frameworks

BACKGROUND

While health information technology (health IT) is able to prevent medication errors in many ways, it may also potentially introduce new paths to errors. To understand the impact of health IT induced medication errors, this study aims to conduct a retrospective analysis of medication safety reports.

METHODS

From the U.S. Food and Drug Administration (FDA) Manufacturer and User Facility Device Experience database, we identified reports in which health IT is a contributing factor to medication errors. We applied two conceptual frameworks, Sittig and Singh's sociotechnical model and Coiera's information value chain, to examine the identified reports.

RESULTS

We identified 152 unique reports on health IT induced medication errors as the final report set for review. The majority (65.13%) of the reports involved multiple contributing factors according to the sociotechnical model. Three dimensions, that is, clinical content, human-computer interface, and people, were involved in more reports than the others. The transition of the effects of health IT on medication practice was summarized using information value chain. Health IT related contributing factors may lead to receiving wrong information, missing information, receiving partial information and delayed information, and receiving wrong information and missing information tend to cause the commission errors in decision-making.

CONCLUSION

The two frameworks provide an opportunity to understand a comprehensive context of safety event and the impact of health IT induced errors on medication safety. The sociotechnical model helps identify the aspects causing medication safety issues. The information value chain helps uncover the effect of the health IT induced medication errors on health care process and patient outcomes.

Current challenges in health information technology-related patient safety

We identify and describe nine key, short-term, challenges to help healthcare organizations, health information technology developers, researchers, policymakers, and funders focus their efforts on health information technology-related patient safety. Categorized according to the stage of the health information technology lifecycle where they appear, these challenges relate to (1) developing models, methods, and tools to enable risk assessment; (2) developing standard user interface design features and functions; (3) ensuring the safety of software in an interfaced, network-enabled clinical environment; (4) implementing a method for unambiguous patient identification (1-4 Design and Development stage); (5) developing and implementing decision support which improves safety; (6) identifying practices to safely manage information technology system transitions (5 and 6 Implementation and Use stage); (7) developing real-time methods to enable automated surveillance and monitoring of system performance and safety; (8) establishing the cultural and legal framework/safe harbor to allow sharing information about hazards and adverse events; and (9) developing models and methods for consumers/patients to improve health information technology safety (7-9 Monitoring, Evaluation, and Optimization stage). These challenges represent key "to-do's" that must be completed before we can expect to have safe, reliable, and efficient health information technology-based systems required to care for patients.

Prospective evaluation of medication-related clinical decision support over-rides in the intensive care unit

Background

Clinical decision support (CDS) displayed in electronic health records has been found to reduce the incidence of medication errors and adverse drug events (ADE). Recent data suggested that medication-related CDS alerts were frequently over-ridden, often inappropriately. Patients in the intensive care unit (ICU) are at an increased risk of ADEs; however, limited data exist on the benefits of CDS in the ICU. This study aims to evaluate potential harm associated with medication-related CDS over-rides in the ICU.

Methods

This was a prospective observational study of adults admitted to any of six ICUs between July 2016 and April 2017 at our institution. Patients with provider-overridden CDS for dose (orders for scheduled frequency and not pro re nata), drug allergy, drug–drug interaction, geriatric and renal alerts (contraindicated medications for renal function or renal dosing) were included. The primary outcome was the appropriateness of over-rides, which were evaluated by two independent reviewers. Secondary outcomes included incidence of ADEs following alert over-ride and risk of ADEs based on over-ride appropriateness.

Results

A total of 2448 over-ridden alerts from 712 unique patient encounters met inclusion criteria. The overall appropriateness rate for over-rides was 81.6% and varied by alert type. More ADEs (potential and definite) were identified following inappropriate over-rides compared with appropriate over-rides (16.5 vs 2.74 per 100 over-ridden alerts, Fisher’s exact test P<0.001). An adjusted logistic regression model showed that inappropriate over-rides were associated with an increased risk of ADEs (OR 6.14, 95% CI 4.63 to 7.71, P<0.001).

Conclusions

Approximately four of five identified CDS over-rides were appropriately over-ridden, with the rate varying by alert type. However, inappropriate over-rides were six times as likely to be associated with potential and definite ADEs, compared with appropriate over-rides. Further efforts should be targeted at improving the positive predictive value of CDS such as by suppressing alerts that are appropriately over-ridden.