Development and validation of a clinical decision support system to prevent anticoagulant duplications

BACKGROUND AND OBJECTIVE

Unintended duplicate prescriptions of anticoagulants increase the risk of serious adverse events. Clinical Decision Support Systems (CDSSs) can help prevent such medication errors; however, sophisticated algorithms are needed to avoid alert fatigue. This article describes the steps taken in our hospital to develop a CDSS to prevent anticoagulant duplication (AD).

METHODS

The project was composed of three phases. In phase I, the status quo was established. In phase II, a clinical pharmacist developed an algorithm to detect ADs using daily data exports. In phase III, the algorithm was integrated into the hospital's electronic health record system. Alerts were reviewed by clinical pharmacists before being sent to the prescribing physician. We conducted a retrospective analysis of all three phases to assess the impact of the interventions on the occurrence and duration of ADs. Phase III was analyzed in more detail regarding the acceptance rate, sensitivity, and specificity of the alerts.

RESULTS

We identified 91 ADs in 1581 patients receiving two or more anticoagulants during phase I, 70 ADs in 1692 patients in phase II, and 57 ADs in 1575 patients in phase III. Mean durations of ADs were 1.8, 1.4, and 1.1 calendar days during phases I, II, and III, respectively. In comparison to the baseline in phase I, the relative risk reduction of AD in patients treated with at least two different anticoagulants during phase III was 42% (RR: 0.58, CI: 0.42-0.81). A total of 429 alerts were generated during phase III, many of which were self-limiting, and 186 alerts were sent to the respective prescribing physician. The acceptance rate was high at 97%. We calculated a sensitivity of 87.4% and a specificity of 87.9%.

CONCLUSION

The stepwise development of a CDSS for the detection of AD markedly reduced the frequency and duration of medication errors in our hospital, thereby improving patient safety.

Optimising interruptive clinical decision support alerts for antithrombotic duplicate prescribing in hospital

INTRODUCTION

Duplicate prescribing clinical decision support alerts can prevent important prescribing errors but are frequently the cause of much alert fatigue. Stat dose prescriptions are a known reason for overriding these alerts. This study aimed to evaluate the effect of excluding stat dose prescriptions from duplicate prescribing alerts for antithrombotic medicines on alert burden, prescriber adherence, and prescribing.

MATERIALS AND METHODS

A before (January 1st, 2017 to August 31st, 2022) and after (October 5th, 2022 to September 30th, 2023) study was undertaken of antithrombotic duplicate prescribing alerts and prescribing following a change in alert settings. Alert and prescribing data for antithrombotic medicines were joined, processed, and analysed to compare alert rates, adherence, and prescribing. Alert burden was assessed as alerts per 100 prescriptions. Adherence was measured at the point of the alert as whether the prescriber accepted the alert and following the alert as whether a relevant prescription was ceased within an hour. Co-prescribing of antithrombotic stat dose prescriptions was assessed pre- and post-alert reconfiguration.

RESULTS

Reconfiguration of the alerts reduced the alert rate by 29 % (p < 0.001). The proportion of alerts associated with cessation of antithrombotic duplication significantly increased (32.8 % to 44.5 %, p < 0.001). Adherence at the point of the alert increased 1.2 % (4.8 % to 6.0 %, p = 0.012) and 11.5 % (29.4 % to 40.9 %, p < 0.001) within one hour of the alert. When ceased after the alert over 80 % of duplicate prescriptions were ceased within 2 min of overriding. Antithrombotic stat dose co-prescribing was unchanged for 4 out of 5 antithrombotic duplication alert rules.

CONCLUSION

By reconfiguring our antithrombotic duplicate prescribing alerts, we reduced alert burden and increased alert adherence. Many prescribers ceased duplicate prescribing within 2 min of alert override highlighting the importance of incorporating post-alert measures in accurately determining prescriber alert adherence.

Why do users override alerts? Utilizing large language model to summarize comments and optimize clinical decision support

OBJECTIVES

To evaluate the capability of using generative artificial intelligence (AI) in summarizing alert comments and to determine if the AI-generated summary could be used to improve clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We extracted user comments to alerts generated from September 1, 2022 to September 1, 2023 at Vanderbilt University Medical Center. For a subset of 8 alerts, comment summaries were generated independently by 2 physicians and then separately by GPT-4. We surveyed 5 CDS experts to rate the human-generated and AI-generated summaries on a scale from 1 (strongly disagree) to 5 (strongly agree) for the 4 metrics: clarity, completeness, accuracy, and usefulness.

RESULTS

Five CDS experts participated in the survey. A total of 16 human-generated summaries and 8 AI-generated summaries were assessed. Among the top 8 rated summaries, five were generated by GPT-4. AI-generated summaries demonstrated high levels of clarity, accuracy, and usefulness, similar to the human-generated summaries. Moreover, AI-generated summaries exhibited significantly higher completeness and usefulness compared to the human-generated summaries (AI: 3.4 ± 1.2, human: 2.7 ± 1.2, P = .001).

CONCLUSION

End-user comments provide clinicians' immediate feedback to CDS alerts and can serve as a direct and valuable data resource for improving CDS delivery. Traditionally, these comments may not be considered in the CDS review process due to their unstructured nature, large volume, and the presence of redundant or irrelevant content. Our study demonstrates that GPT-4 is capable of distilling these comments into summaries characterized by high clarity, accuracy, and completeness. AI-generated summaries are equivalent and potentially better than human-generated summaries. These AI-generated summaries could provide CDS experts with a novel means of reviewing user comments to rapidly optimize CDS alerts both online and offline.

The use of artificial intelligence to optimize medication alerts generated by clinical decision support systems: a scoping review

OBJECTIVE

Current Clinical Decision Support Systems (CDSSs) generate medication alerts that are of limited clinical value, causing alert fatigue. Artificial Intelligence (AI)-based methods may help in optimizing medication alerts. Therefore, we conducted a scoping review on the current state of the use of AI to optimize medication alerts in a hospital setting. Specifically, we aimed to identify the applied AI methods used together with their performance measures and main outcome measures.

MATERIALS AND METHODS

We searched Medline, Embase, and Cochrane Library database on May 25, 2023 for studies of any quantitative design, in which the use of AI-based methods was investigated to optimize medication alerts generated by CDSSs in a hospital setting. The screening process was supported by ASReview software.

RESULTS

Out of 5625 citations screened for eligibility, 10 studies were included. Three studies (30%) reported on both statistical performance and clinical outcomes. The most often reported performance measure was positive predictive value ranging from 9% to 100%. Regarding main outcome measures, alerts optimized using AI-based methods resulted in a decreased alert burden, increased identification of inappropriate or atypical prescriptions, and enabled prediction of user responses. In only 2 studies the AI-based alerts were implemented in hospital practice, and none of the studies conducted external validation.

DISCUSSION AND CONCLUSION

AI-based methods can be used to optimize medication alerts in a hospital setting. However, reporting on models' development and validation should be improved, and external validation and implementation in hospital practice should be encouraged.

Adoption by clinicians of electronic order communications in NHS secondary care: a descriptive account

BACKGROUND

Due to the rapid advancement in information technology, changes to communication modalities are increasingly implemented in healthcare. One such modality is Computerised Provider Order Entry (CPOE) systems which replace paper, verbal or telephone orders with electronic booking of requests. We aimed to understand the uptake, and user acceptability, of CPOE in a large National Health Service hospital system.

METHODS

This retrospective single-centre study investigates the longitudinal uptake of communications through the Prescribing, Information and Communication System (PICS). The development and configuration of PICS are led by the doctors, nurses and allied health professionals that use it and requests for CPOE driven by clinical need have been described.Records of every request (imaging, specialty review, procedure, laboratory) made through PICS were collected between October 2008 and July 2019 and resulting counts were presented. An estimate of the proportion of completed requests made through the system has been provided for three example requests. User surveys were completed.

RESULTS

In the first 6 months of implementation, a total of 832 new request types (imaging types and specialty referrals) were added to the system. Subsequently, an average of 6.6 new request types were added monthly. In total, 8 035 132 orders were requested through PICS. In three example request types (imaging, endoscopy and full blood count), increases in the proportion of requests being made via PICS were seen. User feedback at 6 months reported improved communications using the electronic system.

CONCLUSION

CPOE was popular, rapidly adopted and diversified across specialties encompassing wide-ranging requests.

[Inability of hospital computerised physician order entry systems to secure the use of concentrated potassium intravenous solutions]

OBJECTIVES

To determine whether hospital computerised physician order entry (CPOE) systems contribute to securing intravenous potassium chloride (KCl) prescriptions with reference to the recommendations issued by French healthcare agencies.

METHODS

We sent a questionnaire to the members of the Association pour le Digital et l'Information en Pharmacie.

RESULTS

More than three quarters of the 84 responses received involving 23 CPOE systems indicate that it is possible to: prescribe an ampoule of concentrated potassium chloride 10% 10mL intravenously without any diluents (80%); prescribe 4g of KCl in a bag of 500mL of NaCl 0,9% (98%); prescribe a solution that contains 6 grams of KCl per liter (94%); prescribe the administration of an injectable ampoule orally by means of a free text comment (83%). Nearly half of the responses indicate that it is possible to prescribe: concentrated KCl ampoules as administration solvent (50%); an injectable vial to be administered by oral route (52%).

CONCLUSION

At least 23 hospital CPOE systems are unable to secure the prescriptions of injectable KCl. This finding lifts the veil on an unthought, namely the role of CPOE systems in securing high-risk medications. In order to solve this problem, it should be mandatory that health information technology vendors pay particular attention to these drugs. With regard to injectable KCl, the utilisation of a dilution vehicle, maximum concentration and maximum infusion flow rate are the first four constraints to be satisfied.

Randomized Comparison of Electronic Health Record Alert Types in Eliciting Responses about Prognosis in Gynecologic Oncology Patients

OBJECTIVES

 To compare the ability of different electronic health record alert types to elicit responses from users caring for cancer patients benefiting from goals of care (GOC) conversations.

METHODS

 A validated question asking if the user would be surprised by the patient's 6-month mortality was built as an Epic BestPractice Advisory (BPA) alert in three versions-(1) Required on Open chart (pop-up BPA), (2) Required on Close chart (navigator BPA), and (3) Optional Persistent (Storyboard BPA)-randomized using patient medical record number. Meaningful responses were defined as "Yes" or "No," rather than deferral. Data were extracted over 6 months.

RESULTS

 Alerts appeared for 685 patients during 1,786 outpatient encounters. Measuring encounters where a meaningful response was elicited, rates were highest for Required on Open (94.8% of encounters), compared with Required on Close (90.1%) and Optional Persistent (19.7%) (p < 0.001). Measuring individual alerts to which responses were given, they were most likely meaningful with Optional Persistent (98.3% of responses) and least likely with Required on Open (68.0%) (p < 0.001). Responses of "No," suggesting poor prognosis and prompting GOC, were more likely with Optional Persistent (13.6%) and Required on Open (10.3%) than with Required on Close (7.0%) (p = 0.028).

CONCLUSION

 Required alerts had response rates almost five times higher than optional alerts. Timing of alerts affects rates of meaningful responses and possibly the response itself. The alert with the most meaningful responses was also associated with the most interruptions and deferral responses. Considering tradeoffs in these metrics is important in designing clinical decision support to maximize success.

Use of Computerized Physician Order Entry with Clinical Decision Support to Prevent Dose Errors in Pediatric Medication Orders: A Systematic Review

BACKGROUND

Prescribing is a high-risk task within the pediatric medication-use process and requires defenses to prevent errors. Such system-centric defenses include electronic health record systems with computerized physician order entry (CPOE) and clinical decision support (CDS) tools that assist safe prescribing. The objective of this study was to examine the effects of CPOE systems with CDS functions in preventing dose errors in pediatric medication orders.

MATERIAL AND METHODS

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 criteria and Synthesis Without Meta-Analysis (SWiM) items. The study protocol was registered in PROSPERO (CRD42021277413). The final literature search on MEDLINE (Ovid), Scopus, Web of Science, and EMB Reviews was conducted on 10 September 2023. Only peer-reviewed studies considering both CPOE and CDS systems in pediatric inpatient or outpatient settings were included. Study selection, data extraction, and evidence quality assessment (JBI critical appraisal tool assessment and GRADE approach) were carried out by two individual reviewers. Vote counting method was used to evaluate the effects of CPOE-CDS systems on dose errors rates.

RESULTS

A total of 17 studies published in 2007-2021 met the inclusion criteria. The most used CDS tools were dose range check (n = 14), dose calculator (n = 8), and dosing frequency check (n = 8). Alerts were recorded in 15 studies. A statistically significant reduction in dose errors was found in eight studies, whereas an increase of dose errors was not reported.

CONCLUSIONS

The CPOE-CDS systems have the potential to reduce pediatric dose errors. Most beneficial interventions seem to be system customization, implementing CDS alerts, and the use of dose range check. While human factors are still present within the medication use process, further studies and development activities are needed to optimize the usability of CPOE-CDS systems.

Retrospective analysis of the impact of electronic medical record alerts on low value care in a pediatric hospital

OBJECTIVES

Hospital costs continue to rise unsustainably. Up to 20% of care is wasteful including low value care (LVC). This study aimed to understand whether electronic medical record (EMR) alerts are effective at reducing pediatric LVC and measure the impact on hospital costs.

MATERIALS AND METHODS

Using EMR data over a 76-month period, we evaluated changes in 4 LVC practices following the implementation of EMR alerts, using time series analysis to control for underlying time-based trends, in a large pediatric hospital in Australia. The main outcome measure was the change in rate of each LVC practice. Balancing measures included the rate of alert adherence as a proxy measure for risk of alert fatigue. Hospital costs were calculated by the volume of LVC avoided multiplied by the unit costs. Costs of the intervention were calculated from clinician and analyst time required.

RESULTS

All 4 LVC practices showed a statistically significant reduction following alert implementation. Two LVC practices (blood tests) showed an abrupt change, associated with high rates of alert adherence. The other 2 LVC practices (bronchodilator use in bronchiolitis and electrocardiogram ordering for sleeping bradycardia) showed an accelerated rate of improvement compared to baseline trends with lower rates of alert adherence. Hospital savings were $325 to $180 000 per alert.

DISCUSSION AND CONCLUSION

EMR alerts are effective in reducing pediatric LVC practices and offer a cost-saving opportunity to the hospital. Further efforts to leverage EMR alerts in pediatric settings to reduce LVC are likely to support future sustainable healthcare delivery.

A Practical Approach to Optimize Computerized Provider Order Entry Systems and Reduce Clinician Burden: Pre-Post Evaluation of Vendor-Derived "Order Friction" Data

Computerized provider order entry (CPOE) systems have been cited as a significant contributor to clinician burden. Vendor-derived measures and data sets have been developed to help with optimization of CPOE systems. We describe how we analyzed vendor-derived Order Friction (OF) EHR log data at our health system and propose a practical approach for optimizing CPOE systems by reducing OF. We also conducted a pre-post intervention study using OF data to evaluate the impact of defaulting the frequency of urine, stool and nasal swab tests and found that all modified orders had significantly fewer changes required per order (p<0.01). Our proposed approach is a six-step process: 1) understand the ordering process, 2) understand OF data elements contextually, 3) explore ordering user-level factors, 4) evaluate order volume and friction from different order sources, 5) optimize order-level design, 6) identify high volume alerts to evaluate for appropriateness.

Detecting atypical alert behavior through statistical process control: Clinical decision support alert frequency visualizations

Clinical decision support (CDS) alerts are designed to work according to a set of clearly defined criteria and have the potential to improve clinical care. To efficiently and proactively review abnormally functioning CDS alerts, we postulate that the introduction of a dashboard with statistical process control (SPC) charting will lead to effective detection of erratic alert behavior. We identified custom CDS alerts from an academic medical center that were recorded and monitored in a longitudinal fashion and the data warehouses where this information was stored. We created a dashboard of alert frequency using SPC charts, applied SPC rules for classification of variation, and validated dashboard data. From June-August 2022, the dashboard effectively pulled in data to visually depict alert behavior. SPC-defined parameters for standard deviation from the mean were applied to visualizations and allowed for rapid review of alerts with greatest variation. These alerts were subsequently investigated, and it was determined that they were functioning correctly. The most profound abnormalities detected during implementation reflected changes in practice and not system errors, though further investigation into thresholds for statistical significance will benefit this field. We conclude that SPC visualizations are a time-efficient and effective method of identifying CDS malfunctions.

Addressing Alert Fatigue by Replacing a Burdensome Interruptive Alert with Passive Clinical Decision Support

BACKGROUND

Recognizing that alert fatigue poses risks to patient safety and clinician wellness, there is a growing emphasis on evaluation and governance of electronic health record clinical decision support (CDS). This is particularly critical for interruptive alerts to ensure that they achieve desired clinical outcomes while minimizing the burden on clinicians. This study describes an improvement effort to address a problematic interruptive alert intended to notify clinicians about patients needing coronavirus disease 2019 (COVID) precautions and how we collaborated with operational leaders to develop an alternative passive CDS system in acute care areas.

OBJECTIVES

Our dual aim was to reduce the alert burden by redesigning the CDS to adhere to best practices for decision support while also improving the percent of admitted patients with symptoms of possible COVID who had appropriate and timely infection precautions orders.

METHODS

Iterative changes to CDS design included adjustment to alert triggers and acknowledgment reasons and development of a noninterruptive rule-based order panel for acute care areas. Data on alert burden and appropriate precautions orders on symptomatic admitted patients were followed over time on run and attribute (p) and individuals-moving range control charts.

RESULTS

At baseline, the COVID alert fired on average 8,206 times per week with an alert per encounter rate of 0.36. After our interventions, the alerts per week decreased to 1,449 and alerts per encounter to 0.07 equating to an 80% reduction for both metrics. Concurrently, the percentage of symptomatic admitted patients with COVID precautions ordered increased from 23 to 61% with a reduction in the mean time between COVID test and precautions orders from 19.7 to -1.3 minutes.

CONCLUSION

CDS governance, partnering with operational stakeholders, and iterative design led to successful replacement of a frequently firing interruptive alert with less burdensome passive CDS that improved timely ordering of COVID precautions.

Sharpening clinical decision support alert and reminder designs with MINDSPACE: A systematic review

BACKGROUND

Clinical decision support (CDS) alerts and reminders aim to influence clinical decisions, yet they are often designed without considering human decision-making behaviour. While this behaviour is comprehensively described by behavioural economics (BE), the sheer volume of BE literature poses a challenge to designers when identifying behavioural effects with utility to alert and reminder designs. This study tackles this challenge by focusing on the MINDSPACE framework for behaviour change, which collates nine behavioural effects that profoundly influence human decision-making behaviour: Messenger, Incentives, Norms, Defaults, Salience, Priming, Affect, Commitment, and Ego.

METHOD

A systematic review searching MEDLINE, Embase, PsycINFO, and CINAHL Plus to explore (i) the usage of MINDSPACE effects in alert and reminder designs and (ii) the efficacy of those alerts and reminders in influencing clinical decisions. The search queries comprised ten Boolean searches, with nine focusing on the MINDSPACE effects and one focusing on the term mindspace.

RESULTS

50 studies were selected from 1791 peer-reviewed journal articles in English from 1970 to 2022. Except for ego, eight of nine MINDSPACE effects were utilised to design alerts and reminders, with defaults and norms utilised the most in alerts and reminders, respectively. Overall, alerts and reminders informed by MINDSPACE effects showed an average 71% success rate in influencing clinical decisions (alerts 73%, reminders 69%). Most studies utilised a single effect in their design, with higher efficacy for alerts (64%) than reminders (41%). Others utilised multiple effects, showing higher efficacy for reminders (28%) than alerts (9%).

CONCLUSION

This review presents sufficient evidence demonstrating the MINDSPACE framework's merits for designing CDS alerts and reminders with human decision-making considerations. The framework can adequately address challenges in identifying behavioural effects pertinent to the effective design of CDS alerts and reminders. The review also identified opportunities for future research into other relevant effects (e.g., framing).

Ability of machine-learning based clinical decision support system to reduce alert fatigue, wrong-drug errors, and alert users about look alike, sound alike medication

BACKGROUND AND OBJECTIVE

The overall benefits of using clinical decision support systems (CDSSs) can be restrained if physicians inadvertently ignore clinically useful alerts due to "alert fatigue" caused by an excessive number of clinically irrelevant warnings. Moreover, inappropriate drug errors, look-alike/sound-alike (LASA) drug errors, and problem list documentation are common, costly, and potentially harmful. This study sought to evaluate the overall performance of a machine learning-based CDSS (MedGuard) for triggering clinically relevant alerts, acceptance rate, and to intercept inappropriate drug errors as well as LASA drug errors.

METHODS

We conducted a retrospective study that evaluated MedGuard alerts, the alert acceptance rate, and the rate of LASA alerts between July 1, 2019, and June 31, 2021, from outpatient settings at an academic hospital. An expert pharmacist checked the suitability of the alerts, rate of acceptance, wrong-drug errors, and confusing drug pairs.

RESULTS

Over the two-year study period, 1,206,895 prescriptions were ordered and a total of 28,536 alerts were triggered (alert rate: 2.36 %). Of the 28,536 alerts presented to physicians, 13,947 (48.88 %) were accepted. A total of 8,014 prescriptions were changed/modified (28.08 %, 8,014/28,534) with the most common reasons being adding and/or deleting diseases (52.04 %, 4,171/8,014), adding and/or deleting drugs (21.89 %, 1,755/8,014) and others (35.48 %, 2,844/ 8,014). However, the rate of drug error interception was 1.64 % (470 intercepted errors out of 28,536 alerts), which equates to 16.4 intercepted errors per 1000 alerted orders.

CONCLUSION

This study shows that machine learning based CDSS, MedGuard, has an ability to improve patients' safety by triggering clinically valid alerts. This system can also help improve problem list documentation and intercept inappropriate drug errors and LASA drug errors, which can improve medication safety. Moreover, high acceptance of alert rates can help reduce clinician burnout and adverse events.

Monitoring for adverse drug events of high-risk medications with a computerized clinical decision support system: a prospective cohort study

Monitoring is recommended to prevent severe adverse drug events, but such examinations are often missed. To increase the number of monitoring that should be ordered for high-risk medications, we introduced a clinical decision support system (CDSS) that alerts and orders the monitoring for high-risk medications in an outpatient setting. We conducted a 2-year prospective cohort study at a tertiary care teaching hospital before (phase 1) and after (phase 2) the activation of a CDSS. The CDSS automatically provided alerts for liver function tests for vildagliptin, thyroid function tests for immune checkpoint inhibitors (ICIs) and multikinase inhibitors (MKIs), and a slit-lamp examination of the eyes for oral amiodarone when outpatients were prescribed the medications but not examined for a fixed period. The order of laboratory tests automatically appeared if alert was accepted. The alerts were hidden and did not appear on the display before activation of the CDSS. The outcomes were the number of prescriptions with alerts and examinations. During the study period, 330 patients in phase 1 and 307 patients in phase 2 were prescribed vildagliptin, 20 patients in phase 1 and 19 patients in phase 2 were prescribed ICIs or MKIs, and 72 patients in phase 1 and 66 patients in phase 2 were prescribed oral amiodarone. The baseline characteristics were similar between the phases. In patients prescribed vildagliptin, the proportion of alerts decreased significantly (38% vs 27%, P < 0.0001), and the proportion of examinations increased significantly (0.9% vs 4.0%, P < 0.0001) after activation of the CDSS. In patients prescribed ICIs or MKIs, the proportion of alerts decreased significantly (43% vs 11%, P < 0.0001), and the proportion of examinations increased numerically, but not significantly (2.6% vs 7.0%, P = 0.13). In patients prescribed oral amiodarone, the proportion of alerts decreased (86% vs 81%, P = 0.055), and the proportion of examinations increased (2.2% and 3.0%, P = 0.47); neither was significant. The CDSS has potential to increase the monitoring for high-risk medications. Our study also highlighted the limited acceptance rate of monitoring by CDSS. Further studies are needed to explore the generalizability to other medications and the cause of the limited acceptance rates among physicians.

Clinician Response to Pharmacogenetic Clinical Decision Support Alerts

The objective of this study was to characterize clinician response following standardization of pharmacogenetic (PGx) clinical decision support alerts at University of Florida (UF) Health. A retrospective analysis of all PGx alerts that fired at a tertiary academic medical center from August 2020 through May 2022 was performed. Alert acceptance rate was calculated and compared across six gene-drug pairs, patient care setting, and clinician specialty. The disposition of the triggering medication was compared with the alert response and evaluated for congruence. There were a total of 818 alerts included for analysis of alert response, 557 alerts included in acceptance rate calculations, and 392 alerts with sufficient information to assess clinical response. The overall acceptance rate was 63%. The alert response and clinical response were congruent for 47% of alerts. Alert response was influenced by the triggering gene-drug pair, clinician specialty, patient care setting, and specialty of the provider who initially ordered the PGx test. Clinical response was mostly incongruent with alert response. Alert acceptance is influenced by the triggering gene-drug pair, clinician specialty, and care setting. Alert response is not a reliable surrogate marker for clinical action. Any future research into the impact of clinical decision support (CDS) alerts should focus on clinical response rather than alert response. Given the reliance on CDS alerts to enhance uptake of PGx, it is worthwhile to further investigate their impact on prescribing and patient outcomes.

Effect of knowledgebase transition of a clinical decision support system on medication order and alert patterns in an emergency department

A knowledgebase (KB) transition of a clinical decision support (CDS) system occurred at the study site. The transition was made from one commercial database to another, provided by a different vendor. The change was applied to all medications in the institute. The aim of this study was to analyze the effect of KB transition on medication-related orders and alert patterns in an emergency department (ED). Data of patients, medication-related orders and alerts, and physicians in the ED from January 2018 to December 2020 were analyzed in this study. A set of definitions was set to define orders, alerts, and alert overrides. Changes in order and alert patterns before and after the conversion, which took place in May 2019, were assessed. Overall, 101,450 patients visited the ED, and 1325 physicians made 829,474 prescription orders to patients during visit and at discharge. Alert rates (alert count divided by order count) for periods A and B were 12.6% and 14.1%, and override rates (alert override count divided by alert count) were 60.8% and 67.4%, respectively. Of the 296 drugs that were used more than 100 times during each period, 64.5% of the drugs had an increase in alert rate after the transition. Changes in alert rates were tested using chi-squared test and Fisher's exact test. We found that the CDS system knowledgebase transition was associated with a significant change in alert patterns at the medication level in the ED. Careful consideration is advised when such a transition is performed.

Impact of Predictive Text Clinical Decision Support on Imaging Order Entry in the Emergency Department

PURPOSE

Imaging clinical decision support (CDS) is designed to assist providers in selecting appropriate imaging studies and is now federally required. The aim of this study was to understand the effect of CDS on decisions and workflows in the emergency department (ED).

METHODS

The authors' institution's order entry platform serves up structured indications for imaging orders. Imaging orders are scored by CDS on the basis of appropriate use criteria (AUC). CDS triggers alerts for imaging orders with low AUC scores. Because free text alone cannot be scored by CDS, an artificial intelligence predictive text (AIPT) module was implemented to guide the selection of structured indications when free-text indications are entered. A total of 17,355 imaging orders in the ED over 6 months were retrospectively analyzed.

RESULTS

CDS alerts for low AUC scores were triggered for 3% of all imaging study orders (522 of 17,355). Providers spent an average of 24 seconds interacting with alerts. In 18 of 522 imaging orders with alerts, alternative studies were ordered. After AIPT implementation, the percentage of unscored studies significantly decreased from 81% to 45% (P < .001).

CONCLUSIONS

In a quaternary academic ED, CDS alerts triggered by low AUC scores caused minimal increase in time spent on imaging order entry but had a relatively marginal impact on imaging study selection. AIPT implementation increased the number of scored studies and could potentially enhance CDS effects. CDS implementation enables the collection of novel data regarding which imaging studies receive low AUC scores. Future work could include exploring alternative models of CDS implementation to maximize its impact.

Evaluation of clinical decision support systems in oncology: An updated systematic review

With increasing reliance on technology in oncology, the impact of digital clinical decision support (CDS) tools needs to be examined. A systematic review update was conducted and peer-reviewed literature from 2016 to 2022 were included if CDS tools were used for live decision making and comparatively assessed quantitative outcomes. 3369 studies were screened and 19 were included in this updated review. Combined with a previous review of 24 studies, a total of 43 studies were analyzed. Improvements in outcomes were observed in 42 studies, and 34 of these were of statistical significance. Computerized physician order entry and clinical practice guideline systems comprise the greatest number of evaluated CDS tools (13 and 10 respectively), followed by those that utilize patient-reported outcomes (8), clinical pathway systems (8) and prescriber alerts for best-practice advisories (4). Our review indicates that CDS can improve guideline adherence, patient-centered care, and care delivery processes in oncology.

Impact of Artificial Intelligence-Assisted Indication Selection on Appropriateness Order Scoring for Imaging Clinical Decision Support

PURPOSE

The aim of this study was to assess appropriateness scoring and structured order entry after the implementation of an artificial intelligence (AI) tool for analysis of free-text indications.

METHODS

Advanced outpatient imaging orders in a multicenter health care system were recorded 7 months before (March 1, 2020, to September 21, 2020) and after (October 20, 2020, to May 13, 2021) the implementation of an AI tool targeting free-text indications. Clinical decision support score (not appropriate, may be appropriate, appropriate, or unscored) and indication type (structured, free-text, both, or none) were assessed. The χ2 and multivariate logistic regression adjusting for covariables with bootstrapping were used.

RESULTS

In total, 115,079 orders before and 150,950 orders after AI tool deployment were analyzed. The mean patient age was 59.3 ± 15.5 years, and 146,035 (54.9%) were women; 49.9% of orders were for CT, 38.8% for MR, 5.9% for nuclear medicine, and 5.4% for PET. After deployment, scored orders increased to 52% from 30% (P < .001). Orders with structured indications increased to 67.3% from 34.6% (P < .001). On multivariate analysis, orders were more likely to be scored after tool deployment (odds ratio [OR], 2.7, 95% CI, 2.63-2.78; P < .001). Compared with physicians, orders placed by nonphysician providers were less likely to be scored (OR, 0.80; 95% CI, 0.78-0.83; P < .001). MR (OR, 0.84; 95% CI, 0.82-0.87) and PET (OR, 0.12; 95% CI, 0.10-0.13) were less likely to be scored than CT (; P < .001). After AI tool deployment, 72,083 orders (47.8%) remained unscored, 45,186 (62.7%) with free-text-only indications.

CONCLUSIONS

Embedding AI assistance within imaging clinical decision support was associated with increased structured indication orders and independently predicted a higher likelihood of scored orders. However, 48% of orders remained unscored, driven by both provider behavior and infrastructure-related barriers.

Post-implementation optimization of medication alerts in hospital computerized provider order entry systems: a scoping review

OBJECTIVES

A scoping review identified interventions for optimizing hospital medication alerts post-implementation, and characterized the methods used, the populations studied, and any effects of optimization.

MATERIALS AND METHODS

A structured search was undertaken in the MEDLINE and Embase databases, from inception to August 2023. Articles providing sufficient information to determine whether an intervention was conducted to optimize alerts were included in the analysis. Snowball analysis was conducted to identify additional studies.

RESULTS

Sixteen studies were identified. Most were based in the United States and used a wide range of clinical software. Many studies used inpatient cohorts and conducted more than one intervention during the trial period. Alert types studied included drug-drug interactions, drug dosage alerts, and drug allergy alerts. Six types of interventions were identified: alert inactivation, alert severity reclassification, information provision, use of contextual information, threshold adjustment, and encounter suppression. The majority of interventions decreased alert quantity and enhanced alert acceptance. Alert quantity decreased with alert inactivation by 1%-25.3%, and with alert severity reclassification by 1%-16.5% in 6 of 7 studies. Alert severity reclassification increased alert acceptance by 4.2%-50.2% and was associated with a 100% acceptance rate for high-severity alerts when implemented. Clinical errors reported in 4 studies were seen to remain stable or decrease.

DISCUSSION

Post-implementation medication optimization interventions have positive effects for clinicians when applied in a variety of settings. Less well reported are the impacts of these interventions on the clinical care of patients, and how endpoints such as alert quantity contribute to changes in clinician and pharmacist perceptions of alert fatigue.

CONCLUSION

Well conducted alert optimization can reduce alert fatigue by reducing overall alert quantity, improving clinical acceptance, and enhancing clinical utility.

A Data-Driven Approach to Evaluate Barcode-Assisted Medication Preparation Alerts at a Large Academic Medical Center

BACKGROUND

The purpose of this study was to develop a data-driven process to analyze barcode-assisted medication preparation alert data with a goal of minimizing inaccurate alerts.

METHODS

Medication preparation data for the prior three-month period was obtained from an electronic health record system. A dashboard was developed to identify recurrent, high-volume alerts and associated medication records. A randomization tool was used to obtain a prespecified proportion of the alerts to review for appropriateness. Alert root causes were identified by chart review. Depending on the alert's cause(s), targeted informatics build changes, workflow and purchasing changes, and/or staff education were implemented. The rate of alerts was measured postintervention for select drugs.

RESULTS

The institution averaged 31,000 medication preparation alerts per month. The "barcode not recognized" alert (13,000) was the highest volume over the study period. Eighty-five medication records were identified as contributing to a high volume of alerts (5,200/31,000), representing 49 unique drugs. Of the 85 medication records triggering alerts, 36 required staff education, 22 required informatics build changes, and 8 required workflow changes. Targeted interventions for 2 medications, resulted in reducing the rate of the "barcode not recognized" alert from 26.6% to 1.3% for polyethylene glycol and from 48.7% to 0% for cyproheptadine.

CONCLUSION

This quality improvement project highlighted opportunities to improve medication purchasing, storage, and preparation through development of a standard process to evaluate barcode-assisted medication preparation alert data. A data-driven approach can help identify and minimize inaccurate alerts ("noise") and promote medication safety.

Beyond the override: Using evidence of previous drug tolerance to suppress drug allergy alerts; a retrospective study of opioid alerts

OBJECTIVE

Despite the extensive literature exploring alert fatigue, most studies have focused on describing the phenomenon, but not on fixing it. The authors aimed to identify data useful to avert clinically irrelevant alerts to inform future research on clinical decision support (CDS) design.

METHODS

We conducted a retrospective observational study of opioid drug allergy alert (DAA) overrides for the calendar year of 2019 at a large academic medical center, to identify data elements useful to find irrelevant alerts to be averted.

RESULTS

Overall, 227,815 DAAs were fired in 2019, with an override rate of 91 % (n = 208196). Opioids represented nearly two-thirds of these overrides (n = 129063; 62 %) and were the drug class with the highest override rate (96 %). On average, 29 opioid DAAs were overridden per patient. While most opioid alerts (97.1 %) are fired for a possible match (the drug class of the allergen matches the drug class of the prescribed drug), they are overridden significantly less frequently for definite match (exact match between allergen and prescribed drug) (88 % vs. 95.9 %, p < 0.001). When comparing the triggering drug with previously administered drugs, override rates were equally high for both definite match (95.9 %), no match (95.5 %), and possible match (95.1 %). Likewise, when comparing to home medications, overrides were excessively high for possible match (96.3 %), no match (96 %), and definite match (94.4 %).

CONCLUSION

We estimate that 74.5% of opioid DAAs (46.4% of all DAAs) at our institution could be relatively safely averted, since they either have a definite match for previous inpatient administrations suggesting drug tolerance or are fired as possible match with low risk of cross-sensitivity. Future research should focus on identifying other relevant data elements ideally with automated methods and use of emerging standards to empower CDS systems to suppress false-positive alerts while avoiding safety hazards.

A systematic review of data elements of computerized physician order entry (CPOE): mapping the data to FHIR

OBJECTIVE

Medication errors are the third leading cause of death. There are several methods to prevent prescription errors, one of which is to use a Computerized Physician Order Entry system (CPOE). In a CPOE system, necessary data needs to be collected so that making decisions about prescribing medications and treatment plans could be made. Although many CPOE systems have been developed worldwide, studies have yet to identify the necessary data and data elements of CPOE systems. This study aims to identify data elements of CPOE and standardize these data with Fast Healthcare Interoperability Resources (FHIR) to facilitate data sharing and integration with the electronic health record (EHR) system and reduce data diversity.

METHODS

PubMed, Web of Science, Embase, and Scopus databases for studies up to October 2019 were searched. Two reviewers independently assessed original articles to determine eligibility for inclusion in this review. All articles describing data elements of a COPE system were included. Data elements were obtained from the included articles' text, tables, and figures.Classification of the extracted data elements and mapping them to FHIR was done to facilitate data sharing and integration with the electronic health record (EHR) system and reduce data diversity. The final data elements of CPOE were categorized into five main categories of FHIR (foundation, base, clinical, financial, and specialized) and 146 resources, where possible. One of the researchers did mapping and checked and verified by the second researcher. If a data element could not be mapped to any FHIR resources, this data element was considered an extension to the most relevant resource.

RESULTS

We retrieved 5162 articles through database searches. After the full-text assessment, 21 articles were included. In total, 270 data elements were identified and mapped to the FHIR standard. These elements have been reported in 26 FHIR resources of 146 ones (18%). In total, 71 data elements were considered an extension.

CONCLUSIONS

The results of this study showed that the same data elements were not used in the CPOE systems, and the degree of homogeneity of these systems is limited. The mapping of extracted data with data elements used in the FHIR standard shows the extent to which these systems comply with existing standards. Considering the standards in these systems' design helps developers design more coherent systems that can share data with other systems.

Design, development, implementation, and evaluation of a severe drug-drug interaction alert system in the ICU: An analysis of acceptance and override rates

BACKGROUND AND OBJECTIVE

The override rate of Drug-Drug Interaction Alerts (DDIA) in Intensive Care Units (ICUs) is very high. Therefore, this study aimed to design, develop, implement, and evaluate a severe Drug-Drug Alert System (DDIAS) in a system of ICUs and measure the override rate of this system.

METHODS

This is a cross-sectional study that details the design, development, implementation, and evaluation of a DDIAS for severe interactions into a Computerized Provider Order Entry (CPOE) system in the ICUs of Nemazee general teaching hospitals in 2021. The patients exposed to the volume of DDIAS, acceptance and overridden of DDIAS, and usability of DDIAS have been collected. The study was approved by the local Institutional Review Board (IRB) and; the ethics committee of Shiraz University of Medical Science on date: 2019-11-23 (Approval ID: IR.SUMS.REC.1398.1046).

RESULTS

The knowledge base of the DDIAS contains 9,809 severe potential drug-drug interactions (pDDIs). A total of 2672 medications were prescribed in the population study. The volume and acceptance rate for the DDIAS were 81 % and 97.5 %, respectively. The override rate was 2.5 %. The mean System Usability Scale (SUS) score of the DDIAS was 75.

CONCLUSION

This study demonstrates that implementing high-risk DDIAS at the point of prescribing in ICUs improves adherence to alerts. In addition, the usability of the DDIAS was reasonable. Further studies are needed to investigate the establishment of severe DDIAS and measure the prescribers' response to DDIAS on a larger scale.

Drug Allergy Alert System in a Spanish University Hospital: Description and Dynamics of Use

INTRODUCTION

The drug allergy alert system reduces the frequency of adverse drug events, although it is subjected to collateral effects, since 80-90% of alerts are not real, and a large percentage of alerts are overridden (46.2-96.2%). We reviewed how the alert system is used at University Hospital Fundación Alcorcon (HUFA).

METHODS

Data were obtained from the drug allergy alert and the alert overriding notification forms (both in the period 2011-20). We also recorded drug allergy diagnoses at HUFA, drug consumption in primary care in 2016. We calculated the incidence of drug allergy alert activation, temporal trends in use, and correlations between the number of drugs in several datasets.

RESULTS

We collected 15,535 alerts. NSAIDs and penicillins were the drugs with the highest number of drug allergy alerts (36.55% and 26.91%, respectively). A correlation was found between the number of drug alerts and the type of drug allergy in HUFA in 2016. Only 6.83% of the alerts were removed, and, of these, 21.77% were reactivated. Approximately 100 overrides were recorded per year from 2016 (6.8% of 8,434 activated alerts during 2014-2020).

CONCLUSIONS

The number of drug allergy alerts recorded via the drug allergy alert system of HUFA correlates with the distribution of drug allergy diagnoses in the hospital, although many of the alerts could be false positives (as per current published evidence). We detected a very low frequency of removed alerts (6.83%), a relevant frequency of reactivations (one quarter), and a very low frequency of overrides (6.8%).

Barriers to Adoption of Tailored Drug-Drug Interaction Clinical Decision Support

OBJECTIVE

Despite the benefits of the tailored drug-drug interaction (DDI) alerts and the broad dissemination strategy, the uptake of our tailored DDI alert algorithms that are enhanced with patient-specific and context-specific factors has been limited. The goal of the study was to examine barriers and health care system dynamics related to implementing tailored DDI alerts and identify the factors that would drive optimization and improvement of DDI alerts.

METHODS

We employed a qualitative research approach, conducting interviews with a participant interview guide framed based on Proctor's taxonomy of implementation outcomes and informed by the Theoretical Domains Framework. Participants included pharmacists with informatics roles within hospitals, chief medical informatics officers, and associate medical informatics directors/officers. Our data analysis was informed by the technique used in grounded theory analysis, and the reporting of open coding results was based on a modified version of the Safety-Related Electronic Health Record Research Reporting Framework.

RESULTS

Our analysis generated 15 barriers, and we mapped the interconnections of these barriers, which clustered around three entities (i.e., users, organizations, and technical stakeholders). Our findings revealed that misaligned interests regarding DDI alert performance and misaligned expectations regarding DDI alert optimizations among these entities within health care organizations could result in system inertia in implementing tailored DDI alerts.

CONCLUSION

Health care organizations primarily determine the implementation and optimization of DDI alerts, and it is essential to identify and demonstrate value metrics that health care organizations prioritize to enable tailored DDI alert implementation. This could be achieved via a multifaceted approach, such as partnering with health care organizations that have the capacity to adopt tailored DDI alerts and identifying specialists who know users' needs, liaise with organizations and vendors, and facilitate technical stakeholders' work. In the future, researchers can adopt the systematic approach to study tailored DDI implementation problems from other system perspectives (e.g., the vendors' system).

An Observational Study to Identify Drug-related Problems (DRP) in Routine Care and An Expert Panel Assessment to Rate Clinical Risk and Preventability by Unit-dose Dispensing Systems (UDDS) with Computerized Physician Order Entry (CPOE) and Clinical Decision-Support Systems (CDSS)

Background and aim: Drug-related problems (DRP) jeopardize patient safety. Unit-dose dispensing systems (UDDS) with computerized-physician-order-entry (CPOE) and clinical-decision-support-systems (CDSS) were reported as a promising concept for preventing DRP. We aimed at identifying and categorizing DRP in peroral drug administration considering their clinical risk and preventability by UDSS/CPOE/CDSS. Investigations: In surgical and internal-medicine departments, we observed routine procedures in peroral drug administration for DRP. An expert panel including pharmaceutical and nursing expertise categorized the identified 18 DRP categories into three levels: DRP that have not yet resulted in medication errors (ME) (Level-I), DRP where ME have occurred but have not yet reached the patient (Level-II), and DRP where ME have occurred and have reached the patient (Level-III). Additionally, the panel categorized DRP according to their clinical risk and whether the implementation of UDSS/CPOE/CDSS can prevent them. Results: In 77 surgical patients, 1,849 peroral drug administration procedures, and in 149 internal-medicine patients, 1,405 procedures were observed. The 18 DRP categories were identified with a frequency of 0.6%-26.7% (Level-I), 0.1%-21.5% (Level-II), and 0.0%-1.0% (Level-III). Of those, four categories were considered of high clinical risk: "Name of the medication is not readable", "Prescribed medication is not prepared for administration", "An incorrect or non-prescribed medication is prepared", and "A medication is prepared for the wrong patient (mix-up)". Twelve DRP categories were categorized as highly preventable by UDSS/CPOE/CDSS. Conclusions:Under routine conditions, we identified a substantial number of DRPs. An expert panel categorized many of those DRPs as clinically highly relevant and highly preventable by UDSS/CPOE/CDSS.

Tackling alert fatigue with a semi-automated clinical decision support system: quantitative evaluation and end-user survey

STUDY AIMS

Clinical decision support systems (CDSS) embedded in hospital electronic health records efficiently reduce medication errors, but there is a risk of low physician adherence due to alert fatigue. At the Cantonal Hospital Aarau, a CDSS is being developed that allows the highly accurate detection and correction of medication errors. The semi-automated CDSS sends its alerts either directly to the physician or to a clinical pharmacist for review first. Our aim was to evaluate the performance of the recently implemented CDSS in terms of acceptance rate and alert burden, as well as physicians' satisfaction with the CDSS.

METHODS

All alerts generated by the clinical decision support systems between January and December 2021 were included in a retrospective quantitative evaluation. A team of clinical pharmacists performed a follow-up to determine whether the recommendation made by the CDSS was implemented by the physician. The acceptance rate was calculated including all alerts for which it was possible to determine an outcome. A web-based survey was conducted amongst physicians to assess their attitude towards the CDSS. The survey questions included overall satisfaction, helpfulness of individual algorithms, and perceived alert burden.

RESULTS

In 2021, a total of 10,556 alerts were generated, of which 619 triggered a direct notification to the physician and 2,231 notifications were send to the physician after evaluation by a clinical pharmacist. The acceptance rates were 89.8% and 68.4%, respectively, which translates as an overall acceptance rate of 72.4%. On average, clinical pharmacists received 17.2 alerts per day, while all of the hospital physicians together received 7.8 notifications per day. In the survey, 94.5% of physicians reported being satisfied or very satisfied with the CDSS. Algorithms addressing potential medication errors concerning anticoagulants received the highest usefulness ratings.

CONCLUSION

The development of this semi-automated clinical decision support system with context-based algorithms resulted in alerts with a high acceptance rate. Involving clinical pharmacists proved a promising approach to limit the alert burden of physicians and thus tackle alert fatigue. The CDSS is well accepted by our physicians.

Impact of automated dispensing cabinets on dispensing errors, interruptions and pillbox preparation time

AIM

This work aimed to evaluate the impact of automated dispensing cabinets on the dispensing error rate, the number of interruptions, and pillbox preparation times.

METHODS

A prospective observational study was conducted across 16 wards in two departments (internal medicine and surgery) of a large teaching hospital. The study compared eight wards using automated dispensing cabinets (ADCs) and eight using a traditional ward stock (TWS) method. A disguised observation technique was used to compare occurrences of dispensing errors and interruptions and pillbox preparation times. The proportion of errors was calculated by dividing the number of doses with one or more errors by the total number of opportunities for error. Wards participating in the 'More time for patients' project-a Lean Management approach-were compared with those not participating. The potential severity of intercepted errors was assessed.

RESULTS

Our observations recorded 2924 opportunities for error in the preparation of 570 pillboxes by 132 nurses. We measured a significantly lower overall error rate (1.0% vs 5.0%, p=0.0001), significantly fewer interruptions per hour (3.2 vs 5.7, p=0.008), and a significantly faster mean preparation time per drug (32 s vs 40 s, p=0.0017) among ADC wards than among TWS wards, respectively. We observed a significantly lower overall error rate (1.4% vs 4.4%, p=0.0268) and a non-significantly lower number of interruptions per hour (3.8 vs 5.1, p=0.0802) among wards participating in the 'More time for patients' project.

CONCLUSIONS

A high dispensing-error rate was observed among wards using TWS methods. Wards using ADCs connected to computerised physician order entry and installed in a dedicated room had fewer dispensing errors and interruptions and their nurses prepared pillboxes faster. Wards participating in a Lean Management project had lower error rates than wards not using this approach.

Reducing Therapeutic Duplication in Inpatient Medication Orders

BACKGROUND

Therapeutic duplication, the presence of multiple agents prescribed for the same indication without clarification for when each should be used, can contribute to serious medical errors. Joint Commission standards require that orders contain clarifying information about when each order should be given. In our system, as needed (PRN) acetaminophen and ibuprofen orders are major contributors to therapeutic duplication.

OBJECTIVE

The objective of this study is to design and evaluate effectiveness of clinical decision support (CDS) to reduce therapeutic duplication with acetaminophen and ibuprofen orders.

METHODS

This study was done in a pediatric health system with three freestanding hospitals. We iteratively designed and implemented two CDS strategies aimed at reducing the therapeutic duplication with these agents: (1) interruptive alert prompting clinicians for clarifying PRN comments at order entry and (2) addition of discrete "first-line" and "second-line" PRN reasons to orders. Therapeutic duplications were measured by manual review of orders for 30-day periods before and after each intervention and 6 months later.

RESULTS

Therapeutic duplications decreased from 1,485 in the 30 days prior to the first alert implementation to 818 in the 30 days after but rose back to 1,208 in the 30 days prior to the second intervention. After discrete reasons were added to the order, therapeutic duplication decreased to 336 in the immediate 30 days and 6 months later remained at 277. Alerts firing rates decreased from 76.0 per 1,000 PRN acetaminophen or ibuprofen orders to 42.9 after the second intervention.

CONCLUSION

Interruptive alerts may reduce therapeutic duplication but are associated with high rates of user frustration and alert fatigue. Leveraging discrete PRN reasons for "first line" and "second line" produced a greater reduction in therapeutic duplication as well as fewer interruptive alerts and less manual entry for providers.

Evaluation of the effectiveness of electronic prescription in reducing medical and medical errors (systematic review study)

INTRODUCTION

The use of electronic systems in prescription is considered as the final solution to overcome the many problems of the paper transcription process, especially with the outbreak of Coronavirus needs more attention than before. But despite the many advantages, its implementation faces many challenges and obstacles. Therefore, the present study was conducted to review the effectiveness of computerized physician order entry systems (CPOE) on relative risk reduction on medication error and adverse drug events (ADE).

METHOD

This study is one of the systematic review studies that was conducted in 2021. In this study, searching for keywords such as E-Electronic Prescription, Patient safety, Medication Errors prescription, Drug Interactions, orginal articles from 2000 to October-2020 in the valid databases such as ISI web of Science PubMed Embase, Scopus and search engines like google was done. The included studies were based on the main objectives of the study and based on the inclusion criteria after several stages of review and quality evaluation. In fact, the main criteria for selecting articles were studies that compared the rate of medication errors with or without assessing the associated harms (real or potential) before and after the implementation of EMS.

RESULTS

Out of 110 selected studies after initial screening, only 16 articles were selected due to their relevance. Among the final studies, there was a significant heterogeneity. Only 6 studies were of good quality. Of the 10 studies prescribing error rates, 9 reported reductions, but variable denominators prevented meta-analysis. Twelve studies provided specific examples of systemic drug errors. 5 cases reported their occurrence slightly. Out of 9 cases that analyzed the effects on drug error rate, 7 cases showed a significant relative reduction between 13 and 99%. Four of the six studies that analyzed the effects on potential ADEs showed a significant relative reduction of between 35 and 98%. Two of the four studies that analyzed the effect of ADEs showed a relative reduction of between 30 and 84%.

CONCLUSION

Finally, e-prescribing seems to reduce the risk of medication errors and ADE. However, the studies differed significantly in terms of setting, design, quality and results. More randomized controlled trials (RCTs) are needed to further improve the evidence of health informatics information.

Examining medication ordering errors using AHRQ network of patient safety databases

BACKGROUND

Studies examining the effects of computerized order entry (CPOE) on medication ordering errors demonstrate that CPOE does not consistently prevent these errors as intended. We used the Agency for Healthcare Research and Quality (AHRQ) Network of Patient Safety Databases (NPSD) to investigate the frequency and degree of harm of reported events that occurred at the ordering stage, characterized by error type.

MATERIALS AND METHODS

This was a retrospective observational study of safety events reported by healthcare systems in participating patient safety organizations from 6/2010 through 12/2020. All medication and other substance ordering errors reported to NPSD via common format v1.2 between 6/2010 through 12/2020 were analyzed. We aggregated and categorized the frequency of reported medication ordering errors by error type, degree of harm, and demographic characteristics.

RESULTS

A total of 12 830 errors were reported during the study period. Incorrect dose accounted for 3812 errors (29.7%), followed by incorrect medication 2086 (16.3%), and incorrect duration 765 (6.0%). Of 5282 events that reached the patient and had a known level of severity, 12 resulted in death, 4 resulted in severe harm, 45 resulted in moderate harm, 341 resulted in mild harm, and 4880 resulted in no harm.

CONCLUSION

Incorrect dose and incorrect drug orders were the most commonly reported and harmful types of medication ordering errors. Future studies should aim to develop and test interventions focused on CPOE to prevent medication ordering errors, prioritizing wrong-dose and wrong-drug errors.

Reducing prescribing errors: making electronic prescribing work for cystic fibrosis inpatients

Children admitted to our hospital with cystic fibrosis had frequent medication errors due to polypharmacy and addition of specialist and high-risk medications despite an electronic prescribing and medicines administration system in place. We describe a multidisciplinary quality improvement project that combined a computerised order entry system (CPOE) with human factor process changes. Over 12 months, our run chart showed a 43% reduction in prescription errors. For medications prescribable via the CPOE, errors reaching the patient reduced from 50% to 29%. Electronic prescribing can be seen by clinicians as a fixed unalterable system contributing to rather than ameliorating errors. Improving safety requires whole team engagement and working closely with programmers to adapt function and influence human factors.

Reducing Medication Errors in Children's Hospitals

OBJECTIVES

Knowledge of the prevalence and characteristics of medication errors in pediatric and neonatal patients is limited. This study aimed to evaluate the incidence and medication error characteristics in a pediatric hospital over 5 years and to determine whether serial error prevention programs to optimize a computerized physician order entry (CPOE) system reduce error incidence.

METHODS

We retrospectively reviewed medication errors documented between January 2015 and December 2019.

RESULTS

A total of 2,591,596 prescriptions were checked, and 255 errors were identified. Wrong dose prescriptions constituted the most common errors (56.9%). Medications with the highest rate of errors were antibiotics/antiviral drugs (36.9%). Oral route medications comprised the highest portion (60.8%), followed by intravenous ones (28.6%). The most common stage for medication errors was physician ordering (93.3%). Junior residents were responsible for most errors (45.9%). Most errors occurred in the pediatric ward (53.7%). In total, 221 (86.7%) errors were near misses. Only 4 errors (1.6%) were considered significant and required active monitoring or intervention. Type of error, stage of error, staff composition, and severity level of errors were significantly related to the number of errors in different years. There was a statistically significant decrease in errors per 100,000 prescriptions across different years after optimizing the CPOE system.

CONCLUSIONS

The incidence of medication errors decreased with extensive use of the CPOE system. Continuous application of the CPOE optimization program can effectively reduce medication errors. Further incorporation of pediatric-specific decision-making and support tools and error prevention measures into CPOE systems is needed.

Implementation of a myasthenia gravis drug-disease interaction clinical decision support tool reduces prescribing of high-risk medications

INTRODUCTION/AIMS

High-risk medication exposure is a modifiable risk factor for myasthenic exacerbation and crisis. We evaluated whether real-time electronic clinical decision support (CDS) was effective in reducing the rate of prescribing potentially high-risk medications to avoid or use with caution in patients with myasthenia gravis.

METHODS

An expert panel reviewed the available drug-disease pairings and associated severity levels to activate the alerts for CDS. All unique alerts activated in both inpatient and outpatient contexts were analyzed over a two-year period. Clinical context, alert severity, medication class, and alert action were collected. The primary outcome was alert override rate. Secondary outcomes included the percentage of unique medication exposures avoided and predictors of alert override.

RESULTS

During the analysis period, 2817 unique alerts fired, representing 830 distinct patient-medication exposures for 577 unique patients. The overall alert override rate was 85% (80.3% for inpatient alerts and 95.8% for outpatient alerts). Of unique medication-patient exposures, 19% were avoided because of the alert. Assigned alert severity of "contraindicated" were less likely to be overridden (odds ratio [OR] 0.42, 95% confidence interval [CI] 0.32-0.56), as well as alerts activated during evening staffing (OR 0.69, 95% CI 0.55-0.87).

DISCUSSION

Implementation of a myasthenia gravis drug-disease interaction alert reduced overall patient exposure to potentially harmful medications by approximately 19%. Future optimization includes enhanced provider and pharmacist education. Further refinement of alert logic criteria to optimize medication risk reduction and reduce alert fatigue is warranted to support clinicians in prescribing and reduce electronic health record time burden.

Perspective: leveraging the electronic health record to improve sleep in the hospital

Inpatient sleep loss can worsen health outcomes, including delirium and falls. Sleep disruptions in the hospital often originate from provider-patient interactions ordered electronically through computerized provider order entry. These orders contain clinical decision support systems with default schedules. These defaults are often around-the-clock, may not align with patients' needs, and cause iatrogenic sleep loss. Optimizing clinical decision support in the electronic health record can decrease unnecessary sleep disruptions and influence sleep-friendly decision-making.

CITATION

Kadura S, Siala T, Arora VM. Perspective: Leveraging the electronic health record to improve sleep in the hospital. J Clin Sleep Med. 2023;19(2):421-423.

Distinct components of alert fatigue in physicians' responses to a noninterruptive clinical decision support alert

OBJECTIVE

Clinical decision support (CDS) alerts may improve health care quality but "alert fatigue" can reduce provider responsiveness. We analyzed how the introduction of competing alerts affected provider adherence to a single depression screening alert.

MATERIALS AND METHODS

We analyzed the audit data from all occurrences of a CDS alert at a large academic health system. For patients who screen positive for depression during ambulatory visits, a noninterruptive alert was presented, offering a number of relevant documentation actions. Alert adherence was defined as the selection of any option offered within the alert. We assessed the effect of competing clinical guidance alerts presented during the same encounter and the total of all CDS alerts that the same provider had seen in the prior 90 days, on the probability of depression screen alert adherence, adjusting for physician and patient characteristics.

RESULTS

The depression alert fired during 55 649 office visits involving 418 physicians and 40 474 patients over 41 months. After adjustment, physicians who had seen the most alerts in the prior 90 days were much less likely to respond (adjusted OR highest-lowest quartile, 0.38; 95% CI 0.35-0.42; P < .001). Competing alerts in the same visit further reduced the likelihood of adherence only among physicians in the middle two quartiles of alert exposure in the prior 90 days.

CONCLUSIONS

Adherence to a noninterruptive depression alert was strongly associated with the provider's cumulative alert exposure over the past quarter. Health systems should monitor providers' recent alert exposure as a measure of alert fatigue.

Interruptive Electronic Alerts for Choosing Wisely Recommendations: A Cluster Randomized Controlled Trial

OBJECTIVE

To assess the efficacy of interruptive electronic alerts in improving adherence to the American Board of Internal Medicine's Choosing Wisely recommendations to reduce unnecessary laboratory testing.

MATERIALS AND METHODS

We administered 5 cluster randomized controlled trials simultaneously, using electronic medical record alerts regarding prostate-specific antigen (PSA) testing, acute sinusitis treatment, vitamin D testing, carotid artery ultrasound screening, and human papillomavirus testing. For each alert, we assigned 5 outpatient clinics to an interruptive alert and 5 were observed as a control. Primary and secondary outcomes were the number of postalert orders per 100 patients at each clinic and number of triggered alerts divided by orders, respectively. Post hoc analysis evaluated whether physicians experiencing interruptive alerts reduced their alert-triggering behaviors.

RESULTS

Median postalert orders per 100 patients did not differ significantly between treatment and control groups; absolute median differences ranging from 0.04 to 0.40 for PSA testing. Median alerts per 100 orders did not differ significantly between treatment and control groups; absolute median differences ranged from 0.004 to 0.03. In post hoc analysis, providers receiving alerts regarding PSA testing in men were significantly less likely to trigger additional PSA alerts than those in the control sites (Incidence Rate Ratio 0.12, 95% CI [0.03-0.52]).

DISCUSSION

Interruptive point-of-care alerts did not yield detectable changes in the overall rate of undesired orders or the order-to-alert ratio between active and silent sites. Complementary behavioral or educational interventions are likely needed to improve efforts to curb medical overuse.

CONCLUSION

Implementation of interruptive alerts at the time of ordering was not associated with improved adherence to 5 Choosing Wisely guidelines.

TRIAL REGISTRATION

NCT02709772.

Allergic Adverse Drug Events After Alert Overrides in Hospitalized Patients

OBJECTIVES

This study aimed to assess how often overridden drug allergy alerts (ODAAs) lead to allergic adverse drug events (All-ADEs) and to evaluate the frequency with which drug allergy alerts (DAAs) were overridden and the reasons, as well as appropriateness of these overrides.

METHODS

A retrospective observational study of DAA generated between 2014 and 2016 was conducted. The corresponding DAA records were reviewed to determine the frequency of alert overrides. A chart review was performed on a subset of 194 ODAA (the first of every 3 chronologically ordered ODAA) to identify All-ADEs and to evaluate the override reasons and the appropriateness of these overrides.

RESULTS

A total of 2044 DAAs were overridden (override rate of 44.8%). Most were triggered by a nonexact match (93.81%), when ordering nervous system (21.1%) and cardiovascular system (19.6%) drugs and were generated by physicians (72.7%). The main override reason was that the patient was already taking the drug or had previously tolerated the drug. Only 9.28% of ODAAs were inappropriately overridden. Six All-ADEs (3.09%) were identified and were due to anti-infective (1), antineoplastic (1), and iodinated-contrast (4) drug administration. Most All-ADEs were cutaneous and were mild. None was life-threatening or fatal. The All-ADEs rate was higher among inappropriately ODAA (15.79%, P = 0.013).

CONCLUSIONS

Alert overrides are not exempt from clinical consequences, although few are associated with All-ADEs. It is necessary to identify the drugs involved in those reactions and to update allergy lists to generate only specific and important DAA and to avoid the negative consequences of overrides.

Free-Text Computerized Provider Order Entry Orders Used as Workaround for Communicating Medication Information

OBJECTIVES

Medication information is frequently communicated via free-text computerized provider order entry (CPOE) orders in electronic health records. When such information is transmitted separately from a structured CPOE medication order, there is a significant risk of medication error. Although prior studies have described the frequency of using free-text CPOE orders for communicating medication information, there is a gap in understanding the nature of the medication information contained in the free-text CPOE orders. The aims of this study are to (1) identify the most common medication names communicated in free-text CPOE orders and their risk levels and (2) identify what actions physicians expect that nurses will complete when they place free-text CPOE orders, and (3) describe differences in these patterns across hospitals.

METHODS

This study was a retrospective analysis of a sample of 26,524 free-text CPOE orders from 6 hospitals in the mid-Atlantic U.S. region.

RESULTS

Free-text CPOE orders contained in the sample mentioned 193 medication names. Free-text CPOE orders were used frequently to communicate information about naloxone, heparin, flumazenil, and dextrose. Twenty-two percent of the free-text CPOE orders related to discontinuing medication(s), whereas 7% of the free-text CPOE orders relate to giving medication(s). There was high variation across hospitals both in the percentage of free-text CPOE orders mentioning medication information and in the proportion of those that referred to high-risk medications.

CONCLUSIONS

The prevalence of medication information in free-text CPOE orders may suggest specific communication challenges in respect to urgency, uncertainty, planning, and other aspects of communication and clinical needs. Understanding and addressing communication challenges around commonly mentioned medication names and actions, especially those that are high risk, can help reduce the risk of medication errors.

Overriding Drug-Drug Interaction Alerts in Clinical Decision Support Systems: A Scoping Review

Ineffective computerized alerts for potential Drug-Drug Interactions (DDI) is a longstanding informatics issue. Prescribing clinicians often ignore or override such alerts due to lack of context and clinical relevance, among various other reasons. In this study, we reveiwed published data on the rate of DDI alert overrides and medications involved in the overrides. We identified 34 eligible studies from sites across Asia, Europe, the United States, and the United Kingdom. The override rate of DDI alerts ranged from 55% to 98%, with more than half of the studies reporting the most common drug pairs or medications involved in acceptance or overriding of alerts. The high prevalance of alert overrides highlights the need for decision support systems that take user, drug, and institutional factors into consideration, as well as actionable metrics to better characterize harm associated with overrides.

Can CPOE Based on Electronic Order Sets Cause Unintended Consequences (Expensive and Unnecessary Tests) at the Emergency Department?

Computerized Provider Order Entry (CPOE) systems may cause unintended consequences. This study aimed to describe the on-going system for CPOE order sets, and to explore an economic evaluation at the Emergency Department. First, we developed a costs dashboard which showed us the significant and excessive use of medical tests per consultation. We identified the top 10 most widely used and most expensive tests. Additionally we noticed that the labs seemed to continually increase. Then, we found that 27% of the consultations have at least one item of laboratory practice between January and February 2020, and this represents more than 80% of the consultation costs. Health care spending has reached epic proportions globally. We think that it is time to rethink effective strategies. Maybe it is time to deactivate/remove electronic order sets (EOSs) and the functionality to develop and create their own "private" order sets, in order to eliminate waste and inefficiencies.

Reducing Pediatric Emergency Department Prescription Errors

BACKGROUND

Prescription errors are a significant cause of iatrogenic harm in the health care system. Pediatric emergency department (ED) patients are particularly vulnerable to error. We sought to decrease prescription errors in an academic pediatric ED by 20% over a 24-month period by implementing identified national best practice guidelines.

METHODS

From 2017 to 2019, a multidisciplinary, fellow-driven quality improvement (QI) project was conducted using the Model for Improvement. Four key drivers were identified including simplifying the electronic order entry into prescription folders, improving knowledge of dosing by indication, increasing error feedback to prescribers, and creating awareness of common prescription pitfalls. Four interventions were subsequently implemented. Outcome measures included prescription errors per 1000 prescriptions written for all medications and top 10 error-prone antibiotics. Process measures included provider awareness and use of prescription folders; the balancing measure was provider satisfaction. Differences in outcome measures were assessed by statistical process control methodology. Process and balancing measures were analyzed using 1-way analysis of variance and χ2 testing.

RESULTS

Before our interventions, 8.6 errors per 1000 prescriptions written were identified, with 62% of errors from the top 10 most error-prone antibiotics. After interventions, error rate per 1000 prescriptions decreased from 8.6 to 4.5 overall and from 20.1 to 8.8 for top 10 error-prone antibiotics. Provider awareness of prescription folders was significantly increased.

CONCLUSION

QI efforts to implement previously defined best practices, including simplifying and standardizing computerized provider order entry (CPOE), significantly reduced prescription errors. Synergistic effect of educational and technological efforts likely contributed to the measured improvement.

Clinician collaboration to improve clinical decision support: the Clickbusters initiative

OBJECTIVE

We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program.

RESULTS

Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS.

CONCLUSIONS

At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

Impact of computerized physician order entry system on parenteral nutrition medication errors in a teaching neonatal intensive care unit

OBJECTIVES

Parenteral nutrition (PN) offers a quality therapeutic option when enteral feeding is non-tolerated or impossible. However, it can be associated with an increased risk of medical errors, especially in sensitive populations such as newborns. This study aimed at determining the impact of the implementation of a computerized physician order entry (CPOE) system on PN medication errors in the neonatology department in the largest teaching hospital in Tunisia. Materiel & Methods: The frequency of medication errors in PN, in a teaching neonatal intensive care unit, was prospectively reviewed by a pharmacist between December 2018 and March 2019 in a two-phase interventional study (pre and post locally developed CPOE System implementation).

RESULTS

Implementation of CPOE system decreased PN order errors from 379 to 147 representing a 61.1% reduction. The decreases on PN order errors per stage, i.e. prescribing and preparation, were form 207 to 22 (89.4%), and from 117 to 66 (43.6%) respectively. Mean nutrients intakes were in conformity to the recommended daily intakes during the CPOE phase of the study. CPOE is a protective tool against prescription and preparation errors. It significantly impacted all items of the ordering process.

CONCLUSIONS

In addition to the rigorous application of the recommendations, the CPOE system allows to reduce the risk of PN medication errors. This improves the safety and quality of medicines in newborns.

A systematic review of drug allergy alert systems

BACKGROUND AND OBJECTIVE

Drug allergy alert systems (DAAS), have been considered an effective strategy to reduce preventable adverse drug events (ADEs), improving patient's safety. To date, no review has been conducted analyzing characteristics of DAAS in the hospital setting. Therefore, the aim of this study is to identify, describe and summarize the DAAS used in hospitals. The secondary objectives are to analyse drug allergy alerts (DAA) characteristics, the override rate (OvR) and the clinical consequences of alert overrides.

METHODS

Searches were conducted in Medline and Cochrane Library to identify studies describing DAAS. Systems characteristics, generated alerts, DAA, OvR, and its clinical consequences were extracted and analyzed.

RESULTS

Twenty-eight articles were included in the review. Seventeen different electronic DAAS were identified, of which 53% were commercially available. Systems differed in drug allergy information and rules for generating alerts. DAA were generally interruptive, triggered by non-exact match at drug prescribing and when ignored, an override reason was mandatory. The OvR ranged from 43.7% to 97%. The main override reason given by providers was that 'patient had previously tolerated or had taken the drug without allergic reaction'. Clinical consequences of overriding DAA were only analyzed in four studies, with an ADE incidence between 0% and 6%.

CONCLUSIONS

Different DAAS are used in hospitals with some degree of heterogeneity. Accurate and updated drug allergy information is important to generate only high value alerts. A regular review of DAAS and a standardization of alert rules, alert information and override reasons are necessary to optimize systems. Future studies should evaluate the impact of the DAAS aspects on preventing ADEs.

Advanced Medication Alert System Decreased Hospital-Based Outpatient Duplicated Medications: A Longitudinal Hospital Cohort Study

OBJECTIVES

This study aimed to examine the associations between adoption of an advanced medication alert system and decreases in hospital-based outpatient duplicated medication rates in Taiwan.

METHODS

The unit of analysis was the hospital. We merged the hospital medication alert system adoption survey data and Taiwan National Health Insurance outpatient claims data. The observation time was 1998 to 2011, divided into 5 periods (T1-T5). The analysis included 216 hospitals, and outcome variable was hospital-based outpatient duplicated medication rates. The system adoption time frame, hospital accreditation level, and number of drugs per prescription were defined as predicted variables. A generalized estimating equation regression model was used.

RESULTS

Adoption of the advanced medication alert system gradually increased, such that 100% of medical centers and 84% of regional hospitals, but less than 50% of district hospitals, had systems by T5. The hospital-based outpatient duplicated medication rate continually decreased, from 29.8% to 11.2%. The generalized estimating equation model showed rates of duplicated medications of b = -8.44 at T2 and b = -17.88 at T5 (P < 0.001) compared with T1. Medical centers and regional hospitals demonstrated much lower duplication rates (b = -13.71, b = -6.82; P < 0.001) compared with district hospitals. Hospitals with more medications per prescription had higher duplication rates than did hospitals with fewer items.

CONCLUSIONS

Hospitals accredited at higher levels tended to have advanced medication alert systems. Hospitals that implemented advanced systems decreased hospital-based outpatient duplicated medications, avoiding a potential risk due to inappropriate medication use.