Pilot evaluation of an optimized context-specific drug–drug interaction alerting system: A controlled pre-post study

Post-implementation optimization of medication alerts in hospital computerized provider order entry systems: a scoping review

OBJECTIVES

A scoping review identified interventions for optimizing hospital medication alerts post-implementation, and characterized the methods used, the populations studied, and any effects of optimization.

MATERIALS AND METHODS

A structured search was undertaken in the MEDLINE and Embase databases, from inception to August 2023. Articles providing sufficient information to determine whether an intervention was conducted to optimize alerts were included in the analysis. Snowball analysis was conducted to identify additional studies.

RESULTS

Sixteen studies were identified. Most were based in the United States and used a wide range of clinical software. Many studies used inpatient cohorts and conducted more than one intervention during the trial period. Alert types studied included drug-drug interactions, drug dosage alerts, and drug allergy alerts. Six types of interventions were identified: alert inactivation, alert severity reclassification, information provision, use of contextual information, threshold adjustment, and encounter suppression. The majority of interventions decreased alert quantity and enhanced alert acceptance. Alert quantity decreased with alert inactivation by 1%-25.3%, and with alert severity reclassification by 1%-16.5% in 6 of 7 studies. Alert severity reclassification increased alert acceptance by 4.2%-50.2% and was associated with a 100% acceptance rate for high-severity alerts when implemented. Clinical errors reported in 4 studies were seen to remain stable or decrease.

DISCUSSION

Post-implementation medication optimization interventions have positive effects for clinicians when applied in a variety of settings. Less well reported are the impacts of these interventions on the clinical care of patients, and how endpoints such as alert quantity contribute to changes in clinician and pharmacist perceptions of alert fatigue.

CONCLUSION

Well conducted alert optimization can reduce alert fatigue by reducing overall alert quantity, improving clinical acceptance, and enhancing clinical utility.

Context-Aware Medical Systems within Healthcare Environments: A Systematic Scoping Review to Identify Subdomains and Significant Medical Contexts

Context awareness is a field in pervasive computing, which has begun to impact medical systems via an increasing number of healthcare applications that are starting to use context awareness. The present work seeks to determine which contexts are important for medical applications and which domains of context-aware applications exist in healthcare. A systematic scoping review of context-aware medical systems currently used by patients or healthcare providers (inclusion criteria) was conducted between April 2021 and June 2023. A search strategy was designed and applied to Pub Med, EBSCO, IEEE Explore, Wiley, Science Direct, Springer Link, and ACM, articles from the databases were then filtered based on their abstract, and relevant articles were screened using a questionnaire applied to their full texts prior to data extraction. Applications were grouped into context-aware healthcare application domains based on past reviews and screening results. A total of 25 articles passed all screening levels and underwent data extraction. The most common contexts used were user location (8 out of 25 studies), demographic information (6 out of 25 studies), movement status/activity level (7 out of 25 studies), time of day (5 out of 25 studies), phone usage patterns (5 out of 25 studies), lab/vitals (7 out of 25 studies), and patient history data (8 out of 23 studies). Through a systematic review process, the current study determined the key contexts within context-aware healthcare applications that have reached healthcare providers and patients. The present work has illuminated many of the early successful context-aware healthcare applications. Additionally, the primary contexts leveraged by these systems have been identified, allowing future systems to focus on prioritizing the integration of these key contexts.

Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework

Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.

Prescribers’ reported acceptance and use of drug-drug interaction alerts: An Australian survey

Drug-drug interaction (DDI) alerts are frequently included in electronic medical record (eMR) systems to provide users with relevant information and guidance at the point of care. In this study, we aimed to examine views of DDI alerts among prescribers, including junior doctors, registrars and senior doctors, across Australia. A validated survey for assessing prescribers’ reported acceptance and use of DDI alerts was distributed among researcher networks and in newsletters. Fifty useable responses were received, more than half ( n = 28) from senior doctors. Prescribers at all levels expected DDI alerts to improve performance but junior doctors reported that this was at a high cost, with respect to time and effort. Senior doctors and registrars reported rarely reading alerts and rarely changing prescribing decisions based on alerts. Respondents identified a number of problems with current alerts including limited relevance, repetition, and poor design, highlighting some clear areas for alert improvement.

Modulators Influencing Medication Alert Acceptance: An Explorative Review

OBJECTIVES

Clinical decision support systems (CDSSs) use alerts to enhance medication safety and reduce medication error rates. A major challenge of medication alerts is their low acceptance rate, limiting their potential benefit. A structured overview about modulators influencing alert acceptance is lacking. Therefore, we aimed to review and compile qualitative and quantitative modulators of alert acceptance and organize them in a comprehensive model.

METHODS

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, a literature search in PubMed was started in February 2018 and continued until October 2021. From all included articles, qualitative and quantitative parameters and their impact on alert acceptance were extracted. Related parameters were then grouped into factors, allocated to superordinate determinants, and subsequently further allocated into five categories that were already known to influence alert acceptance.

RESULTS

Out of 539 articles, 60 were included. A total of 391 single parameters were extracted (e.g., patients' comorbidity) and grouped into 75 factors (e.g., comorbidity), and 25 determinants (e.g., complexity) were consequently assigned to the predefined five categories, i.e., CDSS, care provider, patient, setting, and involved drug. More than half of all factors were qualitatively assessed (n = 21) or quantitatively inconclusive (n = 19). Furthermore, 33 quantitative factors clearly influenced alert acceptance (positive correlation: e.g., alert type, patients' comorbidity; negative correlation: e.g., number of alerts per care provider, moment of alert display in the workflow). Two factors (alert frequency, laboratory value) showed contradictory effects, meaning that acceptance was significantly influenced both positively and negatively by these factors, depending on the study. Interventional studies have been performed for only 12 factors while all other factors were evaluated descriptively.

CONCLUSION

This review compiles modulators of alert acceptance distinguished by being studied quantitatively or qualitatively and indicates their effect magnitude whenever possible. Additionally, it describes how further research should be designed to comprehensively quantify the effect of alert modulators.

dfgcompare: a library to support process variant analysis through Markov models

BACKGROUND

Data-driven process analysis is an important area that relies on software support. Process variant analysis is a sort of analysis technique in which analysts compare executed process variants, a.k.a. process cohorts. This comparison can help to identify insights for improving processes. There are a few software supports to enable process cohort comparison based on the frequencies of process activities and performance metrics. These metrics are effective in cohort analysis, but they cannot support cohort comparison based on the probability of transitions among states, which is an important enabler for cohort analysis in healthcare.

RESULTS

This paper defines an approach to compare process cohorts using Markov models. The approach is formalized, and it is implemented as an open-source python library, named dfgcompare. This library can be used by other researchers to compare process cohorts. The implementation is also used to compare caregivers' behavior when prescribing drugs in the Stockholm Region. The result shows that the approach enables the comparison of process cohorts in practice.

CONCLUSIONS

We conclude that dfgcompare supports identifying differences among process cohorts.

Effect of Computerized Physician Order Entry and Clinical Decision Support System on Adverse Drug Events Prevention in the Emergency Department: A Systematic Review

Objective: An adverse drug event (ADE) is an injury resulting from a medical intervention related to a drug. The emergency department (ED) is a ward vulnerable to more ADEs because of overcrowding. Information technologies such as computerized physician order entry (CPOE) and clinical decision support system (CDSS) may decrease the occurrence of ADEs. This study aims to review research that reported the evaluation of the effectiveness of CPOE and CDSS on lowering the occurrence of ADEs in the ED. Data Sources: PubMed, EMBASE, and Web of Science databases were used to find studies published from 2003 to 2018. The search was conducted in November 2018. Study Selection and Data Extraction: The search resulted in 1700 retrieved articles. After applying inclusion and exclusion criteria, 11 articles were included. Data on the date, country, type of system, medication process stages, study design, participants, sample size, and outcomes were extracted. Data Synthesis: Results showed that CPOE and CDSS may prevent ADEs in the ED through significantly decreasing the rate of errors, ADEs, excessive dose, and inappropriate prescribing (in 54.5% of articles); furthermore, CPOE and CDSS may significantly increase the rate of appropriate prescribing and dosing in compliance with established guidelines (45.5% of articles). Conclusion: This study revealed that the use of CPOE and CDSS can lower the occurrence of ADEs in the ED; however, further randomized controlled trials are needed to address the effect of a CDSS, with basic or advanced features, on the occurrence of ADEs in the ED.

The Acceptance of Interruptive Medication Alerts in an Electronic Decision Support System Differs between Different Alert Types

BACKGROUND

 Through targeted medication alerts, clinical decision support systems (CDSS) help users to identify medication errors such as disregarded drug-drug interactions (DDIs). Override rates of such alerts are high; however, they can be mitigated by alert tailoring or workflow-interrupting display of severe alerts that need active user acceptance or overriding. Yet, the extent to which the displayed alert interferes with the prescribers' workflow showed inconclusive impact on alert acceptance.

OBJECTIVES

 We aimed to assess whether and how often prescriptions were changed as a potential result of interruptive alerts on different (contraindicated) prescription constellations with particularly high risks for adverse drug events (ADEs).

METHODS

 We retrospectively collected data of all interruptive alerts issued between March 2016 and August 2020 in the local CDSS (AiDKlinik) at Heidelberg University Hospital. The alert battery consisted of 31 distinct alerts for contraindicated DDI with simvastatin, potentially inappropriate medication for patients > 65 years (PIM, N = 14 drugs and 36 drug combinations), and contraindicated drugs in hyperkalemia (N = 5) that could be accepted or overridden giving a reason in free-text form.

RESULTS

 In 935 prescribing sessions of 500 274 total sessions, at least one interruptive alert was fired. Of all interruptive alerts, about half of the sessions were evaluable whereof in total 57.5% (269 of 468 sessions) were accepted while 42.5% were overridden. The acceptance rate of interruptive alerts differed significantly depending on the alert type (p <0.0001), reaching 85.7% for DDI alerts (N = 185), 65.3% for contraindicated drugs in hyperkalemia (N = 98), and 25.1% for PIM alerts (N = 185).

CONCLUSION

 A total of 57.5% of the interruptive medication alerts with particularly high risks for ADE in our setting were accepted while the acceptance rate differed according to the alert type with contraindicated simvastatin DDI alerts being accepted most frequently.

Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review

OBJECTIVE

Alert fatigue limits the effectiveness of medication safety alerts, a type of computerized clinical decision support (CDS). Researchers have suggested alternative interactive designs, as well as tailoring alerts to clinical roles. As examples, alerts may be tiered to convey risk, and certain alerts may be sent to pharmacists. We aimed to evaluate which variants elicit less alert fatigue.

MATERIALS AND METHODS

We searched for articles published between 2007 and 2017 using the PubMed, Embase, CINAHL, and Cochrane databases. We included articles documenting peer-reviewed empirical research that described the interactive design of a CDS system, to which clinical role it was presented, and how often prescribers accepted the resultant advice. Next, we compared the acceptance rates of conventional CDS-presenting prescribers with interruptive modal dialogs (ie, "pop-ups")-with alternative designs, such as role-tailored alerts.

RESULTS

Of 1011 articles returned by the search, we included 39. We found different methods for measuring acceptance rates; these produced incomparable results. The most common type of CDS-in which modals interrupted prescribers-was accepted the least often. Tiering by risk, providing shortcuts for common corrections, requiring a reason to override, and tailoring CDS to match the roles of pharmacists and prescribers were the most common alternatives. Only 1 alternative appeared to increase prescriber acceptance: role tailoring. Possible reasons include the importance of etiquette in delivering advice, the cognitive benefits of delegation, and the difficulties of computing "relevance."

CONCLUSIONS

Alert fatigue may be mitigated by redesigning the interactive behavior of CDS and tailoring CDS to clinical roles. Further research is needed to develop alternative designs, and to standardize measurement methods to enable meta-analyses.

Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening

OBJECTIVE

Evaluation of the effect of six optimization strategies in a clinical decision support system (CDSS) for drug-drug interaction (DDI) screening on alert burden and alert acceptance and description of clinical pharmacist intervention acceptance.

METHODS

Optimizations in the new CDSS were the customization of the knowledge base (with addition of 67 extra DDIs and changes in severity classification), a new alert design, required override reasons for the most serious alerts, the creation of DDI-specific screening intervals, patient-specific alerting, and a real-time follow-up system of all alerts by clinical pharmacists with interventions by telephone was introduced. The alert acceptance was evaluated both at the prescription level (i.e. prescription acceptance, was the DDI prescribed?) and at the administration level (i.e. administration acceptance, did the DDI actually take place?). Finally, the new follow-up system was evaluated by assessing the acceptance of clinical pharmacist's interventions.

RESULTS

In the pre-intervention period, 1087 alerts (92.0 % level 1 alerts) were triggered, accounting for 19 different DDIs. In the post-intervention period, 2630 alerts (38.4 % level 1 alerts) were triggered, representing 86 different DDIs. The relative risk forprescription acceptance in the post-intervention period compared to the pre-intervention period was 4.02 (95 % confidence interval (CI) 3.17-5.10; 25.5 % versus 6.3 %). The relative risk for administration acceptance was 1.16 (95 % CI 1.08-1.25; 54.4 % versus 46.7 %). Finally, 86.9 % of the clinical pharmacist interventions were accepted.

CONCLUSION

Six concurrently implemented CDSS optimization strategies resulted in a high alert acceptance and clinical pharmacist intervention acceptance. Administration acceptance was remarkably higher than prescription acceptance.

Clinically relevant potential drug-drug interactions in intensive care patients: A large retrospective observational multicenter study

PURPOSE

Potential drug-drug interactions (pDDIs) may harm patients admitted to the Intensive Care Unit (ICU). Due to the patient's critical condition and continuous monitoring on the ICU, not all pDDIs are clinically relevant. Clinical decision support systems (CDSSs) warning for irrelevant pDDIs could result in alert fatigue and overlooking important signals. Therefore, our aim was to describe the frequency of clinically relevant pDDIs (crpDDIs) to enable tailoring of CDSSs to the ICU setting.

MATERIALS & METHODS

In this multicenter retrospective observational study, we used medication administration data to identify pDDIs in ICU admissions from 13 ICUs. Clinical relevance was based on a Delphi study in which intensivists and hospital pharmacists assessed the clinical relevance of pDDIs for the ICU setting.

RESULTS

The mean number of pDDIs per 1000 medication administrations was 70.1, dropping to 31.0 when considering only crpDDIs. Of 103,871 ICU patients, 38% was exposed to a crpDDI. The most frequently occurring crpDDIs involve QT-prolonging agents, digoxin, or NSAIDs.

CONCLUSIONS

Considering clinical relevance of pDDIs in the ICU setting is important, as only half of the detected pDDIs were crpDDIs. Therefore, tailoring CDSSs to the ICU may reduce alert fatigue and improve medication safety in ICU patients.

A health informatics transformation model based on intelligent cloud computing - exemplified by type 2 diabetes mellitus with related cardiovascular diseases

BACKGROUND AND OBJECTIVE

Many studies regarding health analysis request structured datasets but the legacy resources provide scattered data. This study aims to establish a health informatics transformation model (HITM) based upon intelligent cloud computing with the self-developed analytics modules by open source technique. The model was exemplified by the open data of type 2 diabetes mellitus (DM2) with related cardiovascular diseases.

METHODS

The Apache-SPARK framework was employed to generate the infrastructure of the HITM, which enables the machine learning (ML) algorithms including random forest, multi-layer perceptron classifier, support vector machine, and naïve Bayes classifier as well as the regression analysis for intelligent cloud computing. The modeling applied the MIMIC-III open database as an example to design the health informatics data warehouse, which embeds the PL/SQL-based modules to extract the analytical data for the training processes. A coupling analysis flow can drive the ML modules to train the sample data and validate the results.

RESULTS

The four modes of cloud computation were compared to evaluate the feasibility of the cloud platform in accordance with its system performance for more than 11,500 datasets. Then, the modeling adaptability was validated by simulating the featured datasets of obesity and cardiovascular-related diseases for patients with DM2 and its complications. The results showed that the run-time efficiency of the platform performed in around one minute and the prediction accuracy of the featured datasets reached 90%.

CONCLUSIONS

This study helped contribute the modeling for efficient transformation of health informatics. The HITM can be customized for the actual clinical database, which provides big data for training, with the proper ML modules for a predictable process in the cloud platform. The feedback of intelligent computing can be referred to risk assessment in health promotion.

Development, Implementation, and Evaluation of a Personalized Machine Learning Algorithm for Clinical Decision Support: Case Study With Shingles Vaccination

BACKGROUND

Although clinical decision support (CDS) alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An electronic health record (EHR)-integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact the clinician's interaction with these alerts in general.

OBJECTIVE

This study aimed to describe the development and implementation of an ML-based signal-to-noise optimization system (SmartCDS) to increase the signal of alerts by decreasing the volume of low-value herpes zoster (shingles) vaccination alerts.

METHODS

We built and deployed SmartCDS, which builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield a clinician's interaction. We extracted all records of shingles alerts from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients.

RESULTS

During the 6 weeks of pilot deployment, the SmartCDS system suppressed an average of 43.67% (15,425/35,315) potential shingles alerts (appointments) and maintained stable counts of weekly shingles vaccination orders (326.3 with system active vs 331.3 in the control group; P=.38) and weekly user-alert interactions (1118.3 with system active vs 1166.3 in the control group; P=.20).

CONCLUSIONS

All key statistics remained stable while the system was turned on. Although the results are promising, the characteristics of the system can be subject to future data shifts, which require automated logging and monitoring. We demonstrated that an automated, ML-based method and data architecture to suppress alerts are feasible without detriment to overall order rates. This work is the first alert suppression ML-based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.

Do computerized clinical decision support systems improve the prescribing of oral anticoagulants? A systematic review

BACKGROUND

Serious adverse drug reactions have been associated with the underuse or the misuse of oral anticoagulant therapy. We systematically reviewed the impact of computerized clinical decision support systems (CDSS) on the prescribing of oral anticoagulants and we described CDSS features associated with success or failure.

METHODS

We searched Medline, Embase, CENTRAL, CINHAL, and PsycINFO for studies that compared CDSS for the initiation or monitoring of oral anticoagulants with routine care. Two reviewers performed study selection, data collection, and risk-of-bias assessment. Disagreements were resolved with a third reviewer. Potentially important CDSS features, identified from previous literature, were evaluated.

RESULTS

Sixteen studies were included in our qualitative synthesis. Most trials were performed in primary care (n = 7) or hospitals (n = 6) and included atrial fibrillation (AF) patients (n = 9). Recommendations mainly focused on anticoagulation underuse (n = 11) and warfarin-drug interactions (n = 5). Most CDSS were integrated in electronic records or prescribing and provided support automatically at the time and location of decision-making. Significant improvements in practitioner performance were found in 9 out of 16 studies, while clinical outcomes were poorly reported. CDSS features seemed slightly more common in studies that demonstrated improvement.

CONCLUSIONS

CDSS might positively impact the use of oral anticoagulants in AF patients at high risk of stroke. The scope of CDSS should now evolve to assist prescribers in selecting the most appropriate and tailored medication. Efforts should nevertheless be made to improve the relevance of notifications and to address implementation outcomes.

Evaluation of context-specific alerts for potassium-increasing drug-drug interactions: A pre-post study

OBJECTIVE

To investigate whether context-specific alerts for potassium-increasing drug-drug interactions (DDIs) in a clinical decision support system reduced the alert burden, increased alert acceptance, and had an effect on the occurrence of hyperkalemia.

MATERIALS AND METHODS

In the pre-intervention period all alerts for potassium-increasing DDIs were level 1 alerts advising absolute contraindication, while in the post-intervention period the same drug combinations could trigger a level 1 (absolute contraindication), a level 2 (monitor potassium values), or a level 3 alert (informative, not shown to physicians) based on the patient's recent laboratory value of potassium. Alert acceptance was defined as non-prescription or non-administration of the interacting drug combination for level 1 alerts and as monitoring of the potassium levels for level 2 alerts.

RESULTS

The alert burden decreased by 92.8%. The relative risk (RR) for alert acceptance based on prescription rates for level 1 alerts and monitoring rates for level 2 alerts was 15.048 (86.5% vs 5.7%; 95% CI 12.037-18.811; P < 0.001). With alert acceptance for level 1 alerts based on actual administration and for level 2 alerts on monitoring rates, the RR was 3.597 (87.6% vs 24.4%; 95% CI 3.192-4.053; P < 0.001). In the generalized linear mixed model the effect of the intervention on the occurrence of hyperkalemia was not significant (OR 1.091, 95% CI 0.172-6.919).

CONCLUSION

The proposed strategy seems effective to get a grip on the delicate balance between over- and under alerting.

Optimizing Drug-Drug Interaction Alerts Using a Multidimensional Approach

OBJECTIVES

Excessive alerts are a common concern associated with clinical decision support systems that monitor drug-drug interactions (DDIs). To reduce the number of low-value interruptive DDI alerts at our hospital, we implemented an iterative, multidimensional quality improvement effort, which included an interdisciplinary advisory group, alert metrics, and measurement of perceived clinical value.

METHODS

Alert data analysis indicated that DDIs were the most common interruptive medication alert. An interdisciplinary alert advisory group was formed to provide expert advice and oversight for alert refinement and ongoing review of alert data. Alert data were categorized into drug classes and analyzed to identify DDI alerts for refinement. Refinement strategies included alert suppression and modification of alerts to be contextually aware.

RESULTS

On the basis of historical analysis of classified DDI alerts, 26 alert refinements were implemented, representing 47% of all alerts. Alert refinement efforts resulted in the following substantial decreases in the number of interruptive DDI alerts: 40% for all clinicians (22.9-14 per 100 orders) and as high as 82% for attending physicians (6.5-1.2 per 100 orders). Two patient safety events related to alert refinements were reported during the project period.

CONCLUSIONS

Our quality improvement effort refined 47% of all DDI alerts that were firing during historical analysis, significantly reduced the number of DDI alerts in a 54-week period, and established a model for sustained alert refinements.

Clinical Decision Support Systems for Drug Allergy Checking: Systematic Review

BACKGROUND

Worldwide, the burden of allergies-in particular, drug allergies-is growing. In the process of prescribing, dispensing, or administering a drug, a medication error may occur and can have adverse consequences; for example, a drug may be given to a patient with a documented allergy to that particular drug. Computerized physician order entry (CPOE) systems with built-in clinical decision support systems (CDSS) have the potential to prevent such medication errors and adverse events.

OBJECTIVE

The aim of this review is to provide a comprehensive overview regarding all aspects of CDSS for drug allergy, including documenting, coding, rule bases, alerts and alert fatigue, and outcome evaluation.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed as much as possible and searches were conducted in 5 databases using CPOE, CDSS, alerts, and allergic or allergy as keywords. Bias could not be evaluated according to PRISMA guidelines due to the heterogeneity of study types included in the review.

RESULTS

Of the 3160 articles considered, 60 met the inclusion criteria. A further 9 articles were added based on expert opinion, resulting in a total of 69 articles. An interrater agreement of 90.9% with a reliability Κ=.787 (95% CI 0.686-0.888) was reached. Large heterogeneity across study objectives, study designs, study populations, and reported results was found. Several key findings were identified. Evidence of the usefulness of clinical decision support for drug allergies has been documented. Nevertheless, there are some important problems associated with their use. Accurate and structured documenting of information on drug allergies in electronic health records (EHRs) is difficult, as it is often not clear to healthcare providers how and where to document drug allergies. Besides the underreporting of drug allergies, outdated or inaccurate drug allergy information in EHRs poses an important problem. Research on the use of coding terminologies for documenting drug allergies is sparse. There is no generally accepted standard terminology for structured documentation of allergy information. The final key finding is the consistently reported low specificity of drug allergy alerts. Current systems have high alert override rates of up to 90%, leading to alert fatigue. Important challenges remain for increasing the specificity of drug allergy alerts. We found only one study specifically reporting outcomes related to CDSS for drug allergies. It showed that adverse drug events resulting from overridden drug allergy alerts do not occur frequently.

CONCLUSIONS

Accurate and comprehensive recording of drug allergies is required for good use of CDSS for drug allergy screening. We found considerable variation in the way drug allergy are recorded in EHRs. It remains difficult to reduce drug allergy alert overload while maintaining patient safety as the highest priority. Future research should focus on improving alert specificity, thereby reducing override rates and alert fatigue. Also, the effect on patient outcomes and cost-effectiveness should be evaluated.

High-priority and low-priority drug-drug interactions in different international electronic health record systems: A comparative study

OBJECTIVES

To investigate whether alert warnings for high-priority and low-priority drug-drug interactions (DDIs) were present in five international electronic health record (EHR) systems, to compare and contrast the severity level assigned to them, and to establish the proportion of alerts that were overridden.

METHODS

We conducted a comparative, retrospective, multinational study using a convenience sample of 5 EHRs from the U.S., U.K., Republic of Korea and Belgium.

RESULTS

Of the 15 previously defined, high-priority, class-based DDIs, alert warnings were found to exist for 11 in both the Korean and UK systems, 9 in the Belgian system, and all 15 in the two US systems. The specific combinations that were included in these class-based DDIs varied considerably in number, type and level of severity amongst systems. Alerts were only active for 8.4% (52/619) and 52.4% (111/212) of the specific drug-drug combinations contained in the Belgian and UK systems, respectively. Hard stops (not possible to override) existed in the US and UK systems only. The override rates for high-priority alerts requiring provider action ranged from 56.7% to 83.3%. Of the 33 previously defined low-priority DDIs, active alerts existed only in the US systems, for three class-based DDIs. The majority were non-interruptive.

CONCLUSIONS

Alert warnings existed for most of the high-priority DDIs in the different EHRs but overriding them was easy in most of the systems. In addition to validating the high- and low-priority DDIs, this study reported a lack of standardization in DDI levels across different international knowledge bases.

Better specification of triggers to reduce the number of drug interaction alerts in primary care

OBJECTIVE

Drug interaction alerts (drug-drug and drug-disease interaction alerts) for chronic medications substantially contribute to alert fatigue in primary care. The aim of this study was to determine which events require (re)assessment of a drug interaction and whether using these events as triggers in clinical decision support systems (CDSSs) would affect the alert rate.

METHODS

Two random 5% data samples from the CDSSs of 123 community pharmacies were used: dataset 1 and 2. The top 10 of most frequent drug interaction alerts not involving laboratory values were selected. To reach consensus on events that should trigger alerts (e.g. first time dispensing, dose modification) for these drug interactions, a two-step consensus process was used. An expert panel of community pharmacists participated in an online survey and a subsequent consensus meeting. A CDSS with alerts based on the consensus was simulated in both datasets.

RESULTS

Dataset 1 and 2 together contained 1,672,169 prescriptions which led to 591,073 alerts. Consensus on events requiring alerts was reached for the ten selected drug interactions. The simulation showed a reduction of the alert rate of 93.0% for the ten selected drug interactions (comparable for dataset 1 and 2), corresponding with a 28.3% decrease of the overall drug interaction alert rate.

CONCLUSION

By consensus-based better specification of the events that trigger drug interaction alerts in primary care, the alert rate for these drug interactions was reduced by over 90%. This promising approach deserves further investigation to assess its consequences and applicability in daily practice.

Technologic Distractions (Part 1): Summary of Approaches to Manage Alert Quantity With Intent to Reduce Alert Fatigue and Suggestions for Alert Fatigue Metrics

OBJECTIVE

To provide ICU clinicians with evidence-based guidance on tested interventions that reduce or prevent alert fatigue within clinical decision support systems.

DESIGN

Systematic review of PubMed, Embase, SCOPUS, and CINAHL for relevant literature from 1966 to February 2017.

PATIENTS

Focus on critically ill patients and included evaluations in other patient care settings, as well.

INTERVENTIONS

Identified interventions designed to reduce or prevent alert fatigue within clinical decision support systems.

MEASUREMENTS AND MAIN RESULTS

Study selection was based on one primary key question to identify effective interventions that attempted to reduce alert fatigue and three secondary key questions that covered the negative effects of alert fatigue, potential unintended consequences of efforts to reduce alert fatigue, and ideal alert quantity. Data were abstracted by two reviewers independently using a standardized abstraction tool. Surveys, meeting abstracts, "gray" literature, studies not available in English, and studies with non-original data were excluded. For the primary key question, articles were excluded if they did not provide a comparator as key question 1 was designed as a problem, intervention, comparison, and outcome question. We anticipated that reduction in alert fatigue, including the concept of desensitization may not be directly measured and thus considered interventions that reduced alert quantity as a surrogate marker for alert fatigue. Twenty-six articles met the inclusion criteria.

CONCLUSION

Approaches for managing alert fatigue in the ICU are provided as a result of reviewing tested interventions that reduced alert quantity with the anticipated effect of reducing fatigue. Suggested alert management strategies include prioritizing alerts, developing sophisticated alerts, customizing commercially available alerts, and including end user opinion in alert selection. Alert fatigue itself is studied less frequently, as an outcome, and there is a need for more precise evaluation. Standardized metrics for alert fatigue is needed to advance the field. Suggestions for standardized metrics are provided in this document.

Systematic Review of Medical Informatics-Supported Medication Decision Making

This systematic review sought to assess the applications and implications of current medical informatics-based decision support systems related to medication prescribing and use. Studies published between January 2006 and July 2016 which were indexed in PubMed and written in English were reviewed, and 39 studies were ultimately included. Most of the studies looked at computerized provider order entry or clinical decision support systems. Most studies examined decision support systems as a means of reducing errors or risk, particularly associated with medication prescribing, whereas a few studies evaluated the impact medical informatics-based decision support systems have on workflow or operations efficiency. Most studies identified benefits associated with decision support systems, but some indicate there is room for improvement.

Think time: A novel approach to analysis of clinicians' behavior after reduction of drug-drug interaction alerts

OBJECTIVES

Pharmacologic interaction alerting offers the potential for safer medication prescribing, but research reveals persistent concerns regarding alert fatigue. Research studies have tried various strategies to resolve this problem, with low overall success. We examined the effects of targeted alert reduction on clinician behavior in a resource constrained hospital.

METHODS

A physician and a pharmacy informaticist reduced alert levels of several drug-drug interactions (DDI) that clinicians almost always overrode with approval from and knowledge of the medical staff. This study evaluated the behavioral changes in prescribers and non-prescribers as measured by "think time", a new metric for evaluating the resolution time for an alert, before and after suppression of selected DDI alerts.

RESULTS

The user-seen DDI alert rate decreased from 9.98% of all orders to 9.20% (p=0.0001) with an overall volume reduction of 10.3%. There was no statistical difference in the reduction of cancelled (-10.00%) vs. proceed orders (-11.07%). Think time decreased overall by 0.61s (p<0.0001). Think time unexpectedly increased for cancelled orders 1.00s which while not statistically significant (p=0.28) is generally thought to be clinically noteworthy. For overrides, think time decreased 0.67s which was significant (p<0.0001). Think time lowered for both prescribers and non-prescribers. Targeted specialists had shorter think times initially, which shortened more than non-targeted specialists.

CONCLUSIONS

Targeted DDI alert reductions reduce alert burden overall, and increase net efficiency as measured by think time for all prescribers better than for non-prescribers. Think time may increase when cancelling or changing orders in response to DDI alerts vs. a decision to override an alert.

Evaluation of an intravenous preparation information system for improving the reconstitution and dilution process

BACKGROUND

There are very few studies reporting the impact of providing intravenous (IV) preparation information on quality use of antimicrobials, particularly regarding their reconstitution and dilution. Therefore, to improve these processes in IV antimicrobial administration, an IV preparation information system (IPIS) was implemented in a hospital.

OBJECTIVE

We aimed to evaluate the effect of improving reconstitution and dilution by implementing an IPIS in the electronic medical record (EMR) system.

METHODS

Prescriptions and activity records of nurses for injectable antimicrobials that required reconstitution and dilution for IV preparation from January 2008 to December 2013 were retrieved from EMR, and assessed based on packaging label information for reconstituting and diluting solutions. We defined proper reconstitution and dilution as occurring when the reconstitution and dilution solutions prescribed were consistent with the nurses' acting records. The types of intervention in the IPIS were as follows: a pop-up alert for proper reconstitution and passive guidance for proper dilution. We calculated the monthly proper reconstitution rate (PRR) and proper dilution rate (PDR) and evaluated the changes in these rates and trends using interrupted time series analyses.

RESULTS

Prior to the initiation of the reconstitution alert and dilution information, the PRR and PDR were 12.7 and 46.1%, respectively. The reconstitution alert of the IPIS rapidly increased the PRR by 41% (p<0.001), after which the PRR decreased by 0.9% (p=0.013) per month after several months. However, there was no significant change in the rate or trend of the PDR during the study period.

CONCLUSIONS

This study demonstrated that the provision of reconstitution alerts by the IPIS contributed to improving the reconstitution process of IV antimicrobial injection administration. However, providing passive information on dilution solutions was ineffective. Furthermore, solutions to ensure the continuous effectiveness of alert systems are warranted and should be actively sought.

How Regrouping Alerts in Computerized Physician Order Entry Layout Influences Physicians' Prescription Behavior: Results of a Crossover Randomized Trial

Background

As demonstrated in several publications, low positive predictive value alerts in computerized physician order entry (CPOE) induce fatigue and may interrupt physicians unnecessarily during prescription of medication. Although it is difficult to increase the consideration of medical alerts by physician through an improvement of their predictive value, another approach consists to act on the way they are presented. The interruption management model inspired us to propose an alternative alert display strategy of regrouping the alerts in the screen layout, as a possible solution for reducing the interruption in physicians’ workflow.

Objective

In this study, we compared 2 CPOE designs based on a particular alert presentation strategy: one design involved regrouping the alerts in a single place on the screen, and in the other, the alerts were located next to the triggering information. Our objective was to evaluate experimentally whether the new design led to fewer interruptions in workflow and if it affected alert handling.

Methods

The 2 CPOE designs were compared in a controlled crossover randomized trial. All interactions with the system and eye movements were stored for quantitative analysis.

Results

The study involved a group of 22 users consisting of physicians and medical students who solved medical scenarios containing prescription tasks. Scenario completion time was shorter when the alerts were regrouped (mean 117.29 seconds, SD 36.68) than when disseminated on the screen (mean 145.58 seconds, SD 75.07; P=.045). Eye tracking revealed that physicians fixated longer on alerts in the classic design (mean 119.71 seconds, SD 76.77) than in the centralized alert design (mean 70.58 seconds, SD 33.53; P=.001). Visual switches between prescription and alert areas, indicating interruption, were reduced with centralized alerts (mean 41.29, SD 21.26) compared with the classic design (mean 57.81, SD 35.97; P=.04). Prescription behavior (ie, prescription changes after alerting), however, did not change significantly between the 2 strategies of display. The After-Scenario Questionnaire (ASQ) that was filled out after each scenario showed that overall satisfaction was significantly rated lower when alerts were regrouped (mean 4.37, SD 1.23) than when displayed next to the triggering information (mean 5.32, SD 0.94; P=.02).

Conclusions

Centralization of alerts in a table might be a way to motivate physicians to manage alerts more actively, in a meaningful way, rather than just being interrupted by them. Our study could not provide clear recommendations yet, but provides objective data through a cognitive psychological approach. Future tests should work on standardized scenarios that would enable to not only measure physicians’ behavior (visual fixations and handling of alerts) but also validate those actions using clinical criteria.

Hospital Pharmacy in Belgium: From Moving Boxes to Providing Optimal Therapy

-