Drug interaction alert override rates in the Meaningful Use era: no evidence of progress

Optimising interruptive clinical decision support alerts for antithrombotic duplicate prescribing in hospital

INTRODUCTION

Duplicate prescribing clinical decision support alerts can prevent important prescribing errors but are frequently the cause of much alert fatigue. Stat dose prescriptions are a known reason for overriding these alerts. This study aimed to evaluate the effect of excluding stat dose prescriptions from duplicate prescribing alerts for antithrombotic medicines on alert burden, prescriber adherence, and prescribing.

MATERIALS AND METHODS

A before (January 1st, 2017 to August 31st, 2022) and after (October 5th, 2022 to September 30th, 2023) study was undertaken of antithrombotic duplicate prescribing alerts and prescribing following a change in alert settings. Alert and prescribing data for antithrombotic medicines were joined, processed, and analysed to compare alert rates, adherence, and prescribing. Alert burden was assessed as alerts per 100 prescriptions. Adherence was measured at the point of the alert as whether the prescriber accepted the alert and following the alert as whether a relevant prescription was ceased within an hour. Co-prescribing of antithrombotic stat dose prescriptions was assessed pre- and post-alert reconfiguration.

RESULTS

Reconfiguration of the alerts reduced the alert rate by 29 % (p < 0.001). The proportion of alerts associated with cessation of antithrombotic duplication significantly increased (32.8 % to 44.5 %, p < 0.001). Adherence at the point of the alert increased 1.2 % (4.8 % to 6.0 %, p = 0.012) and 11.5 % (29.4 % to 40.9 %, p < 0.001) within one hour of the alert. When ceased after the alert over 80 % of duplicate prescriptions were ceased within 2 min of overriding. Antithrombotic stat dose co-prescribing was unchanged for 4 out of 5 antithrombotic duplication alert rules.

CONCLUSION

By reconfiguring our antithrombotic duplicate prescribing alerts, we reduced alert burden and increased alert adherence. Many prescribers ceased duplicate prescribing within 2 min of alert override highlighting the importance of incorporating post-alert measures in accurately determining prescriber alert adherence.

Override rate of drug-drug interaction alerts in clinical decision support systems: A brief systematic review and meta-analysis

Primary studies have demonstrated that despite being useful, most of the drug-drug interaction (DDI) alerts generated by clinical decision support systems are overridden by prescribers. To provide more information about this issue, we conducted a systematic review and meta-analysis on the prevalence of DDI alerts generated by CDSS and alert overrides by physicians. The search strategy was implemented by applying the terms and MeSH headings and conducted in the MEDLINE/PubMed, EMBASE, Web of Science, Scopus, LILACS, and Google Scholar databases. Blinded reviewers screened 1873 records and 86 full studies, and 16 articles were included for analysis. The overall prevalence of alert generated by CDSS was 13% (CI95% 5-24%, p-value <0.0001, I^2 = 100%), and the overall prevalence of alert override by physicians was 90% (CI95% 85-95%, p-value <0.0001, I^2 = 100%). This systematic review and meta-analysis presents a high rate of alert overrides, even after CDSS adjustments that significantly reduced the number of alerts. After analyzing the articles included in this review, it was clear that the CDSS alerts physicians about potential DDI should be developed with a focus on the user experience, thus increasing their confidence and satisfaction, which may increase patient clinical safety.

Development and Validation of Inpatient Hypoglycemia Models Centered Around the Insulin Ordering Process

BACKGROUND

The insulin ordering process is an opportunity to provide clinicians with hypoglycemia risk predictions, but few hypoglycemia models centered around the insulin ordering process exist.

METHODS

We used data on adult patients, admitted in 2019 to non-ICU floors of a large teaching hospital, who had orders for subcutaneous insulin. Our outcome was hypoglycemia, defined as a blood glucose (BG) <70 mg/dL within 24 hours after ordering insulin. We trained and evaluated models to predict hypoglycemia at the time of placing an insulin order, using logistic regression, random forest, and extreme gradient boosting (XGBoost). We compared performance using area under the receiver operating characteristic curve (AUCs) and precision-recall curves. We determined recall at our goal precision of 0.30.

RESULTS

Of 21 052 included insulin orders, 1839 (9%) were followed by a hypoglycemic event within 24 hours. Logistic regression, random forest, and XGBoost models had AUCs of 0.81, 0.80, and 0.79, and recall of 0.44, 0.49, and 0.32, respectively. The most significant predictor was the lowest BG value in the 24 hours preceding the order. Predictors related to the insulin order being placed at the time of the prediction were useful to the model but less important than the patient's history of BG values over time.

CONCLUSIONS

Hypoglycemia within the next 24 hours can be predicted at the time an insulin order is placed, providing an opportunity to integrate decision support into the medication ordering process to make insulin therapy safer.

Resident-Driven Clinical Decision Support Governance to Improve the Utility of Clinical Decision Support

OBJECTIVES

 This resident-driven quality improvement project aimed to better understand the known problem of a misaligned clinical decision support (CDS) strategy and improve CDS utilization.

METHODS

 An internal survey was sent to all internal medicine (IM) residents to identify the most bothersome CDS alerts. Survey results were supported by electronic health record (EHR) data of CDS firing rates and response rates which were collected for each of the three most bothersome CDS tools. Changes to firing criteria were created to increase utilization and to better align with the five rights of CDS. Findings and proposed changes were presented to our institution's CDS Governance Committee. Changes were approved and implemented. Postintervention firing rates were then collected for 1 week.

RESULTS

 Twenty nine residents participated in the CDS survey and identified sepsis alerts, lipid profile reminders, and telemetry renewals to be the most bothersome alerts. EHR data showed action rates for these CDS as low as 1%. We implemented changes to focus emergency department (ED)-based sepsis alerts to the right provider, better address the right information for lipid profile reminders, and select the right time in workflow for telemetry renewals to be most effective. With these changes we successfully eliminated ED-based sepsis CDS reminders for IM providers, saw a 97% reduction in firing rates for the lipid profile CDS, and noted a 55% reduction in firing rates for telemetry CDS.

CONCLUSION

 This project highlighted that alert improvements spearheaded by resident teams can be completed successfully using robust CDS governance strategies and can effectively optimize interruptive alerts.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

Clinician Response to Pharmacogenetic Clinical Decision Support Alerts

The objective of this study was to characterize clinician response following standardization of pharmacogenetic (PGx) clinical decision support alerts at University of Florida (UF) Health. A retrospective analysis of all PGx alerts that fired at a tertiary academic medical center from August 2020 through May 2022 was performed. Alert acceptance rate was calculated and compared across six gene-drug pairs, patient care setting, and clinician specialty. The disposition of the triggering medication was compared with the alert response and evaluated for congruence. There were a total of 818 alerts included for analysis of alert response, 557 alerts included in acceptance rate calculations, and 392 alerts with sufficient information to assess clinical response. The overall acceptance rate was 63%. The alert response and clinical response were congruent for 47% of alerts. Alert response was influenced by the triggering gene-drug pair, clinician specialty, patient care setting, and specialty of the provider who initially ordered the PGx test. Clinical response was mostly incongruent with alert response. Alert acceptance is influenced by the triggering gene-drug pair, clinician specialty, and care setting. Alert response is not a reliable surrogate marker for clinical action. Any future research into the impact of clinical decision support (CDS) alerts should focus on clinical response rather than alert response. Given the reliance on CDS alerts to enhance uptake of PGx, it is worthwhile to further investigate their impact on prescribing and patient outcomes.

Beyond the override: Using evidence of previous drug tolerance to suppress drug allergy alerts; a retrospective study of opioid alerts

OBJECTIVE

Despite the extensive literature exploring alert fatigue, most studies have focused on describing the phenomenon, but not on fixing it. The authors aimed to identify data useful to avert clinically irrelevant alerts to inform future research on clinical decision support (CDS) design.

METHODS

We conducted a retrospective observational study of opioid drug allergy alert (DAA) overrides for the calendar year of 2019 at a large academic medical center, to identify data elements useful to find irrelevant alerts to be averted.

RESULTS

Overall, 227,815 DAAs were fired in 2019, with an override rate of 91 % (n = 208196). Opioids represented nearly two-thirds of these overrides (n = 129063; 62 %) and were the drug class with the highest override rate (96 %). On average, 29 opioid DAAs were overridden per patient. While most opioid alerts (97.1 %) are fired for a possible match (the drug class of the allergen matches the drug class of the prescribed drug), they are overridden significantly less frequently for definite match (exact match between allergen and prescribed drug) (88 % vs. 95.9 %, p < 0.001). When comparing the triggering drug with previously administered drugs, override rates were equally high for both definite match (95.9 %), no match (95.5 %), and possible match (95.1 %). Likewise, when comparing to home medications, overrides were excessively high for possible match (96.3 %), no match (96 %), and definite match (94.4 %).

CONCLUSION

We estimate that 74.5% of opioid DAAs (46.4% of all DAAs) at our institution could be relatively safely averted, since they either have a definite match for previous inpatient administrations suggesting drug tolerance or are fired as possible match with low risk of cross-sensitivity. Future research should focus on identifying other relevant data elements ideally with automated methods and use of emerging standards to empower CDS systems to suppress false-positive alerts while avoiding safety hazards.

Design, development, implementation, and evaluation of a severe drug-drug interaction alert system in the ICU: An analysis of acceptance and override rates

BACKGROUND AND OBJECTIVE

The override rate of Drug-Drug Interaction Alerts (DDIA) in Intensive Care Units (ICUs) is very high. Therefore, this study aimed to design, develop, implement, and evaluate a severe Drug-Drug Alert System (DDIAS) in a system of ICUs and measure the override rate of this system.

METHODS

This is a cross-sectional study that details the design, development, implementation, and evaluation of a DDIAS for severe interactions into a Computerized Provider Order Entry (CPOE) system in the ICUs of Nemazee general teaching hospitals in 2021. The patients exposed to the volume of DDIAS, acceptance and overridden of DDIAS, and usability of DDIAS have been collected. The study was approved by the local Institutional Review Board (IRB) and; the ethics committee of Shiraz University of Medical Science on date: 2019-11-23 (Approval ID: IR.SUMS.REC.1398.1046).

RESULTS

The knowledge base of the DDIAS contains 9,809 severe potential drug-drug interactions (pDDIs). A total of 2672 medications were prescribed in the population study. The volume and acceptance rate for the DDIAS were 81 % and 97.5 %, respectively. The override rate was 2.5 %. The mean System Usability Scale (SUS) score of the DDIAS was 75.

CONCLUSION

This study demonstrates that implementing high-risk DDIAS at the point of prescribing in ICUs improves adherence to alerts. In addition, the usability of the DDIAS was reasonable. Further studies are needed to investigate the establishment of severe DDIAS and measure the prescribers' response to DDIAS on a larger scale.

Comparative effectiveness of generic commercial versus locally customized clinical decision support tools to reduce prescription of nonsteroidal anti-inflammatory drugs for patients with heart failure

OBJECTIVE

To compare the effectiveness of 2 clinical decision support (CDS) tools to avoid prescription of nonsteroidal anti-inflammatory drugs (NSAIDs) in patients with heart failure (HF): a "commercial" and a locally "customized" alert.

METHODS

We conducted a retrospective cohort study of 2 CDS tools implemented within a large integrated health system. The commercial CDS tool was designed according to third-party drug content and EHR vendor specifications. The customized CDS tool underwent a user-centered design process informed by implementation science principles, with input from a cross disciplinary team. The customized CDS tool replaced the commercial CDS tool. Data were collected from the electronic health record via analytic reports and manual chart review. The primary outcome was effectiveness, defined as whether the clinician changed their behavior and did not prescribe an NSAID.

RESULTS

A random sample of 366 alerts (183 per CDS tool) was evaluated that represented 355 unique patients. The commercial CDS tool was effective for 7 of 172 (4%) patients, while the customized CDS tool was effective for 81 of 183 (44%) patients. After adjusting for age, chronic kidney disease, ejection fraction, NYHA class, concurrent prescription of an opioid or acetaminophen, visit type (inpatient or outpatient), and clinician specialty, the customized alerts were at 24.3 times greater odds of effectiveness compared to the commercial alerts (OR: 24.3 CI: 10.20-58.06).

CONCLUSION

Investing additional resources to customize a CDS tool resulted in a CDS tool that was more effective at reducing the total number of NSAID orders placed for patients with HF compared to a commercially available CDS tool.

Barriers to Adoption of Tailored Drug-Drug Interaction Clinical Decision Support

OBJECTIVE

Despite the benefits of the tailored drug-drug interaction (DDI) alerts and the broad dissemination strategy, the uptake of our tailored DDI alert algorithms that are enhanced with patient-specific and context-specific factors has been limited. The goal of the study was to examine barriers and health care system dynamics related to implementing tailored DDI alerts and identify the factors that would drive optimization and improvement of DDI alerts.

METHODS

We employed a qualitative research approach, conducting interviews with a participant interview guide framed based on Proctor's taxonomy of implementation outcomes and informed by the Theoretical Domains Framework. Participants included pharmacists with informatics roles within hospitals, chief medical informatics officers, and associate medical informatics directors/officers. Our data analysis was informed by the technique used in grounded theory analysis, and the reporting of open coding results was based on a modified version of the Safety-Related Electronic Health Record Research Reporting Framework.

RESULTS

Our analysis generated 15 barriers, and we mapped the interconnections of these barriers, which clustered around three entities (i.e., users, organizations, and technical stakeholders). Our findings revealed that misaligned interests regarding DDI alert performance and misaligned expectations regarding DDI alert optimizations among these entities within health care organizations could result in system inertia in implementing tailored DDI alerts.

CONCLUSION

Health care organizations primarily determine the implementation and optimization of DDI alerts, and it is essential to identify and demonstrate value metrics that health care organizations prioritize to enable tailored DDI alert implementation. This could be achieved via a multifaceted approach, such as partnering with health care organizations that have the capacity to adopt tailored DDI alerts and identifying specialists who know users' needs, liaise with organizations and vendors, and facilitate technical stakeholders' work. In the future, researchers can adopt the systematic approach to study tailored DDI implementation problems from other system perspectives (e.g., the vendors' system).

Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework

Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.

Potential Drug-Related Problems in Pediatric Patients-Describing the Use of a Clinical Decision Support System at Pharmacies in Sweden

The clinical support system Electronic Expert Support (EES) is available at all pharmacies in Sweden to examine electronic prescriptions when dispensing to prevent drug-related problems (DRPs). DRPs are common, and result in patient suffering and substantial costs for society. The aim of this research was to study the use of EES for the pediatric population (ages 0-12 years), by describing what types of alerts are generated for potential DRPs, how they are handled, and how the use of EES has changed over time. Data on the number and categories of EES analyses, alerts, and resolved alerts were provided by the Swedish eHealth Agency. The study shows that the use of EES has increased. The most common type of alert for a potential DRP among pediatric patients was regarding high doses in children (30.3% of all alerts generated). The most common type of alert for a potential DRP that was resolved among pediatrics was therapy duplication (4.6% of the alerts were resolved). The most common reason for closing an alert was dialogue with patient for verification of the treatment (66.3% of all closed alerts). Knowledge of which type of alerts are the most common may contribute to increased prescriber awareness of important potential DRPs.

Potential Drug-Drug Interactions among Patients with Schizophrenia Spectrum Disorders: Prevalence, Association with Risk Factors, and Replicate Analysis in 2021

Background and Objectives: Patients with schizophrenia are often exposed to polypharmacotherapy, which may lead to drug-drug interactions. The aim of the study was to investigate the prevalence of potential drug-drug interactions (pDDIs) in hospitalized patients with schizophrenia spectrum disorders and to identify factors associated with pDDIs and manifested symptoms and signs. Materials and Methods: This cross-sectional observational study included 311 inpatients admitted to a psychiatric hospital. The LexiComp drug interaction program was used to identify pDDIs in 2014. Factors associated with the prevalence of pDDIs and factors related to clinically observed symptoms and signs were assessed using multivariable regression. In addition, replicate analysis of pDDI was performed using 2021 program updates. Results: The prevalence of pDDIs was 88.7%. Our study showed that more than half of the patients received at least one drug combination that should be avoided. The most common pDDIs involved combinations of two antipsychotics or combinations of antipsychotics and benzodiazepines, which can lead to cardio-respiratory depression, sedation, arrhythmias, anticholinergic effects, and neuroleptic malignant syndrome. The number of prescribed drugs was a risk factor for pDDIs (OR 2.85; 95% CI 1.84-5.73). All groups of clinically observed symptoms and signs were associated with the number of drugs. In addition, symptoms and signs characteristic of the nervous system and psychiatric disorders were associated with antipsychotic dosage (IRR 1.33; 95% CI 1.12-1.58), which could contribute to the development of extrapyramidal syndrome, insomnia, anxiety, agitation, and bipolar mania. The 2021 version of the drug interaction program showed a shift in drug interactions toward a lower risk rating, implying less severe patient management and possibly less alert fatigue. Conclusions: Patients with schizophrenia spectrum disorders are at high risk of developing drug-drug interactions. Optimization of drug therapy, patient monitoring, and use of drug interaction programs could help to prevent pDDIs and subsequent adverse drug events.

Distinct components of alert fatigue in physicians' responses to a noninterruptive clinical decision support alert

OBJECTIVE

Clinical decision support (CDS) alerts may improve health care quality but "alert fatigue" can reduce provider responsiveness. We analyzed how the introduction of competing alerts affected provider adherence to a single depression screening alert.

MATERIALS AND METHODS

We analyzed the audit data from all occurrences of a CDS alert at a large academic health system. For patients who screen positive for depression during ambulatory visits, a noninterruptive alert was presented, offering a number of relevant documentation actions. Alert adherence was defined as the selection of any option offered within the alert. We assessed the effect of competing clinical guidance alerts presented during the same encounter and the total of all CDS alerts that the same provider had seen in the prior 90 days, on the probability of depression screen alert adherence, adjusting for physician and patient characteristics.

RESULTS

The depression alert fired during 55 649 office visits involving 418 physicians and 40 474 patients over 41 months. After adjustment, physicians who had seen the most alerts in the prior 90 days were much less likely to respond (adjusted OR highest-lowest quartile, 0.38; 95% CI 0.35-0.42; P < .001). Competing alerts in the same visit further reduced the likelihood of adherence only among physicians in the middle two quartiles of alert exposure in the prior 90 days.

CONCLUSIONS

Adherence to a noninterruptive depression alert was strongly associated with the provider's cumulative alert exposure over the past quarter. Health systems should monitor providers' recent alert exposure as a measure of alert fatigue.

Drug-Drug-Gene Interactions in Cardiovascular Medicine

Cardiovascular disease remains a leading cause of both morbidity and mortality worldwide. It is widely accepted that both concomitant medications (drug-drug interactions, DDIs) and genomic factors (drug-gene interactions, DGIs) can influence cardiovascular drug-related efficacy and safety outcomes. Although thousands of DDI and DGI (aka pharmacogenomic) studies have been published to date, the literature on drug-drug-gene interactions (DDGIs, cumulative effects of DDIs and DGIs) remains scarce. Moreover, multimorbidity is common in cardiovascular disease patients and is often associated with polypharmacy, which increases the likelihood of clinically relevant drug-related interactions. These, in turn, can lead to reduced drug efficacy, medication-related harm (adverse drug reactions, longer hospitalizations, mortality) and increased healthcare costs. To examine the extent to which DDGIs and other interactions influence efficacy and safety outcomes in the field of cardiovascular medicine, we review current evidence in the field. We describe the different categories of DDIs and DGIs before illustrating how these two interact to produce DDGIs and other complex interactions. We provide examples of studies that have reported the prevalence of clinically relevant interactions and the most implicated cardiovascular medicines before outlining the challenges associated with dealing with these interactions in clinical practice. Finally, we provide recommendations on how to manage the challenges including but not limited to expanding the scope of drug information compendia, interaction databases and clinical implementation guidelines (to include clinically relevant DDGIs and other complex interactions) and work towards their harmonization; better use of electronic decision support tools; using big data and novel computational techniques; using clinically relevant endpoints, preemptive genotyping; ensuring ethnic diversity; and upskilling of clinicians in pharmacogenomics and personalized medicine.

Allergic Adverse Drug Events After Alert Overrides in Hospitalized Patients

OBJECTIVES

This study aimed to assess how often overridden drug allergy alerts (ODAAs) lead to allergic adverse drug events (All-ADEs) and to evaluate the frequency with which drug allergy alerts (DAAs) were overridden and the reasons, as well as appropriateness of these overrides.

METHODS

A retrospective observational study of DAA generated between 2014 and 2016 was conducted. The corresponding DAA records were reviewed to determine the frequency of alert overrides. A chart review was performed on a subset of 194 ODAA (the first of every 3 chronologically ordered ODAA) to identify All-ADEs and to evaluate the override reasons and the appropriateness of these overrides.

RESULTS

A total of 2044 DAAs were overridden (override rate of 44.8%). Most were triggered by a nonexact match (93.81%), when ordering nervous system (21.1%) and cardiovascular system (19.6%) drugs and were generated by physicians (72.7%). The main override reason was that the patient was already taking the drug or had previously tolerated the drug. Only 9.28% of ODAAs were inappropriately overridden. Six All-ADEs (3.09%) were identified and were due to anti-infective (1), antineoplastic (1), and iodinated-contrast (4) drug administration. Most All-ADEs were cutaneous and were mild. None was life-threatening or fatal. The All-ADEs rate was higher among inappropriately ODAA (15.79%, P = 0.013).

CONCLUSIONS

Alert overrides are not exempt from clinical consequences, although few are associated with All-ADEs. It is necessary to identify the drugs involved in those reactions and to update allergy lists to generate only specific and important DAA and to avoid the negative consequences of overrides.

Prescribers’ reported acceptance and use of drug-drug interaction alerts: An Australian survey

Drug-drug interaction (DDI) alerts are frequently included in electronic medical record (eMR) systems to provide users with relevant information and guidance at the point of care. In this study, we aimed to examine views of DDI alerts among prescribers, including junior doctors, registrars and senior doctors, across Australia. A validated survey for assessing prescribers’ reported acceptance and use of DDI alerts was distributed among researcher networks and in newsletters. Fifty useable responses were received, more than half ( n = 28) from senior doctors. Prescribers at all levels expected DDI alerts to improve performance but junior doctors reported that this was at a high cost, with respect to time and effort. Senior doctors and registrars reported rarely reading alerts and rarely changing prescribing decisions based on alerts. Respondents identified a number of problems with current alerts including limited relevance, repetition, and poor design, highlighting some clear areas for alert improvement.

The Use of Electronic Health Records to Study Drug-Induced Hypersensitivity Reactions from 2000 to 2021: A Systematic Review

Electronic health records (EHRs) have revolutionized the field of drug hypersensitivity reaction (DHR) research. In this systematic review, we assessed 140 articles from 2000-2021, classifying them under six themes: observational studies (n=61), clinical documentation (n=27), case management (n=22), clinical decision support (CDS) (n=18), case identification (n=9), and genetic studies (n=3). EHRs provide convenient access to millions of medical records, facilitating epidemiological studies of DHRs. Though the goal of CDS is to promote safe drug prescribing, allergy alerts must be designed and used in a way that supports this effort. Ultimately, accurate allergy documentation is essential for DHR prevention.

Implementation of pharmacogenomic clinical decision support for health systems: a cost-utility analysis

We constructed a cost-effectiveness model to assess the clinical and economic value of a CDS alert program that provides pharmacogenomic (PGx) testing results, compared to no alert program in acute coronary syndrome (ACS) and atrial fibrillation (AF), from a health system perspective. We defaulted that 20% of 500,000 health-system members between the ages of 55 and 65 received PGx testing for CYP2C19 (ACS-clopidogrel) and CYP2C9, CYP4F2 and VKORC1 (AF-warfarin) annually. Clinical events, costs, and quality-adjusted life years (QALYs) were calculated over 20 years with an annual discount rate of 3%. In total, 3169 alerts would be fired. The CDS alert program would help avoid 16 major clinical events and 6 deaths for ACS; and 2 clinical events and 0.9 deaths for AF. The incremental cost-effectiveness ratio was $39,477/QALY. A PGx-CDS alert program was cost-effective, under a willingness-to-pay threshold of $100,000/QALY gained, compared to no alert program.

Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis

A clinical decision support system (CDSS) informs or generates medical recommendations for healthcare practitioners. An alert is the most common way for a CDSS to interact with practitioners. Research about alerts in CDSS has proliferated over the past ten years. The research trend is ongoing with new emerging terms and focus. Bibliometric analysis is ideal for researchers to understand the research trend and future directions. Influential articles, institutes, countries, authors, and commonly used keywords were analyzed to grasp a comprehensive view on our topic, alerts in CDSS. Articles published between 2011 and 2021 were extracted from the Web of Science database. There were 728 articles included for bibliometric analysis, among which 24 papers were selected for content analysis. Our analysis shows that the research direction has shifted from patient safety to system utility, implying the importance of alert usability to be clinically impactful. Finally, we conclude with future research directions such as the optimization of alert mechanisms and comprehensiveness to enhance alert appropriateness and to reduce alert fatigue.

A systematic review of drug allergy alert systems

BACKGROUND AND OBJECTIVE

Drug allergy alert systems (DAAS), have been considered an effective strategy to reduce preventable adverse drug events (ADEs), improving patient's safety. To date, no review has been conducted analyzing characteristics of DAAS in the hospital setting. Therefore, the aim of this study is to identify, describe and summarize the DAAS used in hospitals. The secondary objectives are to analyse drug allergy alerts (DAA) characteristics, the override rate (OvR) and the clinical consequences of alert overrides.

METHODS

Searches were conducted in Medline and Cochrane Library to identify studies describing DAAS. Systems characteristics, generated alerts, DAA, OvR, and its clinical consequences were extracted and analyzed.

RESULTS

Twenty-eight articles were included in the review. Seventeen different electronic DAAS were identified, of which 53% were commercially available. Systems differed in drug allergy information and rules for generating alerts. DAA were generally interruptive, triggered by non-exact match at drug prescribing and when ignored, an override reason was mandatory. The OvR ranged from 43.7% to 97%. The main override reason given by providers was that 'patient had previously tolerated or had taken the drug without allergic reaction'. Clinical consequences of overriding DAA were only analyzed in four studies, with an ADE incidence between 0% and 6%.

CONCLUSIONS

Different DAAS are used in hospitals with some degree of heterogeneity. Accurate and updated drug allergy information is important to generate only high value alerts. A regular review of DAAS and a standardization of alert rules, alert information and override reasons are necessary to optimize systems. Future studies should evaluate the impact of the DAAS aspects on preventing ADEs.

Modulators Influencing Medication Alert Acceptance: An Explorative Review

OBJECTIVES

Clinical decision support systems (CDSSs) use alerts to enhance medication safety and reduce medication error rates. A major challenge of medication alerts is their low acceptance rate, limiting their potential benefit. A structured overview about modulators influencing alert acceptance is lacking. Therefore, we aimed to review and compile qualitative and quantitative modulators of alert acceptance and organize them in a comprehensive model.

METHODS

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, a literature search in PubMed was started in February 2018 and continued until October 2021. From all included articles, qualitative and quantitative parameters and their impact on alert acceptance were extracted. Related parameters were then grouped into factors, allocated to superordinate determinants, and subsequently further allocated into five categories that were already known to influence alert acceptance.

RESULTS

Out of 539 articles, 60 were included. A total of 391 single parameters were extracted (e.g., patients' comorbidity) and grouped into 75 factors (e.g., comorbidity), and 25 determinants (e.g., complexity) were consequently assigned to the predefined five categories, i.e., CDSS, care provider, patient, setting, and involved drug. More than half of all factors were qualitatively assessed (n = 21) or quantitatively inconclusive (n = 19). Furthermore, 33 quantitative factors clearly influenced alert acceptance (positive correlation: e.g., alert type, patients' comorbidity; negative correlation: e.g., number of alerts per care provider, moment of alert display in the workflow). Two factors (alert frequency, laboratory value) showed contradictory effects, meaning that acceptance was significantly influenced both positively and negatively by these factors, depending on the study. Interventional studies have been performed for only 12 factors while all other factors were evaluated descriptively.

CONCLUSION

This review compiles modulators of alert acceptance distinguished by being studied quantitatively or qualitatively and indicates their effect magnitude whenever possible. Additionally, it describes how further research should be designed to comprehensively quantify the effect of alert modulators.

Overall performance of a drug-drug interaction clinical decision support system: quantitative evaluation and end-user survey

BACKGROUND

Clinical decision support systems are implemented in many hospitals to prevent medication errors and associated harm. They are however associated with a high burden of false positive alerts and alert fatigue. The aim of this study was to evaluate a drug-drug interaction (DDI) clinical decision support system in terms of its performance, uptake and user satisfaction and to identify barriers and opportunities for improvement.

METHODS

A quantitative evaluation and end-user survey were performed in a large teaching hospital. First, very severe DDI alerts generated between 2019 and 2021 were evaluated retrospectively. Data collection comprised alert burden, override rates, the number of alert overrides reviewed by pharmacists and the resulting pharmacist recommendations as well as their acceptance rate. Second, an e-survey was carried out among prescribers to assess satisfaction, usefulness and relevance of DDI alerts as well as reasons for overriding.

RESULTS

A total of 38,409 very severe DDI alerts were generated, of which 88.2% were overridden by the prescriber. In 3.2% of reviewed overrides, a recommendation by the pharmacist was provided, of which 79.2% was accepted. False positive alerts were caused by a too broad screening interval and lack of incorporation of patient-specific characteristics, such as QTc values. Co-prescribing of a non-vitamin K oral anticoagulant and a low molecular weight heparin accounted for 49.8% of alerts, of which 92.2% were overridden. In 88 (1.1%) of these overridden alerts, concurrent therapy was still present. Despite the high override rate, the e-survey revealed that the DDI clinical decision support system was found useful by prescribers.

CONCLUSIONS

Identified barriers were the lack of DDI-specific screening intervals and inclusion of patient-specific characteristics, both leading to a high number of false positive alerts and risk for alert fatigue. Despite these barriers, the added value of the DDI clinical decision support system was recognized by prescribers. Hence, integration of DDI-specific screening intervals and patient-specific characteristics is warranted to improve the performance of the DDI software.

Habit and Automaticity in Medical Alert Override: Cohort Study

Background

Prior literature suggests that alert dismissal could be linked to physicians’ habits and automaticity. The evidence for this perspective has been mainly observational data. This study uses log data from an electronic medical records system to empirically validate this perspective.

Objective

We seek to quantify the association between habit and alert dismissal in physicians.

Methods

We conducted a retrospective analysis using the log data comprising 66,049 alerts generated from hospitalized patients in a hospital from March 2017 to December 2018. We analyzed 1152 physicians exposed to a specific clinical support alert triggered in a hospital’s electronic medical record system to estimate the extent to which the physicians’ habit strength, which had been developed from habitual learning, impacted their propensity toward alert dismissal. We further examined the association between a physician’s habit strength and their subsequent incidences of alert dismissal. Additionally, we recorded the time taken by the physician to respond to the alert and collected data on other clinical and environmental factors related to the alerts as covariates for the analysis.

Results

We found that a physician’s prior dismissal of alerts leads to their increased habit strength to dismiss alerts. Furthermore, a physician’s habit strength to dismiss alerts was found to be positively associated with incidences of subsequent alert dismissals after their initial alert dismissal. Alert dismissal due to habitual learning was also found to be pervasive across all physician ranks, from junior interns to senior attending specialists. Further, the dismissal of alerts had been observed to typically occur after a very short processing time. Our study found that 72.5% of alerts were dismissed in under 3 seconds after the alert appeared, and 13.2% of all alerts were dismissed in under 1 second after the alert appeared. We found empirical support that habitual dismissal is one of the key factors associated with alert dismissal. We also found that habitual dismissal of alerts is self-reinforcing, which suggests significant challenges in disrupting or changing alert dismissal habits once they are formed.

Conclusions

Habitual tendencies are associated with the dismissal of alerts. This relationship is pervasive across all levels of physician rank and experience, and the effect is self-reinforcing.

Improving the specificity of drug-drug interaction alerts: Can it be done?

PURPOSE

Inaccurate and nonspecific medication alerts contribute to high override rates, alert fatigue, and ultimately patient harm. Drug-drug interaction (DDI) alerts often fail to account for factors that could reduce risk; further, drugs that trigger alerts are often inconsistently grouped into value sets. Toward improving the specificity of DDI alerts, the objectives of this study were to (1) highlight the inconsistency of drug value sets for triggering DDI alerts and (2) demonstrate a method of classifying factors that can be used to modify the risk of harm from a DDI.

METHODS

This was a proof-of-concept study focused on 15 well-known DDIs. Using 3 drug interaction references, we extracted 2 drug value sets and any available order- and patient-related factors for each DDI. Fleiss' kappa was used to measure the consistency of value sets among references. Risk-modifying factors were classified as order parameters (eg, route and dose) or patient characteristics (eg, comorbidities and laboratory results).

RESULTS

Seventeen value sets (56%) had nonsignificant agreement. Agreement among the remaining 13 value sets was on average moderate. Thirty-three factors that could reduce risk in 14 of 15 DDIs (93%) were identified. Most risk-modifying factors (67%) were classified as order parameters.

CONCLUSION

This study demonstrates the importance of increasing the consistency of drug value sets that trigger DDI alerts and how alert specificity and usefulness can be improved with risk-modifying factors obtained from drug references. It may be difficult to operationalize certain factors to reduce unnecessary alerts; however, factors can be used to support decisions by providing contextual information.

Effect of Replacing Vendor QTc Alerts with a Custom QTc Risk Alert in Inpatients

OBJECTIVE

The aim of the study is to implement a customized QTc interval clinical decision support (CDS) alert strategy in our electronic health record for hospitalized patients and aimed at providers with the following objectives: minimize QTc prolongation, minimize exposure to QTc prolonging medications, and decrease overall QTc-related alerts. A strategy that was based on the validated QTc risk scoring tool and replacing medication knowledge vendor alerts with custom QTc prolongation alerts was implemented.

METHODS

This is a retrospective quasi-experimental study with a pre-intervention period (August 2019 to October 2019) and post-intervention period (December 2019 to February 2020). The custom alert was implemented in November 2019.

RESULTS

In the pre-implementation group, 361 (19.3%) patients developed QTc prolongation, and in the post-implementation group, 357 (19.6%) patients developed QTc prolongation (OR: 1.02, 95% CI: 0.87-1.20, p = 0.81). The odds ratio of an action taken post-implementation compared with pre-implementation was 18.90 (95% CI: 14.03-25.47, p <0. 001). There was also a decrease in total orders for QTc prolonging medications from 7,921 (5.5%) to 7,566 (5.3%) with an odds ratio of 0.96 (95% CI: 0.93-0.99, p = 0.01).

CONCLUSION

We were able to decrease patient exposure to QTc prolonging medications while not increasing the rate of QTc prolongation as well as improving alert action rate. Additionally, there was a decrease in QTc prolonging medication orders which illustrates the benefit of using a validated risk score with a customized CDS approach compared with a traditional vendor-based strategy. Further research is needed to confirm if an approach implemented at our organization can reduce QTc prolongation rates.

Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics

BACKGROUND

Excessive electronic health record (EHR) alerts reduce the salience of actionable alerts. Little is known about the frequency of interruptive alerts across health systems and how the choice of metric affects which users appear to have the highest alert burden.

OBJECTIVE

(1) Analyze alert burden by alert type, care setting, provider type, and individual provider across 6 pediatric health systems. (2) Compare alert burden using different metrics.

MATERIALS AND METHODS

We analyzed interruptive alert firings logged in EHR databases at 6 pediatric health systems from 2016-2019 using 4 metrics: (1) alerts per patient encounter, (2) alerts per inpatient-day, (3) alerts per 100 orders, and (4) alerts per unique clinician days (calendar days with at least 1 EHR log in the system). We assessed intra- and interinstitutional variation and how alert burden rankings differed based on the chosen metric.

RESULTS

Alert burden varied widely across institutions, ranging from 0.06 to 0.76 firings per encounter, 0.22 to 1.06 firings per inpatient-day, 0.98 to 17.42 per 100 orders, and 0.08 to 3.34 firings per clinician day logged in the EHR. Custom alerts accounted for the greatest burden at all 6 sites. The rank order of institutions by alert burden was similar regardless of which alert burden metric was chosen. Within institutions, the alert burden metric choice substantially affected which provider types and care settings appeared to experience the highest alert burden.

CONCLUSION

Estimates of the clinical areas with highest alert burden varied substantially by institution and based on the metric used.

Quantifying the economic burden of unintended pregnancies due to drug–drug interactions with hormonal contraceptives from the United States payer perspective

Background: In the United States of America (USA), nearly 10 million women use oral contraceptives (OCs). Concomitant administration of certain medications can result in contraceptive failure, and consequently unintended pregnancies due to drug–drug interactions (DDIs). The objective of this analysis was to estimate the economic impact of unintended pregnancies due to DDIs among women of reproductive age using an OC alone or in combination with an enzyme inducer co-medication in the USA from a payer perspective. Methods: A Markov model using a cohort of 1,000 reproductive-age women was developed to estimate costs due to contraceptive failure for OC alone versus OC with concomitant enzyme inducer drugs. All women were assumed to begin an initial state, continuing until experiencing an unintended pregnancy. Unintended pregnancies could result in birth, induced abortion, spontaneous abortion, or ectopic pregnancy. The cohort was analyzed over a time horizon of 1 year with a cycle length of 1 month. Estimates of costs and probabilities of unintended pregnancy outcomes were obtained from the literature. Probabilities from the Markov cohort trace was used to estimate number of pregnancy outcomes. Results: On average, enzyme inducers resulted in 20 additional unintended pregnancies with additional unadjusted and adjusted costs median (range) of USD136,304 (USD57,436–USD320,093) and USD65,146 (USD28,491–USD162,635), respectively. The major component of the direct cost is attributed to the cost of births. Considering the full range of events, DDIs with enzyme inducers could result in 16–25 additional unintended pregnancies and total unadjusted and adjusted costs ranging between USD46,041 to USD399,121 and USD22,839 to USD202,788 respectively. Conclusion: The direct costs associated with unintended pregnancies due to DDIs may be substantial and are potentially avoidable. Greater awareness of DDI risk with oral contraceptives among payers, physicians, pharmacists and patients may reduce unintended pregnancies in at-risk populations.

Comparison of potential psychiatric drug interactions in six drug interaction database programs: A replication study after 2 years of updates

OBJECTIVE

Drug interaction database programs are a fundamental clinical tool. In 2018, we compared the category of potential drug-drug interaction (DDI) provided by six drug interaction database programs for 100 drug interaction pairs including psychiatric drugs, and found the category often differed. This study replicated the comparison in 2020 after 2 years of updates to all six drug interaction database programs.

METHODS

The 100 drug pairs included 94 different drugs: 67 pairs with a psychiatric and non-psychiatric drug, and 33 pairs with two psychiatric drugs. The assigned category of potential DDI for the drug pairs was compared using percent agreement and Fleiss kappa statistic of interrater reliability.

RESULTS

Despite 67 updates involving 46 of the 100 drug pairs, differences remained. The overall percent agreement among the six drug interaction database programs for the category of potential DDI was 67%. The interrater agreement results did not change. The Fleiss kappa overall interrater agreement was fair. The kappa agreement for a drug pair with any severe category rating was substantial, and the kappa agreement for a drug pair with any major category rating was fair.

CONCLUSIONS

Physicians should be aware of the inconsistency among drug interaction database programs in the category of potential DDI for drug pairs including psychiatric drugs. Additionally, the category of potential DDI for a drug pair may change over time. This study highlights the importance of ongoing international efforts to standardize methods used to define and classify potential DDI.

A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study

Background

Exposure to life-threatening drug-drug interactions (DDIs) occurs despite the widespread use of clinical decision support. The DDI between warfarin and nonsteroidal anti-inflammatory drugs is common and potentially life-threatening. Patients can play a substantial role in preventing harm from DDIs; however, the current model for DDI decision-making is clinician centric.

Objective

This study aims to design and study the usability of DDInteract, a tool to support shared decision-making (SDM) between a patient and provider for the DDI between warfarin and nonsteroidal anti-inflammatory drugs.

Methods

We used an SDM framework and user-centered design methods to guide the design and usability of DDInteract—an SDM electronic health record app to prevent harm from clinically significant DDIs. The design involved iterative prototypes, qualitative feedback from stakeholders, and a heuristic evaluation. The usability evaluation included patients and clinicians. Patients participated in a simulated SDM discussion using clinical vignettes. Clinicians were asked to complete eight tasks using DDInteract and to assess the tool using a survey adapted from the System Usability Scale.

Results

The designed DDInteract prototype includes the following features: a patient-specific risk profile, dynamic risk icon array, patient education section, and treatment decision tree. A total of 4 patients and 11 clinicians participated in the usability study. After an SDM session where patients and clinicians review the tool concurrently, patients generally favored pain treatments with less risk of gastrointestinal bleeding. Clinicians successfully completed the tasks with a mean of 144 (SD 74) seconds and rated the usability of DDInteract as 4.32 (SD 0.52) of 5.

Conclusions

This study expands the use of SDM to DDIs. The next steps are to determine if DDInteract can improve shared decision-making quality and to implement it across health systems using interoperable technology.

The Impact of Pop-Up Clinical Electronic Health Record Decision Tools on Ordering Pulmonary Embolism Studies in the Emergency Department

BACKGROUND

Emergency physicians make time-sensitive care decisions for life threatening diagnoses and utilize evidence-based decision rules and testing with high sensitivity to ensure that critical diagnoses are not missed. Current literature suggests that there is over testing for pulmonary embolism in the emergency department.

OBJECTIVES

This study aimed to determine whether the addition of a pop-up notification of the Modified Wells Criteria into the workflow would impact the number of total orders for computed tomography pulmonary angiography (CTPA) or the diagnostic yield of those studies.

METHODS

This study was a retrospective observational study comparing CTPA utilization rates and diagnostic yield among physicians at a single academic emergency department in the 1 year prior and 1 year post implementation of an active electronic health recored (EHR) pop-up of Modified Well's scoring when ordering a CTPA.

RESULTS

CTPA utilization rates were statistically equivalent, p <0.0001 within a 0.5% equivalence margin, during the pre and post intervention years. The observed difference was 0.1% (95% CI -0.02%, 0.21%). Despite proving equivalence in the rates of CTPA studies ordered, the diagnostic yield, however, was significantly different (p = 0.001), 32.35% in the pre-intervention year compared to 41.60% in the post-intervention year.

CONCLUSION

There are many barriers to the implementation of successful EHR alerts. These findings support and validate previous studies that have shown a higher diagnostic yield of CT angiography for pulmonary embolism after implementation of active alerts integrated into the EHR with ordering studies. These tools are effective quality improvement initiatives, and their use should be encouraged.

The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review

A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.

Comparing Potential Drug-Drug Interactions in Companion Animal Medications Using Two Electronic Databases

Multiple-drug prescriptions can cause drug–drug interactions (DDIs), which increase risks associated with healthcare in veterinary medicine. Moreover, many human medicines are used in canine patients under the responsibility of veterinarians and may cause severe problems due to off-label use. Currently, many electronic databases are being used as tools for potential DDI prediction, for example, Micromedex and Drugs.com, which may benefit the prediction of potential DDIs for drugs used in canine. The purpose of this study was to examine different abilities for the identification of potential DDIs in companion animal medicine, especially in canine patients, by Micromedex and Drugs.com. Micromedex showed 429 pairs of potential DDIs, while Drugs.com showed 842 pairs of potential DDIs. The analysis comparing results between the two databases showed 139 pairs (12.28%) with the same severity and 993 pairs (87.72%) with different severities. The major mechanisms of contraindicated and major potential DDIs were cytochrome P450 induction–inhibition and QT interval prolongation. Veterinarians should interpret potential DDIs from several databases with caution and keep in mind that the results might not be reliable due to differences in sensitivity to drugs, drug-metabolizing enzymes, and elimination pathway between animals and humans.

A dynamic reaction picklist for improving allergy reaction documentation in the electronic health record

OBJECTIVE

Incomplete and static reaction picklists in the allergy module led to free-text and missing entries that inhibit the clinical decision support intended to prevent adverse drug reactions. We developed a novel, data-driven, "dynamic" reaction picklist to improve allergy documentation in the electronic health record (EHR).

MATERIALS AND METHODS

We split 3 decades of allergy entries in the EHR of a large Massachusetts healthcare system into development and validation datasets. We consolidated duplicate allergens and those with the same ingredients or allergen groups. We created a reaction value set via expert review of a previously developed value set and then applied natural language processing to reconcile reactions from structured and free-text entries. Three association rule-mining measures were used to develop a comprehensive reaction picklist dynamically ranked by allergen. The dynamic picklist was assessed using recall at top k suggested reactions, comparing performance to the static picklist.

RESULTS

The modified reaction value set contained 490 reaction concepts. Among 4 234 327 allergy entries collected, 7463 unique consolidated allergens and 469 unique reactions were identified. Of the 3 dynamic reaction picklists developed, the 1 with the optimal ranking achieved recalls of 0.632, 0.763, and 0.822 at the top 5, 10, and 15, respectively, significantly outperforming the static reaction picklist ranked by reaction frequency.

CONCLUSION

The dynamic reaction picklist developed using EHR data and a statistical measure was superior to the static picklist and suggested proper reactions for allergy documentation. Further studies might evaluate the usability and impact on allergy documentation in the EHR.

Precision Population Medicine in Primary Care: The Sanford Chip Experience

Genetic testing has the potential to revolutionize primary care, but few health systems have developed the infrastructure to support precision population medicine applications or attempted to evaluate its impact on patient and provider outcomes. In 2018, Sanford Health, the nation's largest rural nonprofit health care system, began offering genetic testing to its primary care patients. To date, more than 11,000 patients have participated in the Sanford Chip Program, over 90% of whom have been identified with at least one informative pharmacogenomic variant, and about 1.5% of whom have been identified with a medically actionable predisposition for disease. This manuscript describes the rationale for offering the Sanford Chip, the programs and infrastructure implemented to support it, and evolving plans for research to evaluate its real-world impact.

Reducing drug prescription errors and adverse drug events by application of a probabilistic, machine-learning based clinical decision support system in an inpatient setting

BACKGROUND

Drug prescription errors are made, worldwide, on a daily basis, resulting in a high burden of morbidity and mortality. Existing rule-based systems for prevention of such errors are unsuccessful and associated with substantial burden of false alerts.

OBJECTIVE

In this prospective study, we evaluated the accuracy, validity, and clinical usefulness of medication error alerts generated by a novel system using outlier detection screening algorithms, used on top of a legacy standard system, in a real-life inpatient setting.

MATERIALS AND METHODS

We integrated a novel outlier system into an existing electronic medical record system, in a single medical ward in a tertiary medical center. The system monitored all drug prescriptions written during 16 months. The department's staff assessed all alerts for accuracy, clinical validity, and usefulness. We recorded all physician's real-time responses to alerts generated.

RESULTS

The alert burden generated by the system was low, with alerts generated for 0.4% of all medication orders. Sixty percent of the alerts were flagged after the medication was already dispensed following changes in patients' status which necessitated medication changes (eg, changes in vital signs). Eighty-five percent of the alerts were confirmed clinically valid, and 80% were considered clinically useful. Forty-three percent of the alerts caused changes in subsequent medical orders.

CONCLUSION

A clinical decision support system that used a probabilistic, machine-learning approach based on statistically derived outliers to detect medication errors generated clinically useful alerts. The system had high accuracy, low alert burden and low false-positive rate, and led to changes in subsequent orders.

Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review

OBJECTIVE

Alert fatigue limits the effectiveness of medication safety alerts, a type of computerized clinical decision support (CDS). Researchers have suggested alternative interactive designs, as well as tailoring alerts to clinical roles. As examples, alerts may be tiered to convey risk, and certain alerts may be sent to pharmacists. We aimed to evaluate which variants elicit less alert fatigue.

MATERIALS AND METHODS

We searched for articles published between 2007 and 2017 using the PubMed, Embase, CINAHL, and Cochrane databases. We included articles documenting peer-reviewed empirical research that described the interactive design of a CDS system, to which clinical role it was presented, and how often prescribers accepted the resultant advice. Next, we compared the acceptance rates of conventional CDS-presenting prescribers with interruptive modal dialogs (ie, "pop-ups")-with alternative designs, such as role-tailored alerts.

RESULTS

Of 1011 articles returned by the search, we included 39. We found different methods for measuring acceptance rates; these produced incomparable results. The most common type of CDS-in which modals interrupted prescribers-was accepted the least often. Tiering by risk, providing shortcuts for common corrections, requiring a reason to override, and tailoring CDS to match the roles of pharmacists and prescribers were the most common alternatives. Only 1 alternative appeared to increase prescriber acceptance: role tailoring. Possible reasons include the importance of etiquette in delivering advice, the cognitive benefits of delegation, and the difficulties of computing "relevance."

CONCLUSIONS

Alert fatigue may be mitigated by redesigning the interactive behavior of CDS and tailoring CDS to clinical roles. Further research is needed to develop alternative designs, and to standardize measurement methods to enable meta-analyses.

The anatomy of clinical documentation: an assessment and classification of narrative note sections format and content

Introduction. We systematically analyzed the most commonly used narrative note formats and content found in primary and specialty care visit notes to inform future research and electronic health record (EHR) development. Methods. We extracted data from the history of present illness (HPI) and impression and plan (IP) sections of 80 primary and specialty care visit notes. Two authors iteratively classified the format of the sections and compared the size of each section and the overall note size between primary and specialty care notes. We then annotated the content of these sections to develop a taxonomy of types of data communicated in the narrative note sections. Results. Both HPI and IP were significantly longer in primary care when compared to specialty care notes (HPI: n = 187 words, SD[130] vs. n = 119 words, SD [53]; p = 0.004 / IP: n = 270 words, SD [145] vs. n = 170 words, SD [101]; p < 0.001). Although we did not find a significant difference in the overall note size between the two groups, the proportion of HPI and IP content in relation to the total note size was significantly higher in primary care notes (40%, SD [13] vs. 28%, SD [11]; p < 0.001). We identified five combinations of format of HPI + IP sections respectively: (A) story + list with categories; (B) story + story; (C) list without categories + list with categories; (D) list with categories + list with categories; and (E) list with categories + story. HPI and IP content was significantly smaller in combination C compared to combination A (-172 words, [95% Conf. -326, -17.89]; p = 0.02). We identified seven taxa representing 45 different types of data: finding/condition documented (n = 14), intervention documented (n = 9), general descriptions and definitions (n = 7), temporal information (n = 6), reasons and justifications (n = 4), participants and settings (n = 4), and clinical documentation (n = 1). Conclusion. We identified commonly used narrative note section formats and developed a taxonomy of narrative note content to help researchers to tailor their efforts and design more efficient clinical documentation systems.

Towards The Automated, Empirical Filtering of Drug-Drug Interaction Alerts in Clinical Decision Support Systems: Historical Cohort Study of Vitamin K Antagonists

Background

Drug-drug interactions (DDIs) involving vitamin K antagonists (VKAs) constitute an important cause of in-hospital morbidity and mortality. However, the list of potential DDIs is long; the implementation of all these interactions in a clinical decision support system (CDSS) results in over-alerting and alert fatigue, limiting the benefits provided by the CDSS.

Objective

To estimate the probability of occurrence of international normalized ratio (INR) changes for each DDI rule, via the reuse of electronic health records.

Methods

An 8-year, exhaustive, population-based, historical cohort study including a French community hospital, a group of Danish community hospitals, and a Bulgarian hospital. The study database included 156,893 stays. After filtering against two criteria (at least one VKA administration and at least one INR laboratory result), the final analysis covered 4047 stays. Exposure to any of the 145 drugs known to interact with VKA was tracked and analyzed if at least 3 patients were concerned. The main outcomes are VKA potentiation (defined as an INR≥5) and VKA inhibition (defined as an INR≤1.5). Groups were compared using the Fisher exact test and logistic regression, and the results were expressed as an odds ratio (95% confidence limits).

Results

The drugs known to interact with VKAs either did not have a statistically significant association regarding the outcome (47 drug administrations and 14 discontinuations) or were associated with significant reduction in risk of its occurrence (odds ratio<1 for 18 administrations and 21 discontinuations).

Conclusions

The probabilities of outcomes obtained were not those expected on the basis of our current body of pharmacological knowledge. The results do not cast doubt on our current pharmacological knowledge per se but do challenge the commonly accepted idea whereby this knowledge alone should be used to define when a DDI alert should be displayed. Real-life probabilities should also be considered during the filtration of DDI alerts by CDSSs, as proposed in SPC-CDSS (statistically prioritized and contextualized CDSS). However, these probabilities may differ from one hospital to another and so should probably be calculated locally.

Electronic health record alerts for acute kidney injury: multicenter, randomized clinical trial

OBJECTIVE

To determine whether electronic health record alerts for acute kidney injury would improve patient outcomes of mortality, dialysis, and progression of acute kidney injury.

DESIGN

Double blinded, multicenter, parallel, randomized controlled trial.

SETTING

Six hospitals (four teaching and two non-teaching) in the Yale New Haven Health System in Connecticut and Rhode Island, US, ranging from small community hospitals to large tertiary care centers.

PARTICIPANTS

6030 adult inpatients with acute kidney injury, as defined by the Kidney Disease: Improving Global Outcomes (KDIGO) creatinine criteria.

INTERVENTIONS

An electronic health record based "pop-up" alert for acute kidney injury with an associated acute kidney injury order set upon provider opening of the patient's medical record.

MAIN OUTCOME MEASURES

A composite of progression of acute kidney injury, receipt of dialysis, or death within 14 days of randomization. Prespecified secondary outcomes included outcomes at each hospital and frequency of various care practices for acute kidney injury.

RESULTS

6030 patients were randomized over 22 months. The primary outcome occurred in 653 (21.3%) of 3059 patients with an alert and in 622 (20.9%) of 2971 patients receiving usual care (relative risk 1.02, 95% confidence interval 0.93 to 1.13, P=0.67). Analysis by each hospital showed worse outcomes in the two non-teaching hospitals (n=765, 13%), where alerts were associated with a higher risk of the primary outcome (relative risk 1.49, 95% confidence interval 1.12 to 1.98, P=0.006). More deaths occurred at these centers (15.6% in the alert group v 8.6% in the usual care group, P=0.003). Certain acute kidney injury care practices were increased in the alert group but did not appear to mediate these outcomes.

CONCLUSIONS

Alerts did not reduce the risk of our primary outcome among patients in hospital with acute kidney injury. The heterogeneity of effect across clinical centers should lead to a re-evaluation of existing alerting systems for acute kidney injury.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02753751.

Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening

OBJECTIVE

Evaluation of the effect of six optimization strategies in a clinical decision support system (CDSS) for drug-drug interaction (DDI) screening on alert burden and alert acceptance and description of clinical pharmacist intervention acceptance.

METHODS

Optimizations in the new CDSS were the customization of the knowledge base (with addition of 67 extra DDIs and changes in severity classification), a new alert design, required override reasons for the most serious alerts, the creation of DDI-specific screening intervals, patient-specific alerting, and a real-time follow-up system of all alerts by clinical pharmacists with interventions by telephone was introduced. The alert acceptance was evaluated both at the prescription level (i.e. prescription acceptance, was the DDI prescribed?) and at the administration level (i.e. administration acceptance, did the DDI actually take place?). Finally, the new follow-up system was evaluated by assessing the acceptance of clinical pharmacist's interventions.

RESULTS

In the pre-intervention period, 1087 alerts (92.0 % level 1 alerts) were triggered, accounting for 19 different DDIs. In the post-intervention period, 2630 alerts (38.4 % level 1 alerts) were triggered, representing 86 different DDIs. The relative risk forprescription acceptance in the post-intervention period compared to the pre-intervention period was 4.02 (95 % confidence interval (CI) 3.17-5.10; 25.5 % versus 6.3 %). The relative risk for administration acceptance was 1.16 (95 % CI 1.08-1.25; 54.4 % versus 46.7 %). Finally, 86.9 % of the clinical pharmacist interventions were accepted.

CONCLUSION

Six concurrently implemented CDSS optimization strategies resulted in a high alert acceptance and clinical pharmacist intervention acceptance. Administration acceptance was remarkably higher than prescription acceptance.

Prevalence of Clinically Significant Drug-Drug Interactions Across US Children's Hospitals

BACKGROUND

Little is known about the prescribing of medications with potential drug-drug interactions (DDIs) in the pediatric population. The objective of this study was to determine the prevalence and variation of prescribing medications with clinically significant DDIs across children's hospitals in the United States.

METHODS

We performed a retrospective cohort study of patients <26 years of age who were discharged from 1 of 52 US children's hospitals between January 2016 and December 2018. Fifty-three drug pairings with clinically significant DDIs in children were evaluated. We identified patient-level risk factors associated with DDI using multivariable logistic regression. Adjusted hospital-level rates of DDI exposure were derived by using a generalized linear mixed-effects model, and DDI exposure variations were examined across individual hospitals.

RESULTS

Across 52 children's hospitals, 47 414 (2.0%) hospitalizations included exposure to a DDI pairing (34.9 per 1000 patient-days) during the study period. One-quarter of pairings were considered contraindicated (risk grade X). After adjusting for hospital and clinical factors, there was wide variation in the percentage of DDI prescribing across hospitals, ranging from 1.05% to 4.92%. There was also substantial hospital-level variation of exposures to individual drug pairings. Increasing age, number of complex chronic conditions, length of stay, and surgical encounters were independently associated with an increased odds of DDI exposure.

CONCLUSIONS

Patients hospitalized at US children's hospitals are frequently exposed to medications with clinically significant DDIs. Exposure risk varied substantially across hospitals. Further study is needed to determine the rate of adverse events due to DDI exposures and factors amenable for interventions promoting safer medication use.

Integrating the Practical Robust Implementation and Sustainability Model With Best Practices in Clinical Decision Support Design: Implementation Science Approach

Background

Clinical decision support (CDS) design best practices are intended to provide a narrative representation of factors that influence the success of CDS tools. However, they provide incomplete direction on evidence-based implementation principles.

Objective

This study aims to describe an integrated approach toward applying an existing implementation science (IS) framework with CDS design best practices to improve the effectiveness, sustainability, and reproducibility of CDS implementations.

Methods

We selected the Practical Robust Implementation and Sustainability Model (PRISM) IS framework. We identified areas where PRISM and CDS design best practices complemented each other and defined methods to address each. Lessons learned from applying these methods were then used to further refine the integrated approach.

Results

Our integrated approach to applying PRISM with CDS design best practices consists of 5 key phases that iteratively interact and inform each other: multilevel stakeholder engagement, designing the CDS, design and usability testing, thoughtful deployment, and performance evaluation and maintenance. The approach is led by a dedicated implementation team that includes clinical informatics and analyst builder expertise.

Conclusions

Integrating PRISM with CDS design best practices extends user-centered design and accounts for the multilevel, interacting, and dynamic factors that influence CDS implementation in health care. Integrating PRISM with CDS design best practices synthesizes the many known contextual factors that can influence the success of CDS tools, thereby enhancing the reproducibility and sustainability of CDS implementations. Others can adapt this approach to their situation to maximize and sustain CDS implementation success.

Reducing Alert Fatigue by Sharing Low-Level Alerts With Patients and Enhancing Collaborative Decision Making Using Blockchain Technology: Scoping Review and Proposed Framework (MedAlert)

Background

Clinical decision support (CDS) is a tool that helps clinicians in decision making by generating clinical alerts to supplement their previous knowledge and experience. However, CDS generates a high volume of irrelevant alerts, resulting in alert fatigue among clinicians. Alert fatigue is the mental state of alerts consuming too much time and mental energy, which often results in relevant alerts being overridden unjustifiably, along with clinically irrelevant ones. Consequently, clinicians become less responsive to important alerts, which opens the door to medication errors.

Objective

This study aims to explore how a blockchain-based solution can reduce alert fatigue through collaborative alert sharing in the health sector, thus improving overall health care quality for both patients and clinicians.

Methods

We have designed a 4-step approach to answer this research question. First, we identified five potential challenges based on the published literature through a scoping review. Second, a framework is designed to reduce alert fatigue by addressing the identified challenges with different digital components. Third, an evaluation is made by comparing MedAlert with other proposed solutions. Finally, the limitations and future work are also discussed.

Results

Of the 341 academic papers collected, 8 were selected and analyzed. MedAlert securely distributes low-level (nonlife-threatening) clinical alerts to patients, enabling a collaborative clinical decision. Among the solutions in our framework, Hyperledger (private permissioned blockchain) and BankID (federated digital identity management) have been selected to overcome challenges such as data integrity, user identity, and privacy issues.

Conclusions

MedAlert can reduce alert fatigue by attracting the attention of patients and clinicians, instead of solely reducing the total number of alerts. MedAlert offers other advantages, such as ensuring a higher degree of patient privacy and faster transaction times compared with other frameworks. This framework may not be suitable for elderly patients who are not technology savvy or in-patients. Future work in validating this framework based on real health care scenarios is needed to provide the performance evaluations of MedAlert and thus gain support for the better development of this idea.

Survey of psychiatrist use of digital technology in clinical practice

Background

Psychiatrists were surveyed to obtain an overview of how they currently use technology in clinical practice, with a focus on psychiatrists who treat patients with bipolar disorder.

Methods

Data were obtained using an online-only survey containing 46 questions, completed by a convenience sample of 209 psychiatrists in 19 countries. Descriptive statistics, and analyses of linear associations and to remove country heterogeneity were calculated.

Results

Virtually all psychiatrists seek information online with many benefits, but some experience information overload. 75.2% of psychiatrists use an EMR/EHR at work, and 64.6% communicate with patients using a new technology, primarily email (48.8%). 66.0% do not ask patients if they use the Internet in relation to bipolar disorder. 67.3% of psychiatrists feel it is too early to tell if patient online information seeking about bipolar disorder is improving the quality of care. 66.3% of psychiatrists think technology-based treatments will improve the quality of care for some or many patients. However, 60.0% of psychiatrists do not recommend technology-based treatments to patients, and those who recommend select a variety of treatments. Psychiatrists use technology more frequently when the patients live in urban rather than rural or suburban areas. Only 23.9% of psychiatrists have any formal training in technology.

Conclusions

Digital technology is routinely used by psychiatrists in clinical practice. There is near unanimous agreement about the benefits of psychiatrist online information-seeking, but research on information overload is needed. There is less agreement about the appropriate use of other clinical technologies, especially those involving patients. It is too early to tell if technology-based treatments or patient Internet activities will improve the quality of care. The digital divide remains between use of technology for psychiatrists with patients living in urban and rural or suburban areas. Psychiatrists need more formal training in technology to understand risks, benefits and limitations of clinical products.

Professional fission in medical routines: medical scribes and physicians in two US hospital departments

In this study we argue that professionalism imposed from above can result in a type of fission, leading to the ambiguous emergence of new occupations. Our case focuses on the US’ federally mandated use of electronic health records and the increased use of medical scribes. Data include observations of 571 patient encounters across 48 scribe shifts, and 12 interviews with medical scribes and physicians in the ophthalmology and digestive health departments of a community hospital. We found substantial differences in scribes’ roles based on the pre-existing routines within each department, and that scribes developed agency in the interface between the electronic health record and the physicians’ work. Our study contributes to work on occupations as negotiated orders by drawing attention to external influences, the importance of considering differences across professional task routines, and the personal interactions between professional and technical workers.

Going digital: a checklist in preparing for hospital-wide electronic medical record implementation and digital transformation

Objective In an era of rapid digitisation of Australian hospitals, practical guidance is needed in how to successfully implement electronic medical records (EMRs) as both a technical innovation and a major transformative change in clinical care. The aim of the present study was to develop a checklist that clearly and comprehensively defines the steps that best prepare hospitals for EMR implementation and digital transformation. Methods The checklist was developed using a formal methodological framework comprised of: literature reviews of relevant issues; an interactive workshop involving a multidisciplinary group of digital leads from Queensland hospitals; a draft document based on literature and workshop proceedings; and a review and feedback from senior clinical leads. Results The final checklist comprised 19 questions, 13 related to EMR implementation and six to digital transformation. Questions related to the former included organisational considerations (leadership, governance, change leaders, implementation plan), technical considerations (vendor choice, information technology and project management teams, system and hardware alignment with clinician workflows, interoperability with legacy systems) and training (user training, post-go-live contingency plans, roll-out sequence, staff support at point of care). Questions related to digital transformation included cultural considerations (clinically focused vision statement and communication strategy, readiness for change surveys), management of digital disruption syndromes and plans for further improvement in patient care (post-go-live optimisation of digital system, quality and benefit evaluation, ongoing digital innovation). Conclusion This evidence-based, field-tested checklist provides guidance to hospitals planning EMR implementation and separates readiness for EMR from readiness for digital transformation. What is known about the topic? Many hospitals throughout Australia have implemented, or are planning to implement, hospital wide electronic medical records (EMRs) with varying degrees of functionality. Few hospitals have implemented a complete end-to-end digital system with the ability to bring about major transformation in clinical care. Although the many challenges in implementing EMRs have been well documented, they have not been incorporated into an evidence-based, field-tested checklist that can practically assist hospitals in preparing for EMR implementation as both a technical innovation and a vehicle for major digital transformation of care. What does this paper add? This paper outlines a 19-question checklist that was developed using a formal methodological framework comprising literature review of relevant issues, proceedings from an interactive workshop involving a multidisciplinary group of digital leads from hospitals throughout Queensland, including three hospitals undertaking EMR implementation and one hospital with complete end-to-end EMR, and review of a draft checklist by senior clinical leads within a statewide digital healthcare improvement network. The checklist distinguishes between issues pertaining to EMR as a technical innovation and EMR as a vehicle for digital transformation of patient care. What are the implications for practitioners? Successful implementation of a hospital-wide EMR requires senior managers, clinical leads, information technology teams and project management teams to fully address key operational and strategic issues. Using an issues checklist may help prevent any one issue being inadvertently overlooked or underemphasised in the planning and implementation stages, and ensure the EMR is fully adopted and optimally used by clinician users in an ongoing digital transformation of care.

Adverse events targeted by drug-drug interaction alerts in hospitalized patients

OBJECTIVE

To identify the types of adverse drug events (ADEs) that drug-drug interaction (DDI) alerts are trying to prevent in hospitalized patients.

METHODS

This was a retrospective cross-sectional study conducted in a tertiary referral hospital in Australia. All DDI alerts encountered by prescribers during a 1-month period were evaluated for potential ADEs targeted for prevention. If the same DDI alert occurred for the same patient multiple times during hospitalization, it was counted only once (i.e. first alert). This was termed a 'unique DDI alert' for a given patient. The primary outcome was the type of ADE the alerts were trying to prevent.

RESULTS

There were 715 patients who had 1599 unique DDI alerts. The two most common potential ADEs (not mutually exclusive) that the alerts attempted to prevent were QTc prolongation or torsades de pointes (n = 1028/1599, 64 %), followed by extrapyramidal symptoms or neuroleptic malignant syndrome (n = 463/1599, 29 %). Either of these two potential ADEs were present in 83 % (n = 1329/1599) of unique DDI alerts.

CONCLUSION

Alerting systems are primarily trying to prevent two types of potential ADEs, which were included in more than 80 % of DDI alerts. This has important implications for patient monitoring in hospitals.