Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review

Aligning Patient Safety and Stewardship: A Harm Reduction Strategy for Children

Purpose of review

Review important patient safety and stewardship concepts and use clinical examples to describe how they align to improve patient outcomes and reduce harm for children.

Recent findings

Current evidence indicates that healthcare overuse is substantial. Unnecessary care leads to avoidable adverse events, anxiety and distress, and financial toxicity. Increases in antimicrobial resistance, venous thromboembolism, radiation exposure, and healthcare costs are examples of patient harm associated with a lack of stewardship. Studies indicate that many tools can increase standardization of care, improve resource utilization, and enhance safety culture to better align safety and stewardship.

Summary

The principles of stewardship and parsimonious care can improve patient safety for children.

The Impact of Clinical Decision Support Alerts on Clostridioides difficile Testing: A Systematic Review

BACKGROUND

Several studies have investigated the utility of electronic decision support alerts in diagnostic stewardship for Clostridioides difficile infection (CDI). However, it is unclear if alerts are effective in reducing inappropriate CDI testing and/or CDI rates. The aim of this systematic review was to determine if alerts related to CDI diagnostic stewardship are effective at reducing inappropriate CDI testing volume and CDI rates among hospitalized adult patients.

METHODS

We searched Ovid Medline and 5 other databases for original studies evaluating the association between alerts for CDI diagnosis and CDI testing volume and/or CDI rate. Two investigators independently extracted data on study characteristics, study design, alert triggers, cointerventions, and study outcomes.

RESULTS

Eleven studies met criteria for inclusion. Studies varied significantly in alert triggers and in study outcomes. Six of 11 studies demonstrated a statistically significant decrease in CDI testing volume, 6 of 6 studies evaluating appropriateness of CDI testing found a significant reduction in the proportion of inappropriate testing, and 4 of 7 studies measuring CDI rate demonstrated a significant decrease in the CDI rate in the postintervention vs preintervention period. The magnitude of the increase in appropriate CDI testing varied, with some studies reporting an increase with minimal clinical significance.

CONCLUSIONS

The use of electronic alerts for diagnostic stewardship for C. difficile was associated with reductions in CDI testing, the proportion of inappropriate CDI testing, and rates of CDI in most studies. However, broader concerns related to alerts remain understudied, including unintended adverse consequences and alert fatigue.

Integrated displays to improve chronic disease management in ambulatory care: A SMART on FHIR application informed by mixed-methods user testing

OBJECTIVE

The study sought to evaluate a novel electronic health record (EHR) add-on application for chronic disease management that uses an integrated display to decrease user cognitive load, improve efficiency, and support clinical decision making.

MATERIALS AND METHODS

We designed a chronic disease management application using the technology framework known as SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources). We used mixed methods to obtain user feedback on a prototype to support ambulatory providers managing chronic obstructive pulmonary disease. Each participant managed 2 patient scenarios using the regular EHR with and without access to our prototype in block-randomized order. The primary outcome was the percentage of expert-recommended ideal care tasks completed. Timing, keyboard and mouse use, and participant surveys were also collected. User experiences were captured using a retrospective think-aloud interview analyzed by concept coding.

RESULTS

With our prototype, the 13 participants completed more recommended care (81% vs 48%; P < .001) and recommended tasks per minute (0.8 vs 0.6; P = .03) over longer sessions (7.0 minutes vs 5.4 minutes; P = .006). Keystrokes per task were lower with the prototype (6 vs 18; P < .001). Qualitative themes elicited included the desire for reliable presentation of information which matches participants' mental models of disease and for intuitive navigation in order to decrease cognitive load.

DISCUSSION

Participants completed more recommended care by taking more time when using our prototype. Interviews identified a tension between using the inefficient but familiar EHR vs learning to use our novel prototype. Concept coding of user feedback generated actionable insights.

CONCLUSIONS

Mixed methods can support the design and evaluation of SMART on FHIR EHR add-on applications by enhancing understanding of the user experience.

Impact of Opioid Restrictions During a Critical Drug Shortage Period: Interrupted Time Series for Institutional Opioid Utilization

OBJECTIVES

This study aimed to evaluate the impact of intravenous opioid product restrictions at an academic medical institution in an urban setting during the time of critical opioid shortages. We assessed the effect of ordering restrictions on inpatient opioid utilization measured by 1) changes in intermittent oral and injectable opioid product administration; 2) changes in total institutional opioid administration; and 3) changes in the utilization of individual restricted opioid agents.

METHODS

This study is a single-center retrospective analysis by interrupted time series of institutional opioid utilization from 07/2017 to 06/2018. Utilization was quantified using milligrams of intravenous morphine equivalent administered or dispensed per admitted patient. Restrictions were grouped into 10 distinct phases, which informed the interruptions in linear regression models.

RESULTS

Sequential restrictions during the study period led to shifts in use of individual agents but did not have a significant impact on overall total opioid utilization. "Soft" restrictions did not have a direct, statistically significant impact on medication use but did decrease utilization over time. In situations where a product was restricted with a "soft stop" followed by a "hard stop," the "hard stop" directly reduced usage.

CONCLUSIONS

Targeted ordering restrictions allowed the institution to redirect drug use according to clinical need without affecting the overall utilization. Clinical decision support led providers to choose therapeutically equivalent alternatives. The demonstrated effect of restrictions will guide institutions in the selection of "hard stop" or "soft stop" restrictions in response to future shortages.

Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review

OBJECTIVE

Alert fatigue limits the effectiveness of medication safety alerts, a type of computerized clinical decision support (CDS). Researchers have suggested alternative interactive designs, as well as tailoring alerts to clinical roles. As examples, alerts may be tiered to convey risk, and certain alerts may be sent to pharmacists. We aimed to evaluate which variants elicit less alert fatigue.

MATERIALS AND METHODS

We searched for articles published between 2007 and 2017 using the PubMed, Embase, CINAHL, and Cochrane databases. We included articles documenting peer-reviewed empirical research that described the interactive design of a CDS system, to which clinical role it was presented, and how often prescribers accepted the resultant advice. Next, we compared the acceptance rates of conventional CDS-presenting prescribers with interruptive modal dialogs (ie, "pop-ups")-with alternative designs, such as role-tailored alerts.

RESULTS

Of 1011 articles returned by the search, we included 39. We found different methods for measuring acceptance rates; these produced incomparable results. The most common type of CDS-in which modals interrupted prescribers-was accepted the least often. Tiering by risk, providing shortcuts for common corrections, requiring a reason to override, and tailoring CDS to match the roles of pharmacists and prescribers were the most common alternatives. Only 1 alternative appeared to increase prescriber acceptance: role tailoring. Possible reasons include the importance of etiquette in delivering advice, the cognitive benefits of delegation, and the difficulties of computing "relevance."

CONCLUSIONS

Alert fatigue may be mitigated by redesigning the interactive behavior of CDS and tailoring CDS to clinical roles. Further research is needed to develop alternative designs, and to standardize measurement methods to enable meta-analyses.

Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening

OBJECTIVE

Evaluation of the effect of six optimization strategies in a clinical decision support system (CDSS) for drug-drug interaction (DDI) screening on alert burden and alert acceptance and description of clinical pharmacist intervention acceptance.

METHODS

Optimizations in the new CDSS were the customization of the knowledge base (with addition of 67 extra DDIs and changes in severity classification), a new alert design, required override reasons for the most serious alerts, the creation of DDI-specific screening intervals, patient-specific alerting, and a real-time follow-up system of all alerts by clinical pharmacists with interventions by telephone was introduced. The alert acceptance was evaluated both at the prescription level (i.e. prescription acceptance, was the DDI prescribed?) and at the administration level (i.e. administration acceptance, did the DDI actually take place?). Finally, the new follow-up system was evaluated by assessing the acceptance of clinical pharmacist's interventions.

RESULTS

In the pre-intervention period, 1087 alerts (92.0 % level 1 alerts) were triggered, accounting for 19 different DDIs. In the post-intervention period, 2630 alerts (38.4 % level 1 alerts) were triggered, representing 86 different DDIs. The relative risk forprescription acceptance in the post-intervention period compared to the pre-intervention period was 4.02 (95 % confidence interval (CI) 3.17-5.10; 25.5 % versus 6.3 %). The relative risk for administration acceptance was 1.16 (95 % CI 1.08-1.25; 54.4 % versus 46.7 %). Finally, 86.9 % of the clinical pharmacist interventions were accepted.

CONCLUSION

Six concurrently implemented CDSS optimization strategies resulted in a high alert acceptance and clinical pharmacist intervention acceptance. Administration acceptance was remarkably higher than prescription acceptance.

Health informatics and health equity: improving our reach and impact

Health informatics studies the use of information technology to improve human health. As informaticists, we seek to reduce the gaps between current healthcare practices and our societal goals for better health and healthcare quality, safety, or cost. It is time to recognize health equity as one of these societal goals-a point underscored by this Journal of the American Medical Informatics Association Special Focus Issue, "Health Informatics and Health Equity: Improving our Reach and Impact." This Special Issue highlights health informatics research that focuses on marginalized and underserved groups, health disparities, and health equity. In particular, this Special Issue intentionally showcases high-quality research and professional experiences that encompass a broad range of subdisciplines, methods, marginalized populations, and approaches to disparities. Building on this variety of submissions and other recent developments, we highlight contents of the Special Issue and offer an assessment of the state of research at the intersection of health informatics and health equity.

Impact of Electronic Medical Record Alerts on Emergency Physician Workflow and Medical Management

BACKGROUND

Electronic medical record (EMR) alerts are automated messages that notify the physician of important information. However, little is known about how EMR alerts affect the workflow and decision-making of emergency physicians (EPs).

STUDY OBJECTIVES

This study aimed to quantify the number of EMR alerts EPs receive, the time required to resolve alerts, the types of alerts EPs receive, and the impact of alerts on patient management.

METHODS

We performed a prospective observational study at a tertiary care ED with 130,000 visits annually. Research assistants observed EPs on shift from May to December 2018. They recorded the number of EMR alerts received, time spent addressing the alerts, the types of alerts received, and queried the EP to determine if the alert impacted patient management.

RESULTS

Seven residents and six attending physicians were observed on a total of 17 shifts and 153 patient encounters; 78% (119) of patient encounters involved alerts. These 119 patients triggered 530 EMR alerts. EPs spent a mean of 7.06 s addressing each alert and addressed 3.46 alerts per total patient seen. In total, EPs spent approximately 24 s per patient resolving alerts. Only 12 alerts (2.26%) changed clinical management.

CONCLUSION

EPs frequently receive EMR alerts, however, most alerts were not perceived to impact patient care. These alerts contribute to the high volume of interruptions EPs must contend with in the clinical environment of the ED.

Machine Learning Approach to Reduce Alert Fatigue Using a Disease Medication-Related Clinical Decision Support System: Model Development and Validation

Background

Computerized physician order entry (CPOE) systems are incorporated into clinical decision support systems (CDSSs) to reduce medication errors and improve patient safety. Automatic alerts generated from CDSSs can directly assist physicians in making useful clinical decisions and can help shape prescribing behavior. Multiple studies reported that approximately 90%-96% of alerts are overridden by physicians, which raises questions about the effectiveness of CDSSs. There is intense interest in developing sophisticated methods to combat alert fatigue, but there is no consensus on the optimal approaches so far.

Objective

Our objective was to develop machine learning prediction models to predict physicians’ responses in order to reduce alert fatigue from disease medication–related CDSSs.

Methods

We collected data from a disease medication–related CDSS from a university teaching hospital in Taiwan. We considered prescriptions that triggered alerts in the CDSS between August 2018 and May 2019. Machine learning models, such as artificial neural network (ANN), random forest (RF), naïve Bayes (NB), gradient boosting (GB), and support vector machine (SVM), were used to develop prediction models. The data were randomly split into training (80%) and testing (20%) datasets.

Results

A total of 6453 prescriptions were used in our model. The ANN machine learning prediction model demonstrated excellent discrimination (area under the receiver operating characteristic curve [AUROC] 0.94; accuracy 0.85), whereas the RF, NB, GB, and SVM models had AUROCs of 0.93, 0.91, 0.91, and 0.80, respectively. The sensitivity and specificity of the ANN model were 0.87 and 0.83, respectively.

Conclusions

In this study, ANN showed substantially better performance in predicting individual physician responses to an alert from a disease medication–related CDSS, as compared to the other models. To our knowledge, this is the first study to use machine learning models to predict physician responses to alerts; furthermore, it can help to develop sophisticated CDSSs in real-world clinical settings.

Abnormal Vital Sign Recognition and Provider Notification in the Pediatric Emergency Department

INTRODUCTION

Vital signs measurements aid in the early identification of patients at risk of clinical deterioration and determining the severity of illness. Health care providers rely on registered nurses to document vital signs and communicate abnormalities. The purpose of this project was to improve the provider notification process regarding abnormal vital signs in a pediatric emergency department.

METHOD

A best practice advisory (BPA) was piloted by the advanced practice providers in the pediatric emergency department. To evaluate the effects of the BPA, a mixed-methods study was employed.

RESULTS

Implementation of the BPA improved the provider notification process and enhanced clinical decision making. The percentage of patients discharged home with abnormal respiratory rates (10.9% vs. 5.9%, p = .31), abnormal temperatures (15.6% vs. 7.5%, p = .14), and abnormal heart rates (25% vs. 11.9%, p = .11) improved.

DISCUSSION

Creation and implementation of the BPA improved the abnormal vital sign communication process to providers at this single institution.

Reducing Alert Fatigue by Sharing Low-Level Alerts With Patients and Enhancing Collaborative Decision Making Using Blockchain Technology: Scoping Review and Proposed Framework (MedAlert)

Background

Clinical decision support (CDS) is a tool that helps clinicians in decision making by generating clinical alerts to supplement their previous knowledge and experience. However, CDS generates a high volume of irrelevant alerts, resulting in alert fatigue among clinicians. Alert fatigue is the mental state of alerts consuming too much time and mental energy, which often results in relevant alerts being overridden unjustifiably, along with clinically irrelevant ones. Consequently, clinicians become less responsive to important alerts, which opens the door to medication errors.

Objective

This study aims to explore how a blockchain-based solution can reduce alert fatigue through collaborative alert sharing in the health sector, thus improving overall health care quality for both patients and clinicians.

Methods

We have designed a 4-step approach to answer this research question. First, we identified five potential challenges based on the published literature through a scoping review. Second, a framework is designed to reduce alert fatigue by addressing the identified challenges with different digital components. Third, an evaluation is made by comparing MedAlert with other proposed solutions. Finally, the limitations and future work are also discussed.

Results

Of the 341 academic papers collected, 8 were selected and analyzed. MedAlert securely distributes low-level (nonlife-threatening) clinical alerts to patients, enabling a collaborative clinical decision. Among the solutions in our framework, Hyperledger (private permissioned blockchain) and BankID (federated digital identity management) have been selected to overcome challenges such as data integrity, user identity, and privacy issues.

Conclusions

MedAlert can reduce alert fatigue by attracting the attention of patients and clinicians, instead of solely reducing the total number of alerts. MedAlert offers other advantages, such as ensuring a higher degree of patient privacy and faster transaction times compared with other frameworks. This framework may not be suitable for elderly patients who are not technology savvy or in-patients. Future work in validating this framework based on real health care scenarios is needed to provide the performance evaluations of MedAlert and thus gain support for the better development of this idea.

Use of machine learning in geriatric clinical care for chronic diseases: a systematic literature review

Objectives

Geriatric clinical care is a multidisciplinary assessment designed to evaluate older patients’ (age 65 years and above) functional ability, physical health, and cognitive well-being. The majority of these patients suffer from multiple chronic conditions and require special attention. Recently, hospitals utilize various artificial intelligence (AI) systems to improve care for elderly patients. The purpose of this systematic literature review is to understand the current use of AI systems, particularly machine learning (ML), in geriatric clinical care for chronic diseases.

Materials and Methods

We restricted our search to eight databases, namely PubMed, WorldCat, MEDLINE, ProQuest, ScienceDirect, SpringerLink, Wiley, and ERIC, to analyze research articles published in English between January 2010 and June 2019. We focused on studies that used ML algorithms in the care of geriatrics patients with chronic conditions.

Results

We identified 35 eligible studies and classified in three groups: psychological disorder (n = 22), eye diseases (n = 6), and others (n = 7). This review identified the lack of standardized ML evaluation metrics and the need for data governance specific to health care applications.

Conclusion

More studies and ML standardization tailored to health care applications are required to confirm whether ML could aid in improving geriatric clinical care.

Unique Needs for the Implementation of Emergency Department Human Immunodeficiency Virus Screening in Adolescents

BACKGROUND

The Centers for Disease Control and Prevention (CDC) recommend universal human immunodeficiency virus (HIV) screening starting at 13 years, which has been implemented in many general U.S. emergency departments (EDs) but infrequently in pediatric EDs. We aimed to 1) implement a pilot of routine adolescent HIV screening in a pediatric ED and 2) determine the unique barriers to CDC-recommended screening in this region of high HIV prevalence.

METHODS

This was a prospective 4-month implementation of a routine HIV screening pilot in a convenience sample of adolescents 13 to 18 years at a single pediatric ED, based on study personnel availability. Serum-based fourth-generation HIV testing was run through a central laboratory. Parents were allowed to remain in the room for HIV counseling and testing. Data were collected regarding patient characteristics and HIV testing quality metrics. Comparisons were made using chi-square and Fisher's exact tests. Regression analysis was performed to assess for an association between parent presence at the time of enrollment and adolescent decision to participate in HIV screening.

RESULTS

Over 4 months, 344 of 806 adolescents approached consented to HIV screening (57% female, mean ± SD = 15.1 ± 1.6 years). Adolescents with HIV screening were more likely to be older than those who declined (p = 0.025). Other blood tests were collected with the HIV sample for 21% of adolescents; mean time to result was 105 minutes (interquartile range = 69 to 123) and 79% were discharged before the result was available. Having a parent present for enrollment was not associated with adolescent participation (adjusted odds ratio = 1.07, 95% CI = 0.67 to 1.70). Barriers to testing included: fear of needlestick, time to results, cost, and staff availability. One of 344 tests was positive in a young adolescent with Stage 1 HIV.

CONCLUSIONS

Routine HIV screening in adolescents was able to be implemented in this pediatric ED and led to the identification of early infection in a young adolescent who would have otherwise been undetected at this stage of disease. Addressing the unique barriers to adolescent HIV screening is critical in high-prevalence regions and may lead to earlier diagnosis and treatment in this vulnerable population.

Economic impact of clinical decision support interventions based on electronic health records

Background

Unnecessary healthcare utilization, non-adherence to current clinical guidelines, or insufficient personalized care are perpetual challenges and remain potential major cost-drivers for healthcare systems around the world. Implementing decision support systems into clinical care is promised to improve quality of care and thereby yield substantial effects on reducing healthcare expenditure. In this article, we evaluate the economic impact of clinical decision support (CDS) interventions based on electronic health records (EHR).

Methods

We searched for studies published after 2014 using MEDLINE, CENTRAL, WEB OF SCIENCE, EBSCO, and TUFTS CEA registry databases that encompass an economic evaluation or consider cost outcome measures of EHR based CDS interventions. Thereupon, we identified best practice application areas and categorized the investigated interventions according to an existing taxonomy of front-end CDS tools.

Results and discussion

Twenty-seven studies are investigated in this review. Of those, twenty-two studies indicate a reduction of healthcare expenditure after implementing an EHR based CDS system, especially towards prevalent application areas, such as unnecessary laboratory testing, duplicate order entry, efficient transfusion practice, or reduction of antibiotic prescriptions. On the contrary, order facilitators and undiscovered malfunctions revealed to be threats and could lead to new cost drivers in healthcare. While high upfront and maintenance costs of CDS systems are a worldwide implementation barrier, most studies do not consider implementation cost. Finally, four included economic evaluation studies report mixed monetary outcome results and thus highlight the importance of further high-quality economic evaluations for these CDS systems.

Conclusion

Current research studies lack consideration of comparative cost-outcome metrics as well as detailed cost components in their analyses. Nonetheless, the positive economic impact of EHR based CDS interventions is highly promising, especially with regard to reducing waste in healthcare.

Interrupted Time Series of User-centered Clinical Decision Support Implementation for Emergency Department-initiated Buprenorphine for Opioid Use Disorder

OBJECTIVES

Adoption of emergency department (ED) initiation of buprenorphine (BUP) for opioid use disorder (OUD) into routine emergency care has been slow, partly due to clinicians' unfamiliarity with this practice and perceptions that it is complicated and time-consuming. To address these barriers and guide emergency clinicians through the process of BUP initiation, we implemented a user-centered computerized clinical decision support system (CDS). This study was conducted to assess the feasibility of implementation and to evaluate the preliminary efficacy of the intervention to increase the rate of ED-initiated BUP.

METHODS

An interrupted time series study was conducted in an urban, academic ED from April 2018 to February 2019 (preimplementation phase), March 2019 to August 2019 (implementation phase), and September 2019 to December 2019 (maintenance phase) to study the effect of the intervention on adult ED patients identified by a validated electronic health record (EHR)-based computable phenotype consisting of structured data consistent with potential cases of OUD who would benefit from BUP treatment. The intervention offers flexible CDS for identification of OUD, assessment of opioid withdrawal, and motivation of readiness to start treatment and automates EHR activities related to ED initiation of BUP (including documentation, orders, prescribing, and referral). The primary outcome was the rate of ED-initiated BUP. Secondary outcomes were launch of the intervention, prescription for naloxone at ED discharge, and referral for ongoing addiction treatment.

RESULTS

Of the 141,041 unique patients presenting to the ED over the preimplementation and implementation phases (i.e., the phases used in primary analysis), 906 (574 preimplementation and 332 implementation) met OUD phenotype and inclusion criteria. The rate of BUP initiation increased from 3.5% (20/574) in the preimplementation phase to 6.6% (22/332) in the implementation phase (p = 0.03). After the temporal trend of the number of physician's with X-waiver training and other covariates were adjusted for, the relative risk of BUP initiation rate was 2.73 (95% confidence interval [CI] = 0.62 to 12.0, p = 0.18). Similarly, the number of unique attendings who initiated BUP increased modestly 7/53 (13.0%) to 13/57 (22.8%, p = 0.10) after offering just-in-time training during the implementation period. The rate of naloxone prescribed at discharge also increased (6.5% preimplementation and 11.5% implementation; p < 0.01). The intervention received a system usability scale score of 82.0 (95% CI = 76.7 to 87.2).

CONCLUSION

Implementation of user-centered CDS at a single ED was feasible, acceptable, and associated with increased rates of ED-initiated BUP and naloxone prescribing in patients with OUD and a doubling of the number of unique physicians adopting the practice. We have implemented this intervention across several health systems in an ongoing trial to assess its effectiveness, scalability, and generalizability.

Clinical Decision Support and Implications for the Clinician Burnout Crisis

OBJECTIVES

This survey aimed to review aspects of clinical decision support (CDS) that contribute to burnout and identify key themes for improving the acceptability of CDS to clinicians, with the goal of decreasing said burnout.

METHODS

We performed a survey of relevant articles from 2018-2019 addressing CDS and aspects of clinician burnout from PubMed and Web of Science™. Themes were manually extracted from publications that met inclusion criteria.

RESULTS

Eighty-nine articles met inclusion criteria, including 12 review articles. Review articles were either prescriptive, describing how CDS should work, or analytic, describing how current CDS tools are deployed. The non-review articles largely demonstrated poor relevance and acceptability of current tools, and few studies showed benefits in terms of efficiency or patient outcomes from implemented CDS. Encouragingly, multiple studies highlighted steps that succeeded in improving both acceptability and relevance of CDS.

CONCLUSIONS

CDS can contribute to clinician frustration and burnout. Using the techniques of improving relevance, soliciting feedback, customization, measurement of outcomes and metrics, and iteration, the effects of CDS on burnout can be ameliorated.

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

BACKGROUND

Artificial intelligence (AI) provides opportunities to identify the health risks of patients and thus influence patient safety outcomes.

OBJECTIVE

The purpose of this systematic literature review was to identify and analyze quantitative studies utilizing or integrating AI to address and report clinical-level patient safety outcomes.

METHODS

We restricted our search to the PubMed, PubMed Central, and Web of Science databases to retrieve research articles published in English between January 2009 and August 2019. We focused on quantitative studies that reported positive, negative, or intermediate changes in patient safety outcomes using AI apps, specifically those based on machine-learning algorithms and natural language processing. Quantitative studies reporting only AI performance but not its influence on patient safety outcomes were excluded from further review.

RESULTS

We identified 53 eligible studies, which were summarized concerning their patient safety subcategories, the most frequently used AI, and reported performance metrics. Recognized safety subcategories were clinical alarms (n=9; mainly based on decision tree models), clinical reports (n=21; based on support vector machine models), and drug safety (n=23; mainly based on decision tree models). Analysis of these 53 studies also identified two essential findings: (1) the lack of a standardized benchmark and (2) heterogeneity in AI reporting.

CONCLUSIONS

This systematic review indicates that AI-enabled decision support systems, when implemented correctly, can aid in enhancing patient safety by improving error detection, patient stratification, and drug management. Future work is still needed for robust validation of these systems in prospective and real-world clinical environments to understand how well AI can predict safety outcomes in health care settings.

Appropriateness of Overridden Alerts in Computerized Physician Order Entry: Systematic Review

BACKGROUND

The clinical decision support system (CDSS) has become an indispensable tool for reducing medication errors and adverse drug events. However, numerous studies have reported that CDSS alerts are often overridden. The increase in override rates has raised questions about the appropriateness of CDSS application along with concerns about patient safety and quality of care.

OBJECTIVE

The aim of this study was to conduct a systematic review to examine the override rate, the reasons for the alert override at the time of prescribing, and evaluate the appropriateness of overrides.

METHODS

We searched electronic databases, including Google Scholar, PubMed, Embase, Scopus, and Web of Science, without language restrictions between January 1, 2000 and March 31, 2019. Two authors independently extracted data and crosschecked the extraction to avoid errors. The quality of the included studies was examined following Cochrane guidelines.

RESULTS

We included 23 articles in our systematic review. The range of average override alerts was 46.2%-96.2%. An average of 29.4%-100% of the overrides alerts were classified as appropriate, and the rate of appropriateness varied according to the alert type (drug-allergy interaction 63.4%-100%, drug-drug interaction 0%-95%, dose 43.9%-88.8%, geriatric 14.3%-57%, renal 27%-87.5%). The interrater reliability for the assessment of override alerts appropriateness was excellent (kappa=0.79-0.97). The most common reasons given for the override were "will monitor" and "patients have tolerated before."

CONCLUSIONS

The findings of our study show that alert override rates are high, and certain categories of overrides such as drug-drug interaction, renal, and geriatric were classified as inappropriate. Nevertheless, large proportions of drug duplication, drug-allergy, and formulary alerts were appropriate, suggesting that these groups of alerts can be primary targets to revise and update the system for reducing alert fatigue. Future efforts should also focus on optimizing alert types, providing clear information, and explaining the rationale of the alert so that essential alerts are not inappropriately overridden.

Studying Workflow and Workarounds in Electronic Health Record-Supported Work to Improve Health System Performance

Clinical workflow is the enactment of a series of steps to perform a clinical activity. The transition from paper to electronic health records (EHRs) over the past decade has been characterized by profound challenges supporting clinical workflow, impeding frontline clinicians' ability to deliver safe, efficient, and effective care. In response, there has been substantial effort to study clinical workflow as well as workarounds-exceptions to routine workflow-in order to identify opportunities for improvement. This article describes predominant methods of studying workflow and workarounds and provides examples of the applications of these methods along with the resulting insights. Challenges to studying workflow and workarounds are described, and recommendations for how to approach such studies are given. Although there is not yet a set of standard approaches, this article helps advance workflow research that ultimately serves to inform how to coevolve the design of EHR systems and organizational decisions about processes, roles, and responsibilities in order to support clinical workflow that more consistently delivers on the potential benefits of a digitized health care system.

Effective and efficient usage of big data analytics in public sector

Purpose

This study aims to achieve three goals: present a holistic, flexible and dynamic model; define the model’s factors and explain how these factors lead to effective and efficient usage of big data; and generate indexes based on experts’ input to rank them based on their importance.

Design/methodology/approach

This paper uses the analytic hierarchy process, a quantitative method of decision-making, to evaluate the importance of the factors presented in the model. The fundamental principle of the overall model is that of a dynamo which is borrowed from electromagnetic physics. The model is also based on three IS theories.

Findings

Technological advancements and data security are among the most important factors that may impact the effectiveness and efficiency of big data usage. Authentication, governments’ focus on it and transparency and accountability are the most important factors in techno-centric, governmental-centric and user-centric factors, respectively.

Research limitations/implications

The findings of this paper confirmed earlier findings in the literature and quantitatively assessed some of the factors that were conceptually presented. This paper also presented a framework that can be used in future studies.

Practical implications

Policy and decision-makers may need to upgrade pertinent technologies such as internet security, frame policies toward information technology (IT) and train the users.

Originality/value

This paper fills a gap in the literature by presenting a comprehensive study of how different factors dynamically contribute to the effective usage of big data in the public sector. It also quantitatively presents the importance of the factors based on the data collected from 12 IT experts.

Practice-enhancing publications about the medication-use process in 2018

PURPOSE

This article identifies, prioritizes, and summarizes published literature on the medication-use process (MUP) from calendar year 2018 that can impact health-system pharmacy daily practice. The MUP is the foundational system that provides the framework for safe medication utilization within the healthcare environment. The MUP is defined in this article as having the following steps: prescribing/transcribing, dispensing, administration, and monitoring. Articles that evaluated one of the steps were gauged for their usefulness toward daily practice change.

SUMMARY

A PubMed search was conducted in February 2019 for articles published in calendar year 2018 using targeted Medical Subject Headings (MeSH) keywords, targeted non-MeSH keywords, and the table of contents of selected pharmacy journals, providing a total of 43,977 articles. A thorough review identified 62 potentially significant articles: 9 for prescribing/transcribing, 12 for dispensing, 13 for administration, and 28 for monitoring. Ranking of the articles for importance by peers led to the selection of key articles from each category. The highest-ranked articles are briefly summarized, with a mention of why they are important within health-system pharmacy. The other articles are listed for further review and evaluation.

CONCLUSION

It is important to routinely review the published literature and to incorporate significant findings into daily practice. This article assists in identifying and summarizing recent impactful contributions to the MUP literature. Health-system pharmacists have an active role in improving the MUP in their institution, and awareness of significant published studies can assist in changing practice at the institutional level.

Pilot Study to Improve Goals of Care Conversations Among Hospitalists

CONTEXT

Many hospitalized patients receive care that is not concordant with their goals. Teaching communication skills that better align goals and treatment can improve the care that patients receive.

OBJECTIVE

To develop and test an innovative approach that encourages hospitalists to engage in goals of care (GOC) conversations with their patients.

METHODS

We recruited 14 hospitalists and randomized half to receive electronic health record alerts for patients who might benefit most from a goals-of-care conversation, as well as communication coaching. The coaching required an initial meeting, then audio recording of two GOC conversations and feedback from the coach. Outcomes were the presence of GOC conversations (primary), the quality of the GOC conversations, physician perceptions of the intervention, and hospital metrics (e.g., 30-day readmissions, referrals to palliative care).

RESULTS

We did not increase the frequency of GOC conversations but did improve the quality of the conversations. Patients of physicians who received the intervention had fewer 30-day readmission rates and were less likely to die 90 days after admission than patients of physicians in the control arm. Patients of intervention physicians also had fewer palliative care consults than patients of control physicians.

CONCLUSIONS

Teaching hospitalists to have GOC conversations translated into better skills and outcomes for patients. This pilot study shows promise and should be tested in a larger trial.

Structured override reasons for drug-drug interaction alerts in electronic health records

OBJECTIVE

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

MATERIALS AND METHODS

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

RESULTS

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: "will monitor or take precautions," "not clinically significant," and "benefit outweighs risk."

DISCUSSION

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

CONCLUSIONS

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Appropriate utilisation of cardiac telemetry monitoring: a quality improvement project

For hospitals located in the United States, appropriate use of cardiac telemetry monitoring can be achieved resulting in cost savings to healthcare systems. Our institution has a limited number of telemetry beds, increasing the need for appropriate use of telemetry monitoring to minimise delays in patient care, reduce alarm fatigue, and decrease interruptions in patient care. This quality improvement project was conducted in a single academic medical centre in Kansas City, Kansas. The aim of the project was to reduce inappropriate cardiac telemetry monitoring on intermediate care units. Using the 2004 American Heart Association guidelines to guide appropriate telemetry utilisation, this project team sought to investigate the effects of two distinct interventions to reduce inappropriate telemetry monitoring, huddle intervention and mandatory order entry. Telemetry utilisation was followed prospectively for 2 years. During our initial intervention, we achieved a sharp decline in the number of patients on telemetry monitoring. However, over time the efficacy of the huddle intervention subsided, resulting in a need for a more sustained approach. By requiring physicians to input indication for telemetry monitoring, the second intervention increased adherence to practice guidelines and sustained reductions in inappropriate telemetry use.