Technologic Distractions (Part 1): Summary of Approaches to Manage Alert Quantity With Intent to Reduce Alert Fatigue and Suggestions for Alert Fatigue Metrics

Clinical decision support as an implementation strategy to expand identification and administration of treatment of opioid use disorder in the emergency department

INTRODUCTION

US opioid overdoses and deaths continue to increase, despite historic national investment to mitigate risk and improve access to evidence-based treatment. Unfortunately, implementation of emergency department (ED) buprenorphine - an effective medical treatment for opioid use disorder (OUD) - has been limited. Our objective was to assess the effectiveness of an electronic health record (EHR)-integrated, interruptive clinical decision support (CDS) tool to improve rates of ED initiated OUD treatment.

METHODS

This is an observational, pre-post study of a CDS tool designed to identify and facilitate treatment of patients with OUD using electronic health record data. Patients were included if treated at our urban, academic ED between May 1, 2022, and November 8, 2023. The CDS triggered based on a rules-based algorithm using routinely collected EHR data which were identified from a previously validated EHR OUD phenotype. Outcomes are organized under a modified RE-AIM framework, with the primary outcome, Effectiveness, measured by the proportion of OUD patients receiving buprenorphine (administered/prescribed; filled prescriptions). Secondary outcomes include patient Reach, clinician Adoption, and fidelity to Implementation. Chi Square tests and Bayesian structural time-series models evaluate differences in outcomes before and after CDS implementation (CausalImpact package v1.3.0 in R v4.4.0).

RESULTS

There were 171,221 total ED visits during the study period. Patient characteristics before and after CDS implementation were similar. CDS triggered in 4.7 % (2754/58,173) of encounters after initiation of intervention, reaching 116 unique emergency medicine providers and 2566 ED patients. Clinicians adopted the CDS, accessing the OUD treatment pathway link or ordering a social work consult for substance use, in 27 % (1266/4746) of CDS alerts. When compared to the pre-implementation period, CDS implementation was associated with increased buprenorphine administration in the ED by 31 % (95 % CI: 16-47 %, p = 0.001), buprenorphine prescribing from the ED by 20 % (95 % CI: 5-38 %, p = 0.007), and the buprenorphine fill rate at an affiliated ED pharmacy by 17 % (95 % CI: 1-36 %, p = 0.017).

CONCLUSIONS

Implementation of an EHR-integrated, CDS was associated with increased ED buprenorphine administration, prescribing, and prescription fills among ED patients with OUD. Further efforts are needed to assess maintenance strategies that improve adoption, minimize interruptiveness, and optimize workflow congruence.

Barriers to dental providers' use of a clinical decision support tool for pain management following tooth extractions

Background

De-implementing non-effective or even harmful practices in healthcare is sometimes necessary, as has been the case with opioid prescribing in dentistry over the past decade. One approach to practice transformation is to deploy clinical decision support (CDS) tools. This qualitative study examined barriers to CDS use as part of a cluster randomized trial that aimed to decrease opioid prescribing for pain management following tooth extractions across a large dental practice.

Method

Twenty dental providers who took part in the larger randomized trial were purposively selected to complete a semi-structured qualitative interview. Participants represented a broad range in terms of years of practice, dental specialization, and CDS use patterns. Interviews were conducted via Zoom, audio recorded, transcribed, and analyzed using a content analysis approach in ATLAS.ti following participation in the cluster randomized trial.

Results

Reasons for not using the CDS fell generally into two broad categories: unintentional (i.e., forgetting to use the CDS) and intentional. Providers who forgot to use the CDS after training and implementation either were not sure where to look for the alert on the screen or did not remember to look for it because its use was never incorporated into their workflow. Reasons for deciding not to use the CDS included feeling that it slowed down their workflow, thinking that the information it provided would not be useful, and not trusting the functionality of the system.

Conclusions

There were numerous, interdependent human, organizational, and technological factors that influenced the intentionally and unintentionally low CDS use rates observed in the study. Findings highlight issues to be aware of and address in future implementation efforts that utilize CDS.

Trial registration

Clinicaltrials.gov NCT03584789.

Limitations of a platelet count-based clinical decision support system to facilitate diagnosis of heparin-induced thrombocytopenia

Heparin-induced thrombocytopenia (HIT) is a rare complication of heparin exposure with potential for significant morbidity and mortality. Early identification and treatment can prevent catastrophic thrombosis. Herein, we report the performance of a platelet count-based clinical decision support system (CDSS) where providers received a notification when a patient had a platelet count decline of ≥50 %. In the 90-day study period, the CDSS sent 302 notifications on 270 patients. Notifications were frequently inappropriate; 25 % had an expected platelet count decline (organ donation, stem cell transplant), an inaccurate count, or no heparin exposure. Patient testing for HIT prompted by the CDSS was not in accordance with best practice guidelines in most circumstances. For example, 36 % had a low probability 4Ts score, while 42 % with an intermediate or high probability 4Ts score were not tested. Due to concern for lack of efficacy, the CDSS was discontinued. Analysis of an 8-month period before and after discontinuation showed a significant decrease in the number of enzyme immunoassays ordered (547 vs. 386) without a change in the number of patients with HIT identified (13 vs. 13) or the rate of thrombosis in those with confirmed HIT (62 % vs. 62 %). In conclusion, a CDSS based on platelet count decline contributed to "alert fatigue" via inappropriate notification and did not improve evidence-based HIT testing. In addition, its removal did not decrease or delay HIT identification. Additional efforts are needed to better define how CDSS can support the rapid diagnosis and appropriate treatment of patients with HIT.

10 tips on how to use dynamic risk assessment and alerts for AKI

Acute kidney injury (AKI) is a common syndrome in hospitalized patients and is associated with increased morbidity and mortality. The focus of AKI care requires a shift away from strictly supportive management of established injury to the early identification and timely prevention of worsening renal injury. Identifying patients at risk for developing or progression of severe AKI is crucial for improving patient outcomes, reducing the length of hospitalization and minimizing resource utilization. Implementation of dynamic risk scores and incorporation of novel biomarkers show promise for early detection and minimizing progression of AKI. Like any risk assessment tools, these require further external validation in a variety of clinical settings prior to widespread implementation. Additionally, alerts that may minimize exposure to a variety of nephrotoxic medications or prompt early nephrology consultation are shown to reduce the incidence and progression of AKI severity and enhance renal recovery. While dynamic risk scores and alerts are valuable, implementation requires thoughtfulness and should be used in conjunction with the overall clinical picture in certain situations, particularly when considering the initiation of fluid and diuretic administration or renal replacement therapy. Despite the contemporary challenges encountered with alert fatigue, implementing an alert-based bundle to improve AKI care is associated with improved outcomes, even when implementation is incomplete. Lastly, all alert-based interventions should be validated at an institutional level and assessed for their ability to improve institutionally relevant and clinically meaningful outcomes, reduce resource utilization and provide cost-effective interventions.

Exploring home healthcare clinicians' needs for using clinical decision support systems for early risk warning

OBJECTIVES

To explore home healthcare (HHC) clinicians' needs for Clinical Decision Support Systems (CDSS) information delivery for early risk warning within HHC workflows.

METHODS

Guided by the CDS "Five-Rights" framework, we conducted semi-structured interviews with multidisciplinary HHC clinicians from April 2023 to August 2023. We used deductive and inductive content analysis to investigate informants' responses regarding CDSS information delivery.

RESULTS

Interviews with thirteen HHC clinicians yielded 16 codes mapping to the CDS "Five-Rights" framework (right information, right person, right format, right channel, right time) and 11 codes for unintended consequences and training needs. Clinicians favored risk levels displayed in color-coded horizontal bars, concrete risk indicators in bullet points, and actionable instructions in the existing EHR system. They preferred non-intrusive risk alerts requiring mandatory confirmation. Clinicians anticipated risk information updates aligned with patient's condition severity and their visit pace. Additionally, they requested training to understand the CDSS's underlying logic, and raised concerns about information accuracy and data privacy.

DISCUSSION

While recognizing CDSS's value in enhancing early risk warning, clinicians highlighted concerns about increased workload, alert fatigue, and CDSS misuse. The top risk factors identified by machine learning algorithms, especially text features, can be ambiguous due to a lack of context. Future research should ensure that CDSS outputs align with clinical evidence and are explainable.

CONCLUSION

This study identified HHC clinicians' expectations, preferences, adaptations, and unintended uses of CDSS for early risk warning. Our findings endorse operationalizing the CDS "Five-Rights" framework to optimize CDSS information delivery and integration into HHC workflows.

The Impact of Customized Screening Intervals on the Burden of Drug-Drug Interaction Alerts: An Interrupted Time Series Analysis

Fixed and broad screening intervals for drug-drug interaction (DDI) alerts lead to false positive alerts, thereby contributing to alert fatigue among healthcare professionals. Hence, we aimed to investigate the impact of customized screening intervals on the daily incidence of DDI alerts. An interrupted time series analysis was performed at the University Hospitals Leuven to evaluate the impact of a pragmatic intervention on the daily incidence of DDI alerts per 100 prescriptions. The study period encompassed 100 randomly selected days between April 2021 and December 2022. Preceding the intervention, a fixed and broad screening interval of 7 days before and after prescribing an interacting drug was applied. The intervention involved implementing customized screening intervals for a subset of highly prevalent or clinically relevant DDIs into the hospital information system. Additionally, the sensitivity of the tailored approach was evaluated. During the study period, a mean of 5731 (± 2909) new prescriptions per day was generated. The daily incidence of DDI alerts significantly decreased from 9.8% (95% confidence interval (CI) 8.4;11.1) before the intervention, to 6.3% (95% CI 5.4;7.2) afterwards, p < 0.0001. This corresponded to avoiding 201 (0.035*5731) false positive DDI alerts per day. Sensitivity was not compromised by our intervention. Defining and implementing customized screening intervals was feasible and effective in reducing the DDI alert burden without compromising sensitivity.

Clinical decision support to improve CBC and differential ordering

OBJECTIVES

Complete blood count and differential (CBC diff) is a common laboratory test that may be overused or misordered, particularly in an inpatient setting. We assessed the ability of a clinical decision support (CDS) alert to decrease unnecessary orders for CBC diff and analyzed its impact in the laboratory.

METHODS

We designed 3 CDS alerts to provide guidance to providers ordering CBC diff on inpatients at frequencies of daily, greater than once daily, or as needed.

RESULTS

The 3 alerts were highly effective in reducing orders for CBC diff at the frequencies targeted by the alert. Overall, test volume for CBC diff decreased by 32% (mean of 5257 tests per month) after implementation of the alerts, with a corresponding decrease of 22% in manual differentials performed (mean of 898 per month). Turnaround time for manual differentials decreased by a mean of 41.5 minutes, with a mean decrease of up to 90 minutes during peak morning hours.

CONCLUSIONS

The 3 CDS alerts successfully decreased inpatient orders for CBC diff and improved the quality of patient care by decreasing turnaround time for manual differentials.

Long-term seizure diary tracking habits in clinical studies: Evidence from the Human Epilepsy Project

OBJECTIVE

To characterize seizure tracking patterns of people with focal epilepsy using electronic seizure diary entries, and to assess for risk factors associated with poor tracking.

METHODS

We analyzed electronic seizure diary data from 410 participants with newly diagnosed focal epilepsy in the Human Epilepsy Project 1 (HEP1). Each participant was expected to record data each day during the study, regardless of seizure occurrence. The primary outcome of this post-hoc analysis was whether each participant properly tracked a seizure diary entry each day during their study participation. Using finite mixture modeling, we grouped patient tracking trajectories into data-driven clusters. Once defined, we used multinomial modeling to test for independent risk factors of tracking group membership.

RESULTS

Using over up to three years of daily seizure diary data per subject, we found four distinct seizure tracking groups: consistent, frequent at study onset, occasional, and rare. Participants in the consistent tracking group tracked a median of 92% (interquartile range, IQR: 82%, 99%) of expected days, compared to 47% (IQR:34%, 60%) in the frequent at study onset group, 37% (IQR: 26%, 49%) in the occasional group, and 9% (IQR: 3%, 15%) in the rare group. In multivariable analysis, consistent trackers had lower rates of seizure days per tracked year during their study participation, compared to other groups.

SIGNIFICANCE

Future efforts need to focus on improving seizure diary tracking adherence to improve quality of outcome data, particularly in those with higher seizure burden. In addition, accounting for missing data when using seizure diary data as a primary outcome is important in research trials. If not properly accounted for, total seizure burden may be underestimated and biased, skewing results of clinical trials.

Attention Among Health Care Professionals : A Scoping Review

BACKGROUND

The concept of attention can provide insight into the needs of clinicians and how health systems design can impact patient care quality and medical errors.

PURPOSE

To conduct a scoping review to 1) identify and characterize literature relevant to clinician attention; 2) compile metrics used to measure attention; and 3) create a framework of key concepts.

DATA SOURCES

Cumulated Index to Nursing and Allied Health Literature (CINAHL), Medline (PubMed), and Embase (Ovid) from 2001 to 26 February 2024.

STUDY SELECTION

English-language studies addressing health care worker attention in patient care. At least dual review and data abstraction.

DATA EXTRACTION

Article information, health care professional studied, practice environment, study design and intent, factor type related to attention, and metrics of attention used.

DATA SYNTHESIS

Of 6448 screened articles, 585 met inclusion criteria. Most studies were descriptive (n = 469) versus investigational (n = 116). More studies focused on barriers to attention (n = 387; 342 descriptive and 45 investigational) versus facilitators to improving attention (n = 198; 112 descriptive and 86 investigational). We developed a framework, grouping studies into 6 categories: 1) definitions of attention, 2) the clinical environment and its effect on attention, 3) personal factors affecting attention, 4) relationships between interventions or factors that affect attention and patient outcomes, 5) the effect of clinical alarms and alarm fatigue on attention, and 6) health information technology's effect on attention. Eighty-two metrics were used to measure attention.

LIMITATIONS

Does not synthesize answers to specific questions. Quality of studies was not assessed.

CONCLUSION

This overview may be a resource for researchers, quality improvement experts, and health system leaders to improve clinical environments. Future systematic reviews may synthesize evidence on metrics to measure attention and on the effectiveness of barriers or facilitators related to attention.

PRIMARY FUNDING SOURCE

None.

Extraneous Load, Patient Census, and Patient Acuity Correlate With Cognitive Load During ICU Rounds

BACKGROUND

Cognitive load theory asserts that learning and performance degrade when cognitive load exceeds working memory capacity. This is particularly relevant in the learning environment of ICU rounds, when multidisciplinary providers integrate complex decision-making and teaching in a noisy, high-stress environment prone to cognitive distractions.

RESEARCH QUESTION

What features of ICU rounds correlate with high provider cognitive load?

STUDY DESIGN AND METHODS

This was an observational, multisite study of multidisciplinary providers during ICU rounds. Investigators recorded rounding characteristics and hourly extraneous cognitive load events during rounds (defined as distractions, episodes of split-attention or repetition, and deviations from standard communication format). After rounds, investigators measured each provider's cognitive load using the provider task load (PTL), an instrument derived from the National Aeronautics and Space Administration Task Load Index survey that assesses perceived workload associated with complex tasks. Relationships between rounding characteristics, extraneous load, and PTL score were evaluated using mixed-effects modeling.

RESULTS

A total of 76 providers were observed during 32 rounds from December 2020 to May 2021. The mean rounding census ± SD was 12.5 ± 2.9 patients. The mean rounding time ± SD was 2 h 17 min ± 49 min. The mean extraneous load ± SD was 20.5 ± 4.5 events per hour, or one event every 2 min 51 s. This included 8.6 ± 3.4 distractions, 8.2 ± 4.2 communication deviations, 1.9 ± 1.4 repetitions, and 1.8 ± 1.3 episodes of split-attention per hour. Controlling for covariates, the hourly extraneous load events, number of new patients, and number of higher acuity patients were each associated with increased PTL score (slope, 2.40; 95% CI, 0.76-4.04; slope, 5.23; 95% CI, 2.02-8.43; slope, 3.35; 95% CI, 1.34-5.35, respectively).

INTERPRETATION

Increased extraneous load, new patients, and patient acuity were associated with higher cognitive load during ICU rounds. These results can help direct how the ICU rounding structure may be modified to reduce workload and optimize provider learning and performance.

Factors Influencing Medication Administration Outcomes Among New Graduate Nurses Using Bar Code-Assisted Medication Administration

Paramount to patient safety is the ability for nurses to make clinical decisions free from human error. Yet, the dynamic clinical environment in which nurses work is characterized by uncertainty, urgency, and high consequence, necessitating that nurses make quick and critical decisions. The aim of this study was to examine the influence of human and environmental factors on the decision to administer among new graduate nurses in response to alert generation during bar code-assisted medication administration. The design for this study was a descriptive, longitudinal, observational cohort design using EHR audit log and administrative data. The study was set at a large, urban medical center in the United States and included 132 new graduate nurses who worked on adult, inpatient units. Research variables included human and environmental factors. Data analysis included descriptive and inferential analyses. This study found that participants continued with administration of a medication in 90.75% of alert encounters. When considering the response to an alert, residency cohort, alert category, and previous exposure variables were associated with the decision to proceed with administration. It is important to continue to study factors that influence nurses' decision-making, particularly during the process of medication administration, to improve patient safety and outcomes.

Effect of Best Practice Alert (BPA) on Post-Discharge Opioid Prescribing After Minimally Invasive Hysterectomy: A Quality Improvement Study

Purpose

The aim of this study was to describe the effectiveness of an electronic health record best practice alert (BPA) in decreasing gynecologic post-discharge opioid prescribing following benign minimally invasive hysterectomy.

Patients and Methods

The BPA triggered for opioid orders >15 tablets. Prescribers' options included (1) decrease to 15 ≤ tablets; (2) remove the order/utilize a defaulted order set; or (3) override the alert.

Results

332 patients were included. The BPA triggered 29 times. The following actions were taken among 16 patients for whom the BPA triggered: "override the alert" (n=13); "cancel the alert" (n=2); and 'remove the opioid order set' (n=1). 12/16 patients had discharge prescriptions: one patient received 20 tablets; two received 10 tablets; and nine received 15 tablets. Top reasons for over prescribing included concerns for pain control and lack of alternatives.

Conclusion

Implementing a post-discharge opioid prescribing BPA aligned opioid prescribing following benign minimally invasive hysterectomy with guideline recommendations.

Post-implementation optimization of medication alerts in hospital computerized provider order entry systems: a scoping review

OBJECTIVES

A scoping review identified interventions for optimizing hospital medication alerts post-implementation, and characterized the methods used, the populations studied, and any effects of optimization.

MATERIALS AND METHODS

A structured search was undertaken in the MEDLINE and Embase databases, from inception to August 2023. Articles providing sufficient information to determine whether an intervention was conducted to optimize alerts were included in the analysis. Snowball analysis was conducted to identify additional studies.

RESULTS

Sixteen studies were identified. Most were based in the United States and used a wide range of clinical software. Many studies used inpatient cohorts and conducted more than one intervention during the trial period. Alert types studied included drug-drug interactions, drug dosage alerts, and drug allergy alerts. Six types of interventions were identified: alert inactivation, alert severity reclassification, information provision, use of contextual information, threshold adjustment, and encounter suppression. The majority of interventions decreased alert quantity and enhanced alert acceptance. Alert quantity decreased with alert inactivation by 1%-25.3%, and with alert severity reclassification by 1%-16.5% in 6 of 7 studies. Alert severity reclassification increased alert acceptance by 4.2%-50.2% and was associated with a 100% acceptance rate for high-severity alerts when implemented. Clinical errors reported in 4 studies were seen to remain stable or decrease.

DISCUSSION

Post-implementation medication optimization interventions have positive effects for clinicians when applied in a variety of settings. Less well reported are the impacts of these interventions on the clinical care of patients, and how endpoints such as alert quantity contribute to changes in clinician and pharmacist perceptions of alert fatigue.

CONCLUSION

Well conducted alert optimization can reduce alert fatigue by reducing overall alert quantity, improving clinical acceptance, and enhancing clinical utility.

Beyond the override: Using evidence of previous drug tolerance to suppress drug allergy alerts; a retrospective study of opioid alerts

OBJECTIVE

Despite the extensive literature exploring alert fatigue, most studies have focused on describing the phenomenon, but not on fixing it. The authors aimed to identify data useful to avert clinically irrelevant alerts to inform future research on clinical decision support (CDS) design.

METHODS

We conducted a retrospective observational study of opioid drug allergy alert (DAA) overrides for the calendar year of 2019 at a large academic medical center, to identify data elements useful to find irrelevant alerts to be averted.

RESULTS

Overall, 227,815 DAAs were fired in 2019, with an override rate of 91 % (n = 208196). Opioids represented nearly two-thirds of these overrides (n = 129063; 62 %) and were the drug class with the highest override rate (96 %). On average, 29 opioid DAAs were overridden per patient. While most opioid alerts (97.1 %) are fired for a possible match (the drug class of the allergen matches the drug class of the prescribed drug), they are overridden significantly less frequently for definite match (exact match between allergen and prescribed drug) (88 % vs. 95.9 %, p < 0.001). When comparing the triggering drug with previously administered drugs, override rates were equally high for both definite match (95.9 %), no match (95.5 %), and possible match (95.1 %). Likewise, when comparing to home medications, overrides were excessively high for possible match (96.3 %), no match (96 %), and definite match (94.4 %).

CONCLUSION

We estimate that 74.5% of opioid DAAs (46.4% of all DAAs) at our institution could be relatively safely averted, since they either have a definite match for previous inpatient administrations suggesting drug tolerance or are fired as possible match with low risk of cross-sensitivity. Future research should focus on identifying other relevant data elements ideally with automated methods and use of emerging standards to empower CDS systems to suppress false-positive alerts while avoiding safety hazards.

Evaluating the accuracy and efficacy of an electronic medical record alert to identify paediatric patients with low-risk febrile neutropenia

BACKGROUND

Point-of-care decision support, embedded into electronic medical record (EMR) workflows, has the potential to improve efficiency, reduce unwarranted variation and improve patient outcomes. A clinical-facing best practice advisory (BPA) in the Epic EMR system was developed to identify children admitted with low-risk febrile neutropenia (FN) who should be considered for treatment at home after a brief inpatient stay. We evaluated the accuracy and impact of this BPA and identify areas for improvement.

METHODS

The low-risk FN BPA was co-designed with key-stakeholders and implemented after a one-month testing phase. Mixed methodology was used to collect and analyse data. The sensitivity and positive predictive value of the BPA was calculated using FN episodes captured in a prospectively collected database. Overall effectiveness was defined as the proportion of alerts resulting in completion of a FN risk assessment flowsheet.

RESULTS

Over the 12-month period 176 FN episodes were admitted. Overall, the alert had poor sensitivity (58%) and positive predictive value (75%), failing to trigger in 62 (35%) episodes. In the episodes where the alert did trigger, the alert was frequently dismissed by clinicians (76%) and the overall effectiveness was extremely low (3%). Manual review of each FN episode without a BPA identified important design limitations and incorrect workflow assumptions.

DISCUSSION

Given the poor sensitivity and limited impact on clinician behaviour the low-risk BPA, in its current form, has not been an effective intervention at this site. While work is ongoing to enhance the accuracy of the BPA, alternative EMR workflows are likely required to improve the clinical impact.

Facilitators and Barriers to Interacting With Clinical Decision Support in the ICU: A Mixed-Methods Approach

OBJECTIVES

Clinical decision support systems (CDSSs) are used in various aspects of healthcare to improve clinical decision-making, including in the ICU. However, there is growing evidence that CDSS are not used to their full potential, often resulting in alert fatigue which has been associated with patient harm. Clinicians in the ICU may be more vulnerable to desensitization of alerts than clinicians in less urgent parts of the hospital. We evaluated facilitators and barriers to appropriate CDSS interaction and provide methods to improve currently available CDSS in the ICU.

DESIGN

Sequential explanatory mixed-methods study design, using the BEhavior and Acceptance fRamework.

SETTING

International survey study.

PATIENT/SUBJECTS

Clinicians (pharmacists, physicians) identified via survey, with recent experience with clinical decision support.

INTERVENTIONS

An initial survey was developed to evaluate clinician perspectives on their interactions with CDSS. A subsequent in-depth interview was developed to further evaluate clinician (pharmacist, physician) beliefs and behaviors about CDSS. These interviews were then qualitatively analyzed to determine themes of facilitators and barriers with CDSS interactions.

MEASUREMENTS AND MAIN RESULTS

A total of 48 respondents completed the initial survey (estimated response rate 15.5%). The majority believed that responding to CDSS alerts was part of their job (75%) but felt they experienced alert fatigue (56.5%). In the qualitative analysis, a total of five facilitators (patient safety, ease of response, specificity, prioritization, and feedback) and four barriers (excess quantity, work environment, difficulty in response, and irrelevance) were identified from the in-depth interviews.

CONCLUSIONS

In this mixed-methods survey, we identified areas that institutions should focus on to improve appropriate clinician interactions with CDSS, specific to the ICU. Tailoring of CDSS to the ICU may lead to improvement in CDSS and subsequent improved patient safety outcomes.

Expert Consensus on the Nephrotoxic Potential of 195 Medications in the Non-intensive Care Setting: A Modified Delphi Method

INTRODUCTION

Nephrotoxin exposure is significantly associated with acute kidney injury (AKI) development. A standardized list of nephrotoxic medications to surveil and their perceived nephrotoxic potential (NxP) does not exist for non-critically ill patients.

OBJECTIVE

This study generated consensus on the nephrotoxic effect of 195 medications used in the non-intensive care setting.

METHODS

Potentially nephrotoxic medications were identified through a comprehensive literature search, and 29 participants with nephrology or pharmacist expertise were identified. The primary outcome was NxP by consensus. Participants rated each drug on a scale of 0-3 (not nephrotoxic to definite nephrotoxicity). Group consensus was met if ≥ 75% of responses were one single rating or a combination of two consecutive ratings. If ≥ 50% of responses indicated "unknown" or not used in the non-intensive care setting, the medication was removed for consideration. Medications not meeting consensus for a given round were included in the subsequent round(s).

RESULTS

A total of 191 medications were identified in the literature, with 4 medications added after the first round from participants' recommendations. NxP index rating consensus after three rounds was: 14 (7.2%) no NxP in almost all situations (rating 0); 62 (31.8%) unlikely/possibly nephrotoxic (rating 0.5); 21 (10.8%) possibly nephrotoxic (rating 1); 49 (25.1%) possibly/probably nephrotoxic (rating 1.5); 2 (1.0%) probably nephrotoxic (rating 2); 8 (4.1%) probably/definite nephrotoxic (rating 2.5); 0 (0.0%) definitely nephrotoxic (rating 3); and 39 (20.0%) medications were removed from consideration.

CONCLUSIONS

NxP index rating provides clinical consensus on perceived nephrotoxic medications in the non-intensive care setting and homogeneity for future clinical evaluations and research.

Using AI-generated suggestions from ChatGPT to optimize clinical decision support

OBJECTIVE

To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions.

METHODS

We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked it to generate suggestions. We asked human clinician reviewers to review the AI-generated suggestions as well as human-generated suggestions for improving the same CDS alerts, and rate the suggestions for their usefulness, acceptance, relevance, understanding, workflow, bias, inversion, and redundancy.

RESULTS

Five clinicians analyzed 36 AI-generated suggestions and 29 human-generated suggestions for 7 alerts. Of the 20 suggestions that scored highest in the survey, 9 were generated by ChatGPT. The suggestions generated by AI were found to offer unique perspectives and were evaluated as highly understandable and relevant, with moderate usefulness, low acceptance, bias, inversion, redundancy.

CONCLUSION

AI-generated suggestions could be an important complementary part of optimizing CDS alerts, can identify potential improvements to alert logic and support their implementation, and may even be able to assist experts in formulating their own suggestions for CDS improvement. ChatGPT shows great potential for using large language models and reinforcement learning from human feedback to improve CDS alert logic and potentially other medical areas involving complex, clinical logic, a key step in the development of an advanced learning health system.

Implementing real-time prescription benefit tools: Early experiences from 5 academic medical centers

BACKGROUND

Medication price transparency tools are increasingly available, but data on their use, and their potential effects on prescribing behavior, patient out of pocket (OOP) costs, and clinician workflow integration, is limited.

OBJECTIVE

To describe the implementation experiences with real-time prescription benefit (RTPB) tools at 5 large academic medical centers and their early impact on prescription ordering.

DESIGN

and Participants: In this cross-sectional study, we systematically collected information on the characteristics of RTPB tools through discussions with key stakeholders at each of the five organizations. Quantitative encounter data, prescriptions written, and RTPB alerts/estimates and prescription adjustment rates were obtained at each organization in the first three months after "go-live" of the RTPB system(s) between 2019 and 2020.

MAIN MEASURES

Implementation characteristics, prescription orders, cost estimate retrieval rates, and prescription adjustment rates.

KEY RESULTS

Differences were noted with respect to implementation characteristics related to RTPB tools. All of the organizations with the exception of one chose to display OOP cost estimates and suggested alternative prescriptions automatically. Differences were also noted with respect to a patient cost threshold for automatic display. In the first three months after "go-live," RTPB estimate retrieval rates varied greatly across the five organizations, ranging from 8% to 60% of outpatient prescriptions. The prescription adjustment rate was lower, ranging from 0.1% to 4.9% of all prescriptions ordered.

CONCLUSIONS

In this study reporting on the early experiences with RTPB tools across five academic medical centers, we found variability in implementation characteristics and population coverage. In addition RTPB estimate retrieval rates were highly variable across the five organizations, while rates of prescription adjustment ranged from low to modest.

Assessing the Value of ChatGPT for Clinical Decision Support Optimization

Objective

To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions.

Methods

We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked it to generate suggestions. We asked human clinician reviewers to review the AI-generated suggestions as well as human-generated suggestions for improving the same CDS alerts, and rate the suggestions for their usefulness, acceptance, relevance, understanding, workflow, bias, inversion, and redundancy.

Results

Five clinicians analyzed 36 AI-generated suggestions and 29 human-generated suggestions for 7 alerts. Of the 20 suggestions that scored highest in the survey, 9 were generated by ChatGPT. The suggestions generated by AI were found to offer unique perspectives and were evaluated as highly understandable and relevant, with moderate usefulness, low acceptance, bias, inversion, redundancy.

Conclusion

AI-generated suggestions could be an important complementary part of optimizing CDS alerts, can identify potential improvements to alert logic and support their implementation, and may even be able to assist experts in formulating their own suggestions for CDS improvement. ChatGPT shows great potential for using large language models and reinforcement learning from human feedback to improve CDS alert logic and potentially other medical areas involving complex, clinical logic, a key step in the development of an advanced learning health system.

Improving the intensive care experience from the perspectives of different stakeholders

The intensive care unit (ICU) is a complex environment where patients, family members and healthcare professionals have their own personal experiences. Improving ICU experiences necessitates the involvement of all stakeholders. This holistic approach will invariably improve the care of ICU survivors, increase family satisfaction and staff wellbeing, and contribute to dignified end-of-life care. Inclusive and transparent participation of the industry can be a significant addition to develop tools and strategies for delivering this holistic care. We present a report, which follows a round table on ICU experience at the annual congress of the European Society of Intensive Care Medicine. The aim is to discuss the current evidence on patient, family and healthcare professional experience in ICU is provided, together with the panel’s suggestions on potential improvements. Combined with industry, the perspectives of all stakeholders suggest that ongoing improvement of ICU experience is warranted.

Electronic Advisories Increase Naloxone Prescribing Across Health Care Settings

BACKGROUND

Naloxone is a life-saving, yet underprescribed, medication that is recommended to be provided to patients at high risk of opioid overdose.

OBJECTIVE

We set out to evaluate the changes in prescriber practices due to the use of an electronic health record (EHR) advisory that prompted opioid prescribers to co-prescribe naloxone when prescribing a high-dose opioid. It also provided prescribers with guidance on decreasing opioid doses for safety.

DESIGN

This was a retrospective chart abstraction study looking at all opioid prescriptions and all naloxone prescriptions written as emergency department (ED) discharge, inpatient hospital discharge, or outpatient medications, between July 1, 2018, and February 1, 2020. The EHR advisory went live on June 1, 2019.

SUBJECTS

Included in the analysis were all adult patients seen in the abovementioned settings at a large county hospital and associated outpatient clinics.

MAIN MEASURES

We performed an interrupted time series analysis looking at naloxone prescriptions and daily opioid dosing in morphine milligram equivalents (MMEs), before and after initiation of the EHR advisory.

KEY RESULTS

The EHR advisory was associated with changes in prescribers' behavior, leading to increased naloxone prescriptions and decreased prescribed opioid doses.

CONCLUSIONS

EHR advisories are an effective systems-level intervention to enhance the safety of prescribed opioids and increase rates of naloxone prescribing.

Pilot Findings of Pharmacogenomics in Perioperative Care: Initial Results From the First Phase of the ImPreSS Trial

BACKGROUND

Pharmacogenomics, which offers a potential means by which to inform prescribing and avoid adverse drug reactions, has gained increasing consideration in other medical settings but has not been broadly evaluated during perioperative care.

METHODS

The Implementation of Pharmacogenomic Decision Support in Surgery (ImPreSS) Trial is a prospective, single-center study consisting of a prerandomization pilot and a subsequent randomized phase. We describe findings from the pilot period. Patients planning elective surgeries were genotyped with pharmacogenomic results, and decision support was made available to anesthesia providers in advance of surgery. Pharmacogenomic result access and prescribing records were analyzed. Surveys (Likert-scale) were administered to providers to understand utilization barriers.

RESULTS

Of eligible anesthesiology providers, 166 of 211 (79%) enrolled. A total of 71 patients underwent genotyping and surgery (median, 62 years; 55% female; average American Society of Anesthesiologists (ASA) score, 2.6; 58 inpatients and 13 ambulatories). No patients required postoperative intensive care or pain consultations. At least 1 provider accessed pharmacogenomic results before or during 41 of 71 surgeries (58%). Faculty were more likely to access results (78%) compared to house staff (41%; P = .003) and midlevel practitioners (15%) ( P < .0001). Notably, all administered intraoperative medications had favorable genomic results with the exception of succinylcholine administration to 1 patient with genomically increased risk for prolonged apnea (without adverse outcome). Considering composite prescribing in preoperative, recovery, throughout hospitalization, and at discharge, each patient was prescribed a median of 35 (range 15-83) total medications, 7 (range 1-22) of which had annotated pharmacogenomic results. Of 2371 prescribing events, 5 genomically high-risk medications were administered (all tramadol or omeprazole; with 2 of 5 pharmacogenomic results accessed), and 100 genomically cautionary mediations were administered (hydralazine, oxycodone, and pantoprazole; 61% rate of accessing results). Providers reported that although results were generally easy to access and understand, the most common reason for not considering results was because remembering to access pharmacogenomic information was not yet a part of their normal clinical workflow.

CONCLUSIONS

Our pilot data for result access rates suggest interest in pharmacogenomics by anesthesia providers, even if opportunities to alter prescribing in response to high-risk genotypes were infrequent. This pilot phase has also uncovered unique considerations for implementing pharmacogenomic information in the perioperative care setting, and new strategies including adding the involvement of surgery teams, targeting patients likely to need intensive care and dedicated pain care, and embedding pharmacists within rounding models will be incorporated in the follow-on randomized phase to increase engagement and likelihood of affecting prescribing decisions and clinical outcomes.

Universal HIV screening in the emergency department: an interrupted time series analysis

We performed a calendar-matched, 12-month, before (November 27, 2017 to November 26, 2018) and after (November 27, 2018 to November 26, 2019) study, to assess the utility of an emergency department-based HIV screening program. There were 710 and 14 335 patients screened for HIV during the pre and post-best practice alert (BPA) periods, respectively, representing more than a 20-fold increase in HIV screening following BPA implementation. Total HIV positive tests increased 5-fold following BPA implementation.

Developing a Mobile Health Application to Communicate Adverse Drug Reactions - Preconditions, Assessment of Possible Functionalities and Barriers for Patients and Their General Practitioners

Purpose

Mobile health (mHealth) applications offer structured and timely communication between patients and general practitioners (GPs) about adverse drug reactions (ADR). Preconditions, functionalities and barriers should be studied to ensure safe implementation.

Methods

We performed a cross-sectional questionnaire survey addressing (i) preconditions, (ii) users’ assessment of functionalities and (iii) barriers to mHealth managing ADR communication.

Results

A total of 480 patients and 31 GPs completed the survey. (i) A total of 269 (56%) patients and 13 (42%) GPs were willing to use mHealth for ADR communication. Willingness was negatively correlated with age for both patients (r = −0.231; p < 0.001) and GPs (r = −0.558; p = 0.002). (ii) Most useful functionalities mentioned by patients (>60%) included “Rapid feedback on urgency of face-to-face consultations.” GPs valued information on “Patient’s difficulties in medication administration.” (iii) In free-text answers, the barrier reported most frequently by patients was “preferred personal GP contact” (6%), whereas GPs claimed, “uncomplicated use with low expenditure of time and personnel” (19%).

Conclusion

Older patients and GPs mainly show reservations about mHealth for ADR communication but recognize possible benefits. mHealth implementation should avoid a negative effect on GPs’ time budgets; the primary goal should not be to reduce the number of GP-patient contacts but to optimize them.

Clinical Decision Support for Newborn Weight Loss: A Randomized Controlled Trial

BACKGROUND AND OBJECTIVE

The Newborn Weight Tool (NEWT) can inform newborn feeding decisions and might reduce health care utilization by preventing excess weight loss. Clinical decision support (CDS) displaying NEWT might facilitate its use. Our study's objective is to determine the effect of CDS displaying NEWT on feeding and health care utilization.

METHODS

At an hospital involved in NEWT development, we randomly assigned 2682 healthy infants born ≥36 weeks gestation in 2018-2019 either to CDS displaying NEWT with an electronic flag if most recent weight was ≥75th weight loss centile or to a control of usual care with NEWT accessed at clinician discretion. Our primary outcome was feeding type concordant with weight loss, defined as exclusive breastfeeding for those not flagged, exclusive breastfeeding or supplementation for those flagged once, and supplementation for those flagged more than once. Secondary outcomes included inpatient and outpatient utilization in the first 30 days. We used χ2 and Student's t tests to compare intervention infants with control and to compare trial infants with those born in 2017.

RESULTS

Feeding was concordant with for 1854 (74.5%) trial infants and did not differ between randomized groups (P = .65); concordant feeding was higher for all trial infants than for infants born in 2017 (64.4%; P < .0005). Readmission occurred for 51 (3.8%) CDS infants and 45 (3.4%) control infants (P = .56). Among the 60% of trial infants with outpatient records available, there were 3.5 ± 1.7 visits with no differences between randomized groups (P = .10).

CONCLUSIONS

At an hospital involved in NEWT development, CDS displaying NEWT did not alter either feeding or health care utilization compared with discretionary NEWT access.

The impact of clinical decision support systems on provider behavior in the inpatient setting: A systematic review and meta-analysis

BACKGROUND

Clinical decision support systems (CDSS) are used to improve processes of care. CDSS proliferation may have unintended consequences impacting effectiveness.

OBJECTIVE

To evaluate the effectiveness of CDSS in altering clinician behavior.

DESIGN

Electronic searches were performed in EMBASE, PubMed, and Cochrane Central Register of Control Trials for randomized controlled trials testing the impacted of CDSS on clinician behavior from 2000-2021. Extracted data included study design, CDSS attributed and outcomes, user characteristics, settings, and risk of bias. Eligible studies were analyzed qualitatively to describe CDSS types. Studies with sufficient outcome data were included in the meta-analysis.

SETTING AND PARTICIPANTS

Adult inpatients in the United States.

INTERVENTION

Clinical decision support system versus non-clinical decision support system.

MAIN OUTCOME AND MEASURE

A random-effects model measured the pooled risk difference (RD) and odds ratio of clinicians' adherence to CDSS; subgroup analyses tested differences in CDSS effectiveness over time and by CDSS type.

RESULTS

Qualitative synthesis included 22 studies. Eleven studies reported sufficient outcome data for inclusion in the meta-analysis. CDSS did not result in a statistically significant increase in clinician adoption of desired practicies (RD = 0.04 [95% confidence interval {CI} 0.00, 0.07]). CDSS from 2010-2015 (n = 5) did not increase clinician adoption of desired practice [RD -0.01, (95% CI -0.04, 0.02)].CDSS from 2016-2021 (n = 6) were associated with an increase in targeted practices [RD 0.07 (95% CI0.03, 0.12)], pInteraction = 0.004. EHR [RD 0.04 (95% CI 0.00, 0.08)] vs. non-EHR [RD 0.01 (95% CI -0.01, 0.04)] based CDSS interventions did not result in different adoption of desired practices (pInteraction = 0.27). The meta-analysis did not find an overall positive impact of CDSS on clinician behavior in the inpatient setting.

Comparing Important and Well-documented Potential Drug–Drug Interactions between Emergency, Medical, and Surgical ICUs of a Respiratory Referral Center

Introduction

Drug–drug interaction (DDI) is one of the major healthcare challenges in intensive care units (ICUs). The prevalence of DDIs and interacting drug pairs may vary between different types of ICUs. This study aimed to compare the frequency and nature of important and well-documented potential DDIs (pDDIs) in three types of ICUs.

Materials and methods

A prospective study was conducted in medical (M), surgical (S), and emergency (E) ICUs of a tertiary referral center for respiratory diseases. A pharmacist checked the patients’ files three days in a week for 6 months. The pDDIs were identified using the Lexi-Interact database. Interactions with a severity rating of D (modify regimen) and X (avoid combination) and with a reliability rating of good and excellent were considered important and well-documented. These pDDIs were evaluated in terms of drug combinations, mechanisms of interaction, and clinical management.

Results

One hundred eighty-nine patients admitted to MICU, SICU, and EICU were included in the study. The percentage of patients who experienced at least one important and well-documented pDDI was 18.8% in MICU, 11.1% in SICU, and 11.8% in EICU. The most common drug pairs causing important and well-documented interactions were atracurium + hydrocortisone in MICU, meropenem + valproic acid in MICU and EICU, and aspirin + warfarin in SICU.

Conclusion

The current study shows different frequency and nature of pDDIs between three types of ICUs. We recommend conducting similar studies in other settings to develop evidence-based guidance on clinically relevant pDDIs in different types of ICUs.

How to cite this article

Hosseinpoor Z, Farzanegan B, Baniasadi S. Comparing Important and Well-documented Potential Drug–Drug Interactions between Emergency, Medical, and Surgical ICUs of a Respiratory Referral Center. Indian J Crit Care Med 2022;26(5):574–578.

Consensus Obtained for the Nephrotoxic Potential of 167 Drugs in Adult Critically Ill Patients Using a Modified Delphi Method

Introduction

The approach to evaluating nephrotoxins in studies of drug-associated acute kidney injury varies. Some studies use a list of under ten drugs for evaluation whereas others include over 100 drugs. Drugs are typically assigned a binary classification, nephrotoxic or not nephrotoxic. This oversimplifies the nephrotoxic potential of the drugs under investigation.

Objective

This study aimed to assign a nephrotoxin potential for 167 drugs used in the adult critical care setting.

Methods

A three-round, international, interdisciplinary, web-based modified-Delphi study was used to evaluate nephrotoxins used in adult critically ill patients. Twenty-four international experienced clinicians were identified through the Acute Disease Quality Initiative group and professional affiliations. Included individuals represented the fields of intensive care, nephrology, and pharmacy. One hundred and fifty-nine medications were identified from the literature, with eight additional medications added after the first round, for a total of 167 medications. The primary outcome was consensus achieved for nephrotoxicity ratings. Scores were evaluated each round to determine if a consensus was met.

Results

Our nephrotoxin potential index rating indicated that 20 drugs were nephrotoxicity probable or probable/definite per consensus. Nephrotoxic potential was assessed based on the standard use of medications in intensive care and the following consensus scores: 0 = no nephrotoxic potential, 1 = possible nephrotoxic potential, 2 = probable nephrotoxic potential, 3 = definite nephrotoxic potential.

Conclusions

The nephrotoxin potential index rating allows for prioritization of targeted drugs with greater nephrotoxic potential for institutional nephrotoxin stewardship programs. Furthermore, the nephrotoxin potential index rating provides homogeneity for research and guidance on detailed assessments by severity for each drug.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40264-022-01173-4.

An alert tool to promote lung protective ventilation for possible acute respiratory distress syndrome

Objective

Computer-aided decision tools may speed recognition of acute respiratory distress syndrome (ARDS) and promote consistent, timely treatment using lung-protective ventilation (LPV). This study evaluated implementation and service (process) outcomes with deployment and use of a clinical decision support (CDS) synchronous alert tool associated with existing computerized ventilator protocols and targeted patients with possible ARDS not receiving LPV.

Materials and Methods

We performed an explanatory mixed methods study from December 2019 to November 2020 to evaluate CDS alert implementation outcomes across 13 intensive care units (ICU) in an integrated healthcare system with >4000 mechanically ventilated patients annually. We utilized quantitative methods to measure service outcomes including CDS alert tool utilization, accuracy, and implementation effectiveness. Attitudes regarding the appropriateness and acceptability of the CDS tool were assessed via an electronic field survey of physicians and advanced practice providers.

Results

Thirty-eight percent of study encounters had at least one episode of LPV nonadherence. Addition of LPV treatment detection logic prevented an estimated 1812 alert messages (41%) over use of disease detection logic alone. Forty-eight percent of alert recommendations were implemented within 2 h. Alert accuracy was estimated at 63% when compared to gold standard ARDS adjudication, with sensitivity of 85% and positive predictive value of 62%. Fifty-seven percent of survey respondents observed one or more benefits associated with the alert.

Conclusion

Introduction of a CDS alert tool based upon ARDS risk factors and integrated with computerized ventilator protocol instructions increased visibility to gaps in LPV use and promoted increased adherence to LPV.

Impact of an Automated Best Practice Alert on Sex and Race Disparities in Implantable Cardioverter-Defibrillator Therapy

Background

Implantable cardioverter‐defibrillators (ICDs) are indicated in patients with severe left ventricular dysfunction, but many eligible patients do not receive them, especially women and Black patients. Our group had previously demonstrated that a best practice alert (BPA) improves overall rates of electrophysiology referrals and ICD implantations. This study examined the impact of a BPA by sex and race.

Methods and Results

This is a cluster randomized trial of cardiology (n=106) and primary care (n=89) providers who were randomized to receive (BPA, n=93) or not receive (No BPA, n=102) the alert and managed 1856 patients meeting primary prevention criteria for ICD implantation (965 BPA and 891 No BPA). After a median follow up of 34 months, 630 (34%) patients were referred to electrophysiology, and 522 (28%) patients received an ICD. Compared with the No BPA arm, patients in the BPA arm saw a modest differential increase in the rate of electrophysiology referrals at 18 months in men (+4%) compared with women (+7%) but a profound increase in Black patients (+16%) compared with White patients (+2%), thus closing the sex and race gaps. Similar trends were noted for rates of ICD implantation.

Conclusions

Use of a BPA improves rates of electrophysiology referrals and ICD implantations in all comers with severe cardiomyopathy and no prior ventricular arrhythmias but has a more pronounced impact in women and Black patients. The use of a BPA at the point of care is an effective tool in the fight against sex and race inequities in health care.

Modulators Influencing Medication Alert Acceptance: An Explorative Review

OBJECTIVES

Clinical decision support systems (CDSSs) use alerts to enhance medication safety and reduce medication error rates. A major challenge of medication alerts is their low acceptance rate, limiting their potential benefit. A structured overview about modulators influencing alert acceptance is lacking. Therefore, we aimed to review and compile qualitative and quantitative modulators of alert acceptance and organize them in a comprehensive model.

METHODS

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, a literature search in PubMed was started in February 2018 and continued until October 2021. From all included articles, qualitative and quantitative parameters and their impact on alert acceptance were extracted. Related parameters were then grouped into factors, allocated to superordinate determinants, and subsequently further allocated into five categories that were already known to influence alert acceptance.

RESULTS

Out of 539 articles, 60 were included. A total of 391 single parameters were extracted (e.g., patients' comorbidity) and grouped into 75 factors (e.g., comorbidity), and 25 determinants (e.g., complexity) were consequently assigned to the predefined five categories, i.e., CDSS, care provider, patient, setting, and involved drug. More than half of all factors were qualitatively assessed (n = 21) or quantitatively inconclusive (n = 19). Furthermore, 33 quantitative factors clearly influenced alert acceptance (positive correlation: e.g., alert type, patients' comorbidity; negative correlation: e.g., number of alerts per care provider, moment of alert display in the workflow). Two factors (alert frequency, laboratory value) showed contradictory effects, meaning that acceptance was significantly influenced both positively and negatively by these factors, depending on the study. Interventional studies have been performed for only 12 factors while all other factors were evaluated descriptively.

CONCLUSION

This review compiles modulators of alert acceptance distinguished by being studied quantitatively or qualitatively and indicates their effect magnitude whenever possible. Additionally, it describes how further research should be designed to comprehensively quantify the effect of alert modulators.

Using Machine Learning to Identify Patients at High Risk of Inappropriate Drug Dosing in Periods with Renal Dysfunction

Purpose

Dosing of renally cleared drugs in patients with kidney failure often deviates from clinical guidelines, so we sought to elicit predictors of receiving inappropriate doses of renal risk drugs.

Patients and methods

We combined data from the Danish National Patient Register and in-hospital data on drug administrations and estimated glomerular filtration rates for admissions between 1 October 2009 and 1 June 2016, from a pool of about 2.6 million persons. We trained artificial neural network and linear logistic ridge regression models to predict the risk of five outcomes (>0, ≥1, ≥2, ≥3 and ≥5 inappropriate doses daily) with index set 24 hours after admission. We used time-series validation for evaluating discrimination, calibration, clinical utility and explanations.

Results

Of 52,451 admissions included, 42,250 (81%) were used for model development. The median age was 77 years; 50% of admissions were of women. ≥5 drugs were used between admission start and index in 23,124 admissions (44%); the most common drug classes were analgesics, systemic antibacterials, diuretics, antithrombotics, and antacids. The neural network models had better discriminative power (all AUROCs between 0.77 and 0.81) and were better calibrated than their linear counterparts. The main prediction drivers were use of anti-inflammatory, antidiabetic and anti-Parkinson's drugs as well as having a diagnosis of chronic kidney failure. Sex and age affected predictions but slightly.

Conclusion

Our models can flag patients at high risk of receiving at least one inappropriate dose daily in a controlled in-silico setting. A prospective clinical study may confirm that this holds in real-life settings and translates into benefits in hard endpoints.

The potential for leveraging machine learning to filter medication alerts

OBJECTIVE

To evaluate the potential for machine learning to predict medication alerts that might be ignored by a user, and intelligently filter out those alerts from the user's view.

MATERIALS AND METHODS

We identified features (eg, patient and provider characteristics) proposed to modulate user responses to medication alerts through the literature; these features were then refined through expert review. Models were developed using rule-based and machine learning techniques (logistic regression, random forest, support vector machine, neural network, and LightGBM). We collected log data on alerts shown to users throughout 2019 at University of Utah Health. We sought to maximize precision while maintaining a false-negative rate <0.01, a threshold predefined through discussion with physicians and pharmacists. We developed models while maintaining a sensitivity of 0.99. Two null hypotheses were developed: H1-there is no difference in precision among prediction models; and H2-the removal of any feature category does not change precision.

RESULTS

A total of 3,481,634 medication alerts with 751 features were evaluated. With sensitivity fixed at 0.99, LightGBM achieved the highest precision of 0.192 and less than 0.01 for the pre-defined maximal false-negative rate by subject-matter experts (H1) (P < 0.001). This model could reduce alert volume by 54.1%. We removed different combinations of features (H2) and found that not all features significantly contributed to precision. Removing medication order features (eg, dosage) most significantly decreased precision (-0.147, P = 0.001).

CONCLUSIONS

Machine learning potentially enables the intelligent filtering of medication alerts.

Pediatric sepsis screening in US hospitals

Sepsis is a major cause of morbidity and mortality in children. While adverse outcomes can be reduced through prompt initiation of sepsis protocols including fluid resuscitation and antibiotics, provision of these therapies relies on clinician recognition of sepsis. Recognition is challenging in children because early signs of shock such as tachycardia and tachypnea have low specificity while hypotension often does not occur until late in the clinical course. This narrative review highlights the important context that has led to the rapid growth of pediatric sepsis screening in the United States. In this review, we (1) describe different screening tools used in US emergency department, inpatient, and intensive care unit settings; (2) highlight details of the design, implementation, and evaluation of specific tools; (3) review the available data on the process of integrating sepsis screening into an overall sepsis quality improvement program and on the effect of these screening tools on patient outcomes; (4) discuss potential harms of sepsis screening including alarm fatigue; and (5) highlight several future directions in sepsis screening, such as novel tools that incorporate artificial intelligence and machine learning methods to augment sepsis identification with the ultimate goal of precision-based approaches to sepsis recognition and treatment. IMPACT: This narrative review highlights the context that has led to the rapid growth of pediatric sepsis screening nationally. Screening tools used in US emergency department, inpatient, and intensive care unit settings are described in terms of their design, implementation, and clinical performance. Limitations and potential harms of these tools are highlighted, as well as future directions that may lead to a more precision-based approach to sepsis recognition and treatment.

Preferences for Using a Mobile App in Sickle Cell Disease Self-management: Descriptive Qualitative Study

Background

Individuals with sickle cell disease (SCD) and their caregivers may benefit from technology-based resources to improve disease self-management.

Objective

This study explores the preferences regarding a mobile health (mHealth) app to facilitate self-management in adults with SCD and their caregivers living in urban and rural communities.

Methods

Five community listening sessions were conducted in 2 urban and rural communities among adults with SCD and their caregivers (N=43). Each session comprised 4 to 15 participants. Participants were asked questions on methods of finding information about SCD self-care, satisfaction with current methods for finding SCD management information, support for SCD management, important features for development of an mHealth app, and areas of benefit for using an mHealth app for SCD self-management. An inductive-deductive content analysis approach was implemented to identify the critical themes.

Results

Seven critical themes emerged, including the current methods for receiving self-management information, desired information, recommendations for communicating sickle cell self-management information, challenges of disease management, types of support received for disease management, barriers to and facilitators of using an mHealth app, and feature preferences for an mHealth app. In addition, we found that the participants were receptive to using mHealth apps in SCD self-management.

Conclusions

This study expands our knowledge on the use of mHealth technology to reduce information access barriers pertaining to SCD. The findings can be used to develop a patient-centered, user-friendly mHealth app to facilitate disease self-management, thus increasing access to resources for families of patients with SCD residing in rural communities.

Promoting Critical Thinking in Your Intensive Care Unit Team

Effective and efficient critical thinking skills are necessary to engage in accurate clinical reasoning and to make appropriate clinical decisions. Teaching and promoting critical thinking skills in the intensive care unit is challenging because of the volume of data and the constant distractions of competing obligations. Understanding and acknowledging cognitive biases and their impact on clinical reasoning are necessary to promote and support critical thinking in the ICU. Active educational strategies such as concept or mechanism mapping can help to diagnose disorganized thinking and reinforce key connections and important clinical and pathophysiologic concepts, which are critical for inductive reasoning.

Nursing and precision predictive analytics monitoring in the acute and intensive care setting: An emerging role for responding to COVID-19 and beyond

As the global response to COVID-19 continues, nurses will be tasked with appropriately triaging patients, responding to events of clinical deterioration, and developing family-centered plans of care within a healthcare system exceeding capacity. Predictive analytics monitoring, an artificial intelligence (AI)-based tool that translates streaming clinical data into a real-time visual estimation of patient risks, allows for evolving acuity assessments and detection of clinical deterioration while the patient is in pre-symptomatic states. While nurses are on the frontline for the COVID-19 pandemic, the use of AI-based predictive analytics monitoring may help cognitively complex clinical decision-making tasks and pave a pathway for early detection of patients at risk for decompensation. We must develop strategies and techniques to study the impact of AI-based technologies on patient care outcomes and the clinical workflow. This paper outlines key concepts for the intersection of nursing and precision predictive analytics monitoring.

A Framework to Assess Alarm Fatigue Indicators in Critical Care Staff

This article examines work-related and Personality personality factors that could influence health providers in experiencing alarm fatigue. The purpose of this study is to provide a basis to determine factors that may predict the potential of alarm fatigue in critical care staff.

DESIGN

A questionnaire-based survey and an observational study were conducted to assess factors that could contribute to indicators of alarm fatigue.

INTERVENTIONS

Factors included patient-to-staff ratio, criticality of the alarm, priority of different tasks, and personality traits.

SETTING

The study was conducted at an eight-bed ICU in a mid-size hospital in Montana.

SUBJECTS

Data were collected for six day shifts and six night shifts involving 24 critical care professionals. Within each 12-hour shift, six 15-minute intervals were randomly generated through work sampling for 6 days; a total of 1,080 observations were collected.

MEASUREMENTS

Alarm fatigue was assessed with the subjective workload assessment technique and Boredom, Apathy, and Distrust Affects, which were measured through validated questionnaires. The Big Five Personality model was used to assess personality traits.

MAIN RESULTS

Work factors including task prioritization, nurse-to-patient ratio, and length of shifts were associated with indicators of alarm fatigue. Personality traits of openness, conscientiousness, and neuroticism were also associated.

CONCLUSIONS

We recommend assessing personality traits for critical care staff to be aware of how their individualities can affect their behavior towards alarm fatigue. We also recommend an examination of alternative strategies to reduce alarm fatigue, including examining the use of breaks, work rotation, or shift reduction.

Nurse Cognition, Decision Support, and Barcode Medication Administration: A Conceptual Framework for Research, Practice, and Education

This article synthesizes theoretical perspectives related to nurse cognition. We present a conceptual model that can be used by multiple stakeholders to study and contemplate how nurses use clinical decision support systems, and specifically, Barcode-Assisted Medication Administration, to make decisions during the delivery of care. Theoretical perspectives integrated into the model include dual process theory, the Cognitive Continuum Theory, human factors engineering, and the Recognition-Primed Decision model. The resulting framework illustrates the process of nurse cognition during Barcode-Assisted Medication Administration. Additionally, the model includes individual or human and environmental factors that may influence nurse cognition and decision making. It is important to consider the influence of individual, human, and environmental factors on the process of nurse cognition and decision making. Specifically, it is necessary to explore the impact of heuristics and biases on clinician decision making, particularly related to the development of alarm and alert fatigue. Aided by the proposed framework, stakeholders may begin to identify heuristics and cognitive biases that influence the decision of clinicians to accept or override a clinical decision support system alert and whether heuristics and biases are associated with inappropriate alert override.

The Impact of Clinical Decision Support Alerts on Clostridioides difficile Testing: A Systematic Review

BACKGROUND

Several studies have investigated the utility of electronic decision support alerts in diagnostic stewardship for Clostridioides difficile infection (CDI). However, it is unclear if alerts are effective in reducing inappropriate CDI testing and/or CDI rates. The aim of this systematic review was to determine if alerts related to CDI diagnostic stewardship are effective at reducing inappropriate CDI testing volume and CDI rates among hospitalized adult patients.

METHODS

We searched Ovid Medline and 5 other databases for original studies evaluating the association between alerts for CDI diagnosis and CDI testing volume and/or CDI rate. Two investigators independently extracted data on study characteristics, study design, alert triggers, cointerventions, and study outcomes.

RESULTS

Eleven studies met criteria for inclusion. Studies varied significantly in alert triggers and in study outcomes. Six of 11 studies demonstrated a statistically significant decrease in CDI testing volume, 6 of 6 studies evaluating appropriateness of CDI testing found a significant reduction in the proportion of inappropriate testing, and 4 of 7 studies measuring CDI rate demonstrated a significant decrease in the CDI rate in the postintervention vs preintervention period. The magnitude of the increase in appropriate CDI testing varied, with some studies reporting an increase with minimal clinical significance.

CONCLUSIONS

The use of electronic alerts for diagnostic stewardship for C. difficile was associated with reductions in CDI testing, the proportion of inappropriate CDI testing, and rates of CDI in most studies. However, broader concerns related to alerts remain understudied, including unintended adverse consequences and alert fatigue.

Use of machine learning to predict clinical decision support compliance, reduce alert burden, and evaluate duplicate laboratory test ordering alerts

Objectives

While well-designed clinical decision support (CDS) alerts can improve patient care, utilization management, and population health, excessive alerting may be counterproductive, leading to clinician burden and alert fatigue. We sought to develop machine learning models to predict whether a clinician will accept the advice provided by a CDS alert. Such models could reduce alert burden by targeting CDS alerts to specific cases where they are most likely to be effective.

Materials and Methods

We focused on a set of laboratory test ordering alerts, deployed at 8 hospitals within the Partners Healthcare System. The alerts notified clinicians of duplicate laboratory test orders and advised discontinuation. We captured key attributes surrounding 60 399 alert firings, including clinician and patient variables, and whether the clinician complied with the alert. Using these data, we developed logistic regression models to predict alert compliance.

Results

We identified key factors that predicted alert compliance; for example, clinicians were less likely to comply with duplicate test alerts triggered in patients with a prior abnormal result for the test or in the context of a nonvisit-based encounter (eg, phone call). Likewise, differences in practice patterns between clinicians appeared to impact alert compliance. Our best-performing predictive model achieved an area under the receiver operating characteristic curve (AUC) of 0.82. Incorporating this model into the alerting logic could have averted more than 1900 alerts at a cost of fewer than 200 additional duplicate tests.

Conclusions

Deploying predictive models to target CDS alerts may substantially reduce clinician alert burden while maintaining most or all the CDS benefit.

Nephrotoxin Stewardship

Drugs are the third leading cause of acute kidney injury (AKI) in critically ill patients. Nephrotoxin stewardship ensures a structured and consistent approach to safe medication use and prevention of patient harm. Comprehensive nephrotoxin stewardship requires coordinated patient care management strategies for safe medication use, ensuring kidney health, and avoiding unnecessary costs to improve the use of nephrotoxins, renally eliminated drugs, and kidney disease treatments. Implementing nephrotoxin stewardship reduces medication errors and adverse drug events, prevents or reduces severity of drug-associated AKI, prevents progression to or worsening of chronic kidney disease, and alleviates financial burden on the health care system.

Low Rates of Retesting for Eradication of Helicobacter pylori Infection After Treatment in the Veterans Health Administration

BACKGROUND & AIMS

Expert consensus mandates retesting for eradication of Helicobacter pylori infection after treatment, but it is not clear how many patients are actually retested. We evaluated factors associated with retesting for H pylori in a large, nationwide cohort.

METHODS

We performed a retrospective cohort study of patients with H pylori infection (detected by urea breath test, stool antigen, or pathology) who were prescribed an eradication regimen from January 1, 1994 through December 31, 2018 within the Veterans Health Administration (VHA). We collected data on demographic features, smoking history, socioeconomic status, facility poverty level and academic status, and provider specialties and professions. The primary outcome was retesting for eradication. Statistical analyses included mixed-effects logistic regression.

RESULTS

Of 27,185 patients prescribed an H pylori eradication regimen, 6486 patients (23.9%) were retested. Among 7623 patients for whom we could identify the provider who ordered the test, 2663 patients (34.9%) received the order from a gastroenterological provider. Female sex (odds ratio, 1.22; 95% CI, 1.08-1.38; P = .002) and history of smoking (odds ratio, 1.24; 95% CI, 1.15-1.33; P < .001) were patient factors associated with retesting. There was an interaction between method of initial diagnosis of H pylori infection and provider who ordered the initial test (P < .001). There was significant variation in rates of retesting among VHA facilities (P < .001).

CONCLUSIONS

In an analysis of data from a VHA cohort of patients with H pylori infection, we found low rates of retesting after eradication treatment. There is significant variation in rates of retesting among VHA facilities. H pylori testing is ordered by nongastroenterology specialists two-thirds of the time. Confirming eradication of H pylori is mandatory and widespread quality assurance protocols are needed.

Giving Your Electronic Health Record a Checkup After COVID-19: A Practical Framework for Reviewing Clinical Decision Support in Light of the Telemedicine Expansion

BACKGROUND

The transformation of health care during COVID-19, with the rapid expansion of telemedicine visits, presents new challenges to chronic care and preventive health providers. Clinical decision support (CDS) is critically important to chronic care providers, and CDS malfunction is common during times of change. It is essential to regularly reassess an organization's ambulatory CDS program to maintain care quality. This is especially true after an immense change, like the COVID-19 telemedicine expansion.

OBJECTIVE

Our objective is to reassess the ambulatory CDS program at a large academic medical center in light of telemedicine's expansion in response to the COVID-19 pandemic.

METHODS

Our clinical informatics team devised a practical framework for an intrapandemic ambulatory CDS assessment focused on the impact of the telemedicine expansion. This assessment began with a quantitative analysis comparing CDS alert performance in the context of in-person and telemedicine visits. Board-certified physician informaticists then completed a formal workflow review of alerts with inferior performance in telemedicine visits. Informaticists then reported on themes and optimization opportunities through the existing CDS governance structure.

RESULTS

Our assessment revealed that 10 of our top 40 alerts by volume were not firing as expected in telemedicine visits. In 3 of the top 5 alerts, providers were significantly less likely to take action in telemedicine when compared to office visits. Cumulatively, alerts in telemedicine encounters had an action taken rate of 5.3% (3257/64,938) compared to 8.3% (19,427/233,636) for office visits. Observations from a clinical informaticist workflow review included the following: (1) Telemedicine visits have different workflows than office visits. Some alerts developed for the office were not appearing at the optimal time in the telemedicine workflow. (2) Missing clinical data is a common reason for the decreased alert firing seen in telemedicine visits. (3) Remote patient monitoring and patient-reported clinical data entered through the portal could replace data collection usually completed in the office by a medical assistant or registered nurse.

CONCLUSIONS

In a large academic medical center at the pandemic epicenter, an intrapandemic ambulatory CDS assessment revealed clinically significant CDS malfunctions that highlight the importance of reassessing ambulatory CDS performance after the telemedicine expansion.

Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening

OBJECTIVE

Evaluation of the effect of six optimization strategies in a clinical decision support system (CDSS) for drug-drug interaction (DDI) screening on alert burden and alert acceptance and description of clinical pharmacist intervention acceptance.

METHODS

Optimizations in the new CDSS were the customization of the knowledge base (with addition of 67 extra DDIs and changes in severity classification), a new alert design, required override reasons for the most serious alerts, the creation of DDI-specific screening intervals, patient-specific alerting, and a real-time follow-up system of all alerts by clinical pharmacists with interventions by telephone was introduced. The alert acceptance was evaluated both at the prescription level (i.e. prescription acceptance, was the DDI prescribed?) and at the administration level (i.e. administration acceptance, did the DDI actually take place?). Finally, the new follow-up system was evaluated by assessing the acceptance of clinical pharmacist's interventions.

RESULTS

In the pre-intervention period, 1087 alerts (92.0 % level 1 alerts) were triggered, accounting for 19 different DDIs. In the post-intervention period, 2630 alerts (38.4 % level 1 alerts) were triggered, representing 86 different DDIs. The relative risk forprescription acceptance in the post-intervention period compared to the pre-intervention period was 4.02 (95 % confidence interval (CI) 3.17-5.10; 25.5 % versus 6.3 %). The relative risk for administration acceptance was 1.16 (95 % CI 1.08-1.25; 54.4 % versus 46.7 %). Finally, 86.9 % of the clinical pharmacist interventions were accepted.

CONCLUSION

Six concurrently implemented CDSS optimization strategies resulted in a high alert acceptance and clinical pharmacist intervention acceptance. Administration acceptance was remarkably higher than prescription acceptance.

Effects of monitor alarm management training on nurses' alarm fatigue: A randomised controlled trial

BACKGROUND

Chaotic monitor alarm management generates a large number of alarms, which result in alarm fatigue. Intensive care unit (ICU) nurses are caretakers of critically ill patients, the effect of alarm management affect patient safety directly.

OBJECTIVES

To evaluate the effect of monitor alarm management training based on the theory of planned behaviour for reducing alarm fatigue in intensive care unit nurses.

DESIGN

A randomised, single-blind trial. This article follows the requirements of CONSORT statement.

PARTICIPANTS

The study was conducted from February 2019-May 2019 in a tertiary A-level hospital. 93 ICU clinical nurses were included, and they were randomly assigned into two groups.

INTERVENTION

Nurses in the experimental group (n = 47) received a 12-week alarm management training course based on the theory of planned behaviour. Nurses in the control group (n = 46) received regular training. All nurses' alarm fatigue scores were measured with a questionnaire before and after the study period. Total number of alarms, nonactionable alarms and true crisis alarms were recorded continuously throughout the study period.

RESULTS

For baseline comparisons, no significant differences were found. By the analysis of independent samples one-way ANCOVAs, the nurses' adjusted alarm fatigue scores at the post-test in the experimental group were significantly lower than those in the control group (p < .001). After the study period, adjusted total number of alarms and nonactionable alarms recorded in the experimental group were both significantly lower than those recorded in the control group (p < .001). After the study period, no significant difference between the two groups was noted in the adjusted number of true crisis alarms (p > .05). The interventions did not cause adverse events in either group of patients and did not cause adverse events in patients.

CONCLUSION

Intensive care unit nurses' alarm fatigue was effectively decreased by the monitor alarm management training based on the theory of planned behaviour.

RELEVANCE TO CLINICAL PRACTICE

(1) Monitor alarm training based on the theory of planned behaviour is effective in reducing nonactionable alarms and lowering alarm fatigue in ICU nurses. (2) The intervention considering the social psychological aspects of behaviour is effective in rebuilding the nurses' awareness and behaviour of alarm management. (3) Nurses are the direct users of monitoring technology. Hospital administrators should attach importance to the role of nurses in the medical monitoring system. We suggest that nursing managers implement training programmes in more ICUs in the future to improve alarm management ability and lower alarm fatigue in ICU nurses.