Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics

Optimising interruptive clinical decision support alerts for antithrombotic duplicate prescribing in hospital

INTRODUCTION

Duplicate prescribing clinical decision support alerts can prevent important prescribing errors but are frequently the cause of much alert fatigue. Stat dose prescriptions are a known reason for overriding these alerts. This study aimed to evaluate the effect of excluding stat dose prescriptions from duplicate prescribing alerts for antithrombotic medicines on alert burden, prescriber adherence, and prescribing.

MATERIALS AND METHODS

A before (January 1st, 2017 to August 31st, 2022) and after (October 5th, 2022 to September 30th, 2023) study was undertaken of antithrombotic duplicate prescribing alerts and prescribing following a change in alert settings. Alert and prescribing data for antithrombotic medicines were joined, processed, and analysed to compare alert rates, adherence, and prescribing. Alert burden was assessed as alerts per 100 prescriptions. Adherence was measured at the point of the alert as whether the prescriber accepted the alert and following the alert as whether a relevant prescription was ceased within an hour. Co-prescribing of antithrombotic stat dose prescriptions was assessed pre- and post-alert reconfiguration.

RESULTS

Reconfiguration of the alerts reduced the alert rate by 29 % (p < 0.001). The proportion of alerts associated with cessation of antithrombotic duplication significantly increased (32.8 % to 44.5 %, p < 0.001). Adherence at the point of the alert increased 1.2 % (4.8 % to 6.0 %, p = 0.012) and 11.5 % (29.4 % to 40.9 %, p < 0.001) within one hour of the alert. When ceased after the alert over 80 % of duplicate prescriptions were ceased within 2 min of overriding. Antithrombotic stat dose co-prescribing was unchanged for 4 out of 5 antithrombotic duplication alert rules.

CONCLUSION

By reconfiguring our antithrombotic duplicate prescribing alerts, we reduced alert burden and increased alert adherence. Many prescribers ceased duplicate prescribing within 2 min of alert override highlighting the importance of incorporating post-alert measures in accurately determining prescriber alert adherence.

Clinical Decision Support Principles for Quality Improvement and Research

Pediatric hospitalists frequently interact with clinical decision support (CDS) tools in patient care and use these tools for quality improvement or research. In this method/ology paper, we provide an introduction and practical approach to developing and evaluating CDS tools within the electronic health record. First, we define CDS and describe the types of CDS interventions that exist. We then outline a stepwise approach to CDS development, which begins with defining the problem and understanding the system. We present a framework for metric development and then describe tools that can be used for CDS design (eg, 5 Rights of CDS, "10 commandments," usability heuristics, human-centered design) and testing (eg, validation, simulation, usability testing). We review approaches to evaluating CDS tools, which range from randomized studies to traditional quality improvement methods. Lastly, we discuss practical considerations for implementing CDS, including the assessment of a project team's skills and an organization's information technology resources.

Guidance for reporting analyses of metadata on electronic health record use

INTRODUCTION

Research on how people interact with electronic health records (EHRs) increasingly involves the analysis of metadata on EHR use. These metadata can be recorded unobtrusively and capture EHR use at a scale unattainable through direct observation or self-reports. However, there is substantial variation in how metadata on EHR use are recorded, analyzed and described, limiting understanding, replication, and synthesis across studies.

RECOMMENDATIONS

In this perspective, we provide guidance to those working with EHR use metadata by describing 4 common types, how they are recorded, and how they can be aggregated into higher-level measures of EHR use. We also describe guidelines for reporting analyses of EHR use metadata-or measures of EHR use derived from them-to foster clarity, standardization, and reproducibility in this emerging and critical area of research.

Addressing Alert Fatigue by Replacing a Burdensome Interruptive Alert with Passive Clinical Decision Support

BACKGROUND

Recognizing that alert fatigue poses risks to patient safety and clinician wellness, there is a growing emphasis on evaluation and governance of electronic health record clinical decision support (CDS). This is particularly critical for interruptive alerts to ensure that they achieve desired clinical outcomes while minimizing the burden on clinicians. This study describes an improvement effort to address a problematic interruptive alert intended to notify clinicians about patients needing coronavirus disease 2019 (COVID) precautions and how we collaborated with operational leaders to develop an alternative passive CDS system in acute care areas.

OBJECTIVES

Our dual aim was to reduce the alert burden by redesigning the CDS to adhere to best practices for decision support while also improving the percent of admitted patients with symptoms of possible COVID who had appropriate and timely infection precautions orders.

METHODS

Iterative changes to CDS design included adjustment to alert triggers and acknowledgment reasons and development of a noninterruptive rule-based order panel for acute care areas. Data on alert burden and appropriate precautions orders on symptomatic admitted patients were followed over time on run and attribute (p) and individuals-moving range control charts.

RESULTS

At baseline, the COVID alert fired on average 8,206 times per week with an alert per encounter rate of 0.36. After our interventions, the alerts per week decreased to 1,449 and alerts per encounter to 0.07 equating to an 80% reduction for both metrics. Concurrently, the percentage of symptomatic admitted patients with COVID precautions ordered increased from 23 to 61% with a reduction in the mean time between COVID test and precautions orders from 19.7 to -1.3 minutes.

CONCLUSION

CDS governance, partnering with operational stakeholders, and iterative design led to successful replacement of a frequently firing interruptive alert with less burdensome passive CDS that improved timely ordering of COVID precautions.

Comparative effectiveness of generic commercial versus locally customized clinical decision support tools to reduce prescription of nonsteroidal anti-inflammatory drugs for patients with heart failure

OBJECTIVE

To compare the effectiveness of 2 clinical decision support (CDS) tools to avoid prescription of nonsteroidal anti-inflammatory drugs (NSAIDs) in patients with heart failure (HF): a "commercial" and a locally "customized" alert.

METHODS

We conducted a retrospective cohort study of 2 CDS tools implemented within a large integrated health system. The commercial CDS tool was designed according to third-party drug content and EHR vendor specifications. The customized CDS tool underwent a user-centered design process informed by implementation science principles, with input from a cross disciplinary team. The customized CDS tool replaced the commercial CDS tool. Data were collected from the electronic health record via analytic reports and manual chart review. The primary outcome was effectiveness, defined as whether the clinician changed their behavior and did not prescribe an NSAID.

RESULTS

A random sample of 366 alerts (183 per CDS tool) was evaluated that represented 355 unique patients. The commercial CDS tool was effective for 7 of 172 (4%) patients, while the customized CDS tool was effective for 81 of 183 (44%) patients. After adjusting for age, chronic kidney disease, ejection fraction, NYHA class, concurrent prescription of an opioid or acetaminophen, visit type (inpatient or outpatient), and clinician specialty, the customized alerts were at 24.3 times greater odds of effectiveness compared to the commercial alerts (OR: 24.3 CI: 10.20-58.06).

CONCLUSION

Investing additional resources to customize a CDS tool resulted in a CDS tool that was more effective at reducing the total number of NSAID orders placed for patients with HF compared to a commercially available CDS tool.

Trends in computerized provider order entry: 20-year bibliometric overview

Background

Drug-related problems (DRPs) can lead to serious health issues and have significant economic impacts on healthcare systems. One solution to address this issue is the use of computerized physician order entry systems (CPOE), which can help prevent DRPs by reducing the risk of medication errors.

Objective

The purpose of this study is to provide an analysis on scientific production of the past 20 years in order to describe trends in academic publishing on CPOE and to identify the major topics as well as the predominant actors (journals, countries) involved in this field.

Methods

A PubMed search was carried out to extract articles related to computerized provider order entry during the period January 1st 2003- December 31st 2022 using a specific query. Data were downloaded from PubMed in Extensible Markup Language (XML) and were processed through a dedicated parser.

Results

A total of 2,946 articles were retrieved among 623 journals. One third of these articles were published in eight journals. Publications grew strongly from 2002 to 2006, with a dip in 2008 followed by an increase again in 2009. After 2009, there follows a decreasing until 2022.The most producing countries are the USA with 51.39% of the publication over the period by France (3.80%), and Canada (3.77%). About disciplines, the top 3 is: "medical informatics" (21.62% of articles), "pharmacy" (19.04%), and "pediatrics" (6.56%).

Discussion

This study provides an overview of publication trends related to CPOE, which exhibited a significant increase in the first decade of the 21st century followed by a decline after 2009. Possible reasons for this decline include the emergence of digital health tools beyond CPOE, as well as healthcare professionals experiencing alert fatigue of the current system.

Conclusion

Future research should focus on analyzing publication trends in the field of medical informatics and decision-making tools to identify other areas of interest that may have surpassed the development of CPOE.

Effectiveness, quality and implementation of pain, sedation, delirium, and iatrogenic withdrawal syndrome algorithms in pediatric intensive care: a systematic review and meta-analysis

Background

Pain, sedation, delirium, and iatrogenic withdrawal syndrome are conditions that often coexist, algorithms can be used to assist healthcare professionals in decision making. However, a comprehensive review is lacking. This systematic review aimed to assess the effectiveness, quality, and implementation of algorithms for the management of pain, sedation, delirium, and iatrogenic withdrawal syndrome in all pediatric intensive care settings.

Methods

A literature search was conducted on November 29, 2022, in PubMed, Embase, CINAHL and Cochrane Library, ProQuest Dissertations & Theses, and Google Scholar to identify algorithms implemented in pediatric intensive care and published since 2005. Three reviewers independently screened the records for inclusion, verified and extracted data. Included studies were assessed for risk of bias using the JBI checklists, and algorithm quality was assessed using the PROFILE tool (higher % = higher quality). Meta-analyses were performed to compare algorithms to usual care on various outcomes (length of stay, duration and cumulative dose of analgesics and sedatives, length of mechanical ventilation, and incidence of withdrawal).

Results

From 6,779 records, 32 studies, including 28 algorithms, were included. The majority of algorithms (68%) focused on sedation in combination with other conditions. Risk of bias was low in 28 studies. The average overall quality score of the algorithm was 54%, with 11 (39%) scoring as high quality. Four algorithms used clinical practice guidelines during development. The use of algorithms was found to be effective in reducing length of stay (intensive care and hospital), length of mechanical ventilation, duration of analgesic and sedative medications, cumulative dose of analgesics and sedatives, and incidence of withdrawal. Implementation strategies included education and distribution of materials (95%). Supportive determinants of algorithm implementation included leadership support and buy-in, staff training, and integration into electronic health records. The fidelity to algorithm varied from 8.2% to 100%.

Conclusions

The review suggests that algorithm-based management of pain, sedation and withdrawal is more effective than usual care in pediatric intensive care settings. There is a need for more rigorous use of evidence in the development of algorithms and the provision of details on the implementation process.

Systematic Review Registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021276053, PROSPERO [CRD42021276053].

Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework

Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.

Pseudorandomized Testing of a Discharge Medication Alert to Reduce Free-Text Prescribing

BACKGROUND

Pseudorandomized testing can be applied to perform rigorous yet practical evaluations of clinical decision support tools. We apply this methodology to an interruptive alert aimed at reducing free-text prescriptions. Using free-text instead of structured computerized provider order entry elements can cause medication errors and inequity in care by bypassing medication-based clinical decision support tools and hindering automated translation of prescription instructions.

OBJECTIVE

The objective of this study is to evaluate the effectiveness of an interruptive alert at reducing free-text prescriptions via pseudorandomized testing using native electronic health records (EHR) functionality.

METHODS

Two versions of an EHR alert triggered when a provider attempted to sign a discharge free-text prescription. The visible version displayed an interruptive alert to the user, and a silent version triggered in the background, serving as a control. Providers were assigned to the visible and silent arms based on even/odd EHR provider IDs. The proportion of encounters with a free-text prescription was calculated across the groups. Alert trigger rates were compared in process control charts. Free-text prescriptions were analyzed to identify prescribing patterns.

RESULTS

Over the 28-week study period, 143 providers triggered 695 alerts (345 visible and 350 silent). The proportions of encounters with free-text prescriptions were 83% (266/320) and 90% (273/303) in the intervention and control groups, respectively (p = 0.01). For the active alert, median time to action was 31 seconds. Alert trigger rates between groups were similar over time. Ibuprofen, oxycodone, steroid tapers, and oncology-related prescriptions accounted for most free-text prescriptions. A majority of these prescriptions originated from user preference lists.

CONCLUSION

An interruptive alert was associated with a modest reduction in free-text prescriptions. Furthermore, the majority of these prescriptions could have been reproduced using structured order entry fields. Targeting user preference lists shows promise for future intervention.

Clinical Decision Support in the PICU: Implications for Design and Evaluation

OBJECTIVES

To assess the current landscape of clinical decision support (CDS) tools in PICUs in order to identify priority areas of focus in this field.

DESIGN

International, quantitative, cross-sectional survey.

SETTING

Role-specific, web-based survey administered in November and December 2020.

SUBJECTS

Medical directors, bedside nurses, attending physicians, and residents/advanced practice providers at Pediatric Acute Lung Injury and Sepsis Network-affiliated PICUs.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The survey was completed by 109 respondents from 45 institutions, primarily attending physicians from university-affiliated PICUs in the United States. The most commonly used CDS tools were people-based resources (93% used always or most of the time) and laboratory result highlighting (86%), with order sets, order-based alerts, and other electronic CDS tools also used frequently. The most important goal providers endorsed for CDS tools were a proven impact on patient safety and an evidence base for their use. Negative perceptions of CDS included concerns about diminished critical thinking and the burden of intrusive processes on providers. Routine assessment of existing CDS was rare, with infrequent reported use of observation to assess CDS impact on workflows or measures of individual alert burden.

CONCLUSIONS

Although providers share some consensus over CDS utility, we identified specific priority areas of research focus. Consensus across practitioners exists around the importance of evidence-based CDS tools having a proven impact on patient safety. Despite broad presence of CDS tools in PICUs, practitioners continue to view them as intrusive and with concern for diminished critical thinking. Deimplementing ineffective CDS may mitigate this burden, though postimplementation evaluation of CDS is rare.

Appropriateness of Alerts and Physicians’ Responses with a Medication-related Clinical Decision Support System (Preprint)

Background

Alert fatigue is unavoidable when many irrelevant alerts are generated in response to a small number of useful alerts. It is necessary to increase the effectiveness of the clinical decision support system (CDSS) by understanding physicians’ responses.

Objective

This study aimed to understand the CDSS and physicians’ behavior by evaluating the clinical appropriateness of alerts and the corresponding physicians’ responses in a medication-related passive alert system.

Methods

Data on medication-related orders, alerts, and patients’ electronic medical records were analyzed. The analyzed data were generated between August 2019 and June 2020 while the patient was in the emergency department. We evaluated the appropriateness of alerts and physicians’ responses for a subset of 382 alert cases and classified them.

Results

Of the 382 alert cases, only 7.3% (n=28) of the alerts were clinically appropriate. Regarding the appropriateness of the physicians’ responses about the alerts, 92.4% (n=353) were deemed appropriate. In the classification of alerts, only 3.4% (n=13) of alerts were successfully triggered, and 2.1% (n=8) were inappropriate in both alert clinical relevance and physician’s response. In this study, the override rate was 92.9% (n=355).

Conclusions

We evaluated the appropriateness of alerts and physicians’ responses through a detailed medical record review of the medication-related passive alert system. An excessive number of unnecessary alerts are generated, because the algorithm operates as a rule base without reflecting the individual condition of the patient. It is important to maximize the value of the CDSS by comprehending physicians’ responses.

Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts

Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.