Emergency department monitor alarms rarely change clinical management: An observational study

Real-Time Machine Learning Alerts to Prevent Escalation of Care: A Nonrandomized Clustered Pragmatic Clinical Trial

OBJECTIVES

Machine learning algorithms can outperform older methods in predicting clinical deterioration, but rigorous prospective data on their real-world efficacy are limited. We hypothesized that real-time machine learning generated alerts sent directly to front-line providers would reduce escalations.

DESIGN

Single-center prospective pragmatic nonrandomized clustered clinical trial.

SETTING

Academic tertiary care medical center.

PATIENTS

Adult patients admitted to four medical-surgical units. Assignment to intervention or control arms was determined by initial unit admission.

INTERVENTIONS

Real-time alerts stratified according to predicted likelihood of deterioration sent either to the primary team or directly to the rapid response team (RRT). Clinical care and interventions were at the providers' discretion. For the control units, alerts were generated but not sent, and standard RRT activation criteria were used.

MEASUREMENTS AND MAIN RESULTS

The primary outcome was the rate of escalation per 1000 patient bed days. Secondary outcomes included the frequency of orders for fluids, medications, and diagnostic tests, and combined in-hospital and 30-day mortality. Propensity score modeling with stabilized inverse probability of treatment weight (IPTW) was used to account for differences between groups. Data from 2740 patients enrolled between July 2019 and March 2020 were analyzed (1488 intervention, 1252 control). Average age was 66.3 years and 1428 participants (52%) were female. The rate of escalation was 12.3 vs. 11.3 per 1000 patient bed days (difference, 1.0; 95% CI, -2.8 to 4.7) and IPTW adjusted incidence rate ratio 1.43 (95% CI, 1.16-1.78; p < 0.001). Patients in the intervention group were more likely to receive cardiovascular medication orders (16.1% vs. 11.3%; 4.7%; 95% CI, 2.1-7.4%) and IPTW adjusted relative risk (RR) (1.74; 95% CI, 1.39-2.18; p < 0.001). Combined in-hospital and 30-day-mortality was lower in the intervention group (7% vs. 9.3%; -2.4%; 95% CI, -4.5% to -0.2%) and IPTW adjusted RR (0.76; 95% CI, 0.58-0.99; p = 0.045).

CONCLUSIONS

Real-time machine learning alerts do not reduce the rate of escalation but may reduce mortality.

Fewer emergency department alarms is associated with reduced use of medications for acute agitation

BACKGROUND AND OBJECTIVES

Patient monitoring systems provide critical information but often produce loud, frequent alarms that worsen patient agitation and stress. This may increase the use of physical and chemical restraints with implications for patient morbidity and autonomy. This study analyzes how augmenting alarm thresholds affects the proportion of alarm-free time and the frequency of medications administered to treat acute agitation.

METHODS

Our emergency department's patient monitoring system was modified on June 28, 2022 to increase the tachycardia alarm threshold from 130 to 150 and to remove alarm sounds for several arrhythmias, including bigeminy and premature ventricular beats. A pre-post study was performed lasting 55 days before and 55 days after this intervention. The primary outcome was change in number of daily patient alarms. The secondary outcomes were alarm-free time per day and median number of antipsychotic and benzodiazepine medications administered per day. The safety outcome was the median number of patients transferred daily to the resuscitation area. We used quantile regression to compare outcomes between the pre- and post-intervention period and linear regression to correlate alarm-free time with the number of sedating medications administered.

RESULTS

Between the pre- and post-intervention period, the median number of alarms per day decreased from 1332 to 845 (-37%). This was primarily driven by reduced low-priority arrhythmia alarms from 262 to 21 (-92%), while the median daily census was unchanged (33 vs 32). Median hours per day free from alarms increased from 1.0 to 2.4 (difference 1.4, 95% CI 0.8-2.1). The median number of sedating medications administered per day decreased from 14 to 10 (difference - 4, 95% CI -1 to -7) while the number of escalations in level of care to our resuscitation care area did not change significantly. Multivariable linear regression showed a 60-min increase of alarm-free time per day was associated with 0.8 (95% CI 0.1-1.4) fewer administrations of sedating medication while an additional patient on the behavioral health census was associated with 0.5 (95% CI 0.0-1.1) more administrations of sedating medication.

CONCLUSION

A reasonable change in alarm parameter settings may increase the time patients and healthcare workers spend in the emergency department without alarm noise, which in this study was associated with fewer doses of sedating medications administered.

How do people think about the implementation of speech and video recognition technology in emergency medical practice?

Background

Recently, speech and video information recognition technology (SVRT) has developed rapidly. Introducing SVRT into the emergency medical practice process may lead to improvements in health care. The purpose of this study was to evaluate the level of acceptance of SVRT among patients, caregivers and emergency medical staff.

Methods

Structured questionnaires were developed for the patient or caregiver group and the emergency medical staff group. The survey was performed in one tertiary academic hospital emergency department. Questions were optimized for each specific group, and responses were provided mostly using Likert 5-scales. Additional multivariable logistic regression analyses for the whole cohort and subgroups were conducted to calculate odds ratios (OR) and confidence intervals (CI) to examine the association between individual characteristics and SVRT acceptance.

Results

Of 264 participants, respondents demonstrated a positive attitude and acceptance toward SVRT and artificial intelligence (AI) in future; 179 (67.8%) for video recordings, and 190 (72.0%) for speech recordings. A multivariable logistic regression model revealed that several factors were associated with acceptance of SVRT in emergency medical practice: belief in health care improvement by signal analysis technology (OR, 95% CIs: 2.48 (1.15–5.42)) and AI (OR, 95% CIs: 1.70 (0.91–3.17)), reliability of AI application in emergency medicine (OR, 95% CIs: 2.36 (1.28–4.35)) and the security of personal information (OR, 95% CIs: 1.98 (1.10–3.63)).

Conclusion

A high level of acceptance toward SVRT has been shown in patients or caregivers, and it also appears to be associated with positive attitudes toward new technology, AI and security of personal information.

Nurses’ clinical alarm-related behaviors and influencing factors in China†

Objective

To explore the nurses’ behaviors regarding clinical alarms, analyze the related influencing factors, and provide rationales for alarm management.

Methods

A cross-sectional survey was conducted in China. The self-made questionnaire of nurses’ clinical alarm-related knowledge, attitude, and behavior (NCAKAB) was used.

Results

The valid response rate was 98.66% (n = 2368). The average nurses’ clinical alarm-related behaviors (NCAB) score was 65.14 ± 7.95 (out of 85). The dimension scores of NCAB from high to low were alarm learning (4.02 ± 0.85, out of 5), alarm response (27.99 ± 3.64, out of 35), alarm setting (19.24 ± 3.88, out of 25), alarm recognition (7.63 ± 1.68, out of 10) and alarm notification (6.25 ± 1.84, out of 10). There were significant differences in alarm behavior scores between nurses of different ages (F = 4.619, P = 0.000), nursing stints (F = 9.564, P = 0.000), professional titles (F = 4.425, P = 0.004), departments (F = 9.166, P = 0.000), and hospital levels (t = 2.705, P = 0.007). The study showed that nurses’ total alarm behavior scores were positively correlated with the total alarm knowledge score (r = 0.267; P < 0.001) and the total alarm attitude score (r = 438; P < 0.001).

Conclusions

Nurses scored highest in alarm learning, followed by alarm response, alarm setting, alarm recognition, and alarm notification behavior. The factors that influenced alarm behavior included age, title, department, nursing stint, hospital level, professional title, alarm-related training, willingness to participate in alarm-related training, whether or not departments have improved alarm management over the last 3 years, and whether or not departments have formulated norms for alarm management. Nurses with higher scores for clinical alarm knowledge had higher correlating scores for alarm behavior; similarly, nurses with higher scores for clinical alarm attitude had higher scores for alarm-related behavior.

The Influence of Audible Alarm Loudness and Type on Clinical Multitasking

In high-consequence industries such as health care, auditory alarms are an important aspect of an informatics system that monitors patients and alerts providers attending to multiple concurrent tasks. Alarms levels are unnecessarily high and alarm signals are uninformative. In a laboratory-based task setting, we studied 25 anesthesiology residents' responses to auditory alarms in a multitasking paradigm comprised of three tasks: patient monitoring, speech perception/intelligibility, and visual vigilance. These tasks were in the presence of background noise plus/minus music, which served as an attention-diverting stimulus. Alarms signified clinical decompensation and were either conventional alarms or a novel informative auditory icon alarm. Both alarms were presented at four different levels. Task performance (accuracy and response times) were analyzed using logistic and linear mixed-effects regression. Salient findings were 1), the icon alarm had similar performance to the conventional alarm at a +2 dB signal-to-noise-ratio (SNR) (accuracy: OR 1.21 (95% CI 0.88, 1.67), response time: 0.04 s at 2 dB (95% CI: -0.16, 0.24), which is a much lower level than current clinical environments; 2) the icon alarm was associated with 27% greater odds (95% CI: 18%, 37%) of correctly addressing the vigilance task, regardless of alarm SNR, suggesting crossmodal/multisensory multitasking benefits; and 3) compared to the conventional alarm, the icon alarm was associated with an absolute improvement in speech perception of 4% in the presence of an attention-diverting auditory stimulus (p = 0.031). These findings suggest that auditory icons can provide multitasking benefits in cognitively demanding clinical environments.

Physiological deterioration in the Emergency Department: The SNAP40-ED study

Continuous novel ambulatory monitoring may detect deterioration in Emergency Department (ED) patients more rapidly, prompting treatment and preventing adverse events. Single-centre, open-label, prospective, observational cohort study recruiting high/medium acuity (Manchester triage category 2 and 3) participants, aged over 16 years, presenting to ED. Participants were fitted with a novel wearable monitoring device alongside standard clinical care (wired monitoring and/or manual clinical staff vital sign recording) and observed for up to 4 hours in the ED. Primary outcome was time to detection of deterioration. Two-hundred and fifty (250) patients were enrolled. In 82 patients (32.8%) with standard monitoring (wired monitoring and/or manual clinical staff vital sign recording), deterioration in at least one vital sign was noted during their four-hour ED stay. Overall, the novel device detected deterioration a median of 34 minutes earlier than wired monitoring (Q1, Q3 67,194; n=73, mean difference 39.48, p<0.0001). The novel device detected deterioration a median of 24 minutes (Q1, Q3 2,43; n=42) earlier than wired monitoring and 65 minutes (Q1, Q3 28,114; n=31) earlier than manual vital signs. Deterioration in physiology was common in ED patients. ED staff spent a significant amount of time performing observations and responding to alarms, with many not escalated. The novel device detected deterioration significantly earlier than standard care.

Clinical alarms and alarm fatigue in a University Hospital Emergency Department-A retrospective data analysis

BACKGROUND

Alarm fatigue is hypothesized to be caused by vast amount of patient monitor alarms. Objectives were to study the frequency and types of patient monitor alarms, to evaluate alarm fatigue, and to find unit specific alarm threshold values in a university hospital emergency department.

METHODS

We retrospectively gathered alarm data from 9 September to 6 October 2019, in Jorvi Hospital Emergency department, Finland. The department treats surgical, internal and general medicine patients aged 16 and older. The number of patients is on average 4600 to 5000 per month. Eight out of 46 monitors were used for data gathering and the monitored modalities included electrocardiography, respiratory rate, blood pressure, and pulse oximetry.

RESULTS

Total number of alarms in the study monitors was 28 176. Number of acknowledged alarms (ie acknowledgement indicator pressed in the monitor) was 695 (2.5%). The most common alarm types were: Respiratory rate high, 9077 (32.2%), pulse oximetry low, 4572 (16.2%) and pulse oximetry probe off, 4036 (14.3%). Number of alarms with duration under 10 s was 14 936 (53%). Number of individual alarm sounds was 105 000, 469 per monitor per day. Of respiratory rate high alarms, 2846 (31.4%) had initial value below 30 breaths min^-1 . Of pulse oximetry low alarms, 2421 (53.0%) had initial value above 88%.

CONCLUSIONS

Alarm sound load, from individual alarm sounds, was nearly continuous in an emergency department observation room equipped with nine monitors. Intervention by the staff to the alarms was infrequent. More than half of the alarms were momentary.

Automation failures and patient safety

PURPOSE OF REVIEW

The goal of automation is to decrease the anesthesiologist's workload and to decrease the possibility of human error. Automated systems introduce problems of its own, however, including loss of situation awareness, leaving the physician out of the loop, and training physicians how to monitor autonomous systems. This review will discuss the growing role of automated systems in healthcare and describe two types of automation failures.

RECENT FINDINGS

An automation surprise occurs when an automated system takes an action that is unexpected by the user. Mode confusion occurs when the operator does not understand what an automated system is programmed to do and may prevent the clinician from fully understanding what the device is doing during a critical event. Both types of automation failures can decrease a clinician's trust in the system. They may also prevent a clinician from regaining control of a failed system (e.g., a ventilator that is no longer working) during a critical event.

SUMMARY

Clinicians should receive generalized training on how to manage automation and should also be required to demonstrate competency before using medical equipment that employs automation, including electronic health records, infusion pumps, and ventilators.

MEWS++: Enhancing the Prediction of Clinical Deterioration in Admitted Patients through a Machine Learning Model

Early detection of patients at risk for clinical deterioration is crucial for timely intervention. Traditional detection systems rely on a limited set of variables and are unable to predict the time of decline. We describe a machine learning model called MEWS++ that enables the identification of patients at risk of escalation of care or death six hours prior to the event. A retrospective single-center cohort study was conducted from July 2011 to July 2017 of adult (age > 18) inpatients excluding psychiatric, parturient, and hospice patients. Three machine learning models were trained and tested: random forest (RF), linear support vector machine, and logistic regression. We compared the models’ performance to the traditional Modified Early Warning Score (MEWS) using sensitivity, specificity, and Area Under the Curve for Receiver Operating Characteristic (AUC-ROC) and Precision-Recall curves (AUC-PR). The primary outcome was escalation of care from a floor bed to an intensive care or step-down unit, or death, within 6 h. A total of 96,645 patients with 157,984 hospital encounters and 244,343 bed movements were included. Overall rate of escalation or death was 3.4%. The RF model had the best performance with sensitivity 81.6%, specificity 75.5%, AUC-ROC of 0.85, and AUC-PR of 0.37. Compared to traditional MEWS, sensitivity increased 37%, specificity increased 11%, and AUC-ROC increased 14%. This study found that using machine learning and readily available clinical data, clinical deterioration or death can be predicted 6 h prior to the event. The model we developed can warn of patient deterioration hours before the event, thus helping make timely clinical decisions.