Derivation and Validation of Predictive Factors for Clinical Deterioration after Admission in Emergency Department Patients Presenting with Abnormal Vital Signs Without Shock

Patient stratification based on the risk of severe illness in emergency departments through collaborative machine learning models

OBJECTIVES

Emergency department (ED) overcrowding presents a global challenge that inhibits prompt care for critically ill patients. Traditional 5-level triage system that heavily rely on the judgment of the triage staff could fail to detect subtle symptoms in critical patients, thus leading to delayed treatment. Unlike previous rivalry-focused approaches, our study aimed to establish a collaborative machine learning (ML) model that renders risk scores for severe illness, which may assist the triage staff to provide a better patient stratification for timely critical cares.

METHODS

This retrospective study was conducted at a tertiary teaching hospital. Data were collected from January 2015 to October 2022. Demographic and clinical information were collected at triage. The study focused on severe illness as the outcome. We developed artificial neural network (ANN) models, with or without utilizing the Taiwan Triage and Acuity Scale (TTAS) score as one of the predictors. The model using the TTAS score is termed a machine-human collaborative model (ANN-MH), while the model without it is referred to as a machine-only model (ANN-MO). The predictive power of these models was assessed using the area under the receiver-operating-characteristic (AUROC) and the precision-recall curves (AUPRC); their sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score were compared.

RESULTS

The study analyzed 668,602 ED visits from 2015 to 2022. Among them, 278,724 visits from 2015 to 2018 were used for model training and validation, while 320,201 visits from 2019 to 2022 were for testing model performance. Approximately 2.6% of visits were by severely ill patients, whose TTAS scores ranged from 1 to 5. The ANN-MH model achieved a testing AUROC of 0.918 and AUPRC of 0.369, while for the ANN-MO model the AUROC and AUPRC were 0.909 and 0.339, respectively. Based on these metrics, the ANN-MH model outperformed the ANN-MO model, and both surpassed human triage classification. Subgroup analyses further highlighted the models' capability to identify higher-risk patients within the same triage level.

CONCLUSIONS

The traditional 5-level triage system often falls short, leading to under-triage of critical patients. Our models include a score-based differentiation within a triage level to offer advanced risk stratification, thereby promoting patient safety.

Application of Machine Learning Techniques to Development of Emergency Medical Rapid Triage Prediction Models in Acute Care

Given the critical and complex features of medical emergencies, it is essential to develop models that enable prompt and suitable clinical decision-making based on considerable information. Emergency nurses are responsible for categorizing and prioritizing injuries and illnesses on the frontlines of the emergency room. This study aims to create an Emergency Medical Rapid Triage and Prediction Assistance model using electronic medical records and machine learning techniques. Patient information was retrieved from the emergency department of a large regional teaching hospital in Taiwan, and five supervised learning techniques were used to construct classification models for predicting critical outcomes. Of these models, the model using logistic regression had superior prediction performance, with an F1 score of 0.861 and an area under the receiver operating characteristic curve of 0.855. The Emergency Medical Rapid Triage and Prediction Assistance model demonstrated superior performance in predicting intensive care and hospitalization outcomes compared with the Taiwan Triage and Acuity Scale and three clinical early warning tools. The proposed model has the potential to assist emergency nurses in executing challenging triage assessments and emergency teams in treating critically ill patients promptly, leading to improved clinical care and efficient utilization of medical resources.

Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department

This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review

Background

Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis.

Objective

This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis.

Methods

The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age.

Results

A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions.

Conclusions

The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed.

International Registered Report Identifier (IRRID)

RR2-10.2196/24899

Potential predictors of hospital length of stay and hospital charges among patients with all-terrain vehicle injuries in rural Northeast Texas

Background:

All-Terrain Vehicles (ATVs) have become popular for recreation use in recent years. Texas has had more ATV related fatalities than any other state in the nation, with rural Northeast Texas having even higher rates of injuries. There is limited data examining the relationship between ATV injuries and the length of hospital stay, as well as hospital costs. This paper examines both issues in children as well as adults.

Methods:

The regional trauma registry was analyzed for all ATV related injuries between January 2011- October 2016. Injury Severity Score, Glasgow Coma Scale and if they are seen at a Level I Trauma center are predictive for both hospital length of stay and charges.

Results:

Length of Stay was predicted positively by Injury Severity Score, Emergency Department Respi-ration Rate and facility at which patients were treated and negatively by Glasgow Coma Scale. Hospital charges were predicted positively by age, Injury Severity Score, facility of treatment, means of transportation, and Emergency Department pulse and negatively by Glasgow Coma Scale.

Conclusions:

The study found that vital signs can be useful in predicting length of stay and hospital charges. This study not only confirms the findings of other studies regarding what predictors can be used, but expands the research into rural traumatic injuries. It is hoped that this data can help contribute to the development of algorithms to predict which patients will be most likely to require resource intensive treatment.

Association of Vital Signs and Process Outcomes in Emergency Department Patients

Introduction

We sought to determine the association of abnormal vital signs with emergency department (ED) process outcomes in both discharged and admitted patients.

Methods

We performed a retrospective review of five years of operational data at a single site. We identified all visits for patients 18 and older who were discharged home without ancillary services, and separately identified all visits for patients admitted to a floor (ward) bed. We assessed two process outcomes for discharged visits (returns to the ED within 72 hours and returns to the ED within 72 hours resulting in admission) and two process outcomes for admitted patients (transfer to a higher level of care [intermediate care or intensive care] within either six hours or 24 hours of arrival to floor). Last-recorded ED vital signs were obtained for all patients. We report rates of abnormal vital signs in each group, as well as the relative risk of meeting a process outcome for each individual vital sign abnormality.

Results

Patients with tachycardia, tachypnea, or fever more commonly experienced all measured process outcomes compared to patients without these abnormal vitals; admitted hypotensive patients more frequently required transfer to a higher level of care within 24 hours.

Conclusion

In a single facility, patients with abnormal last-recorded ED vital signs experienced more undesirable process outcomes than patients with normal vitals. Vital sign abnormalities may serve as a useful signal in outcome forecasting.

Repeated vital sign measurements in the emergency department predict patient deterioration within 72 hours: a prospective observational study

BACKGROUND

More than one in five patients presenting to the emergency department (ED) with (suspected) infection or sepsis deteriorate within 72 h from admission. Surprisingly little is known about vital signs in relation to deterioration, especially in the ED. The aim of our study was to determine whether repeated vital sign measurements in the ED can differentiate between patients who will deteriorate within 72 h and patients who will not deteriorate.

METHODS

We performed a prospective observational study in patients presenting with (suspected) infection or sepsis to the ED of our tertiary care teaching hospital. Vital signs (heart rate, mean arterial pressure (MAP), respiratory rate and body temperature) were measured in 30-min intervals during the first 3 h in the ED. Primary outcome was patient deterioration within 72 h from admission, defined as the development of acute kidney injury, liver failure, respiratory failure, intensive care unit admission or in-hospital mortality. We performed a logistic regression analysis using a base model including age, gender and comorbidities. Thereafter, we performed separate logistic regression analyses for each vital sign using the value at admission, the change over time and its variability. For each analysis, the odds ratios (OR) and area under the receiver operator curve (AUC) were calculated.

RESULTS

In total 106 (29.5%) of the 359 patients deteriorated within 72 h from admission. Within this timeframe, 18.3% of the patients with infection and 32.9% of the patients with sepsis at ED presentation deteriorated. Associated with deterioration were: age (OR: 1.02), history of diabetes (OR: 1.90), heart rate (OR: 1.01), MAP (OR: 0.96) and respiratory rate (OR: 1.05) at admission, changes over time of MAP (OR: 1.04) and respiratory rate (OR: 1.44) as well as the variability of the MAP (OR: 1.06). Repeated measurements of heart rate and body temperature were not associated with deterioration.

CONCLUSIONS

Repeated vital sign measurements in the ED are better at identifying patients at risk for deterioration within 72 h from admission than single vital sign measurements at ED admission.

A Sepsis-related Diagnosis Impacts Interventions and Predicts Outcomes for Emergency Patients with Severe Sepsis

Introduction

Many patients meeting criteria for severe sepsis are not given a sepsis-related diagnosis by emergency physicians (EP). This study 1) compares emergency department (ED) interventions and in-hospital outcomes among patients with severe sepsis, based on the presence or absence of sepsis-related diagnosis, and 2) assesses how adverse outcomes relate to three-hour sepsis bundle completion among patients fulfilling severe sepsis criteria but not given a sepsis-related diagnosis.

Methods

We performed a retrospective cohort study using patients meeting criteria for severe sepsis at two urban, academic tertiary care centers from March 2015 through May 2015. We included all ED patients with the following: 1) the 1992 Consensus definition of severe sepsis, including two or more systemic inflammatory response syndrome criteria and evidence of organ dysfunction; or 2) physician diagnosis of severe sepsis or septic shock. We excluded patients transferred to or from another hospital and those <18 years old. Patients with an EP-assigned sepsis diagnosis created the “Physician Diagnosis” group; the remaining patients composed the “Consensus Criteria” group. The primary outcome was in-hospital mortality. Secondary outcomes included completed elements of the current three-hour sepsis bundle; non-elective intubation; vasopressor administration; intensive care unit (ICU) admission from the ED; and transfer to the ICU in < 24 hours. We compared proportions of each outcome between groups using the chi-square test, and we also performed a stratified analysis using chi square to assess the association between failure to complete the three-hour bundle and adverse outcomes in each group.

Results

Of 418 patients identified with severe sepsis we excluded 54, leaving 364 patients for analysis: 121 “Physician Diagnosis” and 243 “Consensus Criteria.” The “Physician Diagnosis” group had a higher in-hospital mortality (12.4% vs 3.3%, P < 0.01) and compliance with the three-hour sepsis bundle (52.1% vs 20.2%, P < 0.01) compared with the “Consensus Criteria” group. An incomplete three-hour sepsis bundle was not associated with a higher incidence of death, intubation, vasopressor use, ICU admission or transfer to the ICU in <24 hours in patients without a sepsis diagnosis.

Conclusion

“Physician Diagnosis” patients more frequently received sepsis-specific interventions and had a higher incidence of mortality. “Consensus Criteria” patients had infrequent adverse outcomes regardless of three-hour bundle compliance. EPs’ sepsis diagnoses reflect risk-stratification beyond the severe sepsis criteria.

Serum Lactate Predicts Adverse Outcomes in Emergency Department Patients With and Without Infection

INTRODUCTION

Lactate levels are increasingly used to risk stratify emergency department (ED) patients with and without infection. Whether a serum lactate provides similar prognostic value across diseases is not fully elucidated. This study assesses the prognostic value of serum lactate in ED patients with and without infection to both report and compare relative predictive value across etiologies.

METHODS

We conducted a prospective, observational study of ED patients displaying abnormal vital signs (AVS) (heart rate ≥130 bpm, respiratory rate ≥24 bpm, shock index ≥1, and/or systolic blood pressure <90 mmHg). The primary outcome, deterioration, was a composite of acute renal failure, non-elective intubation, vasopressor administration or in-hospital mortality.

RESULTS

Of the 1,152 patients with AVS who were screened, 488 patients met the current study criteria: 34% deteriorated and 12.5% died. The deterioration rate was 88/342 (26%, 95% CI: 21 - 30%) for lactate < 2.5 mmol/L, 47/90 (52%, 42 - 63%) for lactate 2.5 - 4.0 mmol/L, and 33/46 (72%, 59 - 85%) for lactate >4.0mmol/L. Trended stratified lactate levels were associated with deterioration for both infected (p<0.01) and non-infected (p<0.01) patients. In the logistic regression models, lactate > 4mmol/L was an independent predictor of deterioration for patients with infection (OR 4.8, 95% CI: 1.7 - 14.1) and without infection (OR 4.4, 1.7 - 11.5).

CONCLUSION

Lactate levels can risk stratify patients with AVS who have increased risk of adverse outcomes regardless of infection status.