Predicting mortality among septic patients presenting to the emergency department-a cross sectional analysis using machine learning

Improving triage performance in emergency departments using machine learning and natural language processing: a systematic review

BACKGROUND

In Emergency Departments (EDs), triage is crucial for determining patient severity and prioritizing care, typically using the Manchester Triage Scale (MTS). Traditional triage systems, reliant on human judgment, are prone to under-triage and over-triage, resulting in variability, bias, and incorrect patient classification. Studies suggest that Machine Learning (ML) and Natural Language Processing (NLP) could enhance triage accuracy and consistency. This review analyzes studies on ML and/or NLP algorithms for ED patient triage.

METHODS

Following Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines, we conducted a systematic review across five databases: Web of Science, PubMed, Scopus, IEEE Xplore, and ACM Digital Library, from their inception of each database to October 2023. The risk of bias was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). Only articles employing at least one ML and/or NLP method for patient triage classification were included.

RESULTS

Sixty studies covering 57 ML algorithms were included. Logistic Regression (LR) was the most used model, while eXtreme Gradient Boosting (XGBoost), decision tree-based algorithms with Gradient Boosting (GB), and Deep Neural Networks (DNNs) showed superior performance. Frequent predictive variables included demographics and vital signs, with oxygen saturation, chief complaints, systolic blood pressure, age, and mode of arrival being the most retained. The ML algorithms showed significant bias risk due to critical bias assessment in classification models.

CONCLUSION

NLP methods improved ML algorithms' classification capability using triage nursing and medical notes and structured clinical data compared to algorithms using only structured data. Feature engineering (FE) and class imbalance correction methods enhanced ML workflows' performance, but FE and eXplainable Artificial Intelligence (XAI) were underexplored in this field. Registration and funding. This systematic review has been registered (registration number: CRD42024604529) in the International Prospective Register of Systematic Reviews (PROSPERO) and can be accessed online at the following URL: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=604529 . Funding for this work was provided by the National Council for Scientific and Technological Development (CNPq), Brazil.

Machine learning-based prognostic model for 30-day mortality prediction in Sepsis-3

BACKGROUND

Sepsis poses a critical threat to hospitalized patients, particularly those in the Intensive Care Unit (ICU). Rapid identification of Sepsis is crucial for improving survival rates. Machine learning techniques offer advantages over traditional methods for predicting outcomes. This study aimed to develop a prognostic model using a Stacking-based Meta-Classifier to predict 30-day mortality risks in Sepsis-3 patients from the MIMIC-III database.

METHODS

A cohort of 4,240 Sepsis-3 patients was analyzed, with 783 experiencing 30-day mortality and 3,457 surviving. Fifteen biomarkers were selected using feature ranking methods, including Extreme Gradient Boosting (XGBoost), Random Forest, and Extra Tree, and the Logistic Regression (LR) model was used to assess their individual predictability with a fivefold cross-validation approach for the validation of the prediction. The dataset was balanced using the SMOTE-TOMEK LINK technique, and a stacking-based meta-classifier was used for 30-day mortality prediction. The SHapley Additive explanations analysis was performed to explain the model's prediction.

RESULTS

Using the LR classifier, the model achieved an area under the curve or AUC score of 0.99. A nomogram provided clinical insights into the biomarkers' significance. The stacked meta-learner, LR classifier exhibited the best performance with 95.52% accuracy, 95.79% precision, 95.52% recall, 93.65% specificity, and a 95.60% F1-score.

CONCLUSIONS

In conjunction with the nomogram, the proposed stacking classifier model effectively predicted 30-day mortality in Sepsis patients. This approach holds promise for early intervention and improved outcomes in treating Sepsis cases.

Development of a Clinical Score for Predicting 28-Day Mortality in Geriatric Sepsis Patients; a Cohort study

Introduction

Sepsis is a significant and common cause of death and burden among critically ill patients, which has increasing incidence and mortality in adults over 60 and advanced age. This study aimed to develop an easy-to-use clinical tool for assessing 28-day mortality risk in older sepsis patients upon their initial assessment in the emergency department (ED).

Method

A retrospective cohort study was conducted using electronic medical records of older (≥60 years) ED patients with suspected sepsis from August 1, 2018, to December 31, 2018. A new prediction score was formulated based on the logistic coefficients of clinical predictors through multivariable regression analyses. Then, the score's screening performance was evaluated and compared to existing scoring systems; Systemic Inflammatory Response Syndrome (SIRS), quick Sequential Organ Failure Assessment (qSOFA), National early warning score (NEWS), and The Ramathibodi early warning score (REWS); using receiver operating characteristic curve analysis (AuROC).

Result

The study included 599 patients with the mean age of 77.13 (range: 60-101) years (56.43% male) and an overall 28-day mortality rate of 7.01%. The newly developed prediction score had seven independent predictors of 28-day mortality: malignancy, dependent status, heart rate, respiratory rate, oxygen saturation, consciousness, and lactate, which demonstrated excellent discriminative ability (AuROC: 0.87, 95% confidence interval (CI): 0.82 - 0.92), significantly outperforming SIRS (AuROC: 0.62), qSOFA (AuROC: 0.72), NEWS (AuROC: 0.74), and REWS (AuROC: 0.71), all with p-values <0.01. The score allowed risk stratification into low-risk (positive likelihood ratio (LR+): 0.37, 95% CI: 0.24 - 0.58) and high-risk (LR+: 4.14, 95% CI: 3.14 - 5.44) groups with sensitivity of 69.0% and specificity of 83.3% at a cut-off point of 6.

Conclusion

The novel prediction score demonstrates a remarkable ability to predict 28-day mortality risk in older sepsis patients during their initial ED assessment, offering potential for improved risk stratification and treatment guidance in older patients.

A comparative study of explainable ensemble learning and logistic regression for predicting in-hospital mortality in the emergency department

This study addresses the challenges associated with emergency department (ED) overcrowding and emphasizes the need for efficient risk stratification tools to identify high-risk patients for early intervention. While several scoring systems, often based on logistic regression (LR) models, have been proposed to indicate patient illness severity, this study aims to compare the predictive performance of ensemble learning (EL) models with LR for in-hospital mortality in the ED. A cross-sectional single-center study was conducted at the ED of Imam Reza Hospital in northeast Iran from March 2016 to March 2017. The study included adult patients with one to three levels of emergency severity index. EL models using Bagging, AdaBoost, random forests (RF), Stacking and extreme gradient boosting (XGB) algorithms, along with an LR model, were constructed. The training and validation visits from the ED were randomly divided into 80% and 20%, respectively. After training the proposed models using tenfold cross-validation, their predictive performance was evaluated. Model performance was compared using the Brier score (BS), The area under the receiver operating characteristics curve (AUROC), The area and precision-recall curve (AUCPR), Hosmer-Lemeshow (H-L) goodness-of-fit test, precision, sensitivity, accuracy, F1-score, and Matthews correlation coefficient (MCC). The study included 2025 unique patients admitted to the hospital's ED, with a total percentage of hospital deaths at approximately 19%. In the training group and the validation group, 274 of 1476 (18.6%) and 152 of 728 (20.8%) patients died during hospitalization, respectively. According to the evaluation of the presented framework, EL models, particularly Bagging, predicted in-hospital mortality with the highest AUROC (0.839, CI (0.802-0.875)) and AUCPR = 0.64 comparable in terms of discrimination power with LR (AUROC (0.826, CI (0.787-0.864)) and AUCPR = 0.61). XGB achieved the highest precision (0.83), sensitivity (0.831), accuracy (0.842), F1-score (0.833), and the highest MCC (0.48). Additionally, the most accurate models in the unbalanced dataset belonged to RF with the lowest BS (0.128). Although all studied models overestimate mortality risk and have insufficient calibration (P > 0.05), stacking demonstrated relatively good agreement between predicted and actual mortality. EL models are not superior to LR in predicting in-hospital mortality in the ED. Both EL and LR models can be considered as screening tools to identify patients at risk of mortality.

Machine learning for the prediction of sepsis-related death: a systematic review and meta-analysis

BACKGROUND AND OBJECTIVES

Sepsis is accompanied by a considerably high risk of mortality in the short term, despite the availability of recommended mortality risk assessment tools. However, these risk assessment tools seem to have limited predictive value. With the gradual integration of machine learning into clinical practice, some researchers have attempted to employ machine learning for early mortality risk prediction in sepsis patients. Nevertheless, there is a lack of comprehensive understanding regarding the construction of predictive variables using machine learning and the value of various machine learning methods. Thus, we carried out this systematic review and meta-analysis to explore the predictive value of machine learning for sepsis-related death at different time points.

METHODS

PubMed, Embase, Cochrane, and Web of Science databases were searched until August 9th, 2022. The risk of bias in predictive models was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). We also performed subgroup analysis according to time of death and type of model and summarized current predictive variables used to construct models for sepsis death prediction.

RESULTS

Fifty original studies were included, covering 104 models. The combined Concordance index (C-index), sensitivity, and specificity of machine learning models were 0.799, 0.81, and 0.80 in the training set, and 0.774, 0.71, and 0.68 in the validation set, respectively. Machine learning outperformed conventional clinical scoring tools and showed excellent C-index, sensitivity, and specificity in different subgroups. Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) are the preferred machine learning models because they showed more favorable accuracy with similar modeling variables. This study found that lactate was the most frequent predictor but was seriously ignored by current clinical scoring tools.

CONCLUSION

Machine learning methods demonstrate relatively favorable accuracy in predicting the mortality risk in sepsis patients. Given the limitations in accuracy and applicability of existing prediction scoring systems, there is an opportunity to explore updates based on existing machine learning approaches. Specifically, it is essential to develop or update more suitable mortality risk assessment tools based on the specific contexts of use, such as emergency departments, general wards, and intensive care units.

Predictors of outcomes in emergency department patients with suspected infections and without fulfillment of the sepsis criteria

BACKGROUND

Data on patient characteristics and determinants of serious outcomes for acutely admitted patients with infections who do not fulfill the sepsis criteria are sparse. The study aimed to characterize acutely admitted emergency department (ED) patients with infections and a composite outcome of in-hospital mortality or transfer to the intensive care unit without fulfilling the criteria for sepsis and to examine predictors of the composite outcome.

METHODS

This was a secondary analysis of data from a prospective observational study of patients with suspected bacterial infection admitted to the ED between October 1, 2017 and March 31, 2018. A National Early Warning Score 2 (NEWS2) ≥ 5 within the first 4 h in the ED was assumed to represent a sepsis-like condition with a high risk for the composite endpoint. Patients who achieved the composite outcome were grouped according to fulfillment of the NEWS2 ≥ 5 criteria. We used logistic regression analysis to estimate the unadjusted and adjusted odds ratio (OR) for the composite endpoint among patients with either NEWS2  < 5 (NEWS2-) or NEWS2  ≥ 5 (NEWS2+).

RESULTS

A total of 2055 patients with a median age of 73 years were included. Of these, 198 (9.6%) achieved the composite endpoint, including 59 (29.8%) NEWS2- and 139 (70.2%) NEWS2+ patients, respectively. Diabetes (OR 2.23;1.23-4.0), a Sequential Organ Failure Assessment (SOFA) score ≥ 2 (OR 2.57;1.37-4.79), and a Do-not-attempt-cardiopulmonary-resuscitation order (DNACPR) on admission (OR 3.70;1.75-7.79) were independent predictive variables for the composite endpoint in NEWS2- patients (goodness-of-fit test P = 0.291; area under the receiver operating characteristic curve for the model (AUROC) = 0.72). The regression model for NEWS2+ patients revealed that a SOFA score ≥ 2 (OR 2.79; 1.59-4.91), hypothermia (OR 2.48;1.30-4.75), and DNACPR order on admission were predictive variables for the composite endpoint (goodness-of-fit test P = 0.62; AUROC for the model = 0.70).

CONCLUSION

Approximately one-third of the patients with infections and serious outcomes during hospitalization did not meet the NEWS2 threshold for likely sepsis. Our study identified factors with independent predictive values for the development of serious outcomes that should be tested in future prediction models.

Machine learning techniques for mortality prediction in emergency departments: a systematic review

OBJECTIVES

This systematic review aimed to assess the performance and clinical feasibility of machine learning (ML) algorithms in prediction of in-hospital mortality for medical patients using vital signs at emergency departments (EDs).

DESIGN

A systematic review was performed.

SETTING

The databases including Medline (PubMed), Scopus and Embase (Ovid) were searched between 2010 and 2021, to extract published articles in English, describing ML-based models utilising vital sign variables to predict in-hospital mortality for patients admitted at EDs. Critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist was used for study planning and data extraction. The risk of bias for included papers was assessed using the prediction risk of bias assessment tool.

PARTICIPANTS

Admitted patients to the ED.

MAIN OUTCOME MEASURE

In-hospital mortality.

RESULTS

Fifteen articles were included in the final review. We found that eight models including logistic regression, decision tree, K-nearest neighbours, support vector machine, gradient boosting, random forest, artificial neural networks and deep neural networks have been applied in this domain. Most studies failed to report essential main analysis steps such as data preprocessing and handling missing values. Fourteen included studies had a high risk of bias in the statistical analysis part, which could lead to poor performance in practice. Although the main aim of all studies was developing a predictive model for mortality, nine articles did not provide a time horizon for the prediction.

CONCLUSION

This review provided an updated overview of the state-of-the-art and revealed research gaps; based on these, we provide eight recommendations for future studies to make the use of ML more feasible in practice. By following these recommendations, we expect to see more robust ML models applied in the future to help clinicians identify patient deterioration earlier.