Developing an Interpretable Machine Learning Model to Predict in-Hospital Mortality in Sepsis Patients: A Retrospective Temporal Validation Study

Machine learning for predicting mortality in adult critically ill patients with Sepsis: A systematic review

INTRODUCTION

Various Machine Learning (ML) models have been used to predict sepsis-associated mortality. We conducted a systematic review to evaluate the methodologies employed in studies to predict mortality among patients with sepsis.

METHODS

Following a pre-established protocol registered at the International Prospective Register of Systematic Reviews, we performed a comprehensive search of databases from inception to February 2024. We included peer-reviewed articles reporting predicting mortality in critically ill adult patients with sepsis.

RESULTS

Among the 1822 articles, 31 were included, involving 1,477,200 adult patients with sepsis. Nineteen studies had a high risk of bias. Among the diverse ML models, Logistic regression and eXtreme Gradient Boosting were the most frequently used, in 22 and 16 studies, respectively. Nine studies performed internal and external validation. Compared with conventional scoring systems such as SOFA, the ML models showed slightly higher performance in predicting mortality (AUROC ranges: 0.62-0.90 vs. 0.47-0.86).

CONCLUSIONS

ML models demonstrate a modest improvement in predicting sepsis-associated mortality. The certainty of these findings remains low due to the high risk of bias and significant heterogeneity. Studies should include comprehensive methodological details on calibration and hyperparameter selection, adopt a standardized definition of sepsis, and conduct multicenter prospective designs along with external validations.

Development and validation of an interpretable machine learning for mortality prediction in patients with sepsis

Introduction

Sepsis is a leading cause of death. However, there is a lack of useful model to predict outcome in sepsis. Herein, the aim of this study was to develop an explainable machine learning (ML) model for predicting 28-day mortality in patients with sepsis based on Sepsis 3.0 criteria.

Methods

We obtained the data from the Medical Information Mart for Intensive Care (MIMIC)-III database (version 1.4). The overall data was randomly assigned to the training and testing sets at a ratio of 3:1. Following the application of LASSO regression analysis to identify the modeling variables, we proceeded to develop models using Extreme Gradient Boost (XGBoost), Logistic Regression (LR), Support Vector Machine (SVM), and Random Forest (RF) techniques with 5-fold cross-validation. The optimal model was selected based on its area under the curve (AUC). Finally, the Shapley additive explanations (SHAP) method was used to interpret the optimal model.

Results

A total of 5,834 septic adults were enrolled, the median age was 66 years (IQR, 54-78 years) and 2,342 (40.1%) were women. After feature selection, 14 variables were included for developing model in the training set. The XGBoost model (AUC: 0.806) showed superior performance with AUC, compared with RF (AUC: 0.794), LR (AUC: 0.782) and SVM model (AUC: 0.687). SHAP summary analysis for XGBoost model showed that urine output on day 1, age, blood urea nitrogen and body mass index were the top four contributors. SHAP dependence analysis demonstrated insightful nonlinear interactive associations between factors and outcome. SHAP force analysis provided three samples for model prediction.

Conclusion

In conclusion, our study successfully demonstrated the efficacy of ML models in predicting 28-day mortality in sepsis patients, while highlighting the potential of the SHAP method to enhance model transparency and aid in clinical decision-making.

Predicting ICU readmission risks in intracerebral hemorrhage patients: Insights from machine learning models using MIMIC databases

BACKGROUND

Intracerebral hemorrhage (ICH) is a stroke subtype characterized by high mortality and complex post-event complications. Research has extensively covered the acute phase of ICH; however, ICU readmission determinants remain less explored. Utilizing the MIMIC-III and MIMIC-IV databases, this investigation develops machine learning (ML) models to anticipate ICU readmissions in ICH patients.

METHODS

Retrospective data from 2242 ICH patients were evaluated using ICD-9 codes. Recursive feature elimination with cross-validation (RFECV) discerned significant predictors of ICU readmissions. Four ML models-AdaBoost, RandomForest, LightGBM, and XGBoost-underwent development and rigorous validation. SHapley Additive exPlanations (SHAP) elucidated the effect of distinct features on model outcomes.

RESULTS

ICU readmission rates were 9.6% for MIMIC-III and 10.6% for MIMIC-IV. The LightGBM model, with an AUC of 0.736 (95% CI: 0.668-0.801), surpassed other models in validation datasets. SHAP analysis revealed hydrocephalus, sex, neutrophils, Glasgow Coma Scale (GCS), specific oxygen saturation (SpO2) levels, and creatinine as significant predictors of readmission.

CONCLUSION

The LightGBM model demonstrates considerable potential in predicting ICU readmissions for ICH patients, highlighting the importance of certain clinical predictors. This research contributes to optimizing patient care and ICU resource management. Further prospective studies are warranted to corroborate and enhance these predictive insights for clinical utilization.

A generalizable and interpretable model for mortality risk stratification of sepsis patients in intensive care unit

PURPOSE

This study aimed to construct a mortality model for the risk stratification of intensive care unit (ICU) patients with sepsis by applying a machine learning algorithm.

METHODS

Adult patients who were diagnosed with sepsis during admission to ICU were extracted from MIMIC-III, MIMIC-IV, eICU, and Zigong databases. MIMIC-III was used for model development and internal validation. The other three databases were used for external validation. Our proposed model was developed based on the Extreme Gradient Boosting (XGBoost) algorithm. The generalizability, discrimination, and validation of our model were evaluated. The Shapley Additive Explanation values were used to interpret our model and analyze the contribution of individual features.

RESULTS

A total of 16,741, 15,532, 22,617, and 1,198 sepsis patients were extracted from the MIMIC-III, MIMIC-IV, eICU, and Zigong databases, respectively. The proposed model had an area under the receiver operating characteristic curve (AUROC) of 0.84 in the internal validation, which outperformed all the traditional scoring systems. In the external validations, the AUROC was 0.87 in the MIMIC-IV database, better than all the traditional scoring systems; the AUROC was 0.83 in the eICU database, higher than the Simplified Acute Physiology Score III and Sequential Organ Failure Assessment (SOFA),equal to 0.83 of the Acute Physiology and Chronic Health Evaluation IV (APACHE-IV), and the AUROC was 0.68 in the Zigong database, higher than those from the systemic inflammatory response syndrome and SOFA. Furthermore, the proposed model showed the best discriminatory and calibrated capabilities and had the best net benefit in each validation.

CONCLUSIONS

The proposed algorithm based on XGBoost and SHAP-value feature selection had high performance in predicting the mortality of sepsis patients within 24 h of ICU admission.

Construction and validation of machine learning models for sepsis prediction in patients with acute pancreatitis

BACKGROUND

This study aimed to construct predictive models for the risk of sepsis in patients with Acute pancreatitis (AP) using machine learning methods and compared optimal one with the logistic regression (LR) model and scoring systems.

METHODS

In this retrospective cohort study, data were collected from the Medical Information Mart for Intensive Care III (MIMIC III) database between 2001 and 2012 and the MIMIC IV database between 2008 and 2019. Patients were randomly divided into training and test sets (8:2). The least absolute shrinkage and selection operator (LASSO) regression plus 5-fold cross-validation were used to screen and confirm the predictive factors. Based on the selected predictive factors, 6 machine learning models were constructed, including support vector machine (SVM), K-nearest neighbour (KNN), multi-layer perceptron (MLP), LR, gradient boosting decision tree (GBDT) and adaptive enhancement algorithm (AdaBoost). The models and scoring systems were evaluated and compared using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and the area under the curve (AUC).

RESULTS

A total of 1, 672 patients were eligible for participation. In the training set, 261 AP patients (19.51%) were diagnosed with sepsis. The predictive factors for the risk of sepsis in AP patients included age, insurance, vasopressors, mechanical ventilation, Glasgow Coma Scale (GCS), heart rate, respiratory rate, temperature, SpO2, platelet, red blood cell distribution width (RDW), International Normalized Ratio (INR), and blood urea nitrogen (BUN). The AUC of the GBDT model for sepsis prediction in the AP patients in the testing set was 0.985. The GBDT model showed better performance in sepsis prediction than the LR, systemic inflammatory response syndrome (SIRS) score, bedside index for severity in acute pancreatitis (BISAP) score, sequential organ failure assessment (SOFA) score, quick-SOFA (qSOFA), and simplified acute physiology score II (SAPS II).

CONCLUSION

The present findings suggest that compared to the classical LR model and SOFA, qSOFA, SAPS II, SIRS, and BISAP scores, the machine learning model-GBDT model had a better performance in predicting sepsis in the AP patients, which is a useful tool for early identification of high-risk patients and timely clinical interventions.