Evaluate prognostic accuracy of SOFA component score for mortality among adults with sepsis by machine learning method

Machine learning for predicting mortality in adult critically ill patients with Sepsis: A systematic review

INTRODUCTION

Various Machine Learning (ML) models have been used to predict sepsis-associated mortality. We conducted a systematic review to evaluate the methodologies employed in studies to predict mortality among patients with sepsis.

METHODS

Following a pre-established protocol registered at the International Prospective Register of Systematic Reviews, we performed a comprehensive search of databases from inception to February 2024. We included peer-reviewed articles reporting predicting mortality in critically ill adult patients with sepsis.

RESULTS

Among the 1822 articles, 31 were included, involving 1,477,200 adult patients with sepsis. Nineteen studies had a high risk of bias. Among the diverse ML models, Logistic regression and eXtreme Gradient Boosting were the most frequently used, in 22 and 16 studies, respectively. Nine studies performed internal and external validation. Compared with conventional scoring systems such as SOFA, the ML models showed slightly higher performance in predicting mortality (AUROC ranges: 0.62-0.90 vs. 0.47-0.86).

CONCLUSIONS

ML models demonstrate a modest improvement in predicting sepsis-associated mortality. The certainty of these findings remains low due to the high risk of bias and significant heterogeneity. Studies should include comprehensive methodological details on calibration and hyperparameter selection, adopt a standardized definition of sepsis, and conduct multicenter prospective designs along with external validations.

SOFA in sepsis: with or without GCS

PURPOSE

Sepsis is a global public health burden. The sequential organ failure assessment (SOFA) is the most commonly used scoring system for diagnosing sepsis and assessing severity. Due to the widespread use of endotracheal intubation and sedative medications in sepsis, the accuracy of the Glasgow Coma Score (GCS) is the lowest in SOFA. We designed this multicenter, cross-sectional study to investigate the predictive efficiency of SOFA with or without GCS on ICU mortality in patients with sepsis.

METHODS

First, 3048 patients with sepsis admitted to Peking Union Medical College Hospital (PUMCH) were enrolled in this survey. The data were collected from June 8, 2013 to October 12, 2022. Second, 18,108 patients with sepsis in the eICU database were enrolled. Third, 2397 septic patients with respiratory system ≥ 3 points in SOFA in the eICU database were included. We investigated the predictive efficiency of SOFA with or without GCS on ICU mortality in patients with sepsis in various ICUs of PUMCH, and then we validated the results in the eICU database.

MAIN RESULTS

In data of ICUs in PUMCH, the predictive efficiency of SOFA without GCS (AUROC [95% CI], 24 h, 0.724 [0.688, 0.760], 48 h, 0.734 [0.699, 0.769], 72 h, 0.748 [0.713, 0.783], 168 h, 0.781 [0.747, 0.815]) was higher than that of SOFA with GCS (AUROC [95% CI], 24 h, 0.708 [0.672, 0.744], 48 h, 0.721 [0.685, 0.757], 72 h, 0.735 [0.700, 0.757], 168 h, 0.770 [0.736, 0.804]) on ICU mortality in patients with sepsis, and the difference was statistically significant (P value, 24 h, 0.001, 48 h, 0.003, 72 h, 0.004, 168 h, 0.005). In septic patients with respiratory system ≥ 3 points in SOFA in the eICU database, although the difference was not statistically significant (P value, 24 h, 0.148, 48 h, 0.178, 72 h, 0.132, 168 h, 0.790), SOFA without GCS (AUROC [95% CI], 24 h, 0.601 [0.576, 0.626], 48 h, 0.625 [0.601, 0.649], 72 h, 0.639 [0.615, 0.663], 168 h, 0.653 [0.629, 0.677]) had a higher predictive efficiency on ICU mortality than SOFA with GCS (AUROC [95% CI], 24 h, 0.591 [0.566, 0.616], 48 h, 0.616 [0.592, 0.640], 72 h, 0.628 [0.604, 0.652], 168 h, 0.651 [0.627, 0.675]).

CONCLUSIONS

In severe sepsis, it is realistic and feasible to discontinue the routine GCS for SOFA in patients with a respiratory system ≥ 3 points, and even better predict ICU mortality.

Assessing SOFA score trajectories in sepsis using machine learning: A pragmatic approach to improve the accuracy of mortality prediction

INTRODUCTION

An increasing amount of longitudinal health data is available on critically ill septic patients in the age of digital medicine, including daily sequential organ failure assessment (SOFA) score measurements. Thus, the assessment in sepsis focuses increasingly on the evaluation of the individual disease's trajectory. Machine learning (ML) algorithms may provide a promising approach here to improve the evaluation of daily SOFA score dynamics. We tested whether ML algorithms can outperform the conventional ΔSOFA score regarding the accuracy of 30-day mortality prediction.

METHODS

We used the multicentric SepsisDataNet.NRW study cohort that prospectively enrolled 252 sepsis patients between 03/2018 and 09/2019 for training ML algorithms, i.e. support vector machine (SVM) with polynomial kernel and artificial neural network (aNN). We used the Amsterdam UMC database covering 1,790 sepsis patients for external and independent validation.

RESULTS

Both SVM (AUC 0.84; 95% CI: 0.71-0.96) and aNN (AUC 0.82; 95% CI: 0.69-0.95) assessing the SOFA scores of the first seven days led to a more accurate prognosis of 30-day mortality compared to the ΔSOFA score between day 1 and 7 (AUC 0.73; 95% CI: 0.65-0.80; p = 0.02 and p = 0.05, respectively). These differences were even more prominent the shorter the time interval considered. Using the SOFA scores of day 1 to 3 SVM (AUC 0.82; 95% CI: 0.68 0.95) and aNN (AUC 0.80; 95% CI: 0.660.93) led to a more accurate prognosis of 30-day mortality compared to the ΔSOFA score (AUC 0.66; 95% CI: 0.58-0.74; p < 0.01 and p < 0.01, respectively). Strikingly, all these findings could be confirmed in the independent external validation cohort.

CONCLUSIONS

The ML-based algorithms using daily SOFA scores markedly improved the accuracy of mortality compared to the conventional ΔSOFA score. Therefore, this approach could provide a promising and automated approach to assess the individual disease trajectory in sepsis. These findings reflect the potential of incorporating ML algorithms as robust and generalizable support tools on intensive care units.

Early Prediction of Mortality for Septic Patients Visiting Emergency Room Based on Explainable Machine Learning: A Real-World Multicenter Study

BACKGROUND

Worldwide, sepsis is the leading cause of death in hospitals. If mortality rates in patients with sepsis can be predicted early, medical resources can be allocated efficiently. We constructed machine learning (ML) models to predict the mortality of patients with sepsis in a hospital emergency department.

METHODS

This study prospectively collected nationwide data from an ongoing multicenter cohort of patients with sepsis identified in the emergency department. Patients were enrolled from 19 hospitals between September 2019 and December 2020. For acquired data from 3,657 survivors and 1,455 deaths, six ML models (logistic regression, support vector machine, random forest, extreme gradient boosting [XGBoost], light gradient boosting machine, and categorical boosting [CatBoost]) were constructed using fivefold cross-validation to predict mortality. Through these models, 44 clinical variables measured on the day of admission were compared with six sequential organ failure assessment (SOFA) components (PaO2/FIO2 [PF], platelets (PLT), bilirubin, cardiovascular, Glasgow Coma Scale score, and creatinine). The confidence interval (CI) was obtained by performing 10,000 repeated measurements via random sampling of the test dataset. All results were explained and interpreted using Shapley's additive explanations (SHAP).

RESULTS

Of the 5,112 participants, CatBoost exhibited the highest area under the curve (AUC) of 0.800 (95% CI, 0.756-0.840) using clinical variables. Using the SOFA components for the same patient, XGBoost exhibited the highest AUC of 0.678 (95% CI, 0.626-0.730). As interpreted by SHAP, albumin, lactate, blood urea nitrogen, and international normalization ratio were determined to significantly affect the results. Additionally, PF and PLTs in the SOFA component significantly influenced the prediction results.

CONCLUSION

Newly established ML-based models achieved good prediction of mortality in patients with sepsis. Using several clinical variables acquired at the baseline can provide more accurate results for early predictions than using SOFA components. Additionally, the impact of each variable was identified.

The critical role of neutrophil-endothelial cell interactions in sepsis: new synergistic approaches employing organ-on-chip, omics, immune cell phenotyping and in silico modeling to identify new therapeutics

Sepsis is a global health concern accounting for more than 1 in 5 deaths worldwide. Sepsis is now defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. Sepsis can develop from bacterial (gram negative or gram positive), fungal or viral (such as COVID) infections. However, therapeutics developed in animal models and traditional in vitro sepsis models have had little success in clinical trials, as these models have failed to fully replicate the underlying pathophysiology and heterogeneity of the disease. The current understanding is that the host response to sepsis is highly diverse among patients, and this heterogeneity impacts immune function and response to infection. Phenotyping immune function and classifying sepsis patients into specific endotypes is needed to develop a personalized treatment approach. Neutrophil-endothelium interactions play a critical role in sepsis progression, and increased neutrophil influx and endothelial barrier disruption have important roles in the early course of organ damage. Understanding the mechanism of neutrophil-endothelium interactions and how immune function impacts this interaction can help us better manage the disease and lead to the discovery of new diagnostic and prognosis tools for effective treatments. In this review, we will discuss the latest research exploring how in silico modeling of a synergistic combination of new organ-on-chip models incorporating human cells/tissue, omics analysis and clinical data from sepsis patients will allow us to identify relevant signaling pathways and characterize specific immune phenotypes in patients. Emerging technologies such as machine learning can then be leveraged to identify druggable therapeutic targets and relate them to immune phenotypes and underlying infectious agents. This synergistic approach can lead to the development of new therapeutics and the identification of FDA approved drugs that can be repurposed for the treatment of sepsis.

Predicting ICU admission and death in the Emergency Department: A comparison of six early warning scores

AIM

To compare the ability of the most used Early Warning Scores (EWS) to identify adult patients at risk of poor outcomes in the emergency department (ED).

METHODS

Single-center, retrospective observational study. We evaluated the digital records of consecutive ED admissions in patients ≥ 18 years from 2010 to 2019 and calculated NEWS, NEWS2, MEWS, RAPS, REMS, and SEWS based on parameters measured on ED arrival. We assessed the discrimination and calibration performance of each EWS in predicting death/ICU admission within 24 hours using ROC analysis and visual calibration. We also measured the relative weight of clinical and physiological derangements that identified patients missed by EWS risk stratification using neural network analysis.

RESULTS

Among 225,369 patients assessed in the ED during the study period, 1941 (0.9%) were admitted to ICU or died within 24 hours. NEWS was the most accurate predictor (area under the receiver operating characteristic [AUROC] curve 0.904 [95% CI 0.805-0.913]), followed by NEWS2 (AUROC 0.901). NEWS was also well calibrated. In patients judged at low risk (NEWS < 2), 359 events occurred (18.5% of the total). Neural network analysis revealed that age, systolic BP, and temperature had the highest relative weight for these NEWS-unpredicted events.

CONCLUSIONS

NEWS is the most accurate EWS for predicting the risk of death/ICU admission within 24 h from ED arrival. The score also had a fair calibration with few events occurring in patients classified at low risk. Neural network analysis suggests the need for further improvements by focusing on the prompt diagnosis of sepsis and the development of practical tools for the measurement of the respiratory rate.