Predicting in-hospital mortality in ICU patients with sepsis using gradient boosting decision tree

Machine learning for predicting mortality in adult critically ill patients with Sepsis: A systematic review

INTRODUCTION

Various Machine Learning (ML) models have been used to predict sepsis-associated mortality. We conducted a systematic review to evaluate the methodologies employed in studies to predict mortality among patients with sepsis.

METHODS

Following a pre-established protocol registered at the International Prospective Register of Systematic Reviews, we performed a comprehensive search of databases from inception to February 2024. We included peer-reviewed articles reporting predicting mortality in critically ill adult patients with sepsis.

RESULTS

Among the 1822 articles, 31 were included, involving 1,477,200 adult patients with sepsis. Nineteen studies had a high risk of bias. Among the diverse ML models, Logistic regression and eXtreme Gradient Boosting were the most frequently used, in 22 and 16 studies, respectively. Nine studies performed internal and external validation. Compared with conventional scoring systems such as SOFA, the ML models showed slightly higher performance in predicting mortality (AUROC ranges: 0.62-0.90 vs. 0.47-0.86).

CONCLUSIONS

ML models demonstrate a modest improvement in predicting sepsis-associated mortality. The certainty of these findings remains low due to the high risk of bias and significant heterogeneity. Studies should include comprehensive methodological details on calibration and hyperparameter selection, adopt a standardized definition of sepsis, and conduct multicenter prospective designs along with external validations.

Machine Learning for Early Discrimination Between Lung Cancer and Benign Nodules Using Routine Clinical and Laboratory Data

BACKGROUND

Lung cancer poses a global health threat necessitating early detection and precise staging for improved patient outcomes. This study focuses on developing and validating a machine learning-based risk model for early lung cancer screening and staging, using routine clinical data.

METHODS

Two medical center, observational, retrospective studies were conducted, involving 2312 lung cancer patients and 653 patients with benign nodules. Machine learning techniques, including differential analysis and feature selection, were employed to identify key factors for modeling. The study focused on variables such as nodule density, carcinoembryonic antigen (CEA), age, and lifestyle habits. The Logistic Regression model was utilized for early diagnoses, and the XGBoost model was utilized for staging based on selected features.

RESULTS

For early diagnoses, the Logistic Regression model achieved an area under the curve (AUC) of 0.716 (95% confidence interval [CI] 0.607-0.826), with 0.703 sensitivity and 0.654 specificity. The XGBoost model excelled in distinguishing late-stage from early-stage lung cancer, exhibiting an AUC of 0.913 (95% CI 0.862-0.963), with 0.909 sensitivity and 0.814 specificity. These findings highlight the model's potential for enhancing diagnostic accuracy and staging in lung cancer.

CONCLUSION

This study introduces a novel machine learning-based risk model for early lung cancer screening and staging, leveraging routine clinical information and laboratory data. The model shows promise in enhancing accuracy, mitigating overdiagnosis, and improving patient outcomes.

Development and validation of an interpretable model for predicting sepsis mortality across care settings

There are numerous prognostic predictive models for evaluating mortality risk, but current scoring models might not fully cater to sepsis patients' needs. This study developed and validated a new model for sepsis patients that is suitable for any care setting and accurately forecasts 28-day mortality. The derivation dataset, gathered from 20 hospitals between September 2019 and December 2021, contrasted with the validation dataset, collected from 15 hospitals from January 2022 to December 2022. In this study, 7436 patients were classified as members of the derivation dataset, and 2284 patients were classified as members of the validation dataset. The point system model emerged as the optimal model among the tested predictive models for foreseeing sepsis mortality. For community-acquired sepsis, the model's performance was satisfactory (derivation dataset AUC: 0.779, 95% CI 0.765-0.792; validation dataset AUC: 0.787, 95% CI 0.765-0.810). Similarly, for hospital-acquired sepsis, it performed well (derivation dataset AUC: 0.768, 95% CI 0.748-0.788; validation dataset AUC: 0.729, 95% CI 0.687-0.770). The calculator, accessible at https://avonlea76.shinyapps.io/shiny_app_up/ , is user-friendly and compatible. The new predictive model of sepsis mortality is user-friendly and satisfactorily forecasts 28-day mortality. Its versatility lies in its applicability to all patients, encompassing both community-acquired and hospital-acquired sepsis.

Establishment and assessment of mortality risk prediction model in patients with sepsis based on early-stage peripheral lymphocyte subsets

This study is aimed to explore the value of lymphocyte subsets in evaluating the severity and prognosis of sepsis. The counts of lymphocytes, CD3+ T cells, CD4+ T cells, CD8+ T cells, CD19+ B cells, and NK cells significantly decreased between day 1 and day 3 in both the survivor and the non-survivor groups. The peripheral lymphocyte subsets (PLS) at day 1 were not significantly different between the survivor and the non-survivor groups. However, at day 3, the counts of lymphocytes, CD3+ T cells, CD4+ T cells, and NK cells were remarkably lower in the non-survivor group. No significant differences in CD8+ T cells, or CD19+ B cells were observed. The PLS index was independently and significantly associated with the 28-day mortality risk in septic patients (OR: 3.08, 95% CI: 1.18-9.67). Based on these clinical parameters and the PLS index, we developed a nomograph for evaluating the individual mortality risk in sepsis. The area under the curve of prediction with the PLS index was significantly higher than that from the model with only clinical parameters (0.912 vs. 0.817). Our study suggests that the decline of PLS occurred in the early stage of sepsis. The new novel PLS index can be an independent predictor of 28-day mortality in septic patients. The prediction model based on clinical parameters and the PLS index has relatively high predicting ability.

Chinese experts' consensus on the application of intensive care big data

The development of intensive care medicine is inseparable from the diversified monitoring data. Intensive care medicine has been closely integrated with data since its birth. Critical care research requires an integrative approach that embraces the complexity of critical illness and the computational technology and algorithms that can make it possible. Considering the need of standardization of application of big data in intensive care, Intensive Care Medicine Branch of China Health Information and Health Care Big Data Society, Standard Committee has convened expert group, secretary group and the external audit expert group to formulate Chinese Experts' Consensus on the Application of Intensive Care Big Data (2022). This consensus makes 29 recommendations on the following five parts: Concept of intensive care big data, Important scientific issues, Standards and principles of database, Methodology in solving big data problems, Clinical application and safety consideration of intensive care big data. The consensus group believes this consensus is the starting step of application big data in the field of intensive care. More explorations and big data based retrospective research should be carried out in order to enhance safety and reliability of big data based models of critical care field.

The glycopatterns of Pseudomonas aeruginosa as a potential biomarker for its carbapenem resistance

IMPORTANCE

Bacterial surface glycans are an attractive therapeutic target in response to antibiotics; however, current knowledge of the corresponding mechanisms is rather limited. Antimicrobial susceptibility testing, genome sequencing, and MALDI-TOF MS, commonly used in recent years to analyze bacterial resistance, are unable to rapidly and efficiently establish associations between glycans and resistance. The discovery of new antimicrobial strategies still requires the introduction of promising analytical methods. In this study, we applied lectin microarray technology and a machine-learning model to screen for important glycan structures associated with carbapenem-resistant P. aeruginosa. This work highlights that specific glycopatterns can be important biomarkers associated with bacterial antibiotic resistance, which promises to provide a rapid entry point for exploring new resistance mechanisms in pathogens.

Machine learning for the prediction of sepsis-related death: a systematic review and meta-analysis

BACKGROUND AND OBJECTIVES

Sepsis is accompanied by a considerably high risk of mortality in the short term, despite the availability of recommended mortality risk assessment tools. However, these risk assessment tools seem to have limited predictive value. With the gradual integration of machine learning into clinical practice, some researchers have attempted to employ machine learning for early mortality risk prediction in sepsis patients. Nevertheless, there is a lack of comprehensive understanding regarding the construction of predictive variables using machine learning and the value of various machine learning methods. Thus, we carried out this systematic review and meta-analysis to explore the predictive value of machine learning for sepsis-related death at different time points.

METHODS

PubMed, Embase, Cochrane, and Web of Science databases were searched until August 9th, 2022. The risk of bias in predictive models was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). We also performed subgroup analysis according to time of death and type of model and summarized current predictive variables used to construct models for sepsis death prediction.

RESULTS

Fifty original studies were included, covering 104 models. The combined Concordance index (C-index), sensitivity, and specificity of machine learning models were 0.799, 0.81, and 0.80 in the training set, and 0.774, 0.71, and 0.68 in the validation set, respectively. Machine learning outperformed conventional clinical scoring tools and showed excellent C-index, sensitivity, and specificity in different subgroups. Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) are the preferred machine learning models because they showed more favorable accuracy with similar modeling variables. This study found that lactate was the most frequent predictor but was seriously ignored by current clinical scoring tools.

CONCLUSION

Machine learning methods demonstrate relatively favorable accuracy in predicting the mortality risk in sepsis patients. Given the limitations in accuracy and applicability of existing prediction scoring systems, there is an opportunity to explore updates based on existing machine learning approaches. Specifically, it is essential to develop or update more suitable mortality risk assessment tools based on the specific contexts of use, such as emergency departments, general wards, and intensive care units.

Predicting COVID-19 Outcomes: Machine Learning Predictions Across Diverse Datasets

Background The COVID-19 infection has spread rapidly since its emergence and has affected a large part of the global population. With the increasing number of cases, researchers are trying to predict the prognosis of patients by using different data with artificial intelligence methods such as machine learning (ML). In this study, we aimed to predict mortality risk in COVID-19 patients using ML algorithms with different datasets. Methodology In this retrospective study, we evaluated the fever, oxygen saturation, laboratory results, thorax computed tomography (CT) findings, and comorbid diseases at admission to the hospital of 404 patients whose diagnosis was confirmed by the reverse transcription polymerase chain reaction test. Different datasets were created by combining the data. The Synthetic Minority Oversampling Technique was used to reduce the imbalance in the dataset. K-nearest neighbors, support vector machine, stochastic gradient descent, random forest, neural network, naive Bayes, logistic regression, gradient boosting, XGBoost, and AdaBoost models were used to create the ML algorithm, and the accuracy rates of mortality prediction were compared. Results When the dataset was created with CT parenchyma score, pulmonary artery and inferior vena cava diameters, and laboratory results, mortality was predicted with an accuracy of 98.4% with the gradient boosting model. Conclusions The study demonstrates that patient prognosis can be accurately predicted using simple measurements from thorax CT scans and laboratory findings.

Machine Learning Differentiation of Autism Spectrum Sub-Classifications

PURPOSE

Disorders on the autism spectrum have characteristics that can manifest as difficulties with communication, executive functioning, daily living, and more. These challenges can be mitigated with early identification. However, diagnostic criteria has changed from DSM-IV to DSM-5, which can make diagnosing a disorder on the autism spectrum complex. We evaluated machine learning to classify individuals as having one of three disorders of the autism spectrum under DSM-IV, or as non-spectrum.

METHODS

We employed machine learning to analyze retrospective data from 38,560 individuals. Inputs encompassed clinical, demographic, and assessment data.

RESULTS

The algorithm achieved AUROCs ranging from 0.863 to 0.980. The model correctly classified 80.5% individuals; 12.6% of individuals from this dataset were misclassified with another disorder on the autism spectrum.

CONCLUSION

Machine learning can classify individuals as having a disorder on the autism spectrum or as non-spectrum using minimal data inputs.

A generalizable and interpretable model for mortality risk stratification of sepsis patients in intensive care unit

PURPOSE

This study aimed to construct a mortality model for the risk stratification of intensive care unit (ICU) patients with sepsis by applying a machine learning algorithm.

METHODS

Adult patients who were diagnosed with sepsis during admission to ICU were extracted from MIMIC-III, MIMIC-IV, eICU, and Zigong databases. MIMIC-III was used for model development and internal validation. The other three databases were used for external validation. Our proposed model was developed based on the Extreme Gradient Boosting (XGBoost) algorithm. The generalizability, discrimination, and validation of our model were evaluated. The Shapley Additive Explanation values were used to interpret our model and analyze the contribution of individual features.

RESULTS

A total of 16,741, 15,532, 22,617, and 1,198 sepsis patients were extracted from the MIMIC-III, MIMIC-IV, eICU, and Zigong databases, respectively. The proposed model had an area under the receiver operating characteristic curve (AUROC) of 0.84 in the internal validation, which outperformed all the traditional scoring systems. In the external validations, the AUROC was 0.87 in the MIMIC-IV database, better than all the traditional scoring systems; the AUROC was 0.83 in the eICU database, higher than the Simplified Acute Physiology Score III and Sequential Organ Failure Assessment (SOFA),equal to 0.83 of the Acute Physiology and Chronic Health Evaluation IV (APACHE-IV), and the AUROC was 0.68 in the Zigong database, higher than those from the systemic inflammatory response syndrome and SOFA. Furthermore, the proposed model showed the best discriminatory and calibrated capabilities and had the best net benefit in each validation.

CONCLUSIONS

The proposed algorithm based on XGBoost and SHAP-value feature selection had high performance in predicting the mortality of sepsis patients within 24 h of ICU admission.

Predicting stroke and mortality in mitral stenosis with atrial flutter: A machine learning approach

BACKGROUND

Our study hypothesized that an intelligent gradient boosting machine (GBM) model can predict cerebrovascular events and all-cause mortality in mitral stenosis (MS) with atrial flutter (AFL) by recognizing comorbidities, electrocardiographic and echocardiographic parameters.

METHODS

The machine learning model was used as a statistical analyzer in recognizing the key risk factors and high-risk features with either outcome of cerebrovascular events or mortality.

RESULTS

A total of 2184 patients with their chart data and imaging studies were included and the GBM analysis demonstrated mitral valve area (MVA), right ventricular systolic pressure, pulmonary artery pressure (PAP), left ventricular ejection fraction (LVEF), New York Heart Association (NYHA) class, and surgery as the most significant predictors of transient ischemic attack (TIA/stroke). MVA, PAP, LVEF, creatinine, hemoglobin, and diastolic blood pressure were predictors for all-cause mortality.

CONCLUSION

The GBM model assimilates clinical data from all diagnostic modalities and significantly improves risk prediction performance and identification of key variables for the outcome of MS with AFL.

Machine learning determination of applied behavioral analysis treatment plan type

BACKGROUND

Applied behavioral analysis (ABA) is regarded as the gold standard treatment for autism spectrum disorder (ASD) and has the potential to improve outcomes for patients with ASD. It can be delivered at different intensities, which are classified as comprehensive or focused treatment approaches. Comprehensive ABA targets multiple developmental domains and involves 20-40 h/week of treatment. Focused ABA targets individual behaviors and typically involves 10-20 h/week of treatment. Determining the appropriate treatment intensity involves patient assessment by trained therapists, however, the final determination is highly subjective and lacks a standardized approach. In our study, we examined the ability of a machine learning (ML) prediction model to classify which treatment intensity would be most suited individually for patients with ASD who are undergoing ABA treatment.

METHODS

Retrospective data from 359 patients diagnosed with ASD were analyzed and included in the training and testing of an ML model for predicting comprehensive or focused treatment for individuals undergoing ABA treatment. Data inputs included demographics, schooling, behavior, skills, and patient goals. A gradient-boosted tree ensemble method, XGBoost, was used to develop the prediction model, which was then compared against a standard of care comparator encompassing features specified by the Behavior Analyst Certification Board treatment guidelines. Prediction model performance was assessed via area under the receiver-operating characteristic curve (AUROC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

The prediction model achieved excellent performance for classifying patients in the comprehensive versus focused treatment groups (AUROC: 0.895; 95% CI 0.811-0.962) and outperformed the standard of care comparator (AUROC 0.767; 95% CI 0.629-0.891). The prediction model also achieved sensitivity of 0.789, specificity of 0.808, PPV of 0.6, and NPV of 0.913. Out of 71 patients whose data were employed to test the prediction model, only 14 misclassifications occurred. A majority of misclassifications (n = 10) indicated comprehensive ABA treatment for patients that had focused ABA treatment as the ground truth, therefore still providing a therapeutic benefit. The three most important features contributing to the model's predictions were bathing ability, age, and hours per week of past ABA treatment.

CONCLUSION

This research demonstrates that the ML prediction model performs well to classify appropriate ABA treatment plan intensity using readily available patient data. This may aid with standardizing the process for determining appropriate ABA treatments, which can facilitate initiation of the most appropriate treatment intensity for patients with ASD and improve resource allocation.

Applying an Improved Stacking Ensemble Model to Predict the Mortality of ICU Patients with Heart Failure

Cardiovascular diseases have been identified as one of the top three causes of death worldwide, with onset and deaths mostly due to heart failure (HF). In ICU, where patients with HF are at increased risk of death and consume significant medical resources, early and accurate prediction of the time of death for patients at high risk of death would enable them to receive appropriate and timely medical care. The data for this study were obtained from the MIMIC-III database, where we collected vital signs and tests for 6699 HF patient during the first 24 h of their first ICU admission. In order to predict the mortality of HF patients in ICUs more precisely, an integrated stacking model is proposed and applied in this paper. In the first stage of dataset classification, the datasets were subjected to first-level classifiers using RF, SVC, KNN, LGBM, Bagging, and Adaboost. Then, the fusion of these six classifier decisions was used to construct and optimize the stacked set of second-level classifiers. The results indicate that our model obtained an accuracy of 95.25% and AUROC of 82.55% in predicting the mortality rate of HF patients, which demonstrates the outstanding capability and efficiency of our method. In addition, the results of this study also revealed that platelets, glucose, and blood urea nitrogen were the clinical features that had the greatest impact on model prediction. The results of this analysis not only improve the understanding of patients' conditions by healthcare professionals but allow for a more optimal use of healthcare resources.

Predicting risk of sepsis, comparison between machine learning methods: a case study of a Virginia hospital

Sepsis is an inflammation caused by the body's systemic response to an infection. The infection could be a result of many diseases, such as pneumonia, urinary tract infection, and other illnesses. Some of its symptoms are fever, tachycardia, tachypnea, etc. Unfortunately, sepsis remains a critical problem at the hospitals and leads to many issues, such as increasing mortality rate, health care costs, and health care utilization. Early detection of sepsis in patients can help respond quickly, take preventive actions, and prevent major issues. The main aim of this study is to predict the risk of sepsis by utilizing the patient’s demographic and clinical information, i.e., patient’s gender, age, severity level, mortality risk, admit type along with hospital length of stay. Six machine learning approaches, Logistic Regression (LR), Naïve Bayes, Support Vector Machine (SVM), Boosted Tree, Classification and Regression Tree (CART), and Bootstrap Forest are used to predict the risk of sepsis. The results showed that different machine learning methods have other performances in terms of various measures. For instance, the Bootstrap Forest machine learning method exhibited the highest performance in AUC and R-square or SVM and Boosted Tree showed the highest performance in terms of misclassification rate. The Bootstrap Forest can be considered the best machine learning method in predicting sepsis regarding applied features in this research, mainly because it showed superior performance and efficiency in two performance measures: AUC and R-square.

Six machine learning methods, Logistic Regression (LR), Naïve Bayes, Support Vector Machine (SVM), Boosted Tree, Classification and Regression Tree (CART), and Bootstrap Forest were compared together in order to predict sepsis.

Early stage of admission data including patient’s gender, age, severity level, mortality risk, admit type along with hospital length of stay were used for predicting sepsis.

The Bootstrap Forest can be considered the best machine learning method in predicting sepsis regarding applied features in this research mainly because it showed superior performance and efficiency in two performance measures, i.e. AUC and R-square.

Predicting Risk of Hypoglycemia in Patients With Type 2 Diabetes by Electronic Health Record-Based Machine Learning: Development and Validation

BACKGROUND

Hypoglycemia is a common adverse event in the treatment of diabetes. To efficiently cope with hypoglycemia, effective hypoglycemia prediction models need to be developed.

OBJECTIVE

The aim of this study was to develop and validate machine learning models to predict the risk of hypoglycemia in adult patients with type 2 diabetes.

METHODS

We used the electronic health records of all adult patients with type 2 diabetes admitted to West China Hospital between November 2019 and December 2021. The prediction model was developed based on XGBoost and natural language processing. F1 score, area under the receiver operating characteristic curve (AUC), and decision curve analysis (DCA) were used as the main criteria to evaluate model performance.

RESULTS

We included 29,843 patients with type 2 diabetes, of whom 2804 patients (9.4%) developed hypoglycemia. In this study, the embedding machine learning model (XGBoost3) showed the best performance among all the models. The AUC and the accuracy of XGBoost are 0.82 and 0.93, respectively. The XGboost3 was also superior to other models in DCA.

CONCLUSIONS

The Paragraph Vector-Distributed Memory model can effectively extract features and improve the performance of the XGBoost model, which can then effectively predict hypoglycemia in patients with type 2 diabetes.

Evaluating machine learning models for sepsis prediction: A systematic review of methodologies

Studies for sepsis prediction using machine learning are developing rapidly in medical science recently. In this review, we propose a set of new evaluation criteria and reporting standards to assess 21 qualified machine learning models for quality analysis based on PRISMA. Our assessment shows that (1.) the definition of sepsis is not consistent among the studies; (2.) data sources and data preprocessing methods, machine learning models, feature engineering, and inclusion types vary widely among the studies; (3.) the closer to the onset of sepsis, the higher the value of AUROC is; (4.) the improvement in AUROC is primarily due to using machine learning as a feature engineering tool; (5.) deep neural networks coupled with Sepsis-3 diagnostic criteria tend to yield better results on the time series data collected from patients with sepsis. The new evaluation criteria and reporting standards will facilitate the development of improved machine learning models for clinical applications.

•

New evaluation/reporting standard for sepsis prediction machine learning models

•

Major limitations in the current models for sepsis prediction have been identified

•

We strongly suggest using machine learning as a feature engineering tool

•

Recommending multilayer neural networks and Sepsis 3.0 for yield better result

The potential for leveraging machine learning to filter medication alerts

OBJECTIVE

To evaluate the potential for machine learning to predict medication alerts that might be ignored by a user, and intelligently filter out those alerts from the user's view.

MATERIALS AND METHODS

We identified features (eg, patient and provider characteristics) proposed to modulate user responses to medication alerts through the literature; these features were then refined through expert review. Models were developed using rule-based and machine learning techniques (logistic regression, random forest, support vector machine, neural network, and LightGBM). We collected log data on alerts shown to users throughout 2019 at University of Utah Health. We sought to maximize precision while maintaining a false-negative rate <0.01, a threshold predefined through discussion with physicians and pharmacists. We developed models while maintaining a sensitivity of 0.99. Two null hypotheses were developed: H1-there is no difference in precision among prediction models; and H2-the removal of any feature category does not change precision.

RESULTS

A total of 3,481,634 medication alerts with 751 features were evaluated. With sensitivity fixed at 0.99, LightGBM achieved the highest precision of 0.192 and less than 0.01 for the pre-defined maximal false-negative rate by subject-matter experts (H1) (P < 0.001). This model could reduce alert volume by 54.1%. We removed different combinations of features (H2) and found that not all features significantly contributed to precision. Removing medication order features (eg, dosage) most significantly decreased precision (-0.147, P = 0.001).

CONCLUSIONS

Machine learning potentially enables the intelligent filtering of medication alerts.

Establishment and validation of the predictive model for the in-hospital death in patients with sepsis

BACKGROUND

Identifying sepsis patients with risk of in-hospital death early can improve the prognosis of patients. This study aimed to develop a model to predict in-hospital death of sepsis patients based on the Medical Information Mart for Intensive Care-Ⅲ (MIMIC-Ⅲ) database, and use clinical data to externally validate the model.

METHODS

A total of 1,839 sepsis patients were used for model development, and 125 clinical cases were used for external validation. The discriminatory ability of the model was determined by calculating the area under the curve (AUC) with 95% confidence intervals (CI).

RESULTS

The AUC of the random forest model and logistic regression model was 0.754 (95%CI, 0.732-0.776) and 0.703 (95%CI, 0.680-0.727), respectively, and the random forest model had higher AUC (Z = 3.070, P = .002). External validation showed that the AUC of the random forest model was 0.539 (95%CI, 0.440-0.628). Further validation was carried out according to gender and SOFA score. The AUC of the model in males and females was 0.648 and 0.412, respectively. In addition, the AUC of the model in the population with SOFA scores of 3-8, 9-12, and 13-15 were 0.705, 0.495, and 0.769, respectively.

CONCLUSIONS

The random forest model had a better predictive ability and a good applicability to external populations with SOFA score of 13-15.