Improvement of APACHE II score system for disease severity based on XGBoost algorithm

Machine-learning models are superior to severity scoring systems for the prediction of the mortality of critically ill patients in a tertiary medical center

BACKGROUND

Intensive care unit (ICU) mortality prediction helps to guide therapeutic decision making for critically ill patients. Several scoring systems based on statistical techniques have been developed for this purpose. In this study, we developed a machine-learning model to predict patient mortality in the very early stage of ICU admission.

METHODS

This study was performed with data from all patients admitted to the intensive care units of a tertiary medical center in Taiwan from 2009 to 2018. The patients' comorbidities, co-medications, vital signs, and laboratory data on the day of ICU admission were obtained from electronic medical records. We constructed random forest and extreme gradient boosting (XGBoost) models to predict ICU mortality, and compared their performance with that of traditional scoring systems.

RESULTS

Data from 12,377 patients was allocated to training (n = 9901) and testing (n = 2476) datasets. The median patient age was 70.0 years; 9210 (74.41%) patients were under mechanical ventilation in the ICU. The areas under receiver operating characteristic curves for the random forest and XGBoost models (0.876 and 0.880, respectively) were larger than those for the Acute Physiology and Chronic Health Evaluation II score (0.738), Sequential Organ Failure Assessment score (0.747), and Simplified Acute Physiology Score II (0.743). The fraction of inspired oxygen on ICU admission was the most important predictive feature across all models.

CONCLUSION

The XGBoost model most accurately predicted ICU mortality and was superior to traditional scoring systems. Our results highlight the utility of machine learning for ICU mortality prediction in the Asian population.

Does older mean flexible? Psychological flexibility and illness cognitions in chronic medical conditions - the moderating effect of age

Adjustment to a Chronic Medical Condition (CMC) is associated with developing hypotheses regarding one's symptoms, known as illness cognition (IC). Aging is associated with a higher rate of CMC. We assessed the effects of aging and psychological flexibility (PF)-one's ability to be open to change, and to alter or persist in behaviors according to environmental circumstances - on IC development in CMC. In a cross-sectional study of hospitalized patients with CMC, 192 patients in four age groups: younger (<50), midlife (50-59), young old (60-69), and elderly (≥70) completed questionnaires sampling IC, PF and demographics. Younger participants reported less helplessness (IC) while lower scores in one PF component (perceiving reality as multifaceted) were reported by the elderly (≥70); older age was associated with a more fixed, narrow perception of reality. Both effects remained significant when using the medical condition severity as a covariate. In general, age was positively associated with IC of acceptance and Helplessness. In regression analysis, CMC severity significantly predicted all IC. Moreover, the interaction of age and perceiving reality as dynamic and changing (PF-RDC component) significantly predicted IC- acceptance of illness; follow-up analysis revealed significant correlations between PF-RDC and acceptance only for younger patients (< age 50). PF-RDC also significantly predicted IC - perceived benefit; among the entire sample higher RDC was associated with less IC - perceived benefit. Implications for theory and practice are discussed.

Doctors' experiences of referring and admitting patients to the intensive care unit: a qualitative study of doctors' practices at two tertiary hospitals in Malawi

OBJECTIVE

To explore doctors' experiences of referring and admitting patients to the intensive care unit (ICU) at two tertiary hospitals in Malawi.

DESIGN

This was a qualitative study that used face-to-face interviews. The interviews were audiotaped and transcribed verbatim into English. The data were analysed manually through conventional content analysis.

SETTING

Two public tertiary hospitals in the central and southern regions of Malawi. Interviews were conducted from January to June 2021.

PARTICIPANTS

Sixteen doctors who were involved in the referral and admission of patients to the ICU.

RESULTS

Four themes were identified namely, lack of clear admission criteria, ICU admission requires a complex chain of consultations, shortage of ICU resources, and lack of an ethical and legal framework for discontinuing treatment of critically ill patients who were too sick to benefit from ICU.

CONCLUSION

Despite the acute disease burden and increased demand for ICU care, the two hospitals lack clear processes for referring and admitting patients to the ICU. Given the limited bed space in ICUs, hospitals in low-income countries, including Malawi, need to improve or develop admission criteria, severity scoring systems, ongoing professional development activities, and legislation for discontinuing intensive care treatments and end-of-life care.

Interpretable machine learning models for predicting in-hospital death in patients in the intensive care unit with cerebral infarction

BACKGROUND

Research on patients with cerebral infarction in the Intensive Care Unit (ICU) is still lacking. Our study aims to develop and validate multiple machine-learning (ML) models using two large ICU databases-Medical Information Mart for Intensive Care version III (MIMIC-III) and eICU Research Institute Database (eRI)-to guide clinical practice.

METHODS

We collected clinical data from patients with cerebral infarction in the MIMIC-III and eRI databases within 24 h of admission. The opinion of neurologists and the Least Absolute Shrinkage and Selection Operator regression was used to screen for relevant clinical features. Using eRI as the training set and MIMIC-III as the test set, we developed and validated six ML models. Based on the results of the model validation, we select the best model and perform the interpretability analysis on it.

RESULTS

A total of 4,338 patients were included in the study (eRI:3002, MIMIC-III:1336), resulting in a total of 18 clinical characteristics through screening. Model validation results showed that random forest (RF) was the best model, with AUC and F1 scores of 0.799 and 0.417 in internal validation and 0.733 and 0.498 in external validation, respectively; moreover, its sensitivity and recall were the highest of the six algorithms for both the internal and external validation. The explanatory analysis of the model showed that the three most important variables in the RF model were Acute Physiology Score-III, Glasgow Coma Scale score, and heart rate, and that the influence of each variable on the judgement of the model was consistent with medical knowledge.

CONCLUSION

Based on a large sample of patients and advanced algorithms, our study bridges the limitations of studies on this area. With our model, physicians can use the admission information of cerebral infarction patients in the ICU to identify high-risk groups among them who are prone to in-hospital death, so that they could be more alert to this group of patients and upgrade medical measures early to minimize the mortality of patients.

Temporal convolutional networks and data rebalancing for clinical length of stay and mortality prediction

It is critical for hospitals to accurately predict patient length of stay (LOS) and mortality in real-time. We evaluate temporal convolutional networks (TCNs) and data rebalancing methods to predict LOS and mortality. This is a retrospective cohort study utilizing the MIMIC-III database. The MIMIC-Extract pipeline processes 24 hour time-series clinical objective data for 23,944 unique patient records. TCN performance is compared to both baseline and state-of-the-art machine learning models including logistic regression, random forest, gated recurrent unit with decay (GRU-D). Models are evaluated for binary classification tasks (LOS > 3 days, LOS > 7 days, mortality in-hospital, and mortality in-ICU) with and without data rebalancing and analyzed for clinical runtime feasibility. Data is split temporally, and evaluations utilize tenfold cross-validation (stratified splits) followed by simulated prospective hold-out validation. In mortality tasks, TCN outperforms baselines in 6 of 8 metrics (area under receiver operating characteristic, area under precision-recall curve (AUPRC), and F-1 measure for in-hospital mortality; AUPRC, accuracy, and F-1 for in-ICU mortality). In LOS tasks, TCN performs competitively to the GRU-D (best in 6 of 8) and the random forest model (best in 2 of 8). Rebalancing improves predictive power across multiple methods and outcome ratios. The TCN offers strong performance in mortality classification and offers improved computational efficiency on GPU-enabled systems over popular RNN architectures. Dataset rebalancing can improve model predictive power in imbalanced learning. We conclude that temporal convolutional networks should be included in model searches for critical care outcome prediction systems.

Which model is superior in predicting ICU survival: artificial intelligence versus conventional approaches

Background

A disease severity classification system is widely used to predict the survival of patients admitted to the intensive care unit with different diagnoses. In the present study, conventional severity classification systems were compared with artificial intelligence predictive models (Artificial Neural Network and Decision Tree) in terms of the prediction of the survival rate of the patients admitted to the intensive care unit.

Methods

This retrospective cohort study was performed on the data of the patients admitted to the ICU of Ghaemshahr’s Razi Teaching Care Center from March 20th, 2017, to September 22nd, 2019. The required data for calculating conventional severity classification models (SOFA, SAPS II, APACHE II, and APACHE IV) were collected from the patients’ medical records. Subsequently, the score of each model was calculated. Artificial intelligence predictive models (Artificial Neural Network and Decision Tree) were developed in the next step. Lastly, the performance of each model in predicting the survival of the patients admitted to the intensive care unit was evaluated using the criteria of sensitivity, specificity, accuracy, F-measure, and area under the ROC curve. Also, each model was validated externally. The R program, version 4.1, was used to create the artificial intelligence models, and SPSS Statistics Software, version 21, was utilized to perform statistical analysis.

Results

The area under the ROC curve of SOFA, SAPS II, APACHE II, APACHE IV, multilayer perceptron artificial neural network, and CART decision tree were 76.0, 77.1, 80.3, 78.5, 84.1, and 80.0, respectively.

Conclusion

The results showed that although the APACHE II model had better results than other conventional models in predicting the survival rate of the patients admitted to the intensive care unit, the other conventional models provided acceptable results too. Moreover, the findings showed that the artificial neural network model had the best performance among all the studied models, indicating the discrimination power of this model in predicting patient survival compared to the other models.

Establishment of ICU Mortality Risk Prediction Models with Machine Learning Algorithm Using MIMIC-IV Database

Objective: The mortality rate of critically ill patients in ICUs is relatively high. In order to evaluate patients’ mortality risk, different scoring systems are used to help clinicians assess prognosis in ICUs, such as the Acute Physiology and Chronic Health Evaluation III (APACHE III) and the Logistic Organ Dysfunction Score (LODS). In this research, we aimed to establish and compare multiple machine learning models with physiology subscores of APACHE III—namely, the Acute Physiology Score III (APS III)—and LODS scoring systems in order to obtain better performance for ICU mortality prediction. Methods: A total number of 67,748 patients from the Medical Information Database for Intensive Care (MIMIC-IV) were enrolled, including 7055 deceased patients, and the same number of surviving patients were selected by the random downsampling technique, for a total of 14,110 patients included in the study. The enrolled patients were randomly divided into a training dataset (n = 9877) and a validation dataset (n = 4233). Fivefold cross-validation and grid search procedures were used to find and evaluate the best hyperparameters in different machine learning models. Taking the subscores of LODS and the physiology subscores that are part of the APACHE III scoring systems as input variables, four machine learning methods of XGBoost, logistic regression, support vector machine, and decision tree were used to establish ICU mortality prediction models, with AUCs as metrics. AUCs, specificity, sensitivity, positive predictive value, negative predictive value, and calibration curves were used to find the best model. Results: For the prediction of mortality risk in ICU patients, the AUC of the XGBoost model was 0.918 (95%CI, 0.915–0.922), and the AUCs of logistic regression, SVM, and decision tree were 0.872 (95%CI, 0.867–0.877), 0.872 (95%CI, 0.867–0.877), and 0.852 (95%CI, 0.847–0.857), respectively. The calibration curves of logistic regression and support vector machine performed better than the other two models in the ranges 0–40% and 70%–100%, respectively, while XGBoost performed better in the range of 40–70%. Conclusions: The mortality risk of ICU patients can be better predicted by the characteristics of the Acute Physiology Score III and the Logistic Organ Dysfunction Score with XGBoost in terms of ROC curve, sensitivity, and specificity. The XGBoost model could assist clinicians in judging in-hospital outcome of critically ill patients, especially in patients with a more uncertain survival outcome.