[Prediction of intensive care unit readmission for critically ill patients based on ensemble learning]

Development and validation of an interpretable 3 day intensive care unit readmission prediction model using explainable boosting machines

Background

Intensive care unit (ICU) readmissions are associated with mortality and poor outcomes. To improve discharge decisions, machine learning (ML) could help to identify patients at risk of ICU readmission. However, as many models are black boxes, dangerous properties may remain unnoticed. Widely used post hoc explanation methods also have inherent limitations. Few studies are evaluating inherently interpretable ML models for health care and involve clinicians in inspecting the trained model.

Methods

An inherently interpretable model for the prediction of 3 day ICU readmission was developed. We used explainable boosting machines that learn modular risk functions and which have already been shown to be suitable for the health care domain. We created a retrospective cohort of 15,589 ICU stays and 169 variables collected between 2006 and 2019 from the University Hospital Münster. A team of physicians inspected the model, checked the plausibility of each risk function, and removed problematic ones. We collected qualitative feedback during this process and analyzed the reasons for removing risk functions. The performance of the final explainable boosting machine was compared with a validated clinical score and three commonly used ML models. External validation was performed on the widely used Medical Information Mart for Intensive Care version IV database.

Results

The developed explainable boosting machine used 67 features and showed an area under the precision-recall curve of 0.119 ± 0.020 and an area under the receiver operating characteristic curve of 0.680 ± 0.025. It performed on par with state-of-the-art gradient boosting machines (0.123 ± 0.016, 0.665 ± 0.036) and outperformed the Simplified Acute Physiology Score II (0.084 ± 0.025, 0.607 ± 0.019), logistic regression (0.092 ± 0.026, 0.587 ± 0.016), and recurrent neural networks (0.095 ± 0.008, 0.594 ± 0.027). External validation confirmed that explainable boosting machines (0.221 ± 0.023, 0.760 ± 0.010) performed similarly to gradient boosting machines (0.232 ± 0.029, 0.772 ± 0.018). Evaluation of the model inspection showed that explainable boosting machines can be useful to detect and remove problematic risk functions.

Conclusions

We developed an inherently interpretable ML model for 3 day ICU readmission prediction that reached the state-of-the-art performance of black box models. Our results suggest that for low- to medium-dimensional datasets that are common in health care, it is feasible to develop ML models that allow a high level of human control without sacrificing performance.

A machine learning-based risk warning platform for potentially inappropriate prescriptions for elderly patients with cardiovascular disease

Potentially inappropriate prescribing (PIP), including potentially inappropriate medications (PIMs) and potential prescribing omissions (PPOs), is a major risk factor for adverse drug reactions (ADRs). Establishing a risk warning model for PIP to screen high-risk patients and implementing targeted interventions would significantly reduce the occurrence of PIP and adverse drug events. Elderly patients with cardiovascular disease hospitalized at the Sichuan Provincial People’s Hospital were included in the study. Information about PIP, PIM, and PPO was obtained by reviewing patient prescriptions according to the STOPP/START criteria (2nd edition). Data were divided into a training set and test set at a ratio of 8:2. Five sampling methods, three feature screening methods, and eighteen machine learning algorithms were used to handle data and establish risk warning models. A 10-fold cross-validation method was employed for internal validation in the training set, and the bootstrap method was used for external validation in the test set. The performances were assessed by area under the receiver operating characteristic curve (AUC), and the risk warning platform was developed based on the best models. The contributions of features were interpreted using SHapley Additive ExPlanation (SHAP). A total of 404 patients were included in the study (318 [78.7%] with PIP; 112 [27.7%] with PIM; and 273 [67.6%] with PPO). After data sampling and feature selection, 15 datasets were obtained and 270 risk warning models were built based on them to predict PIP, PPO, and PIM, respectively. External validation showed that the AUCs of the best model for PIP, PPO, and PIM were 0.8341, 0.7007, and 0.7061, respectively. The results suggested that angina, number of medications, number of diseases, and age were the key factors in the PIP risk warning model. The risk warning platform was established to predict PIP, PIM, and PPO, which has acceptable accuracy, prediction performance, and potential clinical application perspective.

A machine learning-based risk warning platform for potentially inappropriate prescriptions for elderly patients with cardiovascular disease

Potentially inappropriate prescribing (PIP), including potentially inappropriate medications (PIMs) and potential prescribing omissions (PPOs), is a major risk factor for adverse drug reactions (ADRs). Establishing a risk warning model for PIP to screen high-risk patients and implementing targeted interventions would significantly reduce the occurrence of PIP and adverse drug events. Elderly patients with cardiovascular disease hospitalized at the Sichuan Provincial People's Hospital were included in the study. Information about PIP, PIM, and PPO was obtained by reviewing patient prescriptions according to the STOPP/START criteria (2nd edition). Data were divided into a training set and test set at a ratio of 8:2. Five sampling methods, three feature screening methods, and eighteen machine learning algorithms were used to handle data and establish risk warning models. A 10-fold cross-validation method was employed for internal validation in the training set, and the bootstrap method was used for external validation in the test set. The performances were assessed by area under the receiver operating characteristic curve (AUC), and the risk warning platform was developed based on the best models. The contributions of features were interpreted using SHapley Additive ExPlanation (SHAP). A total of 404 patients were included in the study (318 [78.7%] with PIP; 112 [27.7%] with PIM; and 273 [67.6%] with PPO). After data sampling and feature selection, 15 datasets were obtained and 270 risk warning models were built based on them to predict PIP, PPO, and PIM, respectively. External validation showed that the AUCs of the best model for PIP, PPO, and PIM were 0.8341, 0.7007, and 0.7061, respectively. The results suggested that angina, number of medications, number of diseases, and age were the key factors in the PIP risk warning model. The risk warning platform was established to predict PIP, PIM, and PPO, which has acceptable accuracy, prediction performance, and potential clinical application perspective.