1
|
Danay L, Ramon-Gonen R, Gorodetski M, Schwartz DG. Evaluating the effectiveness of a sliding window technique in machine learning models for mortality prediction in ICU cardiac arrest patients. Int J Med Inform 2024; 191:105565. [PMID: 39094548 DOI: 10.1016/j.ijmedinf.2024.105565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 05/22/2024] [Accepted: 07/21/2024] [Indexed: 08/04/2024]
Abstract
Extensive research has been devoted to predicting ICU mortality, to assist clinical teams managing critical patients. Electronic health records (EHR) contain both static and dynamic medical data, with the latter accumulating during ICU stays. Existing models often rely on a fixed time window (e.g., first 24 h) for prediction, potentially missing vital post-24-hour data. The present study aims to improve mortality prediction for ICU patients following Cardiac Arrest (CA) using a dynamic sliding window approach that accommodates evolving data characteristics. Our cohort included 2331 CA patients, of whom 684 died in the ICU and 1647 survived. Applying the sliding window technique, we created six different time windows and used each separately for model training and validation. We compared our results to a baseline accumulative window. The different time windows created by the sliding window technique differed in their prediction performance and outperformed the baseline 24-hour window significantly. The XGBoost model outperformed all other models, with the 30-42 h time window achieving the best results (AUC = 0.8, accuracy = 0.77). Our work shows that the sliding window technique is effective in improving mortality prediction. We demonstrated how important time-window selection is and showed that enhancing it can save time and thus improve mortality prediction. These findings promise to improve the clinical team's efficiency in prioritizing patients and giving greater attention to higher-risk patients. To conclude, mortality prediction in the ICU can be improved if we consider alternative time windows instead of the 24-hour window, which is currently the most widely accepted among scoring systems today.
Collapse
Affiliation(s)
- Lihi Danay
- The Graduate School of Business Administration, Bar-Ilan University, Ramat-Gan, Israel
| | - Roni Ramon-Gonen
- The Graduate School of Business Administration, Bar-Ilan University, Ramat-Gan, Israel.
| | | | - David G Schwartz
- The Graduate School of Business Administration, Bar-Ilan University, Ramat-Gan, Israel
| |
Collapse
|
2
|
Tang J, Huang J, He X, Zou S, Gong L, Yuan Q, Peng Z. The prediction of in-hospital mortality in elderly patients with sepsis-associated acute kidney injury utilizing machine learning models. Heliyon 2024; 10:e26570. [PMID: 38420451 PMCID: PMC10901004 DOI: 10.1016/j.heliyon.2024.e26570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 02/14/2024] [Accepted: 02/15/2024] [Indexed: 03/02/2024] Open
Abstract
Background Sepsis-associated acute kidney injury (SA-AKI) is a severe complication associated with poorer prognosis and increased mortality, particularly in elderly patients. Currently, there is a lack of accurate mortality risk prediction models for these patients in clinic. Objectives This study aimed to develop and validate machine learning models for predicting in-hospital mortality risk in elderly patients with SA-AKI. Methods Machine learning models were developed and validated using the public, high-quality Medical Information Mart for Intensive Care (MIMIC)-IV critically ill database. The recursive feature elimination (RFE) algorithm was employed for key feature selection. Eleven predictive models were compared, with the best one selected for further validation. Shapley Additive Explanations (SHAP) values were used for visualization and interpretation, making the machine learning models clinically interpretable. Results There were 16,154 patients with SA-AKI in the MIMIC-IV database, and 8426 SA-AKI patients were included in this study (median age: 77.0 years; female: 45%). 7728 patients excluded based on these criteria. They were randomly divided into a training cohort (5,934, 70%) and a validation cohort (2,492, 30%). Nine key features were selected by the RFE algorithm. The CatBoost model achieved the best performance, with an AUC of 0.844 in the training cohort and 0.804 in the validation cohort. SHAP values revealed that AKI stage, PaO2, and lactate were the top three most important features contributing to the CatBoost model. Conclusion We developed a model capable of predicting the risk of in-hospital mortality in elderly patients with SA-AKI.
Collapse
Affiliation(s)
- Jie Tang
- Department of Gastroenterology, Xiangya Hospital, Central South University, Changsha, Hunan, China
- Organ Fibrosis Key Lab of Hunan Province, Central South University, Changsha, Hunan, China
| | - Jian Huang
- Department of Diagnostic Ultrasound and Echocardiography, Sir Run Run Shaw Hospital, Zhejiang University College of Medicine, Hangzhou, Zhejiang, China
| | - Xin He
- Organ Fibrosis Key Lab of Hunan Province, Central South University, Changsha, Hunan, China
- Department of Nephrology, Xiangya Hospital, Central South University, Changsha, Hunan, China
| | - Sijue Zou
- Organ Fibrosis Key Lab of Hunan Province, Central South University, Changsha, Hunan, China
- Department of Nephrology, Xiangya Hospital, Central South University, Changsha, Hunan, China
| | - Li Gong
- Organ Fibrosis Key Lab of Hunan Province, Central South University, Changsha, Hunan, China
- Department of Nephrology, Xiangya Hospital, Central South University, Changsha, Hunan, China
| | - Qiongjing Yuan
- Organ Fibrosis Key Lab of Hunan Province, Central South University, Changsha, Hunan, China
- Department of Nephrology, Xiangya Hospital, Central South University, Changsha, Hunan, China
- National International Joint Research Center for Medical Metabolomices, Xiangya Hospital, Central South University, Changsha, Hunan, China
- National Clinical Medical Research Center for Geriatric Diseases, Xiangya Hospital of Central South University, Changsha, Hunan, China
| | - Zhangzhe Peng
- Organ Fibrosis Key Lab of Hunan Province, Central South University, Changsha, Hunan, China
- Department of Nephrology, Xiangya Hospital, Central South University, Changsha, Hunan, China
- National International Joint Research Center for Medical Metabolomices, Xiangya Hospital, Central South University, Changsha, Hunan, China
| |
Collapse
|
3
|
Prediction of acute hypertensive episodes in critically ill patients. Artif Intell Med 2023; 139:102525. [PMID: 37100504 DOI: 10.1016/j.artmed.2023.102525] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 01/19/2023] [Accepted: 03/06/2023] [Indexed: 03/16/2023]
Abstract
Prevention and treatment of complications are the backbone of medical care, particularly in critical care settings. Early detection and prompt intervention can potentially prevent complications from occurring and improve outcomes. In this study, we use four longitudinal vital signs variables of intensive care unit patients, focusing on predicting acute hypertensive episodes (AHEs). These episodes represent elevations in blood pressure and may result in clinical damage or indicate a change in a patient's clinical situation, such as an elevation in intracranial pressure or kidney failure. Prediction of AHEs may allow clinicians to anticipate changes in the patient's condition and respond early on to prevent these from occurring. Temporal abstraction was employed to transform the multivariate temporal data into a uniform representation of symbolic time intervals, from which frequent time-intervals-related patterns (TIRPs) are mined and used as features for AHE prediction. A novel TIRP metric for classification, called coverage, is introduced that measures the coverage of a TIRP's instances in a time window. For comparison, several baseline models were applied on the raw time series data, including logistic regression and sequential deep learning models, are used. Our results show that using frequent TIRPs as features outperforms the baseline models, and the use of the coverage, metric outperforms other TIRP metrics. Two approaches to predicting AHEs in real-life application conditions are evaluated: using a sliding window to continuously predict whether a patient would experience an AHE within a specific prediction time period ahead, our models produced an AUC-ROC of 82%, but with low AUPRC. Alternatively, predicting whether an AHE would generally occur during the entire admission resulted in an AUC-ROC of 74%.
Collapse
|
4
|
Liu M, Guo C, Guo S. An explainable knowledge distillation method with XGBoost for ICU mortality prediction. Comput Biol Med 2023; 152:106466. [PMID: 36566626 DOI: 10.1016/j.compbiomed.2022.106466] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 11/15/2022] [Accepted: 12/19/2022] [Indexed: 12/24/2022]
Abstract
BACKGROUND AND OBJECTIVE Mortality prediction is an important task in intensive care unit (ICU) for quantifying the severity of patients' physiological condition. Currently, scoring systems are widely applied for mortality prediction, while the performance is unsatisfactory in many clinical conditions due to the non-specificity and linearity characteristics of the used model. As the availability of the large volume of data recorded in electronic health records (EHRs), deep learning models have achieved state-of-art predictive performance. However, deep learning models are hard to meet the requirement of explainability in clinical conditions. Hence, an explainable Knowledge Distillation method with XGBoost (XGB-KD) is proposed to improve the predictive performance of XGBoost while supporting better explainability. METHODS In this method, we first use outperformed deep learning teacher models to learn the complex patterns hidden in high-dimensional multivariate time series data. Then, we distill knowledge from soft labels generated by the ensemble of teacher models to guide the training of XGBoost student model, whose inputs are meaningful features obtained from feature engineering. Finally, we conduct model calibration to obtain predicted probabilities reflecting the true posterior probabilities and use SHapley Additive exPlanations (SHAP) to obtain insights about the trained model. RESULTS We conduct comprehensive experiments on MIMIC-III dataset to evaluate our method. The results demonstrate that our method achieves better predictive performance than vanilla XGBoost, deep learning models and several state-of-art baselines from related works. Our method can also provide intuitive explanations. CONCLUSIONS Our method is useful for improving the predictive performance of XGBoost by distilling knowledge from deep learning models and can provide meaningful explanations for predictions.
Collapse
Affiliation(s)
- Mucan Liu
- Institute of Systems Engineering, Dalian University of Technology, Dalian 116024, China
| | - Chonghui Guo
- Institute of Systems Engineering, Dalian University of Technology, Dalian 116024, China.
| | - Sijia Guo
- Institute of Systems Engineering, Dalian University of Technology, Dalian 116024, China
| |
Collapse
|
5
|
Novitski P, Cohen CM, Karasik A, Hodik G, Moskovitch R. Temporal patterns selection for All-Cause Mortality prediction in T2D with ANNs. J Biomed Inform 2022; 134:104198. [PMID: 36100163 DOI: 10.1016/j.jbi.2022.104198] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Revised: 08/10/2022] [Accepted: 09/03/2022] [Indexed: 01/02/2023]
Abstract
Mortality prevention in T2D elderly population having Chronic Kidney Disease (CKD) may be possible thorough risk assessment and predictive modeling. In this study we investigate the ability to predict mortality using heterogeneous Electronic Health Records data. Temporal abstraction is employed to transform the heterogeneous multivariate temporal data into a uniform representation of symbolic time intervals, from which then frequent Time Intervals Related Patterns (TIRPs) are discovered. However, in this study a novel representation of the TIRPs is introduced, which enables to incorporate them in Deep Learning Networks. We describe here the use of iTirps and bTirps, in which the TIRPs are represented by a integer and binary vector representing the time respectively. While bTirp represents whether a TIRP's instance was present, iTirp represents whether multiple instances were present. While the framework showed encouraging results, a major challenge is often the large number of TIRPs, which may cause the models to under-perform. We introduce a novel method for TIRPs' selection method, called TIRP Ranking Criteria (TRC), which is consists on the TIRP's metrics, such as the differences in its recurrences, its frequencies, and the average duration difference between the classes. Additionally, we introduce an advanced version, called TRC Redundant TIRP Removal (TRC-RTR), TIRPs that highly correlate are candidates for removal. Then the selected subset of iTirp/bTirps is fed into a Deep Learning architecture like a Recurrent Neural Network or a Convolutional Neural Network. Furthermore, a predictive committee is utilized in which raw data and iTirp data are both used as input. Our results show that iTirps-based models that use a subset of iTirps based on the TRC-RTR method outperform models that use raw data or models that use full set of discovered iTirps.
Collapse
Affiliation(s)
- Pavel Novitski
- Software and Information Systems Engineering, Ben Gurion University, Beer-Sheva, Israel.
| | - Cheli Melzer Cohen
- Maccabi Data Science Institute, Maccabi Healthcare Services, Tel-Aviv, Israel.
| | - Avraham Karasik
- Maccabi Data Science Institute, Maccabi Healthcare Services, Tel-Aviv, Israel.
| | - Gabriel Hodik
- Maccabi Data Science Institute, Maccabi Healthcare Services, Tel-Aviv, Israel.
| | - Robert Moskovitch
- Software and Information Systems Engineering, Ben Gurion University, Beer-Sheva, Israel; Population Health and Science, Ichan Medical School at Mount Sinai, NYC, USA.
| |
Collapse
|