1
|
Zhang R, Yin M, Jiang A, Zhang S, Liu L, Xu X. Application Value of the Automated Machine Learning Model Based on Modified Computed Tomography Severity Index Combined With Serological Indicators in the Early Prediction of Severe Acute Pancreatitis. J Clin Gastroenterol 2024; 58:692-701. [PMID: 37646502 PMCID: PMC11219072 DOI: 10.1097/mcg.0000000000001909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 07/16/2023] [Indexed: 09/01/2023]
Abstract
BACKGROUND AND AIMS Machine learning (ML) algorithms are widely applied in building models of medicine due to their powerful studying and generalizing ability. To assess the value of the Modified Computed Tomography Severity Index (MCTSI) combined with serological indicators for early prediction of severe acute pancreatitis (SAP) by automated ML (AutoML). PATIENTS AND METHODS The clinical data, of the patients with acute pancreatitis (AP) hospitalized in Hospital 1 and hospital 2 from January 2017 to December 2021, were retrospectively analyzed. Serological indicators within 24 hours of admission were collected. MCTSI score was completed by noncontrast computed tomography within 24 hours of admission. Data from the hospital 1 were adopted for training, and data from the hospital 2 were adopted for external validation. The diagnosis of AP and SAP was based on the 2012 revised Atlanta classification of AP. Models were built using traditional logistic regression and AutoML analysis with 4 types of algorithms. The performance of models was evaluated by the receiver operating characteristic curve, the calibration curve, and the decision curve analysis based on logistic regression and decision curve analysis, feature importance, SHapley Additive exPlanation Plot, and Local Interpretable Model Agnostic Explanation based on AutoML. RESULTS A total of 499 patients were used to develop the models in the training data set. An independent data set of 201 patients was used to test the models. The model developed by the Deep Neural Net (DL) outperformed other models with an area under the receiver operating characteristic curve (areas under the curve) of 0.907 in the test set. Furthermore, among these AutoML models, the DL and gradient boosting machine models achieved the highest sensitivity values, both exceeding 0.800. CONCLUSION The AutoML model based on the MCTSI score combined with serological indicators has good predictive value for SAP in the early stage.
Collapse
Affiliation(s)
- Rufa Zhang
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu No. 1 People’s Hospital
| | - Minyue Yin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu, China
| | - Anqi Jiang
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu No. 1 People’s Hospital
| | - Shihou Zhang
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu No. 1 People’s Hospital
| | - Luojie Liu
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu No. 1 People’s Hospital
| | - Xiaodan Xu
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu No. 1 People’s Hospital
| |
Collapse
|
2
|
Zhang C, Peng J, Wang L, Wang Y, Chen W, Sun MW, Jiang H. A deep learning-powered diagnostic model for acute pancreatitis. BMC Med Imaging 2024; 24:154. [PMID: 38902660 PMCID: PMC11188273 DOI: 10.1186/s12880-024-01339-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2024] [Accepted: 06/17/2024] [Indexed: 06/22/2024] Open
Abstract
BACKGROUND Acute pancreatitis is one of the most common diseases requiring emergency surgery. Rapid and accurate recognition of acute pancreatitis can help improve clinical outcomes. This study aimed to develop a deep learning-powered diagnostic model for acute pancreatitis. MATERIALS AND METHODS In this investigation, we enrolled a cohort of 190 patients with acute pancreatitis who were admitted to Sichuan Provincial People's Hospital between January 2020 and December 2021. Abdominal computed tomography (CT) scans were obtained from both patients with acute pancreatitis and healthy individuals. Our model was constructed using two modules: (1) the acute pancreatitis classifier module; (2) the pancreatitis lesion segmentation module. Each model's performance was assessed based on precision, recall rate, F1-score, Area Under the Curve (AUC), loss rate, frequency-weighted accuracy (fwavacc), and Mean Intersection over Union (MIOU). RESULTS Upon admission, significant variations were observed between patients with mild and severe acute pancreatitis in inflammatory indexes, liver, and kidney function indicators, as well as coagulation parameters. The acute pancreatitis classifier module exhibited commendable diagnostic efficacy, showing an impressive AUC of 0.993 (95%CI: 0.978-0.999) in the test set (comprising healthy examination patients vs. those with acute pancreatitis, P < 0.001) and an AUC of 0.850 (95%CI: 0.790-0.898) in the external validation set (healthy examination patients vs. patients with acute pancreatitis, P < 0.001). Furthermore, the acute pancreatitis lesion segmentation module demonstrated exceptional performance in the validation set. For pancreas segmentation, peripancreatic inflammatory exudation, peripancreatic effusion, and peripancreatic abscess necrosis, the MIOU values were 86.02 (84.52, 87.20), 61.81 (56.25, 64.83), 57.73 (49.90, 68.23), and 66.36 (55.08, 72.12), respectively. These findings underscore the robustness and reliability of the developed models in accurately characterizing and assessing acute pancreatitis. CONCLUSION The diagnostic model for acute pancreatitis, driven by deep learning, exhibits excellent efficacy in accurately evaluating the severity of the condition. TRIAL REGISTRATION This is a retrospective study.
Collapse
Affiliation(s)
- Chi Zhang
- Department of Intensive Care Medicine, The Third People's Hospital of Chengdu, Affiliated Hospital of Southwest Jiaotong University, Chengdu, China
| | - Jin Peng
- Institute for Emergency and Disaster Medicine, School of Medicine, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
- Department of Histology and Neuroscience, Sichuan University, Chengdu, China
| | - Lu Wang
- Institute for Emergency and Disaster Medicine, School of Medicine, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
- Department of Emergency Medicine, School of Medicine, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
- Sichuan Provincial Clinical Research Center for Emergency and Critical Care, School of Medicine, University of Electronic Science and Technology of China, Chengdu, China
| | - Yu Wang
- Sichuan Provincial Clinical Research Center for Emergency and Critical Care, School of Medicine, University of Electronic Science and Technology of China, Chengdu, China
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Beijing, China
| | - Wei Chen
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Beijing, China
| | - Ming-Wei Sun
- Institute for Emergency and Disaster Medicine, School of Medicine, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Hua Jiang
- Institute for Emergency and Disaster Medicine, School of Medicine, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China.
- Department of Emergency Medicine, School of Medicine, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China.
- Sichuan Provincial Clinical Research Center for Emergency and Critical Care, School of Medicine, University of Electronic Science and Technology of China, Chengdu, China.
| |
Collapse
|
3
|
Zhang R, Yin M, Jiang A, Zhang S, Xu X, Liu L. Automated machine learning for early prediction of acute kidney injury in acute pancreatitis. BMC Med Inform Decis Mak 2024; 24:16. [PMID: 38212745 PMCID: PMC10785491 DOI: 10.1186/s12911-024-02414-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 01/01/2024] [Indexed: 01/13/2024] Open
Abstract
BACKGROUND Acute kidney injury (AKI) represents a frequent and grave complication associated with acute pancreatitis (AP), substantially elevating both mortality rates and the financial burden of hospitalization. The aim of our study is to construct a predictive model utilizing automated machine learning (AutoML) algorithms for the early prediction of AKI in patients with AP. METHODS We retrospectively analyzed patients who were diagnosed with AP in our hospital from January 2017 to December 2021. These patients were randomly allocated into a training set and a validation set at a ratio of 7:3. To develop predictive models for each set, we employed the least absolute shrinkage and selection operator (LASSO) algorithm along with AutoML. A nomogram was developed based on multivariate logistic regression analysis outcomes. The model's efficacy was assessed using receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA). Additionally, the performance of the model constructed via AutoML was evaluated using decision curve analysis (DCA), feature importance, SHapley Additive exPlanations (SHAP) plots, and locally interpretable model-agnostic explanations (LIME). RESULTS This study incorporated a total of 437 patients who met the inclusion criteria. Out of these, 313 were assigned to the training cohort and 124 to the validation cohort. In the training and validation cohorts, AKI occurred in 68 (21.7%) and 29(23.4%) patients, respectively. Comparative analysis revealed that the AutoML models exhibited enhanced performance over traditional logistic regression (LR). Furthermore, the deep learning (DL) model demonstrated superior predictive accuracy, evidenced by an area under the ROC curve of 0.963 in the training set and 0.830 in the validation set, surpassing other comparative models. The key variables identified as significant in the DL model within the training dataset included creatinine (Cr), urea (Urea), international normalized ratio (INR), etiology, smoking, alanine aminotransferase (ALT), hypertension, prothrombin time (PT), lactate dehydrogenase (LDH), and diabetes. CONCLUSION The AutoML model, utilizing DL algorithm, offers considerable clinical significance in the early detection of AKI among patients with AP.
Collapse
Affiliation(s)
- Rufa Zhang
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu NO.1 People's Hospital, No. 1 Shuyuan Street, 215500, Suzhou, Jiangsu, China
| | - Minyue Yin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu, China
| | - Anqi Jiang
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu NO.1 People's Hospital, No. 1 Shuyuan Street, 215500, Suzhou, Jiangsu, China
| | - Shihou Zhang
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu NO.1 People's Hospital, No. 1 Shuyuan Street, 215500, Suzhou, Jiangsu, China
| | - Xiaodan Xu
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu NO.1 People's Hospital, No. 1 Shuyuan Street, 215500, Suzhou, Jiangsu, China.
| | - Luojie Liu
- Department of Gastroenterology, Changshu Hospital Affiliated to Soochow University, Changshu NO.1 People's Hospital, No. 1 Shuyuan Street, 215500, Suzhou, Jiangsu, China.
| |
Collapse
|
4
|
Li JN, Mu D, Zheng SC, Tian W, Wu ZY, Meng J, Wang RF, Zheng TL, Zhang YL, Windsor J, Lu GT, Wu D. Machine learning improves prediction of severity and outcomes of acute pancreatitis: a prospective multi-center cohort study. SCIENCE CHINA. LIFE SCIENCES 2023; 66:1934-1937. [PMID: 37209250 DOI: 10.1007/s11427-022-2333-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Accepted: 03/21/2023] [Indexed: 05/22/2023]
Affiliation(s)
- Jia-Ning Li
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 100730, China
| | - Dong Mu
- Department of Gastroenterology, General Hospital of Western Theater Command, Chengdu, 610083, China
| | - Shi-Cheng Zheng
- Department of Gastroenterology, West China Longquan Hospital Sichuan University, Chengdu, 610100, China
| | - Wei Tian
- Department of Gastroenterology, Fangshan Liangxiang Hospital of Beijing, Beijing, 102400, China
| | - Zuo-Yan Wu
- Department of Gastroenterology, The Sixth Hospital of Beijing, Beijing, 100048, China
| | - Jie Meng
- Department of Gastroenterology, Affiliated Hospital of Hebei University, Baoding, 071030, China
| | - Rui-Feng Wang
- Department of Gastroenterology, The Fourth Affiliated Hospital of Harbin Medical University, Harbin, 150001, China
| | - Tian-Lei Zheng
- Artificial Intelligence Unit, Department of Medical Equipment Management, Affiliated Hospital of Xuzhou Medical University, Xuzhou, 221004, China
| | - Yue-Lun Zhang
- Medical Research Center, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 100730, China
| | - John Windsor
- Surgical and Translational Research Centre, Faculty of Medical and Health Sciences, University of Auckland, Auckland, Auckland, 1142, New Zealand
| | - Guo-Tao Lu
- Department of Gastroenterology, Affiliated Hospital of Yangzhou University, Yangzhou University, Yangzhou, 225003, China.
| | - Dong Wu
- Department of Gastroenterology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 100730, China.
- State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 100730, China.
| |
Collapse
|
5
|
Early prediction of the severe course, survival, and ICU requirements in acute pancreatitis by artificial intelligence. Pancreatology 2023; 23:176-186. [PMID: 36610872 DOI: 10.1016/j.pan.2022.12.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/07/2022] [Revised: 11/20/2022] [Accepted: 12/12/2022] [Indexed: 01/09/2023]
Abstract
OBJECTIVE To evaluate the success of artificial intelligence for early prediction of severe course, survival, and intensive care unit(ICU) requirement in patients with acute pancreatitis(AP). METHODS Retrospectively, 1334 patients were included the study. Severity is determined according to the Revised Atlanta Classification(RAC). The success of machine learning(ML) method was evaluated by 13 simple demographic, clinical, etiologic, and laboratory features obtained on ER admission. Additionally, it was evaluated whether Balthazar-computerized tomography severity index(CTSI) at 48-h contributed to success. The dataset was split into two parts, 90% for ML(of which 70% for learning and 30% for testing) and 10% for validation and 5-fold stratified sampling has been utilized. Variable Importance was used in the selection of features during training phase of machine. The Gradient Boost Algorithm trained the machine by KNIME analytics platform. SMOTE has been applied to increase the minority classes for training. The combined effects of the measured features were examined by multivariate logistic regression analysis and reciever operating curve curves of the prediction and confidence of the target variables were obtained. RESULTS Accuracy values for the early estimation of Atlanta severity score, ICU requirement, and survival were found as 88.20%, 98.25%, and 92.77% respectively. When Balthazar-CTSI score is used, results were found as 91.02%, 92.25%, and 98% respectively. CONCLUSIONS The ML method we used successfully predicted the severe course, ICU requirement and survival, with promising accuracy values of over 88%. If 48-h Balthazar-CTSI is included in the calculation, the severity score and survival rates increase even more.
Collapse
|
6
|
Luo J, Lan L, Huang S, Zeng X, Xiang Q, Li M, Yang S, Zhao W, Zhou X. Real-time prediction of organ failures in patients with acute pancreatitis using longitudinal irregular data. J Biomed Inform 2023; 139:104310. [PMID: 36773821 DOI: 10.1016/j.jbi.2023.104310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 01/10/2023] [Accepted: 02/06/2023] [Indexed: 02/12/2023]
Abstract
It is extremely important to identify patients with acute pancreatitis who are at high risk for developing persistent organ failures early in the course of the disease. Due to the irregularity of longitudinal data and the poor interpretability of complex models, many models used to identify acute pancreatitis patients with a high risk of organ failure tended to rely on simple statistical models and limited their application to the early stages of patient admission. With the success of recurrent neural networks in modeling longitudinal medical data and the development of interpretable algorithms, these problems can be well addressed. In this study, we developed a novel model named Multi-task and Time-aware Gated Recurrent Unit RNN (MT-GRU) to directly predict organ failure in patients with acute pancreatitis based on irregular medical EMR data. Our proposed end-to-end multi-task model achieved significantly better performance compared to two-stage models. In addition, our model not only provided an accurate early warning of organ failure for patients throughout their hospital stay, but also demonstrated individual and population-level important variables, allowing physicians to understand the scientific basis of the model for decision-making. By providing early warning of the risk of organ failure, our proposed model is expected to assist physicians in improving outcomes for patients with acute pancreatitis.
Collapse
Affiliation(s)
- Jiawei Luo
- West China Biomedical Big Data Center, West China Hospital/West China School of Medicine, Sichuan University, Chengdu, China.
| | - Lan Lan
- IT Center, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| | - Shixin Huang
- School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing, China.
| | - Xiaoxi Zeng
- West China Biomedical Big Data Center, West China Hospital/West China School of Medicine, Sichuan University, Chengdu, China.
| | - Qu Xiang
- West China Biomedical Big Data Center, West China Hospital/West China School of Medicine, Sichuan University, Chengdu, China.
| | - Mengjiao Li
- West China Biomedical Big Data Center, West China Hospital/West China School of Medicine, Sichuan University, Chengdu, China.
| | - Shu Yang
- College of Medical Information Engineering, Chengdu University of Traditional Chinese Medicine, Chengdu, China.
| | - Weiling Zhao
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, USA.
| | - Xiaobo Zhou
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, USA.
| |
Collapse
|
7
|
Yin M, Zhang R, Zhou Z, Liu L, Gao J, Xu W, Yu C, Lin J, Liu X, Xu C, Zhu J. Automated Machine Learning for the Early Prediction of the Severity of Acute Pancreatitis in Hospitals. Front Cell Infect Microbiol 2022; 12:886935. [PMID: 35755847 PMCID: PMC9226483 DOI: 10.3389/fcimb.2022.886935] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Accepted: 04/29/2022] [Indexed: 11/13/2022] Open
Abstract
Background Machine learning (ML) algorithms are widely applied in building models of medicine due to their powerful studying and generalizing ability. This study aims to explore different ML models for early identification of severe acute pancreatitis (SAP) among patients hospitalized for acute pancreatitis. Methods This retrospective study enrolled patients with acute pancreatitis (AP) from multiple centers. Data from the First Affiliated Hospital and Changshu No. 1 Hospital of Soochow University were adopted for training and internal validation, and data from the Second Affiliated Hospital of Soochow University were adopted for external validation from January 2017 to December 2021. The diagnosis of AP and SAP was based on the 2012 revised Atlanta classification of acute pancreatitis. Models were built using traditional logistic regression (LR) and automated machine learning (AutoML) analysis with five types of algorithms. The performance of models was evaluated by the receiver operating characteristic (ROC) curve, the calibration curve, and the decision curve analysis (DCA) based on LR and feature importance, SHapley Additive exPlanation (SHAP) Plot, and Local Interpretable Model Agnostic Explanation (LIME) based on AutoML. Results A total of 1,012 patients were included in this study to develop the AutoML models in the training/validation dataset. An independent dataset of 212 patients was used to test the models. The model developed by the gradient boost machine (GBM) outperformed other models with an area under the ROC curve (AUC) of 0.937 in the validation set and an AUC of 0.945 in the test set. Furthermore, the GBM model achieved the highest sensitivity value of 0.583 among these AutoML models. The model developed by eXtreme Gradient Boosting (XGBoost) achieved the highest specificity value of 0.980 and the highest accuracy of 0.958 in the test set. Conclusions The AutoML model based on the GBM algorithm for early prediction of SAP showed evident clinical practicability.
Collapse
Affiliation(s)
- Minyue Yin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Rufa Zhang
- Department of Gastroenterology, The Changshu No. 1 Hospital of Soochow University, Suzhou, China
| | - Zhirun Zhou
- Department of Obstetrics and Gynecology, The Second Affiliated Hospital of Soochow University, Suzhou, China
| | - Lu Liu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jingwen Gao
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Wei Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Chenyan Yu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jiaxi Lin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Xiaolin Liu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Chunfang Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jinzhou Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| |
Collapse
|
8
|
Yu Y, Peng C, Zhang Z, Shen K, Zhang Y, Xiao J, Xi W, Wang P, Rao J, Jin Z, Wang Z. Machine Learning Methods for Predicting Long-Term Mortality in Patients After Cardiac Surgery. Front Cardiovasc Med 2022; 9:831390. [PMID: 35592400 PMCID: PMC9110683 DOI: 10.3389/fcvm.2022.831390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Accepted: 03/21/2022] [Indexed: 11/21/2022] Open
Abstract
Objective: This study aims to construct and validate several machine learning (ML) algorithms to predict long-term mortality and identify risk factors in unselected patients post-cardiac surgery. Methods The Medical Information Mart for Intensive Care (MIMIC-III) database was used to perform a retrospective administrative database study. Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Four-year mortality was set as the study outcome. We used the ML methods of logistic regression (LR), artificial neural network (NNET), naïve bayes (NB), gradient boosting machine (GBM), adapting boosting (Ada), random forest (RF), bagged trees (BT), and eXtreme Gradient Boosting (XGB). The prognostic capacity and clinical utility of these ML models were compared using the area under the receiver operating characteristic curves (AUC), calibration curves, and decision curve analysis (DCA). Results Of 7,368 patients in MIMIC-III included in the final cohort, a total of 1,337 (18.15%) patients died during a 4-year follow-up. Among 65 variables extracted from the database, a total of 25 predictors were selected using recursive feature elimination and included in the subsequent analysis. The Ada model performed best among eight models in both discriminatory ability with the highest AUC of 0.801 and goodness of fit (visualized by calibration curve). Moreover, the DCA shows that the net benefit of the RF, Ada, and BT models surpassed that of other ML models for almost all threshold probability values. Additionally, through the Ada technique, we determined that red blood cell distribution width (RDW), blood urea nitrogen (BUN), SAPS II, anion gap (AG), age, urine output, chloride, creatinine, congestive heart failure, and SOFA were the Top 10 predictors in the feature importance rankings. Conclusions The Ada model performs best in predicting 4-year mortality after cardiac surgery among the eight ML models, which might have significant application in the development of early warning systems for patients following operations.
Collapse
Affiliation(s)
- Yue Yu
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Chi Peng
- Department of Health Statistics, Naval Medical University, Shanghai, China
| | - Zhiyuan Zhang
- Department of Cardiothoracic Surgery, No. 988 Hospital of Joint Logistic Support Force of PLA, Zhengzhou, China
| | - Kejia Shen
- Department of Personnel Administration, Second Affiliated Hospital of Naval Medical University, Shanghai, China
| | - Yufeng Zhang
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Jian Xiao
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Wang Xi
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Pei Wang
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Jin Rao
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
| | - Zhichao Jin
- Department of Health Statistics, Naval Medical University, Shanghai, China
- *Correspondence: Zhichao Jin
| | - Zhinong Wang
- Department of Cardiothoracic Surgery, Changzheng Hospital, Naval Medical University, Shanghai, China
- Zhinong Wang
| |
Collapse
|
9
|
Predicting the Need for Therapeutic Intervention and Mortality in Acute Pancreatitis: A Two-Center International Study Using Machine Learning. J Pers Med 2022; 12:jpm12040616. [PMID: 35455733 PMCID: PMC9031087 DOI: 10.3390/jpm12040616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 04/05/2022] [Accepted: 04/06/2022] [Indexed: 11/29/2022] Open
Abstract
Background: Current approaches to predicting intervention needs and mortality have reached 65–85% accuracy, which falls below clinical decision-making requirements in patients with acute pancreatitis (AP). We aimed to accurately predict therapeutic intervention needs and mortality on admission, in AP patients, using machine learning (ML). Methods: Data were obtained from three databases of patients admitted with AP: one retrospective (Chengdu) and two prospective (Liverpool and Chengdu) databases. Intervention and mortality differences, as well as potential predictors, were investigated. Univariate analysis was conducted, followed by a random forest ML algorithm used in multivariate analysis, to identify predictors. The ML performance matrix was applied to evaluate the model’s performance. Results: Three datasets of 2846 patients included 25 potential clinical predictors in the univariate analysis. The top ten identified predictors were obtained by ML models, for predicting interventions and mortality, from the training dataset. The prediction of interventions includes death in non-intervention patients, validated with high accuracy (96%/98%), the area under the receiver-operating-characteristic curve (0.90/0.98), and positive likelihood ratios (22.3/69.8), respectively. The post-test probabilities in the test set were 55.4% and 71.6%, respectively, which were considerably superior to existing prognostic scores. The ML model, for predicting mortality in intervention patients, performed better or equally with prognostic scores. Conclusions: ML, using admission clinical predictors, can accurately predict therapeutic interventions and mortality in patients with AP.
Collapse
|
10
|
Hong W, Lu Y, Zhou X, Jin S, Pan J, Lin Q, Yang S, Basharat Z, Zippi M, Goyal H. Usefulness of Random Forest Algorithm in Predicting Severe Acute Pancreatitis. Front Cell Infect Microbiol 2022; 12:893294. [PMID: 35755843 PMCID: PMC9226542 DOI: 10.3389/fcimb.2022.893294] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 04/29/2022] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND AND AIMS This study aimed to develop an interpretable random forest model for predicting severe acute pancreatitis (SAP). METHODS Clinical and laboratory data of 648 patients with acute pancreatitis were retrospectively reviewed and randomly assigned to the training set and test set in a 3:1 ratio. Univariate analysis was used to select candidate predictors for the SAP. Random forest (RF) and logistic regression (LR) models were developed on the training sample. The prediction models were then applied to the test sample. The performance of the risk models was measured by calculating the area under the receiver operating characteristic (ROC) curves (AUC) and area under precision recall curve. We provide visualized interpretation by using local interpretable model-agnostic explanations (LIME). RESULTS The LR model was developed to predict SAP as the following function: -1.10-0.13×albumin (g/L) + 0.016 × serum creatinine (μmol/L) + 0.14 × glucose (mmol/L) + 1.63 × pleural effusion (0/1)(No/Yes). The coefficients of this formula were utilized to build a nomogram. The RF model consists of 16 variables identified by univariate analysis. It was developed and validated by a tenfold cross-validation on the training sample. Variables importance analysis suggested that blood urea nitrogen, serum creatinine, albumin, high-density lipoprotein cholesterol, low-density lipoprotein cholesterol, calcium, and glucose were the most important seven predictors of SAP. The AUCs of RF model in tenfold cross-validation of the training set and the test set was 0.89 and 0.96, respectively. Both the area under precision recall curve and the diagnostic accuracy of the RF model were higher than that of both the LR model and the BISAP score. LIME plots were used to explain individualized prediction of the RF model. CONCLUSIONS An interpretable RF model exhibited the highest discriminatory performance in predicting SAP. Interpretation with LIME plots could be useful for individualized prediction in a clinical setting. A nomogram consisting of albumin, serum creatinine, glucose, and pleural effusion was useful for prediction of SAP.
Collapse
Affiliation(s)
- Wandong Hong
- Department of Gastroenterology and Hepatology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
- *Correspondence: Wandong Hong,
| | - Yajing Lu
- Department of Gastroenterology and Hepatology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Xiaoying Zhou
- School of the First Clinical Medical Sciences, Wenzhou Medical University, Wenzhou, China
| | - Shengchun Jin
- School of the First Clinical Medical Sciences, Wenzhou Medical University, Wenzhou, China
| | - Jingyi Pan
- School of the First Clinical Medical Sciences, Wenzhou Medical University, Wenzhou, China
| | - Qingyi Lin
- School of the First Clinical Medical Sciences, Wenzhou Medical University, Wenzhou, China
| | - Shaopeng Yang
- School of the First Clinical Medical Sciences, Wenzhou Medical University, Wenzhou, China
| | - Zarrin Basharat
- Jamil-ur-Rahman Center for Genome Research, Dr. Panjwani Centre for Molecular Medicine and Drug Research, International Center for Chemical and Biological Sciences, University of Karachi, Karachi, Pakistan
| | - Maddalena Zippi
- Unit of Gastroenterology and Digestive Endoscopy, Sandro Pertini Hospital, Rome, Italy
| | - Hemant Goyal
- Department of Medicine, The Wright Center for Graduate Medical Education, Scranton, PA, United States
| |
Collapse
|
11
|
Machine learning predictive models for acute pancreatitis: A systematic review. Int J Med Inform 2021; 157:104641. [PMID: 34785488 DOI: 10.1016/j.ijmedinf.2021.104641] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 11/03/2021] [Accepted: 11/05/2021] [Indexed: 12/14/2022]
Abstract
INTRODUCTION Acute pancreatitis (AP) is a common clinical pancreatic disease. Patients with different severity levels have different clinical outcomes. With the advantages of algorithms, machine learning (ML) has gradually emerged in the field of disease prediction, assisting doctors in decision-making. METHODS A systematic review was conducted using the PubMed, Web of Science, Scopus, and Embase databases, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Publication time was limited from inception to 29 May 2021. Studies that have used ML to establish predictive tools for AP were eligible for inclusion. Quality assessment of the included studies was conducted in accordance with the IJMEDI checklist. RESULTS In this systematic review, 24 of 2,913 articles, with a total of 8,327 patients and 47 models, were included. The studies could be divided into five categories: 10 studies (42%) reported severity prediction; 10 studies (42%), complication prediction; 3 studies (13%), mortality prediction; 2 studies (8%), recurrence prediction; and 2 studies (8%), surgery timing prediction. ML showed great accuracy in several prediction tasks. However, most of the included studies were retrospective in nature, conducted at a single centre, based on database data, and lacked external validation. According to the IJMEDI checklist and our scoring criteria, two studies were considered to be of high quality. Most studies had an obvious bias in the quality of data preparation, validation, and deployment dimensions. CONCLUSION In the prediction tasks for AP, ML has shown great potential in assisting decision-making. However, the existing studies still have some deficiencies in the process of model construction. Future studies need to optimize the deficiencies and further evaluate the comparability of the ML systems and model performance, so as to consequently develop high-quality ML-based models that can be used in clinical practice.
Collapse
|
12
|
Yang DJ, Li M, Yue C, Hu WM, Lu HM. Development and validation of a prediction model for deep vein thrombosis in older non-mild acute pancreatitis patients. World J Gastrointest Surg 2021. [DOI: 10.4240/wjgs.v13.i10.1257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/06/2023] Open
|
13
|
Yang DJ, Li M, Yue C, Hu WM, Lu HM. Development and validation of a prediction model for deep vein thrombosis in older non-mild acute pancreatitis patients. World J Gastrointest Surg 2021; 13:1258-1266. [PMID: 34754393 PMCID: PMC8554725 DOI: 10.4240/wjgs.v13.i10.1258] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 07/01/2021] [Accepted: 09/19/2021] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Deep vein thrombosis (DVT) may cause pulmonary embolus, leading to late deaths. The systemic inflammatory and hypercoagulable state of moderate and severe acute pancreatitis (non-mild acute pancreatitis, NMAP) patients may contribute to the development of venous thromboembolism. Accurate prediction of DVT is conducive to clinical decisions.
AIM To develop and validate a potential new prediction nomogram model for the occurrence of DVT in NMAP.
METHODS NMAP patient admission between 2013.1.1 and 2018.12.31 at the West China Hospital of Sichuan University was collected. A total of 220 patients formed the training set for nomogram development, and a validation set was constructed using bootstrapping with 100 resamplings. Univariate and multivariate logistic regression analyses were used to estimate independent risk factors associated with DVT. The independent risk factors were included in the nomogram. The accuracy and utility of the nomogram were evaluated by calibration curve and decision curve analysis, respectively.
RESULTS A total of 220 NMAP patients over 60 years old were enrolled for this analysis. DVT was detected in 80 (36.4%) patients. The final nomogram included age, sex, surgery times, D-dimer, neutrophils, any organ failure, blood culture, and classification. This model achieved good concordance indexes of 0.827 (95%CI: 0.769-0.885) and 0.803 (95%CI: 0.743-0.860) in the training and validation sets, respectively.
CONCLUSION We developed and validated a prediction nomogram model for DVT in older patients with NMAP. This may help guide doctors in making sound decisions regarding the administration of DVT prophylaxis.
Collapse
Affiliation(s)
- Du-Jiang Yang
- Department of Gastrointestinal Surgery, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| | - Mao Li
- Department of Pancreatic Surgery, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| | - Chao Yue
- Department of Pancreatic Surgery, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| | - Wei-Ming Hu
- Department of Pancreatic Surgery, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| | - Hui-Min Lu
- Department of Pancreatic Surgery, West China Hospital, Sichuan University, Chengdu 610041, Sichuan Province, China
| |
Collapse
|