1
|
Wang J, Tozzi F, Ashraf Ganjouei A, Romero-Hernandez F, Feng J, Calthorpe L, Castro M, Davis G, Withers J, Zhou C, Chaudhary Z, Adam M, Berrevoet F, Alseidi A, Rashidian N. Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis. J Gastrointest Surg 2024; 28:956-965. [PMID: 38556418 DOI: 10.1016/j.gassur.2024.03.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 03/04/2024] [Accepted: 03/08/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery. METHODS A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model. RESULTS A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009). CONCLUSION We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.
Collapse
Affiliation(s)
- Jane Wang
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Francesca Tozzi
- Department of General, HPB Surgery and Liver Transplantation, Ghent University Hospital, Ghent, Belgium
| | - Amir Ashraf Ganjouei
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Fernanda Romero-Hernandez
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Jean Feng
- Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, California, United States
| | - Lucia Calthorpe
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Maria Castro
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Greta Davis
- Department of Surgery, Division of Plastic and Reconstructive Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Jacquelyn Withers
- Department of Surgery, Division of Plastic and Reconstructive Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Connie Zhou
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Zaim Chaudhary
- University of California, Berkeley, Berkeley, California, United States
| | - Mohamed Adam
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Frederik Berrevoet
- Department of General, HPB Surgery and Liver Transplantation, Ghent University Hospital, Ghent, Belgium
| | - Adnan Alseidi
- Department of Surgery, University of California, San Francisco, San Francisco, California, United States
| | - Nikdokht Rashidian
- Department of General, HPB Surgery and Liver Transplantation, Ghent University Hospital, Ghent, Belgium.
| |
Collapse
|
2
|
Calleja R, Durán M, Ayllón MD, Ciria R, Briceño J. Machine learning in liver surgery: Benefits and pitfalls. World J Clin Cases 2024; 12:2134-2137. [PMID: 38680268 PMCID: PMC11045503 DOI: 10.12998/wjcc.v12.i12.2134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Revised: 02/08/2024] [Accepted: 03/29/2024] [Indexed: 04/16/2024] Open
Abstract
The application of machine learning (ML) algorithms in various fields of hepatology is an issue of interest. However, we must be cautious with the results. In this letter, based on a published ML prediction model for acute kidney injury after liver surgery, we discuss some limitations of ML models and how they may be addressed in the future. Although the future faces significant challenges, it also holds a great potential.
Collapse
Affiliation(s)
- Rafael Calleja
- Hepatobiliary Surgery and Liver Transplantation Unit, Hospital Universitario Reina Sofía, Maimonides Biomedical Research Institute of Cordoba, Córdoba 14004, Spain
| | - Manuel Durán
- Hepatobiliary Surgery and Liver Transplantation Unit, Hospital Universitario Reina Sofía, Maimonides Biomedical Research Institute of Cordoba, Córdoba 14004, Spain
| | - María Dolores Ayllón
- Hepatobiliary Surgery and Liver Transplantation Unit, Hospital Universitario Reina Sofía, Maimonides Biomedical Research Institute of Cordoba, Córdoba 14004, Spain
| | - Ruben Ciria
- Hepatobiliary Surgery and Liver Transplantation Unit, Hospital Universitario Reina Sofía, Maimonides Biomedical Research Institute of Cordoba, Córdoba 14004, Spain
| | - Javier Briceño
- Hepatobiliary Surgery and Liver Transplantation Unit, Hospital Universitario Reina Sofía, Maimonides Biomedical Research Institute of Cordoba, Córdoba 14004, Spain
| |
Collapse
|
3
|
Yu X, Ji Y, Huang M, Feng Z. Machine learning for acute kidney injury: Changing the traditional disease prediction mode. Front Med (Lausanne) 2023; 10:1050255. [PMID: 36817768 PMCID: PMC9935708 DOI: 10.3389/fmed.2023.1050255] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 01/17/2023] [Indexed: 02/05/2023] Open
Abstract
Acute kidney injury (AKI) is a serious clinical comorbidity with clear short-term and long-term prognostic implications for inpatients. The diversity of risk factors for AKI has been recognized in previous studies, and a series of predictive models have been developed using traditional statistical methods in conjunction with its preventability, but they have failed to meet the expectations in limited clinical applications, the rapid spread of electronic health records and artificial intelligence machine learning technology has brought new hope for the construction of AKI prediction models. In this article, we systematically review the definition and classification of machine learning methods, modeling ideas and evaluation methods, and the characteristics and current status of modeling studies. According to the modeling objectives, we subdivided them into critical care medical setting models, all medical environment models, special surgery models, special disease models, and special nephrotoxin exposure models. As the first review article to comprehensively summarize and analyze machine learning prediction models for AKI, we aim to objectively describe the advantages and disadvantages of machine learning approaches to modeling, and help other researchers more quickly and intuitively understand the current status of modeling research, inspire ideas and learn from experience, so as to guide and stimulate more research and more in-depth exploration in the future, which will ultimately provide greater help to improve the overall status of AKI diagnosis and treatment.
Collapse
|
4
|
Fan T, Wang J, Li L, Kang J, Wang W, Zhang C. Predicting the risk factors of diabetic ketoacidosis-associated acute kidney injury: A machine learning approach using XGBoost. Front Public Health 2023; 11:1087297. [PMID: 37089510 PMCID: PMC10117643 DOI: 10.3389/fpubh.2023.1087297] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 03/17/2023] [Indexed: 04/25/2023] Open
Abstract
Objective The purpose of this study was to develop and validate a predictive model based on a machine learning (ML) approach to identify patients with DKA at increased risk of AKI within 1 week of hospitalization in the intensive care unit (ICU). Methods Patients diagnosed with DKA from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database according to the International Classification of Diseases (ICD)-9/10 code were included. The patient's medical history is extracted, along with data on their demographics, vital signs, clinical characteristics, laboratory results, and therapeutic measures. The best-performing model is chosen by contrasting the 8 Ml models. The area under the receiver operating characteristic curve (AUC), sensitivity, accuracy, and specificity were calculated to select the best-performing ML model. Results The final study enrolled 1,322 patients with DKA in total, randomly split into training (1,124, 85%) and validation sets (198, 15%). 497 (37.5%) of them experienced AKI within a week of being admitted to the ICU. The eXtreme Gradient Boosting (XGBoost) model performed best of the 8 Ml models, and the AUC of the training and validation sets were 0.835 and 0.800, respectively. According to the result of feature importance, the top 5 main features contributing to the XGBoost model were blood urea nitrogen (BUN), urine output, weight, age, and platelet count (PLT). Conclusion An ML-based individual prediction model for DKA-associated AKI (DKA-AKI) was developed and validated. The model performs robustly, identifies high-risk patients early, can assist in clinical decision-making, and can improve the prognosis of DKA patients to some extent.
Collapse
Affiliation(s)
- Tingting Fan
- Department of Endocrinology, Second Affiliated Hospital of Jilin University, Changchun, China
| | - Jiaxin Wang
- Department of Endocrinology, Second Affiliated Hospital of Jilin University, Changchun, China
| | - Luyao Li
- Department of Endocrinology, Second Affiliated Hospital of Jilin University, Changchun, China
| | - Jing Kang
- Department of Endocrinology, Second Affiliated Hospital of Jilin University, Changchun, China
| | - Wenrui Wang
- Digestive Diseases Center, Department of Hepatopancreatobiliary Medicine, Second Affiliated Hospital of Jilin University, Changchun, China
| | - Chuan Zhang
- Department of Endocrinology, Second Affiliated Hospital of Jilin University, Changchun, China
- *Correspondence: Chuan Zhang,
| |
Collapse
|