1
|
Alsaber AR, Al-Herz A, Alawadhi B, Doush IA, Setiya P, AL-Sultan AT, Saleh K, Al-Awadhi A, Hasan E, Al-Kandari W, Mokaddem K, Ghanem AA, Attia Y, Hussain M, AlHadhood N, Ali Y, Tarakmeh H, Aldabie G, AlKadi A, Alhajeri H. Machine learning-based remission prediction in rheumatoid arthritis patients treated with biologic disease-modifying anti-rheumatic drugs: findings from the Kuwait rheumatic disease registry. Front Big Data 2024; 7:1406365. [PMID: 39421133 PMCID: PMC11484091 DOI: 10.3389/fdata.2024.1406365] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Accepted: 09/12/2024] [Indexed: 10/19/2024] Open
Abstract
Background Rheumatoid arthritis (RA) is a common condition treated with biological disease-modifying anti-rheumatic medicines (bDMARDs). However, many patients exhibit resistance, necessitating the use of machine learning models to predict remissions in patients treated with bDMARDs, thereby reducing healthcare costs and minimizing negative effects. Objective The study aims to develop machine learning models using data from the Kuwait Registry for Rheumatic Diseases (KRRD) to identify clinical characteristics predictive of remission in RA patients treated with biologics. Methods The study collected follow-up data from 1,968 patients treated with bDMARDs from four public hospitals in Kuwait from 2013 to 2022. Machine learning techniques like lasso, ridge, support vector machine, random forest, XGBoost, and Shapley additive explanation were used to predict remission at a 1-year follow-up. Results The study used the Shapley plot in explainable Artificial Intelligence (XAI) to analyze the effects of predictors on remission prognosis across different types of bDMARDs. Top clinical features were identified for patients treated with bDMARDs, each associated with specific mean SHAP values. The findings highlight the importance of clinical assessments and specific treatments in shaping treatment outcomes. Conclusion The proposed machine learning model system effectively identifies clinical features predicting remission in bDMARDs, potentially improving treatment efficacy in rheumatoid arthritis patients.
Collapse
Affiliation(s)
- Ahmad R. Alsaber
- College of Business and Economics, American University of Kuwait, Salmiya, Kuwait
| | - Adeeba Al-Herz
- Department of Rheumatology, Al-Amiri Hospital, Kuwait City, Kuwait
| | - Balqees Alawadhi
- Department of Food and Nutritional Sciences, The Public Authority for Applied Education & Training, Shuwaikh Industrial, Kuwait
| | - Iyad Abu Doush
- College of Engineering and Applied Sciences, American University of Kuwait, Salmiya, Kuwait
- Computer Science Department, Yarmouk University, Irbid, Jordan
| | - Parul Setiya
- College of Agriculture, Govind Ballabh Pant University of Agriculture and Technology, Pantnagar, India
| | - Ahmad T. AL-Sultan
- Department of Community Medicine and Behavioral Sciences, Kuwait University, Safat, Kuwait
| | - Khulood Saleh
- Department of Rheumatology, Farwaniya Hospital, Kuwait City, Kuwait
| | - Adel Al-Awadhi
- Department of Rheumatology, Al-Amiri Hospital, Kuwait City, Kuwait
| | - Eman Hasan
- Department of Rheumatology, Al-Amiri Hospital, Kuwait City, Kuwait
| | | | - Khalid Mokaddem
- Department of Rheumatology, Al-Amiri Hospital, Kuwait City, Kuwait
| | - Aqeel A. Ghanem
- Department of Rheumatology, Mubarak Al-Kabeer Hospital, Kuwait City, Kuwait
| | - Yousef Attia
- Department of Rheumatology, Al-Amiri Hospital, Kuwait City, Kuwait
| | - Mohammed Hussain
- Department of Rheumatology, Al-Amiri Hospital, Kuwait City, Kuwait
| | - Naser AlHadhood
- Department of Rheumatology, Farwaniya Hospital, Kuwait City, Kuwait
| | - Yaser Ali
- Department of Rheumatology, Mubarak Al-Kabeer Hospital, Kuwait City, Kuwait
| | - Hoda Tarakmeh
- Department of Rheumatology, Mubarak Al-Kabeer Hospital, Kuwait City, Kuwait
| | - Ghaydaa Aldabie
- Department of Rheumatology, Farwaniya Hospital, Kuwait City, Kuwait
| | - Amjad AlKadi
- Department of Rheumatology, Al-Sabah Hospital, Kuwait City, Kuwait
| | - Hebah Alhajeri
- Department of Rheumatology, Mubarak Al-Kabeer Hospital, Kuwait City, Kuwait
| |
Collapse
|
2
|
Holtenius J, Mosfeldt M, Enocson A, Berg HE. Prediction of mortality among severely injured trauma patients A comparison between TRISS and machine learning-based predictive models. Injury 2024; 55:111702. [PMID: 38936227 DOI: 10.1016/j.injury.2024.111702] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2024] [Revised: 06/13/2024] [Accepted: 06/19/2024] [Indexed: 06/29/2024]
Abstract
BACKGROUND Given the huge impact of trauma on hospital systems around the world, several attempts have been made to develop predictive models for the outcomes of trauma victims. The most used, and in many studies most accurate predictive model, is the "Trauma Score and Injury Severity Score" (TRISS). Although it has proven to be fairly accurate and is widely used, it has faced criticism for its inability to classify more complex cases. In this study, we aimed to develop machine learning models that better than TRISS could predict mortality among severely injured trauma patients, something that has not been studied using data from a nationwide register before. METHODS Patient data was collected from the national trauma register in Sweden, SweTrau. The studied period was from the 1st of January 2015 to 31st of December 2019. After feature selection and multiple imputation of missing data three machine learning (ML) methods (Random Forest, eXtreme Gradient Boosting, and a Generalized Linear Model) were used to create predictive models. The ML models and TRISS were then tested on predictive ability for 30-day mortality. RESULTS The ML models were well-calibrated and outperformed TRISS in all the tested measurements. Among the ML models, the eXtreme Gradient Boosting model performed best with an AUC of 0.91 (0.88-0.93). CONCLUSION This study showed that all the developed ML-based prediction models were superior to TRISS for the prediction of trauma mortality.
Collapse
Affiliation(s)
- Jonas Holtenius
- Department of Clinical Science, Intervention and Technology, Karolinska Institute, 14152 Stockholm, Sweden; Department of Trauma, Acute Surgery and Orthopaedics, Karolinska University Hospital, 17177 Stockholm, Sweden.
| | - Mathias Mosfeldt
- Department of Molecular Medicine and Surgery, Karolinska Institute, 17176 Stockholm, Sweden; Department of Trauma, Acute Surgery and Orthopaedics, Karolinska University Hospital, 17177 Stockholm, Sweden
| | - Anders Enocson
- Department of Molecular Medicine and Surgery, Karolinska Institute, 17176 Stockholm, Sweden; Department of Trauma, Acute Surgery and Orthopaedics, Karolinska University Hospital, 17177 Stockholm, Sweden
| | - Hans E Berg
- Department of Clinical Science, Intervention and Technology, Karolinska Institute, 14152 Stockholm, Sweden; Department of Trauma, Acute Surgery and Orthopaedics, Karolinska University Hospital, 17177 Stockholm, Sweden
| |
Collapse
|
3
|
Mulenga C, Kaonga P, Hamoonga R, Mazaba ML, Chabala F, Musonda P. Predicting Mortality in Hospitalized COVID-19 Patients in Zambia: An Application of Machine Learning. Glob Health Epidemiol Genom 2023; 2023:8921220. [PMID: 37260675 PMCID: PMC10228226 DOI: 10.1155/2023/8921220] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Revised: 02/23/2023] [Accepted: 04/27/2023] [Indexed: 06/02/2023] Open
Abstract
The coronavirus disease 2019 (COVID-19) has wreaked havoc globally, resulting in millions of cases and deaths. The objective of this study was to predict mortality in hospitalized COVID-19 patients in Zambia using machine learning (ML) methods based on factors that have been shown to be predictive of mortality and thereby improve pandemic preparedness. This research employed seven powerful ML models that included decision tree (DT), random forest (RF), support vector machines (SVM), logistic regression (LR), Naïve Bayes (NB), gradient boosting (GB), and XGBoost (XGB). These classifiers were trained on 1,433 hospitalized COVID-19 patients from various health facilities in Zambia. The performances achieved by these models were checked using accuracy, recall, F1-Score, area under the receiver operating characteristic curve (ROC_AUC), area under the precision-recall curve (PRC_AUC), and other metrics. The best-performing model was the XGB which had an accuracy of 92.3%, recall of 94.2%, F1-Score of 92.4%, and ROC_AUC of 97.5%. The pairwise Mann-Whitney U-test analysis showed that the second-best model (GB) and the third-best model (RF) did not perform significantly worse than the best model (XGB) and had the following: GB had an accuracy of 91.7%, recall of 94.2%, F1-Score of 91.9%, and ROC_AUC of 97.1%. RF had an accuracy of 90.8%, recall of 93.6%, F1-Score of 91.0%, and ROC_AUC of 96.8%. Other models showed similar results for the same metrics checked. The study successfully derived and validated the selected ML models and predicted mortality effectively with reasonably high performance in the stated metrics. The feature importance analysis found that knowledge of underlying health conditions about patients' hospital length of stay (LOS), white blood cell count, age, and other factors can help healthcare providers offer lifesaving services on time, improve pandemic preparedness, and decongest health facilities in Zambia and other countries with similar settings.
Collapse
Affiliation(s)
- Clyde Mulenga
- Department of Epidemiology and Biostatistics, University of Zambia, Lusaka, Zambia
- Institute of Basic and Biomedical Sciences, Levy Mwanawasa Medical University, Lusaka, Zambia
| | - Patrick Kaonga
- Department of Epidemiology and Biostatistics, University of Zambia, Lusaka, Zambia
| | - Raymond Hamoonga
- The Health Press, Zambia National Public Health Institute, Lusaka, Zambia
| | - Mazyanga Lucy Mazaba
- Communication Information and Research, Zambia National Public Health Institute, Lusaka, Zambia
| | - Freeman Chabala
- Institute of Basic and Biomedical Sciences, Levy Mwanawasa Medical University, Lusaka, Zambia
| | - Patrick Musonda
- Department of Epidemiology and Biostatistics, University of Zambia, Lusaka, Zambia
| |
Collapse
|
4
|
Peng HT, Siddiqui MM, Rhind SG, Zhang J, da Luz LT, Beckett A. Artificial intelligence and machine learning for hemorrhagic trauma care. Mil Med Res 2023; 10:6. [PMID: 36793066 PMCID: PMC9933281 DOI: 10.1186/s40779-023-00444-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Accepted: 02/01/2023] [Indexed: 02/17/2023] Open
Abstract
Artificial intelligence (AI), a branch of machine learning (ML) has been increasingly employed in the research of trauma in various aspects. Hemorrhage is the most common cause of trauma-related death. To better elucidate the current role of AI and contribute to future development of ML in trauma care, we conducted a review focused on the use of ML in the diagnosis or treatment strategy of traumatic hemorrhage. A literature search was carried out on PubMed and Google scholar. Titles and abstracts were screened and, if deemed appropriate, the full articles were reviewed. We included 89 studies in the review. These studies could be grouped into five areas: (1) prediction of outcomes; (2) risk assessment and injury severity for triage; (3) prediction of transfusions; (4) detection of hemorrhage; and (5) prediction of coagulopathy. Performance analysis of ML in comparison with current standards for trauma care showed that most studies demonstrated the benefits of ML models. However, most studies were retrospective, focused on prediction of mortality, and development of patient outcome scoring systems. Few studies performed model assessment via test datasets obtained from different sources. Prediction models for transfusions and coagulopathy have been developed, but none is in widespread use. AI-enabled ML-driven technology is becoming integral part of the whole course of trauma care. Comparison and application of ML algorithms using different datasets from initial training, testing and validation in prospective and randomized controlled trials are warranted for provision of decision support for individualized patient care as far forward as possible.
Collapse
Affiliation(s)
- Henry T Peng
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada.
| | - M Musaab Siddiqui
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada
| | - Shawn G Rhind
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada
| | - Jing Zhang
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada
| | | | - Andrew Beckett
- St. Michael's Hospital, Toronto, ON, M5B 1W8, Canada
- Royal Canadian Medical Services, Ottawa, K1A 0K2, Canada
| |
Collapse
|
5
|
Takkavatakarn K, Hofer IS. Artificial Intelligence and Machine Learning in Perioperative Acute Kidney Injury. ADVANCES IN KIDNEY DISEASE AND HEALTH 2023; 30:53-60. [PMID: 36723283 DOI: 10.1053/j.akdh.2022.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 09/30/2022] [Accepted: 10/28/2022] [Indexed: 12/24/2022]
Abstract
Acute kidney injury (AKI) is a common complication after a surgery, especially in cardiac and aortic procedures, and has a significant impact on morbidity and mortality. Early identification of high-risk patients and providing effective prevention and therapeutic approach are the main strategies for reducing the possibility of perioperative AKI. Consequently, several risk-prediction models and risk assessment scores have been developed for the prediction of perioperative AKI. However, a majority of these risk scores are only derived from preoperative data while the intraoperative time-series monitoring data such as heart rate and blood pressure were not included. Moreover, the complexity of the pathophysiology of AKI, as well as its nonlinear and heterogeneous nature, imposes limitations on the use of linear statistical techniques. The development of clinical medicine's digitization, the widespread availability of electronic medical records, and the increase in the use of continuous monitoring have generated vast quantities of data. Machine learning has recently shown promise as a method for automatically integrating large amounts of data in predicting the risk of perioperative outcomes. In this article, we discussed the development, limitations of existing work, and the potential future direction of models using machine learning techniques to predict AKI after a surgery.
Collapse
Affiliation(s)
- Kullaya Takkavatakarn
- Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY; Division of Nephrology, Department of Medicine, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
| | - Ira S Hofer
- Department of Anesthesiology, Pain and Perioperative Medicine, Icahn School of Medicine at Mount, Sinai, NY.
| |
Collapse
|
6
|
Lee KC, Hsu CC, Lin TC, Chiang HF, Horng GJ, Chen KT. Prediction of Prognosis in Patients with Trauma by Using Machine Learning. MEDICINA (KAUNAS, LITHUANIA) 2022; 58:medicina58101379. [PMID: 36295540 PMCID: PMC9606956 DOI: 10.3390/medicina58101379] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Revised: 09/21/2022] [Accepted: 09/28/2022] [Indexed: 11/06/2022]
Abstract
Background and Objectives: We developed a machine learning algorithm to analyze trauma-related data and predict the mortality and chronic care needs of patients with trauma. Materials and Methods: We recruited admitted patients with trauma during 2015 and 2016 and collected their clinical data. Then, we subjected this database to different machine learning techniques and chose the one with the highest accuracy by using cross-validation. The primary endpoint was mortality, and the secondary endpoint was requirement for chronic care. Results: Data of 5871 patients were collected. We then used the eXtreme Gradient Boosting (xGBT) machine learning model to create two algorithms: a complete model and a short-term model. The complete model exhibited an 86% recall for recovery, 30% for chronic care, 67% for mortality, and 80% for complications; the short-term model fitted for ED displayed an 89% recall for recovery, 25% for chronic care, and 41% for mortality. Conclusions: We developed a machine learning algorithm that displayed good recall for the healthy recovery group but unsatisfactory results for those requiring chronic care or having a risk of mortality. The prediction power of this algorithm may be improved by implementing features such as age group classification, severity selection, and score calibration of trauma-related variables.
Collapse
Affiliation(s)
- Kuo-Chang Lee
- Emergency Department, Chi-Mei Medical Center, Tainan 710402, Taiwan
| | - Chien-Chin Hsu
- Emergency Department, Chi-Mei Medical Center, Tainan 710402, Taiwan
- Department of Biotechnology, Southern Taiwan University of Science and Technology, Tainan 71005, Taiwan
| | - Tzu-Chieh Lin
- Department of Computer Science and Information Engineering, Southern Taiwan University of Science and Technology, Tainan 71005, Taiwan
| | - Hsiu-Fen Chiang
- Department of Computer Science and Information Engineering, Southern Taiwan University of Science and Technology, Tainan 71005, Taiwan
| | - Gwo-Jiun Horng
- Department of Computer Science and Information Engineering, Southern Taiwan University of Science and Technology, Tainan 71005, Taiwan
| | - Kuo-Tai Chen
- Emergency Department, Chi-Mei Medical Center, Tainan 710402, Taiwan
- Correspondence: ; Tel.: +886-6-2812811 (ext. 57196); Fax: +886-6-2816161
| |
Collapse
|
7
|
Machine Learning in the Prediction of Trauma Outcomes: A Systematic Review. Ann Emerg Med 2022; 80:440-455. [PMID: 35842343 DOI: 10.1016/j.annemergmed.2022.05.011] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 03/20/2022] [Accepted: 05/04/2022] [Indexed: 11/23/2022]
Abstract
STUDY OBJECTIVE Machine learning models carry unique potential as decision-making aids and prediction tools for improving patient care. Traumatically injured patients provide a uniquely heterogeneous population with severe injuries that can be difficult to predict. Given the relative infancy of machine learning applications in medicine, this systematic review aimed to better understand the current state of machine learning development and implementation to help create a basis for future research. METHODS We conducted a systematic review from inception to May 2021, using Embase, MEDLINE through Ovid, Web of Science, Google Scholar, and relevant gray literature, for uses of machine learning in predicting the outcomes of trauma patients. The screening and data extraction were performed by 2 independent reviewers. RESULTS Of the 14,694 identified articles screened, 67 were included for data extraction. Artificial neural networks comprised the most commonly used model, and mortality was the most prevalent outcome of interest. In terms of machine learning model development, there was a lack of studies that employed external validation, feature selection methods, and performed formal calibration testing. Significant heterogeneity in reporting was also observed between the machine learning models employed, patient populations, performance metrics, and features employed. CONCLUSION This review highlights the heterogeneity in the development and reporting of machine learning models for the prediction of trauma outcomes. While these models present an area of opportunity as an ancillary to clinical decision-making, we recommend more standardization and rigorous guidelines for the development of future models.
Collapse
|
8
|
Crabb BT, Hamrick F, Campbell JM, Vignolles-Jeong J, Magill ST, Prevedello DM, Carrau RL, Otto BA, Hardesty DA, Couldwell WT, Karsy M. Machine Learning-Based Analysis and Prediction of Unplanned 30-Day Readmissions After Pituitary Adenoma Resection: A Multi-Institutional Retrospective Study With External Validation. Neurosurgery 2022; 91:263-271. [PMID: 35384923 DOI: 10.1227/neu.0000000000001967] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2021] [Accepted: 02/05/2022] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Unplanned readmission after transsphenoidal resection of pituitary adenoma can occur in up to 10% of patients but is unpredictable. OBJECTIVE To develop a reliable system for predicting unplanned readmission and create a validated method for stratifying patients by risk. METHODS Data sets were retrospectively collected from the National Surgical Quality Improvement Program and 2 tertiary academic medical centers. Eight machine learning classifiers were fit to the National Surgical Quality Improvement Program data, optimized using Bayesian parameter optimization and evaluated on the external data. Permutation analysis identified the relative importance of predictive variables, and a risk stratification system was built using the trained machine learning models. RESULTS Readmissions were accurately predicted by several classification models with an area under the receiving operator characteristic curve of 0.76 (95% CI 0.68-0.83) on the external data set. Permutation analysis identified the most important variables for predicting readmission as preoperative sodium level, returning to the operating room, and total operation time. High-risk and medium-risk patients, as identified by the proposed risk stratification system, were more likely to be readmitted than low-risk patients, with relative risks of 12.2 (95% CI 5.9-26.5) and 4.2 (95% CI 2.3-8.7), respectively. Overall risk stratification showed high discriminative capability with a C-statistic of 0.73. CONCLUSION In this multi-institutional study with outside validation, unplanned readmissions after pituitary adenoma resection were accurately predicted using machine learning techniques. The features identified in this study and the risk stratification system developed could guide clinical and surgical decision making, reduce healthcare costs, and improve the quality of patient care by better identifying high-risk patients for closer perioperative management.
Collapse
Affiliation(s)
- Brendan T Crabb
- Department of Neurosurgery, University of Utah, Salt Lake City, Utah, USA
| | - Forrest Hamrick
- Department of Neurosurgery, University of Utah, Salt Lake City, Utah, USA
| | - Justin M Campbell
- Department of Neurosurgery, University of Utah, Salt Lake City, Utah, USA
| | | | - Stephen T Magill
- Department of Neurosurgery, The Ohio State University, Columbus, Ohio, USA
| | | | - Ricardo L Carrau
- Department of Neurosurgery, The Ohio State University, Columbus, Ohio, USA
| | - Bradley A Otto
- Department of Neurosurgery, The Ohio State University, Columbus, Ohio, USA
| | - Douglas A Hardesty
- Department of Neurosurgery, The Ohio State University, Columbus, Ohio, USA
| | | | - Michael Karsy
- Department of Neurosurgery, University of Utah, Salt Lake City, Utah, USA
| |
Collapse
|
9
|
Somboon S, Phunghassaporn N, Tansawet A, Lolak S. Accuracy of machine learning logistic regression in death prediction in road traffic injury patients. Asian J Surg 2021; 45:537-538. [PMID: 34657787 DOI: 10.1016/j.asjsur.2021.09.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2021] [Accepted: 09/23/2021] [Indexed: 11/16/2022] Open
Affiliation(s)
| | - Naralin Phunghassaporn
- Department of Surgery, Faculty of Medicine Vajira Hospital, Navamindradhiraj University, Bangkok, Thailand
| | - Amarit Tansawet
- Department of Surgery, Faculty of Medicine Vajira Hospital, Navamindradhiraj University, Bangkok, Thailand.
| | - Sermkiat Lolak
- Section of Data Science for Healthcare, Department of Clinical Epidemiology and Biostatistics, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, Thailand.
| |
Collapse
|
10
|
Elfanagely O, Toyoda Y, Othman S, Mellia JA, Basta M, Liu T, Kording K, Ungar L, Fischer JP. Machine Learning and Surgical Outcomes Prediction: A Systematic Review. J Surg Res 2021; 264:346-361. [PMID: 33848833 DOI: 10.1016/j.jss.2021.02.045] [Citation(s) in RCA: 44] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Revised: 02/13/2021] [Accepted: 02/27/2021] [Indexed: 12/20/2022]
Abstract
BACKGROUND Machine learning (ML) has garnered increasing attention as a means to quantitatively analyze the growing and complex medical data to improve individualized patient care. We herein aim to critically examine the current state of ML in predicting surgical outcomes, evaluate the quality of currently available research, and propose areas of improvement for future uses of ML in surgery. METHODS A systematic review was conducted in accordance with the Preferred Reporting Items for a Systematic Review and Meta-Analysis (PRISMA) checklist. PubMed, MEDLINE, and Embase databases were reviewed under search syntax "machine learning" and "surgery" for papers published between 2015 and 2020. RESULTS Of the initial 2677 studies, 45 papers met inclusion and exclusion criteria. Fourteen different subspecialties were represented with neurosurgery being most common. The most frequently used ML algorithms were random forest (n = 19), artificial neural network (n = 17), and logistic regression (n = 17). Common outcomes included postoperative mortality, complications, patient reported quality of life and pain improvement. All studies which compared ML algorithms to conventional studies which used area under the curve (AUC) to measure accuracy found improved outcome prediction with ML models. CONCLUSIONS While still in its early stages, ML models offer surgeons an opportunity to capitalize on the myriad of clinical data available and improve individualized patient care. Limitations included heterogeneous outcome and imperfect quality of some of the papers. We therefore urge future research to agree upon methods of outcome reporting and require basic quality standards.
Collapse
Affiliation(s)
- Omar Elfanagely
- Division of Plastic Surgery, Department of Surgery, University of Pennsylvania, Philadelphia, Pennsylvania.
| | - Yoshiko Toyoda
- Division of Plastic Surgery, Department of Surgery, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Sammy Othman
- Division of Plastic Surgery, Department of Surgery, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Joseph A Mellia
- Division of Plastic Surgery, Department of Surgery, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Marten Basta
- Department of Plastic and Reconstructive Surgery, Brown University, Providence, Rhode Island
| | - Tony Liu
- Department of Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Konrad Kording
- Department of Neuroscience, University of Pennsylvania, Philadelphia, Pennsylvania; Department of Bioengineering, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Lyle Ungar
- Department of Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania
| | - John P Fischer
- Division of Plastic Surgery, Department of Surgery, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
11
|
Staziaki PV, Wu D, Rayan JC, Santo IDDO, Nan F, Maybury A, Gangasani N, Benador I, Saligrama V, Scalera J, Anderson SW. Machine learning combining CT findings and clinical parameters improves prediction of length of stay and ICU admission in torso trauma. Eur Radiol 2021; 31:5434-5441. [PMID: 33475772 DOI: 10.1007/s00330-020-07534-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Revised: 11/12/2020] [Accepted: 11/17/2020] [Indexed: 11/30/2022]
Abstract
OBJECTIVE To develop machine learning (ML) models capable of predicting ICU admission and extended length of stay (LOS) after torso (chest, abdomen, or pelvis) trauma, by using clinical and/or imaging data. MATERIALS AND METHODS This was a retrospective study of 840 adult patients admitted to a level 1 trauma center after injury to the torso over the course of 1 year. Clinical parameters included age, sex, vital signs, clinical scores, and laboratory values. Imaging data consisted of any injury present on CT. The two outcomes of interest were ICU admission and extended LOS, defined as more than the median LOS in the dataset. We developed and tested artificial neural network (ANN) and support vector machine (SVM) models, and predictive performance was evaluated by area under the receiver operating characteristic (ROC) curve (AUC). RESULTS The AUCs of SVM and ANN models to predict ICU admission were up to 0.87 ± 0.03 and 0.78 ± 0.02, respectively. The AUCs of SVM and ANN models to predict extended LOS were up to 0.80 ± 0.04 and 0.81 ± 0.05, respectively. Predictions based on imaging alone or imaging with clinical parameters were consistently more accurate than those based solely on clinical parameters. CONCLUSIONS The best performing models incorporated imaging findings and outperformed those with clinical findings alone. ML models have the potential to help predict outcomes in trauma by integrating clinical and imaging findings, although further research may be needed to optimize their performance. KEY POINTS • Artificial neural network and support vector machine-based models were used to predict the intensive care unit admission and extended length of stay after trauma to the torso. • Our input data consisted of clinical parameters and CT imaging findings derived from radiology reports, and we found that combining the two significantly enhanced the prediction of both outcomes with either model. • The highest accuracy (83%) and highest area under the receiver operating characteristic curve (0.87) were obtained for artificial neural networks and support vector machines, respectively, by combining clinical and imaging features in the prediction of intensive care unit admission.
Collapse
Affiliation(s)
- Pedro Vinícius Staziaki
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA.
| | - Di Wu
- Department of Electrical and Computer Engineering, Boston University College of Engineering, Boston, MA, USA
| | - Jesse C Rayan
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| | - Irene Dixe de Oliveira Santo
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| | - Feng Nan
- Department of Electrical and Computer Engineering, Boston University College of Engineering, Boston, MA, USA
| | - Aaron Maybury
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| | - Neha Gangasani
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| | - Ilan Benador
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| | - Venkatesh Saligrama
- Department of Electrical and Computer Engineering, Boston University College of Engineering, Boston, MA, USA
| | - Jonathan Scalera
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| | - Stephan W Anderson
- Department of Radiology, Boston Medical Center, Boston University School of Medicine, 820 Harrison Ave, FGH Building, 4th Floor, Boston, MA, 02114, USA
| |
Collapse
|
12
|
Jaotombo F, Pauly V, Auquier P, Orleans V, Boucekine M, Fond G, Ghattas B, Boyer L. Machine-learning prediction of unplanned 30-day rehospitalization using the French hospital medico-administrative database. Medicine (Baltimore) 2020; 99:e22361. [PMID: 33285668 PMCID: PMC7717815 DOI: 10.1097/md.0000000000022361] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/23/2023] Open
Abstract
Predicting unplanned rehospitalizations has traditionally employed logistic regression models. Machine learning (ML) methods have been introduced in health service research and may improve the prediction of health outcomes. The objective of this work was to develop a ML model to predict 30-day all-cause rehospitalizations based on the French hospital medico-administrative database.This was a retrospective cohort study of all discharges in the year 2015 from acute-care inpatient hospitalizations in a tertiary-care university center comprising 4 French hospitals. The study endpoint was unplanned 30-day all-cause rehospitalization. Logistic regression (LR), classification and regression trees (CART), random forest (RF), gradient boosting (GB), and neural networks (NN) were applied to the collected data. The predictive performance of the models was evaluated using the H-measure and the area under the ROC curve (AUC).Our analysis included 118,650 hospitalizations, of which 4127 (3.5%) led to rehospitalizations via emergency departments. The RF model was the most performant model according to the H-measure (0.29) and the AUC (0.79). The performances of the RF, GB and NN models (H-measures ranged from 0.18 to 0. 29, AUC ranged from 0.74 to 0.79) were better than those of the LR model (H-measure = 0.18, AUC = 0.74); all P values <.001. In contrast, LR was superior to CART (H-measure = 0.16, AUC = 0.70), P < .0001.The use of ML may be an alternative to regression models to predict health outcomes. The integration of ML, particularly the RF algorithm, in the prediction of unplanned rehospitalization may help health service providers target patients at high risk of rehospitalizations and propose effective interventions at the hospital level.
Collapse
Affiliation(s)
- Franck Jaotombo
- Aix-Marseille University, EA 3279 - Public Health, Chronic Diseases and Quality of Life - Research Unit, La Timone Medical University, 27, boulevard Jean-Moulin
- Mathematics Institute of Marseille, Aix-Marseille University, Marseille, France
| | - Vanessa Pauly
- Aix-Marseille University, EA 3279 - Public Health, Chronic Diseases and Quality of Life - Research Unit, La Timone Medical University, 27, boulevard Jean-Moulin
- Service d’Information Médicale, Public Health Department, La Conception Hospital, Assistance Publique - Hôpitaux de Marseille, 147 Boulevard Baille, Marseille, France
| | - Pascal Auquier
- Aix-Marseille University, EA 3279 - Public Health, Chronic Diseases and Quality of Life - Research Unit, La Timone Medical University, 27, boulevard Jean-Moulin
| | - Veronica Orleans
- Service d’Information Médicale, Public Health Department, La Conception Hospital, Assistance Publique - Hôpitaux de Marseille, 147 Boulevard Baille, Marseille, France
| | - Mohamed Boucekine
- Aix-Marseille University, EA 3279 - Public Health, Chronic Diseases and Quality of Life - Research Unit, La Timone Medical University, 27, boulevard Jean-Moulin
| | - Guillaume Fond
- Aix-Marseille University, EA 3279 - Public Health, Chronic Diseases and Quality of Life - Research Unit, La Timone Medical University, 27, boulevard Jean-Moulin
| | - Badih Ghattas
- Mathematics Institute of Marseille, Aix-Marseille University, Marseille, France
| | - Laurent Boyer
- Aix-Marseille University, EA 3279 - Public Health, Chronic Diseases and Quality of Life - Research Unit, La Timone Medical University, 27, boulevard Jean-Moulin
- Service d’Information Médicale, Public Health Department, La Conception Hospital, Assistance Publique - Hôpitaux de Marseille, 147 Boulevard Baille, Marseille, France
| |
Collapse
|
13
|
Muscas G, Matteuzzi T, Becattini E, Orlandini S, Battista F, Laiso A, Nappini S, Limbucci N, Renieri L, Carangelo BR, Mangiafico S, Della Puppa A. Development of machine learning models to prognosticate chronic shunt-dependent hydrocephalus after aneurysmal subarachnoid hemorrhage. Acta Neurochir (Wien) 2020; 162:3093-3105. [PMID: 32642833 PMCID: PMC7593274 DOI: 10.1007/s00701-020-04484-6] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2020] [Accepted: 07/02/2020] [Indexed: 01/06/2023]
Abstract
BACKGROUND Shunt-dependent hydrocephalus significantly complicates subarachnoid hemorrhage (SAH), and reliable prognosis methods have been sought in recent years to reduce morbidity and costs associated with delayed treatment or neglected onset. Machine learning (ML) defines modern data analysis techniques allowing accurate subject-based risk stratifications. We aimed at developing and testing different ML models to predict shunt-dependent hydrocephalus after aneurysmal SAH. METHODS We consulted electronic records of patients with aneurysmal SAH treated at our institution between January 2013 and March 2019. We selected variables for the models according to the results of the previous works on this topic. We trained and tested four ML algorithms on three datasets: one containing binary variables, one considering variables associated with shunt-dependency after an explorative analysis, and one including all variables. For each model, we calculated AUROC, specificity, sensitivity, accuracy, PPV, and also, on the validation set, the NPV and the Matthews correlation coefficient (ϕ). RESULTS Three hundred eighty-six patients were included. Fifty patients (12.9%) developed shunt-dependency after a mean follow-up of 19.7 (± 12.6) months. Complete information was retrieved for 32 variables, used to train the models. The best models were selected based on the performances on the validation set and were achieved with a distributed random forest model considering 21 variables, with a ϕ = 0.59, AUC = 0.88; sensitivity and specificity of 0.73 (C.I.: 0.39-0.94) and 0.92 (C.I.: 0.84-0.97), respectively; PPV = 0.59 (0.38-0.77); and NPV = 0.96 (0.90-0.98). Accuracy was 0.90 (0.82-0.95). CONCLUSIONS Machine learning prognostic models allow accurate predictions with a large number of variables and a more subject-oriented prognosis. We identified a single best distributed random forest model, with an excellent prognostic capacity (ϕ = 0.58), which could be especially helpful in identifying low-risk patients for shunt-dependency.
Collapse
Affiliation(s)
- Giovanni Muscas
- Neurosurgery Clinic, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Largo Piero Palagi 1, 50137, Florence, Italy.
| | - Tommaso Matteuzzi
- Institute of Physics, Alma Mater Studiorum, University of Bologna, Bologna, Italy
| | - Eleonora Becattini
- Neurosurgery Clinic, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Largo Piero Palagi 1, 50137, Florence, Italy
| | - Simone Orlandini
- Neurosurgery Clinic, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Largo Piero Palagi 1, 50137, Florence, Italy
| | - Francesca Battista
- Neurosurgery Clinic, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Largo Piero Palagi 1, 50137, Florence, Italy
| | - Antonio Laiso
- Neurosurgery Clinic, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Largo Piero Palagi 1, 50137, Florence, Italy
- Interventional Neuroradiology Unit, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Florence, Italy
| | - Sergio Nappini
- Interventional Neuroradiology Unit, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Florence, Italy
| | - Nicola Limbucci
- Interventional Neuroradiology Unit, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Florence, Italy
| | - Leonardo Renieri
- Interventional Neuroradiology Unit, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Florence, Italy
| | | | - Salvatore Mangiafico
- Interventional Neuroradiology Unit, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Florence, Italy
| | - Alessandro Della Puppa
- Neurosurgery Clinic, Department of Neuroscience, Psychology, Pharmacology and Child Health, Careggi University Hospital and University of Florence, Largo Piero Palagi 1, 50137, Florence, Italy
| |
Collapse
|
14
|
Abstract
Commercial applications of artificial intelligence and machine learning have made remarkable progress recently, particularly in areas such as image recognition, natural speech processing, language translation, textual analysis, and self-learning. Progress had historically languished in these areas, such that these skills had come to seem ineffably bound to intelligence. However, these commercial advances have performed best at single-task applications in which imperfect outputs and occasional frank errors can be tolerated.The practice of anesthesiology is different. It embodies a requirement for high reliability, and a pressured cycle of interpretation, physical action, and response rather than any single cognitive act. This review covers the basics of what is meant by artificial intelligence and machine learning for the practicing anesthesiologist, describing how decision-making behaviors can emerge from simple equations. Relevant clinical questions are introduced to illustrate how machine learning might help solve them-perhaps bringing anesthesiology into an era of machine-assisted discovery.
Collapse
|
15
|
Matsuo K, Aihara H, Nakai T, Morishita A, Tohma Y, Kohmura E. Machine Learning to Predict In-Hospital Morbidity and Mortality after Traumatic Brain Injury. J Neurotrauma 2019; 37:202-210. [PMID: 31359814 DOI: 10.1089/neu.2018.6276] [Citation(s) in RCA: 60] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023] Open
Abstract
Recently, successful predictions using machine learning (ML) algorithms have been reported in various fields. However, in traumatic brain injury (TBI) cohorts, few studies have examined modern ML algorithms. To develop a simple ML model for TBI outcome prediction, we conducted a performance comparison of nine algorithms: ridge regression, least absolute shrinkage and selection operator (LASSO) regression, random forest, gradient boosting, extra trees, decision tree, Gaussian naïve Bayes, multi-nomial naïve Bayes, and support vector machine. Fourteen feasible parameters were introduced in the ML models, including age, Glasgow Coma Scale (GCS), systolic blood pressure (SBP), abnormal pupillary response, major extracranial injury, computed tomography (CT) findings, and routinely collected laboratory values (glucose, C-reactive protein [CRP], and fibrin/fibrinogen degradation products [FDP]). Data from 232 patients with TBI were randomly divided into a training sample (80%) for hyperparameter tuning and validation sample (20%). The bootstrap method was used for validation. Random forest demonstrated the best performance for in-hospital poor outcome prediction and ridge regression for in-hospital mortality prediction: the mean statistical measures were 100% sensitivity, 72.3% specificity, 91.7% accuracy, and 0.895 area under the receiver operating characteristic curve (AUC); and 88.4% sensitivity, 88.2% specificity, 88.6% accuracy, and 0.875 AUC, respectively. Based on the feature selection method using the tree-based ensemble algorithm, age, Glasgow Coma Scale, fibrin/fibrinogen degradation products, and glucose were identified as the most important prognostic factors for poor outcome and mortality. Our results indicate the relatively good predictive performance of modern ML for TBI outcome. Further external validation is required for more heterogeneous samples to confirm our results.
Collapse
Affiliation(s)
- Kazuya Matsuo
- Department of Neurosurgery, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| | - Hideo Aihara
- Department of Neurosurgery, Hyogo Prefectural Kakogawa Medical Center, Kakogawa, Hyogo, Japan
| | - Tomoaki Nakai
- Department of Neurosurgery, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| | - Akitsugu Morishita
- Department of Neurosurgery, Hyogo Prefectural Kakogawa Medical Center, Kakogawa, Hyogo, Japan
| | - Yoshiki Tohma
- Department of Emergency and Critical Care Medicine, Hyogo Prefectural Kakogawa Medical Center, Kakogawa, Hyogo, Japan
| | - Eiji Kohmura
- Department of Neurosurgery, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| |
Collapse
|
16
|
Rau CS, Wu SC, Chuang JF, Huang CY, Liu HT, Chien PC, Hsieh CH. Machine Learning Models of Survival Prediction in Trauma Patients. J Clin Med 2019; 8:jcm8060799. [PMID: 31195670 PMCID: PMC6616432 DOI: 10.3390/jcm8060799] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2019] [Revised: 06/03/2019] [Accepted: 06/03/2019] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND We aimed to build a model using machine learning for the prediction of survival in trauma patients and compared these model predictions to those predicted by the most commonly used algorithm, the Trauma and Injury Severity Score (TRISS). METHODS Enrolled hospitalized trauma patients from 2009 to 2016 were divided into a training dataset (70% of the original data set) for generation of a plausible model under supervised classification, and a test dataset (30% of the original data set) to test the performance of the model. The training and test datasets comprised 13,208 (12,871 survival and 337 mortality) and 5603 (5473 survival and 130 mortality) patients, respectively. With the provision of additional information such as pre-existing comorbidity status or laboratory data, logistic regression (LR), support vector machine (SVM), and neural network (NN) (with the Stuttgart Neural Network Simulator (RSNNS)) were used to build models of survival prediction and compared to the predictive performance of TRISS. Predictive performance was evaluated by accuracy, sensitivity, and specificity, as well as by area under the curve (AUC) measures of receiver operating characteristic curves. RESULTS In the validation dataset, NN and the TRISS presented the highest score (82.0%) for balanced accuracy, followed by SVM (75.2%) and LR (71.8%) models. In the test dataset, NN had the highest balanced accuracy (75.1%), followed by the TRISS (70.2%), SVM (70.6%), and LR (68.9%) models. All four models (LR, SVM, NN, and TRISS) exhibited a high accuracy of more than 97.5% and a sensitivity of more than 98.6%. However, NN exhibited the highest specificity (51.5%), followed by the TRISS (41.5%), SVM (40.8%), and LR (38.5%) models. CONCLUSIONS These four models (LR, SVM, NN, and TRISS) exhibited a similar high accuracy and sensitivity in predicting the survival of the trauma patients. In the test dataset, the NN model had the highest balanced accuracy and predictive specificity.
Collapse
Affiliation(s)
- Cheng-Shyuan Rau
- Department of Neurosurgery, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| | - Shao-Chun Wu
- Department of Anesthesiology, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| | - Jung-Fang Chuang
- Department of Trauma Surgery, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| | - Chun-Ying Huang
- Department of Trauma Surgery, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| | - Hang-Tsung Liu
- Department of Trauma Surgery, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| | - Peng-Chen Chien
- Department of Plastic Surgery, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| | - Ching-Hua Hsieh
- Department of Plastic Surgery, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung 833, Taiwan.
| |
Collapse
|
17
|
Lee HC, Yoon SB, Yang SM, Kim WH, Ryu HG, Jung CW, Suh KS, Lee KH. Prediction of Acute Kidney Injury after Liver Transplantation: Machine Learning Approaches vs. Logistic Regression Model. J Clin Med 2018; 7:jcm7110428. [PMID: 30413107 PMCID: PMC6262324 DOI: 10.3390/jcm7110428] [Citation(s) in RCA: 110] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2018] [Revised: 10/29/2018] [Accepted: 11/06/2018] [Indexed: 12/15/2022] Open
Abstract
Acute kidney injury (AKI) after liver transplantation has been reported to be associated with increased mortality. Recently, machine learning approaches were reported to have better predictive ability than the classic statistical analysis. We compared the performance of machine learning approaches with that of logistic regression analysis to predict AKI after liver transplantation. We reviewed 1211 patients and preoperative and intraoperative anesthesia and surgery-related variables were obtained. The primary outcome was postoperative AKI defined by acute kidney injury network criteria. The following machine learning techniques were used: decision tree, random forest, gradient boosting machine, support vector machine, naïve Bayes, multilayer perceptron, and deep belief networks. These techniques were compared with logistic regression analysis regarding the area under the receiver-operating characteristic curve (AUROC). AKI developed in 365 patients (30.1%). The performance in terms of AUROC was best in gradient boosting machine among all analyses to predict AKI of all stages (0.90, 95% confidence interval [CI] 0.86–0.93) or stage 2 or 3 AKI. The AUROC of logistic regression analysis was 0.61 (95% CI 0.56–0.66). Decision tree and random forest techniques showed moderate performance (AUROC 0.86 and 0.85, respectively). The AUROC of support the vector machine, naïve Bayes, neural network, and deep belief network was smaller than that of the other models. In our comparison of seven machine learning approaches with logistic regression analysis, the gradient boosting machine showed the best performance with the highest AUROC. An internet-based risk estimator was developed based on our model of gradient boosting. However, prospective studies are required to validate our results.
Collapse
Affiliation(s)
- Hyung-Chul Lee
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
| | - Soo Bin Yoon
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
| | - Seong-Mi Yang
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
| | - Won Ho Kim
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul 03080, Korea.
| | - Ho-Geol Ryu
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul 03080, Korea.
| | - Chul-Woo Jung
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul 03080, Korea.
| | - Kyung-Suk Suh
- Department of Surgery, Seoul National University Hospital, Seoul National University College of Medicine, Seoul 03080, Korea.
| | - Kook Hyun Lee
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul 03080, Korea.
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul 03080, Korea.
| |
Collapse
|
18
|
Derivation and Validation of Machine Learning Approaches to Predict Acute Kidney Injury after Cardiac Surgery. J Clin Med 2018; 7:jcm7100322. [PMID: 30282956 PMCID: PMC6210196 DOI: 10.3390/jcm7100322] [Citation(s) in RCA: 78] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2018] [Revised: 10/01/2018] [Accepted: 10/02/2018] [Indexed: 12/18/2022] Open
Abstract
Machine learning approaches were introduced for better or comparable predictive ability than statistical analysis to predict postoperative outcomes. We sought to compare the performance of machine learning approaches with that of logistic regression analysis to predict acute kidney injury after cardiac surgery. We retrospectively reviewed 2010 patients who underwent open heart surgery and thoracic aortic surgery. Baseline medical condition, intraoperative anesthesia, and surgery-related data were obtained. The primary outcome was postoperative acute kidney injury (AKI) defined according to the Kidney Disease Improving Global Outcomes criteria. The following machine learning techniques were used: decision tree, random forest, extreme gradient boosting, support vector machine, neural network classifier, and deep learning. The performance of these techniques was compared with that of logistic regression analysis regarding the area under the receiver-operating characteristic curve (AUC). During the first postoperative week, AKI occurred in 770 patients (38.3%). The best performance regarding AUC was achieved by the gradient boosting machine to predict the AKI of all stages (0.78, 95% confidence interval (CI) 0.75–0.80) or stage 2 or 3 AKI. The AUC of logistic regression analysis was 0.69 (95% CI 0.66–0.72). Decision tree, random forest, and support vector machine showed similar performance to logistic regression. In our comprehensive comparison of machine learning approaches with logistic regression analysis, gradient boosting technique showed the best performance with the highest AUC and lower error rate. We developed an Internet–based risk estimator which could be used for real-time processing of patient data to estimate the risk of AKI at the end of surgery.
Collapse
|
19
|
Ouchi K, Lindvall C, Chai PR, Boyer EW. Machine Learning to Predict, Detect, and Intervene Older Adults Vulnerable for Adverse Drug Events in the Emergency Department. J Med Toxicol 2018; 14:248-252. [PMID: 29858745 PMCID: PMC6097964 DOI: 10.1007/s13181-018-0667-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2018] [Revised: 05/15/2018] [Accepted: 05/21/2018] [Indexed: 02/08/2023] Open
Abstract
Adverse drug events (ADEs) are common and have serious consequences in older adults. ED visits are opportunities to identify and alter the course of such vulnerable patients. Current practice, however, is limited by inaccurate reporting of medication list, time-consuming medication reconciliation, and poor ADE assessment. This manuscript describes a novel approach to predict, detect, and intervene vulnerable older adults at risk of ADE using machine learning. Toxicologists' expertise in ADE is essential to creating the machine learning algorithm. Leveraging the existing electronic health records to better capture older adults at risk of ADE in the ED may improve their care.
Collapse
Affiliation(s)
- Kei Ouchi
- Department of Emergency Medicine, Brigham and Women's Hospital, 75 Francis St, Neville 200, Boston, MA, 02125, USA.
- Department of Emergency Medicine, Harvard Medical School, Boston, MA, USA.
- Serious Illness Care Program, Ariadne Labs, Boston, MA, USA.
- Department of Psychosocial Oncology and Palliative Care, Dana Farber Cancer Institute, Boston, MA, USA.
| | - Charlotta Lindvall
- Department of Psychosocial Oncology and Palliative Care, Dana Farber Cancer Institute, Boston, MA, USA
- Division of Palliative Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
| | - Peter R Chai
- Department of Emergency Medicine, Brigham and Women's Hospital, 75 Francis St, Neville 200, Boston, MA, 02125, USA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA, USA
- The Fenway Institute, Boston, MA, USA
| | - Edward W Boyer
- Department of Emergency Medicine, Brigham and Women's Hospital, 75 Francis St, Neville 200, Boston, MA, 02125, USA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA, USA
- The Fenway Institute, Boston, MA, USA
| |
Collapse
|
20
|
Kuo CY, Yu LC, Chen HC, Chan CL. Comparison of Models for the Prediction of Medical Costs of Spinal Fusion in Taiwan Diagnosis-Related Groups by Machine Learning Algorithms. Healthc Inform Res 2018; 24:29-37. [PMID: 29503750 PMCID: PMC5820083 DOI: 10.4258/hir.2018.24.1.29] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2017] [Revised: 01/16/2018] [Accepted: 01/22/2018] [Indexed: 12/22/2022] Open
Abstract
Objectives The aims of this study were to compare the performance of machine learning methods for the prediction of the medical costs associated with spinal fusion in terms of profit or loss in Taiwan Diagnosis-Related Groups (Tw-DRGs) and to apply these methods to explore the important factors associated with the medical costs of spinal fusion. Methods A data set was obtained from a regional hospital in Taoyuan city in Taiwan, which contained data from 2010 to 2013 on patients of Tw-DRG49702 (posterior and other spinal fusion without complications or comorbidities). Naïve-Bayesian, support vector machines, logistic regression, C4.5 decision tree, and random forest methods were employed for prediction using WEKA 3.8.1. Results Five hundred thirty-two cases were categorized as belonging to the Tw-DRG49702 group. The mean medical cost was US $4,549.7, and the mean age of the patients was 62.4 years. The mean length of stay was 9.3 days. The length of stay was an important variable in terms of determining medical costs for patients undergoing spinal fusion. The random forest method had the best predictive performance in comparison to the other methods, achieving an accuracy of 84.30%, a sensitivity of 71.4%, a specificity of 92.2%, and an AUC of 0.904. Conclusions Our study demonstrated that the random forest model can be employed to predict the medical costs of Tw-DRG49702, and could inform hospital strategy in terms of increasing the financial management efficiency of this operation.
Collapse
Affiliation(s)
- Ching-Yen Kuo
- Institute of Information Management, Yuan-Ze University, Taoyuan, Taiwan.,Department of Medical Administration, Taoyuan General Hospital, Ministry of Health and Welfare, Taoyuan, Taiwan
| | - Liang-Chin Yu
- Institute of Information Management, Yuan-Ze University, Taoyuan, Taiwan
| | - Hou-Chaung Chen
- Department of Orthopedics, Taoyuan General Hospital, Ministry of Health and Welfare, Taoyuan, Taiwan
| | - Chien-Lung Chan
- Institute of Information Management, Yuan-Ze University, Taoyuan, Taiwan.,Innovation Center for Big Data and Digital Convergence, Yuan-Ze University, Taoyuan, Taiwan
| |
Collapse
|