1
|
Li B, Eisenberg N, Beaton D, Lee DS, Al-Omran L, Wijeysundera DN, Hussain MA, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using Machine Learning to Predict Outcomes Following Transfemoral Carotid Artery Stenting. J Am Heart Assoc 2024; 13:e035425. [PMID: 39189482 DOI: 10.1161/jaha.124.035425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/08/2024] [Accepted: 07/23/2024] [Indexed: 08/28/2024]
Abstract
BACKGROUND Transfemoral carotid artery stenting (TFCAS) carries important perioperative risks. Outcome prediction tools may help guide clinical decision-making but remain limited. We developed machine learning algorithms that predict 1-year stroke or death following TFCAS. METHODS AND RESULTS The VQI (Vascular Quality Initiative) database was used to identify patients who underwent TFCAS for carotid artery stenosis between 2005 and 2024. We identified 112 features from the index hospitalization (82 preoperative [demographic/clinical], 13 intraoperative [procedural], and 17 postoperative [in-hospital course/complications]). The primary outcome was 1-year postprocedural stroke or death. The data were divided into training (70%) and test (30%) sets. Six machine learning models were trained using preoperative features with 10-fold cross-validation. The primary model evaluation metric was area under the receiver operating characteristic curve. The algorithm with the best performance was further trained using intra- and postoperative features. Model robustness was assessed using calibration plots and Brier scores. Overall, 35 214 patients underwent TFCAS during the study period and 3257 (9.2%) developed 1-year stroke or death. The best preoperative prediction model was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.94 (95% CI, 0.93-0.95). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67). The extreme gradient boosting model maintained excellent performance at the intra- and postoperative stages, with area under the receiver operating characteristic curve values of 0.94 (95% CI, 0.93-0.95) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted/observed event probabilities with Brier scores of 0.11 (preoperative), 0.11 (intraoperative), and 0.09 (postoperative). CONCLUSIONS Machine learning can accurately predict 1-year stroke or death following TFCAS, performing better than logistic regression.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Institute of Medical Science, University of Toronto Ontario Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM) University of Toronto Ontario Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre University Health Network Toronto Ontario Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto University of Toronto Ontario Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre University Health Network Toronto Ontario Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
| | - Leen Al-Omran
- School of Medicine Alfaisal University Riyadh Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
- Department of Anesthesia St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health Brigham and Women's Hospital, Harvard Medical School Boston MA USA
| | - Ori D Rotstein
- Department of Surgery University of Toronto Ontario Canada
- Institute of Medical Science, University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Division of General Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
| | - Charles de Mestral
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto Ontario Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM) University of Toronto Ontario Canada
- Data Science & Advanced Analytics, Unity Health Toronto University of Toronto Ontario Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Leslie Dan Faculty of Pharmacy University of Toronto Ontario Canada
| | - Graham Roche-Nagle
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre University Health Network Toronto Ontario Canada
- Division of Vascular and Interventional Radiology University Health Network Toronto Ontario Canada
| | - Mohammed Al-Omran
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Institute of Medical Science, University of Toronto Ontario Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM) University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Department of Surgery King Faisal Specialist Hospital and Research Center Riyadh Saudi Arabia
| |
Collapse
|
2
|
Li B, Eisenberg N, Beaton D, Lee DS, Al-Omran L, Wijeysundera DN, Hussain MA, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Predicting inferior vena cava filter complications using machine learning. J Vasc Surg Venous Lymphat Disord 2024:101943. [PMID: 39084408 DOI: 10.1016/j.jvsv.2024.101943] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2024] [Revised: 06/03/2024] [Accepted: 06/26/2024] [Indexed: 08/02/2024]
Abstract
OBJECTIVE Inferior vena cava (IVC) filter placement is associated with important long-term complications. Predictive models for filter-related complications may help guide clinical decision-making but remain limited. We developed machine learning (ML) algorithms that predict 1-year IVC filter complications using preoperative data. METHODS The Vascular Quality Initiative database was used to identify patients who underwent IVC filter placement between 2013 and 2024. We identified 77 preoperative demographic and clinical features from the index hospitalization when the filter was placed. The primary outcome was 1-year filter-related complications (composite of filter thrombosis, migration, angulation, fracture, and embolization or fragmentation, vein perforation, new caval or iliac vein thrombosis, new pulmonary embolism, access site thrombosis, or failed retrieval). The data were divided into training (70%) and test (30%) sets. Six ML models were trained using preoperative features with 10-fold cross-validation (Extreme Gradient Boosting, random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was assessed using calibration plot and Brier score. Performance was evaluated across subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, planned duration of filter, landing site of filter, and presence of prior IVC filter placement. RESULTS Overall, 14,476 patients underwent IVC filter placement and 584 (4.0%) experienced 1-year filter-related complications. Patients with a primary outcome were younger (59.3 ± 16.7 years vs 63.8 ± 16.0 years; P < .001) and more likely to have thrombotic risk factors including thrombophilia, prior venous thromboembolism (VTE), and family history of VTE. The best prediction model was Extreme Gradient Boosting, achieving an AUROC of 0.93 (95% confidence interval, 0.92-0.94). In comparison, logistic regression had an AUROC of 0.63 (95% confidence interval, 0.61-0.65). Calibration plot showed good agreement between predicted/observed event probabilities with a Brier score of 0.07. The top 10 predictors of 1-year filter-related complications were (1) thrombophilia, (2) prior VTE, (3) antiphospholipid antibodies, (4) factor V Leiden mutation, (5) family history of VTE, (6) planned duration of IVC filter (temporary), (7) unable to maintain therapeutic anticoagulation, (8) malignancy, (9) recent or active bleeding, and (10) age. Model performance remained robust across all subgroups. CONCLUSIONS We developed ML models that can accurately predict 1-year IVC filter complications, performing better than logistic regression. These algorithms have potential to guide patient selection for filter placement, counselling, perioperative management, and follow-up to mitigate filter-related complications and improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Institute of Medical Science, University of Toronto, Toronto, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada
| | - Leen Al-Omran
- School of Medicine, Alfaisal University, Riyadh, Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, MA
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, Canada; Institute of Medical Science, University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada; Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada; Division of Vascular and Interventional Radiology, University Health Network, Toronto, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Institute of Medical Science, University of Toronto, Toronto, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia.
| |
Collapse
|
3
|
Guo Y, Huang C, Sheng Y, Zhang W, Ye X, Lian H, Xu J, Chen Y. Improve the efficiency and accuracy of ophthalmologists' clinical decision-making based on AI technology. BMC Med Inform Decis Mak 2024; 24:192. [PMID: 38982465 PMCID: PMC11234671 DOI: 10.1186/s12911-024-02587-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 06/24/2024] [Indexed: 07/11/2024] Open
Abstract
BACKGROUND As global aging intensifies, the prevalence of ocular fundus diseases continues to rise. In China, the tense doctor-patient ratio poses numerous challenges for the early diagnosis and treatment of ocular fundus diseases. To reduce the high risk of missed or misdiagnosed cases, avoid irreversible visual impairment for patients, and ensure good visual prognosis for patients with ocular fundus diseases, it is particularly important to enhance the growth and diagnostic capabilities of junior doctors. This study aims to leverage the value of electronic medical record data to developing a diagnostic intelligent decision support platform. This platform aims to assist junior doctors in diagnosing ocular fundus diseases quickly and accurately, expedite their professional growth, and prevent delays in patient treatment. An empirical evaluation will assess the platform's effectiveness in enhancing doctors' diagnostic efficiency and accuracy. METHODS In this study, eight Chinese Named Entity Recognition (NER) models were compared, and the SoftLexicon-Glove-Word2vec model, achieving a high F1 score of 93.02%, was selected as the optimal recognition tool. This model was then used to extract key information from electronic medical records (EMRs) and generate feature variables based on diagnostic rule templates. Subsequently, an XGBoost algorithm was employed to construct an intelligent decision support platform for diagnosing ocular fundus diseases. The effectiveness of the platform in improving diagnostic efficiency and accuracy was evaluated through a controlled experiment comparing experienced and junior doctors. RESULTS The use of the diagnostic intelligent decision support platform resulted in significant improvements in both diagnostic efficiency and accuracy for both experienced and junior doctors (P < 0.05). Notably, the gap in diagnostic speed and precision between junior doctors and experienced doctors narrowed considerably when the platform was used. Although the platform also provided some benefits to experienced doctors, the improvement was less pronounced compared to junior doctors. CONCLUSION The diagnostic intelligent decision support platform established in this study, based on the XGBoost algorithm and NER, effectively enhances the diagnostic efficiency and accuracy of junior doctors in ocular fundus diseases. This has significant implications for optimizing clinical diagnosis and treatment.
Collapse
Affiliation(s)
- Yingxuan Guo
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Changke Huang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Yaying Sheng
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Wenjie Zhang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Xin Ye
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China
| | - Hengli Lian
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Jiahao Xu
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Yiqi Chen
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, Zhejiang, China.
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China.
| |
Collapse
|
4
|
Huyghebaert TA, Wallner C, Montemurro P. Implementation of a Machine Learning Approach Evaluating Risk Factors for Complications after Single-Stage Augmentation Mastopexy. Aesthetic Plast Surg 2024:10.1007/s00266-024-04142-7. [PMID: 38849552 DOI: 10.1007/s00266-024-04142-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 05/13/2024] [Indexed: 06/09/2024]
Abstract
BACKGROUND Single-stage mastopexy augmentation is a much-debated intervention due to its complexity and the associated relatively high complication rates. This study aimed to reevaluate the risk factors for these complications using a novel approach based on artificial intelligence and to demonstrate its possible limitations. PATIENTS AND METHODS Complete datasets of patients who underwent single-staged augmentation mastopexy during 2014-2023 at one institution by a single surgeon were collected retrospectively. These were subsequently processed and analyzed by CART, RF and XGBoost algorithms. RESULTS A total of 342 patients were included in the study, of which 43 (12.57%) reported surgery-associated complications, whereby capsular contracture (n = 19) was the most common. BMI represented the most important variable for the development of complications (FIS = 0.44 in CART). 2.9% of the patients expressed the desire for implant change in the course, with absence of any complications. A statistically significant correlation between smoking and the desire for implant change (p < 0.001) was revealed. CONCLUSION The importance of implementing artificial intelligence into clinical research could be underpinned by this study, as risk variables can be reclassified based on factors previously considered less or even irrelevant. Thereby we encountered limitations using ML approaches. Further studies will be needed to investigate the association between smoking, BMI and the current implant size with the desire for implant change without any complications. Moreover, we could show that the procedure can be performed safely without high risk of developing major complications. LEVEL OF EVIDENCE IV This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266.
Collapse
Affiliation(s)
- Tom Alexander Huyghebaert
- Department of Plastic Surgery, BG University Hospital Bergmannsheil, Ruhr University Bochum, Bürkle-de-la-Camp Platz 1, 44789, Bochum, Germany.
| | - Christoph Wallner
- Department of Plastic Surgery, BG University Hospital Bergmannsheil, Ruhr University Bochum, Bürkle-de-la-Camp Platz 1, 44789, Bochum, Germany
| | | |
Collapse
|
5
|
Vitt JR, Mainali S. Artificial Intelligence and Machine Learning Applications in Critically Ill Brain Injured Patients. Semin Neurol 2024; 44:342-356. [PMID: 38569520 DOI: 10.1055/s-0044-1785504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/05/2024]
Abstract
The utilization of Artificial Intelligence (AI) and Machine Learning (ML) is paving the way for significant strides in patient diagnosis, treatment, and prognostication in neurocritical care. These technologies offer the potential to unravel complex patterns within vast datasets ranging from vast clinical data and EEG (electroencephalogram) readings to advanced cerebral imaging facilitating a more nuanced understanding of patient conditions. Despite their promise, the implementation of AI and ML faces substantial hurdles. Historical biases within training data, the challenge of interpreting multifaceted data streams, and the "black box" nature of ML algorithms present barriers to widespread clinical adoption. Moreover, ethical considerations around data privacy and the need for transparent, explainable models remain paramount to ensure trust and efficacy in clinical decision-making.This article reflects on the emergence of AI and ML as integral tools in neurocritical care, discussing their roles from the perspective of both their scientific promise and the associated challenges. We underscore the importance of extensive validation in diverse clinical settings to ensure the generalizability of ML models, particularly considering their potential to inform critical medical decisions such as withdrawal of life-sustaining therapies. Advancement in computational capabilities is essential for implementing ML in clinical settings, allowing for real-time analysis and decision support at the point of care. As AI and ML are poised to become commonplace in clinical practice, it is incumbent upon health care professionals to understand and oversee these technologies, ensuring they adhere to the highest safety standards and contribute to the realization of personalized medicine. This engagement will be pivotal in integrating AI and ML into patient care, optimizing outcomes in neurocritical care through informed and data-driven decision-making.
Collapse
Affiliation(s)
- Jeffrey R Vitt
- Department of Neurological Surgery, UC Davis Medical Center, Sacramento, California
| | - Shraddha Mainali
- Department of Neurology, Virginia Commonwealth University, Richmond, Virginia
| |
Collapse
|
6
|
Liu W, Jia L, Xu L, Yang F, Guo Z, Li J, Zhang D, Liu Y, Xiang H, Cheng H, Hou J, Li S, Li H. Prediction of early neurologic deterioration in patients with perforating artery territory infarction using machine learning: a retrospective study. Front Neurol 2024; 15:1368902. [PMID: 38841697 PMCID: PMC11150528 DOI: 10.3389/fneur.2024.1368902] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Accepted: 04/24/2024] [Indexed: 06/07/2024] Open
Abstract
Background Early neurological deterioration (END) is a frequent complication in patients with perforating artery territory infarction (PAI), leading to poorer outcomes. Therefore, we aimed to apply machine learning (ML) algorithms to predict the occurrence of END in PAI and investigate related risk factors. Methods This retrospective study analyzed a cohort of PAI patients, excluding those with severe stenosis of the parent artery. We included demographic characteristics, clinical features, laboratory data, and imaging variables. Recursive feature elimination with cross-validation (RFECV) was performed to identify critical features. Seven ML algorithms, namely logistic regression, random forest, adaptive boosting, gradient boosting decision tree, histogram-based gradient boosting, extreme gradient boosting, and category boosting, were developed to predict END in PAI patients using these critical features. We compared the accuracy of these models in predicting outcomes. Additionally, SHapley Additive exPlanations (SHAP) values were introduced to interpret the optimal model and assess the significance of input features. Results The study enrolled 1,020 PAI patients with a mean age of 60.46 (range 49.11-71.81) years. Of these, 30.39% were women, and 129 (12.65%) experienced END. RFECV selected 13 critical features, including blood urea nitrogen (BUN), total cholesterol (TC), low-density-lipoprotein cholesterol (LDL-C), apolipoprotein B (apoB), atrial fibrillation, loading dual antiplatelet therapy (DAPT), single antiplatelet therapy (SAPT), argatroban, the basal ganglia, the thalamus, the posterior choroidal arteries, maximal axial infarct diameter (measured at < 15 mm), and stroke subtype. The gradient-boosting decision tree had the highest area under the curve (0.914) among the seven ML algorithms. The SHAP analysis identified apoB as the most significant variable for END. Conclusion Our results suggest that ML algorithms, especially the gradient-boosting decision tree, are effective in predicting the occurrence of END in PAI patients.
Collapse
Affiliation(s)
- Wei Liu
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Longbin Jia
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Lina Xu
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Fengbing Yang
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Zixuan Guo
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Jinna Li
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Dandan Zhang
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Yan Liu
- The First Clinical College of Changzhi Medical College, Changzhi, China
| | - Han Xiang
- The First Clinical College of Changzhi Medical College, Changzhi, China
| | - Hongjiang Cheng
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Jing Hou
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Shifang Li
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| | - Huimin Li
- Department of Neurology, Jincheng People's Hospital, Jincheng, China
| |
Collapse
|
7
|
Li B, Aljabri B, Verma R, Beaton D, Hussain MA, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al‐Omran M. Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning. J Am Heart Assoc 2024; 13:e033194. [PMID: 38639373 PMCID: PMC11179886 DOI: 10.1161/jaha.123.033194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 03/01/2024] [Indexed: 04/20/2024]
Abstract
BACKGROUND Lower extremity endovascular revascularization for peripheral artery disease carries nonnegligible perioperative risks; however, outcome prediction tools remain limited. Using machine learning, we developed automated algorithms that predict 30-day outcomes following lower extremity endovascular revascularization. METHODS AND RESULTS The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity endovascular revascularization (angioplasty, stent, or atherectomy) for peripheral artery disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day postprocedural major adverse limb event (composite of major reintervention, untreated loss of patency, or major amputation) or death. Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Overall, 21 886 patients were included, and 30-day major adverse limb event/death occurred in 1964 (9.0%) individuals. The best performing model for predicting 30-day major adverse limb event/death was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.94). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.72 (95% CI, 0.70-0.74). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.09. The top 3 predictive features in our algorithm were (1) chronic limb-threatening ischemia, (2) tibial intervention, and (3) congestive heart failure. CONCLUSIONS Our machine learning models accurately predict 30-day outcomes following lower extremity endovascular revascularization using preoperative data with good discrimination and calibration. Prospective validation is warranted to assess for generalizability and external validity.
Collapse
Affiliation(s)
- Ben Li
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
| | - Badr Aljabri
- Department of SurgeryKing Saud UniversityRiyadhSaudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in IrelandUniversity of Medicine and Health SciencesDublinIreland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoTorontoCanada
| | - Mohamad A. Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women’s HospitalHarvard Medical SchoolBostonMAUSA
| | - Douglas S. Lee
- Division of Cardiology, Peter Munk Cardiac CentreUniversity Health NetworkTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
| | - Duminda N. Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Department of AnesthesiaSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Charles de Mestral
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Muhammad Mamdani
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Leslie Dan Faculty of PharmacyUniversity of TorontoTorontoCanada
| | - Mohammed Al‐Omran
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Department of SurgeryKing Faisal Specialist Hospital and Research CenterRiyadhSaudi Arabia
| |
Collapse
|
8
|
Li B, Eisenberg N, Beaton D, Lee DS, Aljabri B, Verma R, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using Machine Learning (XGBoost) to Predict Outcomes After Infrainguinal Bypass for Peripheral Artery Disease. Ann Surg 2024; 279:705-713. [PMID: 38116648 DOI: 10.1097/sla.0000000000006181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2023]
Abstract
OBJECTIVE To develop machine learning (ML) algorithms that predict outcomes after infrainguinal bypass. BACKGROUND Infrainguinal bypass for peripheral artery disease carries significant surgical risks; however, outcome prediction tools remain limited. METHODS The Vascular Quality Initiative database was used to identify patients who underwent infrainguinal bypass for peripheral artery disease between 2003 and 2023. We identified 97 potential predictor variables from the index hospitalization [68 preoperative (demographic/clinical), 13 intraoperative (procedural), and 16 postoperative (in-hospital course/complications)]. The primary outcome was 1-year major adverse limb event (composite of surgical revision, thrombectomy/thrombolysis, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained 6 ML models using preoperative features. The primary model evaluation metric was the area under the receiver operating characteristic curve (AUROC). The top-performing algorithm was further trained using intraoperative and postoperative features. Model robustness was evaluated using calibration plots and Brier scores. RESULTS Overall, 59,784 patients underwent infrainguinal bypass, and 15,942 (26.7%) developed 1-year major adverse limb event/death. The best preoperative prediction model was XGBoost, achieving an AUROC (95% CI) of 0.94 (0.93-0.95). In comparison, logistic regression had an AUROC (95% CI) of 0.61 (0.59-0.63). Our XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs (95% CI's) of 0.94 (0.93-0.95) and 0.96 (0.95-0.97), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.08 (preoperative), 0.07 (intraoperative), and 0.05 (postoperative). CONCLUSIONS ML models can accurately predict outcomes after infrainguinal bypass, outperforming logistic regression.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Kingdom of Saudi Arabia
| |
Collapse
|
9
|
Schweingruber N, Bremer J, Wiehe A, Mader MMD, Mayer C, Woo MS, Kluge S, Grensemann J, Quandt F, Gempt J, Fischer M, Thomalla G, Gerloff C, Sauvigny J, Czorlich P. Early prediction of ventricular peritoneal shunt dependency in aneurysmal subarachnoid haemorrhage patients by recurrent neural network-based machine learning using routine intensive care unit data. J Clin Monit Comput 2024:10.1007/s10877-024-01151-4. [PMID: 38512361 DOI: 10.1007/s10877-024-01151-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Accepted: 03/08/2024] [Indexed: 03/23/2024]
Abstract
Aneurysmal subarachnoid haemorrhage (aSAH) can lead to complications such as acute hydrocephalic congestion. Treatment of this acute condition often includes establishing an external ventricular drainage (EVD). However, chronic hydrocephalus develops in some patients, who then require placement of a permanent ventriculoperitoneal (VP) shunt. The aim of this study was to employ recurrent neural network (RNN)-based machine learning techniques to identify patients who require VP shunt placement at an early stage. This retrospective single-centre study included all patients who were diagnosed with aSAH and treated in the intensive care unit (ICU) between November 2010 and May 2020 (n = 602). More than 120 parameters were analysed, including routine neurocritical care data, vital signs and blood gas analyses. Various machine learning techniques, including RNNs and gradient boosting machines, were evaluated for their ability to predict VP shunt dependency. VP-shunt dependency could be predicted using an RNN after just one day of ICU stay, with an AUC-ROC of 0.77 (CI: 0.75-0.79). The accuracy of the prediction improved after four days of observation (Day 4: AUC-ROC 0.81, CI: 0.79-0.84). At that point, the accuracy of the prediction was 76% (CI: 75.98-83.09%), with a sensitivity of 85% (CI: 83-88%) and a specificity of 74% (CI: 71-78%). RNN-based machine learning has the potential to predict VP shunt dependency on Day 4 after ictus in aSAH patients using routine data collected in the ICU. The use of machine learning may allow early identification of patients with specific therapeutic needs and accelerate the execution of required procedures.
Collapse
Affiliation(s)
- Nils Schweingruber
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Jan Bremer
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Anton Wiehe
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
- Department of Informatics, University of Hamburg, 22527, Hamburg, Germany
| | - Marius Marc-Daniel Mader
- Department of Neurosurgery, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
- Institute for Stem Cell Biology and Regenerative Medicine, Stanford University School of Medicine, Stanford, CA, 94305, USA
| | - Christina Mayer
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
- Institute of Neuroimmunology and Multiple Sclerosis (INIMS), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Marcel Seungsu Woo
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
- Institute of Neuroimmunology and Multiple Sclerosis (INIMS), Center for Molecular Neurobiology Hamburg (ZMNH), University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Stefan Kluge
- Department of Intensive Care Medicine, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Jörn Grensemann
- Department of Intensive Care Medicine, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Fanny Quandt
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Jens Gempt
- Department of Neurosurgery, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
| | - Marlene Fischer
- Department of Intensive Care Medicine, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Götz Thomalla
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Christian Gerloff
- Department of Neurology, University Medical Center Hamburg-Eppendorf, 20246, Hamburg, Germany
| | - Jennifer Sauvigny
- Department of Neurosurgery, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany
| | - Patrick Czorlich
- Department of Neurosurgery, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246, Hamburg, Germany.
| |
Collapse
|
10
|
Li B, Warren BE, Eisenberg N, Beaton D, Lee DS, Aljabri B, Verma R, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Machine Learning to Predict Outcomes of Endovascular Intervention for Patients With PAD. JAMA Netw Open 2024; 7:e242350. [PMID: 38483388 PMCID: PMC10940965 DOI: 10.1001/jamanetworkopen.2024.2350] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 01/19/2024] [Indexed: 03/17/2024] Open
Abstract
Importance Endovascular intervention for peripheral artery disease (PAD) carries nonnegligible perioperative risks; however, outcome prediction tools are limited. Objective To develop machine learning (ML) algorithms that can predict outcomes following endovascular intervention for PAD. Design, Setting, and Participants This prognostic study included patients who underwent endovascular intervention for PAD between January 1, 2004, and July 5, 2023, with 1 year of follow-up. Data were obtained from the Vascular Quality Initiative (VQI), a multicenter registry containing data from vascular surgeons and interventionalists at more than 1000 academic and community hospitals. From an initial cohort of 262 242 patients, 26 565 were excluded due to treatment for acute limb ischemia (n = 14 642) or aneurysmal disease (n = 3456), unreported symptom status (n = 4401) or procedure type (n = 2319), or concurrent bypass (n = 1747). Data were split into training (70%) and test (30%) sets. Exposures A total of 112 predictive features (75 preoperative [demographic and clinical], 24 intraoperative [procedural], and 13 postoperative [in-hospital course and complications]) from the index hospitalization were identified. Main Outcomes and Measures Using 10-fold cross-validation, 6 ML models were trained using preoperative features to predict 1-year major adverse limb event (MALE; composite of thrombectomy or thrombolysis, surgical reintervention, or major amputation) or death. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intraoperative and postoperative data. Results Overall, 235 677 patients who underwent endovascular intervention for PAD were included (mean [SD] age, 68.4 [11.1] years; 94 979 [40.3%] female) and 71 683 (30.4%) developed 1-year MALE or death. The best preoperative prediction model was extreme gradient boosting (XGBoost), achieving the following performance metrics: AUROC, 0.94 (95% CI, 0.93-0.95); accuracy, 0.86 (95% CI, 0.85-0.87); sensitivity, 0.87; specificity, 0.85; positive predictive value, 0.85; and negative predictive value, 0.87. In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). The XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs of 0.94 (95% CI, 0.93-0.95) and 0.98 (95% CI, 0.97-0.99), respectively. Conclusions and Relevance In this prognostic study, ML models were developed that accurately predicted outcomes following endovascular intervention for PAD, which performed better than logistic regression. These algorithms have potential for important utility in guiding perioperative risk-mitigation strategies to prevent adverse outcomes following endovascular intervention for PAD.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
| | - Blair E. Warren
- Division of Vascular and Interventional Radiology, Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Ontario, Canada
| | - Douglas S. Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Duminda N. Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Department of Anesthesia, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Ori D. Rotstein
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Division of General Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular and Interventional Radiology, Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia
| |
Collapse
|
11
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al-Omran M. Predicting Outcomes Following Endovascular Abdominal Aortic Aneurysm Repair Using Machine Learning. Ann Surg 2024; 279:521-527. [PMID: 37389890 DOI: 10.1097/sla.0000000000005978] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/01/2023]
Abstract
OBJECTIVE To develop machine learning (ML) models that predict outcomes following endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA). BACKGROUND EVAR carries non-negligible perioperative risks; however, there are no widely used outcome prediction tools. METHODS The National Surgical Quality Improvement Program targeted database was used to identify patients who underwent EVAR for infrarenal AAA between 2011 and 2021. Input features included 36 preoperative variables. The primary outcome was 30-day major adverse cardiovascular event (composite of myocardial infarction, stroke, or death). Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 ML models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Subgroup analysis was performed to assess model performance based on age, sex, race, ethnicity, and prior AAA repair. RESULTS Overall, 16,282 patients were included. The primary outcome of 30-day major adverse cardiovascular event occurred in 390 (2.4%) patients. Our best-performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.95 (0.94-0.96) compared with logistic regression [0.72 [0.70-0.74)]. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.06. Model performance remained robust on all subgroup analyses. CONCLUSIONS Our newer ML models accurately predict 30-day outcomes following EVAR using preoperative data and perform better than logistic regression. Our automated algorithms can guide risk mitigation strategies for patients being considered for EVAR.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, MA
| | - Jamal J Hoballah
- Department of Surgery, Division of Vascular and Endovascular Surgery, American University of Beirut Medical Center, Beirut, Lebanon
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
- Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia
| |
Collapse
|
12
|
Li B, Eisenberg N, Beaton D, Lee DS, Aljabri B, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following suprainguinal bypass. J Vasc Surg 2024; 79:593-608.e8. [PMID: 37804954 DOI: 10.1016/j.jvs.2023.09.037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Revised: 09/20/2023] [Accepted: 09/24/2023] [Indexed: 10/09/2023]
Abstract
OBJECTIVE Suprainguinal bypass for peripheral artery disease (PAD) carries important surgical risks; however, outcome prediction tools remain limited. We developed machine learning (ML) algorithms that predict outcomes following suprainguinal bypass. METHODS The Vascular Quality Initiative database was used to identify patients who underwent suprainguinal bypass for PAD between 2003 and 2023. We identified 100 potential predictor variables from the index hospitalization (68 preoperative [demographic/clinical], 13 intraoperative [procedural], and 19 postoperative [in-hospital course/complications]). The primary outcomes were major adverse limb events (MALE; composite of untreated loss of patency, thrombectomy/thrombolysis, surgical revision, or major amputation) or death at 1 year following suprainguinal bypass. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The best performing algorithm was further trained using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, symptom status, procedure type, prior intervention for PAD, concurrent interventions, and urgency. RESULTS Overall, 16,832 patients underwent suprainguinal bypass, and 3136 (18.6%) developed 1-year MALE or death. Patients with 1-year MALE or death were older (mean age, 64.9 vs 63.5 years; P < .001) with more comorbidities, had poorer functional status (65.7% vs 80.9% independent at baseline; P < .001), and were more likely to have chronic limb-threatening ischemia (67.4% vs 47.6%; P < .001) than those without an outcome. Despite being at higher cardiovascular risk, they were less likely to receive acetylsalicylic acid or statins preoperatively and at discharge. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.92 (95% confidence interval [CI], 0.91-0.93). In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). Our XGBoost model maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.93 (95% CI, 0.92-0.94) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (preoperative), 0.11 (intraoperative), and 0.10 (postoperative). Of the top 10 predictors, nine were preoperative features including chronic limb-threatening ischemia, previous procedures, comorbidities, and functional status. Model performance remained robust on all subgroup analyses. CONCLUSIONS We developed ML models that accurately predict outcomes following suprainguinal bypass, performing better than logistic regression. Our algorithms have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes following suprainguinal bypass.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
13
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al-Omran M. Predicting outcomes following lower extremity open revascularization using machine learning. Sci Rep 2024; 14:2899. [PMID: 38316811 PMCID: PMC10844206 DOI: 10.1038/s41598-024-52944-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 01/25/2024] [Indexed: 02/07/2024] Open
Abstract
Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Institute of Medical Science, University of Toronto, Toronto, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, USA
| | - Jamal J Hoballah
- Division of Vascular and Endovascular Surgery, Department of Surgery, American University of Beirut Medical Center, Beirut, Lebanon
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Canada.
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada.
- Institute of Medical Science, University of Toronto, Toronto, Canada.
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada.
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia.
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
14
|
Wu J, Duan C, Han C, Hou X. Identification of CXC Chemokine Receptor 2 (CXCR2) as a Novel Eosinophils-Independent Diagnostic Biomarker of Pediatric Eosinophilic Esophagitis by Integrated Bioinformatic and Machine-Learning Analysis. Immunotargets Ther 2024; 13:55-74. [PMID: 38328342 PMCID: PMC10849108 DOI: 10.2147/itt.s439289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 01/17/2024] [Indexed: 02/09/2024] Open
Abstract
Background Eosinophilic esophagitis (EoE) is a complex allergic condition frequently accompanied by various atopic comorbidities in children, which significantly affects their life qualities. Therefore, this study aimed to evaluate pivotal molecular markers that may facilitate the diagnosis of EoE in pediatric patients. Methods Three available EoE-associated gene expression datasets in children: GSE184182, GSE 197702, GSE55794, along with GSE173895 were downloaded from the GEO database. Differentially expressed genes (DEGs) identified by "limma" were intersected with key module genes identified by weighted gene co-expression network analysis (WGCNA), and the shared genes went through functional enrichment analysis. The protein-protein interaction (PPI) network and the machine learning algorithms: least absolute shrinkage and selection operator (LASSO), random forest (RF), and XGBoost were used to reveal candidate diagnostic markers for EoE. The receiver operating characteristic (ROC) curve showed the efficacy of differential diagnosis of this marker, along with online databases predicting its molecular regulatory network. Finally, we performed gene set enrichment analysis (GSEA) and assessed immune cell infiltration of EoE/control samples by using the CIBERSORT algorithm. The correlations between the key diagnostic biomarker and immune cells were also investigated. Results The intersection of 936 DEGs and 1446 key module genes in EoE generated 567 genes, which were primarily enriched in immune regulation. Following the construction of the PPI network and filtration by machine learning, CXCR2 served as a potential diagnostic biomarker of pediatric EoE with a perfect diagnostic efficacy (AUC = ~1.00) in regional tissue/peripheral whole blood samples. Multiple infiltrated immune cells were observed to participate in disrupting the homeostasis of esophageal epithelium to varying degrees. Conclusion The immune-correlated CXCR2 gene was proved to be a promising diagnostic indicator for EoE, and dysregulated regulatory T cells (Tregs)/neutrophils might play a crucial role in the pathogenesis of EoE in children.
Collapse
Affiliation(s)
- Junhao Wu
- Division of Gastroenterology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, People’s Republic of China
| | - Caihan Duan
- Division of Gastroenterology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, People’s Republic of China
| | - Chaoqun Han
- Division of Gastroenterology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, People’s Republic of China
| | - Xiaohua Hou
- Division of Gastroenterology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, People’s Republic of China
| |
Collapse
|
15
|
Li B, Aljabri B, Verma R, Beaton D, Eisenberg N, Lee DS, Wijeysundera DN, Forbes TL, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following open abdominal aortic aneurysm repair. J Vasc Surg 2023; 78:1426-1438.e6. [PMID: 37634621 DOI: 10.1016/j.jvs.2023.08.121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 08/15/2023] [Accepted: 08/19/2023] [Indexed: 08/29/2023]
Abstract
OBJECTIVE Prediction of outcomes following open abdominal aortic aneurysm (AAA) repair remains challenging with a lack of widely used tools to guide perioperative management. We developed machine learning (ML) algorithms that predict outcomes following open AAA repair. METHODS The Vascular Quality Initiative (VQI) database was used to identify patients who underwent elective open AAA repair between 2003 and 2023. Input features included 52 preoperative demographic/clinical variables. All available preoperative variables from VQI were used to maximize predictive performance. The primary outcome was in-hospital major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death). Secondary outcomes were individual components of the primary outcome, other in-hospital complications, and 1-year mortality and any reintervention. We split our data into training (70%) and test (30%) sets. Using 10-fold cross-validation, six ML models were trained using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. The top 10 predictive features in our final model were determined based on variable importance scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median area deprivation index, proximal clamp site, prior aortic surgery, and concomitant procedures. RESULTS Overall, 12,027 patients were included. The primary outcome of in-hospital MACE occurred in 630 patients (5.2%). Compared with patients without a primary outcome, those who developed in-hospital MACE were older with more comorbidities, demonstrated poorer functional status, had more complex aneurysms, and were more likely to require concomitant procedures. Our best performing prediction model for in-hospital MACE was XGBoost, achieving an AUROC of 0.93 (95% confidence interval, 0.92-0.94). Comparatively, logistic regression had an AUROC of 0.71 (95% confidence interval, 0.70-0.73). For secondary outcomes, XGBoost achieved AUROCs between 0.84 and 0.94. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.05. These findings highlight the excellent predictive performance of the XGBoost model. The top three predictive features in our algorithm for in-hospital MACE following open AAA repair were: (1) coronary artery disease; (2) American Society of Anesthesiologists classification; and (3) proximal clamp site. Model performance remained robust on all subgroup analyses. CONCLUSIONS Open AAA repair outcomes can be accurately predicted using preoperative data with our ML models, which perform better than logistic regression. Our automated algorithms can help guide risk-mitigation strategies for patients being considered for open AAA repair to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Thomas L Forbes
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
16
|
Zhu G, Yuan A, Yu D, Zha A, Wu H. Machine learning to predict mortality for aneurysmal subarachnoid hemorrhage (aSAH) using a large nationwide EHR database. PLOS DIGITAL HEALTH 2023; 2:e0000400. [PMID: 38055677 DOI: 10.1371/journal.pdig.0000400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 10/29/2023] [Indexed: 12/08/2023]
Abstract
Aneurysmal subarachnoid hemorrhage (aSAH) develops quickly once it occurs and threatens the life of patients. We aimed to use machine learning to predict mortality for SAH patients at an early stage which can help doctors make clinical decisions. In our study, we applied different machine learning methods to an aSAH cohort extracted from a national EHR database, the Cerner Health Facts EHR database (2000-2018). The outcome of interest was in-hospital mortality, as either passing away while still in the hospital or being discharged to hospice care. Machine learning-based models were primarily evaluated by the area under the receiver operating characteristic curve (AUC). The population size of the SAH cohort was 6728. The machine learning methods achieved an average of AUCs of 0.805 for predicting mortality with only the initial 24 hours' EHR data. Without losing the prediction power, we used the logistic regression to identify 42 risk factors, -examples include age and serum glucose-that exhibit a significant correlation with the mortality of aSAH patients. Our study illustrates the potential of utilizing machine learning techniques as a practical prognostic tool for predicting aSAH mortality at the bedside.
Collapse
Affiliation(s)
- Gen Zhu
- Global Health & Analytics, Development, Novartis Pharmaceuticals, East Hanover, New Jersey, United States of America
| | - Anthony Yuan
- Department of Internal Medicine, The University of Texas Southwestern, Texas, United States of America
| | - Duo Yu
- Division of Biostatistics, Institute for Health & Equity, Medical College of Wisconsin, Milwaukee, Wisconsin, United States of America
| | - Alicia Zha
- Department of Neurology, The Ohio State University Wexner Medical Center, Columbus, Ohio, United States of America
| | - Hulin Wu
- Department of Biostatistics and Data Science, The University of Texas Health Science Center at Houston, Houston, Texas, United States of America
| |
Collapse
|
17
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al-Omran M. Predicting outcomes following open revascularization for aortoiliac occlusive disease using machine learning. J Vasc Surg 2023; 78:1449-1460.e7. [PMID: 37454952 DOI: 10.1016/j.jvs.2023.07.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 06/12/2023] [Accepted: 07/08/2023] [Indexed: 07/18/2023]
Abstract
OBJECTIVE Open surgical treatment options for aortoiliac occlusive disease carry significant perioperative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following open aortoiliac revascularization. METHODS The National Surgical Quality Improvement Program (NSQIP) targeted vascular database was used to identify patients who underwent open aortoiliac revascularization for atherosclerotic disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. The 30-day secondary outcomes were individual components of the primary outcome, major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death), individual components of MACE, wound complication, bleeding, other morbidity, non-home discharge, and unplanned readmission. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. Variable importance scores were calculated to determine the top 10 predictive features. Performance was assessed on subgroups based on age, sex, race, ethnicity, symptom status, procedure type, and urgency. RESULTS Overall, 9649 patients were included. The primary outcome of 30-day MALE or death occurred in 1021 patients (10.6%). Our best performing prediction model for 30-day MALE or death was XGBoost, achieving an AUROC of 0.95 (95% confidence interval [CI], 0.94-0.96). In comparison, logistic regression had an AUROC of 0.79 (95% CI, 0.77-0.81). For 30-day secondary outcomes, XGBoost achieved AUROCs between 0.87 and 0.97 (untreated loss of patency [0.95], major reintervention [0.88], major amputation [0.96], death [0.97], MACE [0.95], myocardial infarction [0.88], stroke [0.93], wound complication [0.94], bleeding [0.87], other morbidity [0.96], non-home discharge [0.90], and unplanned readmission [0.91]). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.05. The strongest predictive feature in our algorithm was chronic limb-threatening ischemia. Model performance remained robust on all subgroup analyses of specific demographic/clinical populations. CONCLUSIONS Our ML models accurately predict 30-day outcomes following open aortoiliac revascularization using preoperative data, performing better than logistic regression. They have potential for important utility in guiding risk-mitigation strategies for patients being considered for open aortoiliac revascularization to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Department of Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon; College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, MA
| | - Jamal J Hoballah
- Division of Vascular and Endovascular Surgery, Department of Surgery, American University of Beirut Medical Center, Beirut, Lebanon
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Department of Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
18
|
Wang H, Bothe TL, Deng C, Lv S, Khedkar PH, Kovacs R, Patzak A, Wu L. Comparison of Prognostic Models for Functional Outcome in Aneurysmal Subarachnoid Hemorrhage Based on Machine Learning. World Neurosurg 2023; 180:e686-e699. [PMID: 37821029 DOI: 10.1016/j.wneu.2023.10.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 10/03/2023] [Indexed: 10/13/2023]
Abstract
BACKGROUND Controversy exists regarding the superiority of the performance of prognostic tools based on advanced machine learning (ML) algorithms for patients with aneurysmal subarachnoid hemorrhage (aSAH). However, it is unclear whether ML prognostic models will benefit patients due to the lack of a comprehensive assessment. We aimed to develop and evaluate ML models for predicting unfavorable functional outcomes for aSAH patients and identify the model with the greatest performance. METHODS In this retrospective study, a dataset of 955 patients with aSAH was used to construct and validate prognostic models for functional outcomes assessed using the modified Rankin scale during a follow-up period of 3-6 months. Clinical scores and clinical and radiological features on admission and secondary complications were used to construct models based on 5 ML algorithms (i.e., logistic regression [LR], k-nearest neighbor, extreme gradient boosting, random forest, and artificial neural network). For evaluation among the models, the area under the receiver operating characteristic curve, area under the precision-recall curve, calibration curve, and decision curve analysis were used. RESULTS Composite models had significantly higher area under the receiver operating characteristic curves than did simple models in predicting unfavorable functional outcomes. Compared with other composite models (random forest and extreme gradient boosting) with good calibration, LR had the highest area under the precision-recall score and showed the greatest benefit in decision curve analysis. CONCLUSIONS Of the 5 studied ML models, the conventional LR model outperformed the advanced algorithms in predicting the prognosis and could be a useful tool for health care professionals.
Collapse
Affiliation(s)
- Han Wang
- Institute of Translational Physiology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu, Berlin, Germany; Department of Neurosurgery, Nanjing Drum Tower Hospital, The Affiliated Hospital of Nanjing University Medical School, Nanjing, China
| | - Tomas L Bothe
- Institute of Translational Physiology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu, Berlin, Germany
| | - Chulei Deng
- Department of Neurosurgery, Jinling Hospital, Nanjing, China
| | - Shengyin Lv
- Department of Neurology, Second Hospital of Nanjing, Nanjing University of Chinese Medicine, Nanjing, China
| | - Pratik H Khedkar
- Institute of Translational Physiology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu, Berlin, Germany
| | - Richard Kovacs
- Institute of Neurophysiology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu, Berlin, Germany
| | - Andreas Patzak
- Institute of Translational Physiology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu, Berlin, Germany
| | - Lingyun Wu
- Department of Neurosurgery, Nanjing Drum Tower Hospital, The Affiliated Hospital of Nanjing University Medical School, Nanjing, China.
| |
Collapse
|
19
|
Zhang Y, Zeng H, Zhou H, Li J, Wang T, Guo Y, Cai L, Hu J, Zhang X, Chen G. Predicting the Outcome of Patients with Aneurysmal Subarachnoid Hemorrhage: A Machine-Learning-Guided Scorecard. J Clin Med 2023; 12:7040. [PMID: 38002653 PMCID: PMC10671848 DOI: 10.3390/jcm12227040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Revised: 11/05/2023] [Accepted: 11/08/2023] [Indexed: 11/26/2023] Open
Abstract
Aneurysmal subarachnoid hemorrhage (aSAH) frequently causes long-term disability, but predicting outcomes remains challenging. Routine parameters such as demographics, admission status, CT findings, and blood tests can be used to predict aSAH outcomes. The aim of this study was to compare the performance of traditional logistic regression with several machine learning algorithms using readily available indicators and to generate a practical prognostic scorecard based on machine learning. Eighteen routinely available indicators were collected as outcome predictors for individuals with aSAH. Logistic regression (LR), random forest (RF), support vector machines (SVMs), and fully connected neural networks (FCNNs) were compared. A scorecard system was established based on predictor weights. The results show that machine learning models and a scorecard achieved 0.75~0.8 area under the curve (AUC) predicting aSAH outcomes (LR 0.739, RF 0.749, SVM 0.762~0.793, scorecard 0.794). FCNNs performed best (~0.95) but lacked interpretability. The scorecard model used only five factors, generating a clinically useful tool with a total cutoff score of ≥5, indicating poor prognosis. We developed and validated machine learning models proven to predict outcomes more accurately in individuals with aSAH. The parameters found to be the most strongly predictive of outcomes were NLR, lymphocyte count, monocyte count, hypertension status, and SEBES. The scorecard system provides a simplified means of applying predictive analytics at the bedside using a few key indicators.
Collapse
Affiliation(s)
- Yi Zhang
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Hanhai Zeng
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Hang Zhou
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Jingbo Li
- Department of Neurointensive Care Unit, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
| | - Tingting Wang
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Yinghan Guo
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Lingxin Cai
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Junwen Hu
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| | - Xiaotong Zhang
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- College of Electrical Engineering, Zhejiang University, Hangzhou 310020, China
- Interdisciplinary Institute of Neuroscience and Technology, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou 310020, China
- MOE Frontier Science Center for Brain Science and Brain-Machine Integration, Zhejiang University, Hangzhou 310058, China
| | - Gao Chen
- Department of Neurosurgery, Second Affiliated Hospital of Zhejiang University School of Medicine, Zhejiang University, Hangzhou 310016, China
- Key Laboratory of Precise Treatment and Clinical Translational Research of Neurological Diseases, Hangzhou 310016, China
| |
Collapse
|
20
|
Li B, Aljabri B, Verma R, Beaton D, Eisenberg N, Lee DS, Wijeysundera DN, Forbes TL, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Machine learning to predict outcomes following endovascular abdominal aortic aneurysm repair. Br J Surg 2023; 110:1840-1849. [PMID: 37710397 DOI: 10.1093/bjs/znad287] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 07/27/2023] [Accepted: 08/27/2023] [Indexed: 09/16/2023]
Abstract
BACKGROUND Endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA) carries important perioperative risks; however, there are no widely used outcome prediction tools. The aim of this study was to apply machine learning (ML) to develop automated algorithms that predict 1-year mortality following EVAR. METHODS The Vascular Quality Initiative database was used to identify patients who underwent elective EVAR for infrarenal AAA between 2003 and 2023. Input features included 47 preoperative demographic/clinical variables. The primary outcome was 1-year all-cause mortality. Data were split into training (70 per cent) and test (30 per cent) sets. Using 10-fold cross-validation, 6 ML models were trained using preoperative features with logistic regression as the baseline comparator. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. RESULTS Some 63 655 patients were included. One-year mortality occurred in 3122 (4.9 per cent) patients. The best performing prediction model for 1-year mortality was XGBoost, achieving an AUROC (95 per cent c.i.) of 0.96 (0.95-0.97). Comparatively, logistic regression had an AUROC (95 per cent c.i.) of 0.69 (0.68-0.71). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.04. The top 3 predictive features in the algorithm were 1) unfit for open AAA repair, 2) functional status, and 3) preoperative dialysis. CONCLUSIONS In this data set, machine learning was able to predict 1-year mortality following EVAR using preoperative data and outperformed standard logistic regression models.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Ontario, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Thomas L Forbes
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia
| |
Collapse
|
21
|
Tian CW, Chen XX, Shi L, Zhu HY, Dai GC, Chen H, Rui YF. Machine learning applications for the prediction of extended length of stay in geriatric hip fracture patients. World J Orthop 2023; 14:741-754. [PMID: 37970626 PMCID: PMC10642403 DOI: 10.5312/wjo.v14.i10.741] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 09/08/2023] [Accepted: 09/28/2023] [Indexed: 10/16/2023] Open
Abstract
BACKGROUND Geriatric hip fractures are one of the most common fractures in elderly individuals, and prolonged hospital stays increase the risk of death and complications. Machine learning (ML) has become prevalent in clinical data processing and predictive models. This study aims to develop ML models for predicting extended length of stay (eLOS) among geriatric patients with hip fractures and to identify the associated risk factors. AIM To develop ML models for predicting the eLOS among geriatric patients with hip fractures, identify associated risk factors, and compare the performance of each model. METHODS A retrospective study was conducted at a single orthopaedic trauma centre, enrolling all patients who underwent hip fracture surgery between January 2018 and December 2022. The study collected various patient characteristics, encompassing demographic data, general health status, injury-related data, laboratory examinations, surgery-related data, and length of stay. Features that exhibited significant differences in univariate analysis were integrated into the ML model establishment and subsequently cross-verified. The study compared the performance of the ML models and determined the risk factors for eLOS. RESULTS The study included 763 patients, with 380 experiencing eLOS. Among the models, the decision tree, random forest, and extreme Gradient Boosting models demonstrated the most robust performance. Notably, the artificial neural network model also exhibited impressive results. After cross-validation, the support vector machine and logistic regression models demonstrated superior performance. Predictors for eLOS included delayed surgery, D-dimer level, American Society of Anaesthesiologists (ASA) classification, type of surgery, and sex. CONCLUSION ML proved to be highly accurate in predicting the eLOS for geriatric patients with hip fractures. The identified key risk factors were delayed surgery, D-dimer level, ASA classification, type of surgery, and sex. This valuable information can aid clinicians in allocating resources more efficiently to meet patient demand effectively.
Collapse
Affiliation(s)
- Chu-Wei Tian
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| | - Xiang-Xu Chen
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| | - Liu Shi
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| | - Huan-Yi Zhu
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| | - Guang-Chun Dai
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| | - Hui Chen
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| | - Yun-Feng Rui
- Department of Orthopaedics, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Multidisciplinary Team for Geriatric Hip Fracture Comprehensive Management, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
- Trauma Center, Zhongda Hospital, School of Medicine, Southeast University, Nanjing 210009, Jiangsu Province, China
| |
Collapse
|
22
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al‐Omran M. Predicting Major Adverse Cardiovascular Events Following Carotid Endarterectomy Using Machine Learning. J Am Heart Assoc 2023; 12:e030508. [PMID: 37804197 PMCID: PMC10757546 DOI: 10.1161/jaha.123.030508] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Accepted: 08/28/2023] [Indexed: 10/09/2023]
Abstract
Background Carotid endarterectomy (CEA) is a major vascular operation for stroke prevention that carries significant perioperative risks; however, outcome prediction tools remain limited. The authors developed machine learning algorithms to predict outcomes following CEA. Methods and Results The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent CEA between 2011 and 2021. Input features included 36 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse cardiovascular events (composite of stroke, myocardial infarction, or death). The data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary metric for evaluating model performance was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Overall, 38 853 patients underwent CEA during the study period. Thirty-day major adverse cardiovascular events occurred in 1683 (4.3%) patients. The best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve of 0.91 (95% CI, 0.90-0.92). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.62 (95% CI, 0.60-0.64), and existing tools in the literature demonstrate area under the receiver operating characteristic curve values ranging from 0.58 to 0.74. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.02. The strongest predictive feature in our algorithm was carotid symptom status. Conclusions The machine learning models accurately predicted 30-day outcomes following CEA using preoperative data and performed better than existing tools. They have potential for important utility in guiding risk-mitigation strategies to improve outcomes for patients being considered for CEA.
Collapse
Affiliation(s)
- Ben Li
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health TorontoUniversity of TorontoCanada
- Institute of Medical ScienceUniversity of TorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoCanada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in IrelandUniversity of Medicine and Health SciencesDublinIreland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoCanada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research InstituteAmerican University of Beirut Medical CenterBeirutLebanon
- College of MedicineAlfaisal UniversityRiyadhKingdom of Saudi Arabia
| | - Mohamad A. Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women’s HospitalHarvard Medical SchoolBostonMAUSA
| | - Jamal J. Hoballah
- Division of Vascular and Endovascular Surgery, Department of SurgeryAmerican University of Beirut Medical CenterBeirutLebanon
| | - Douglas S. Lee
- Division of Cardiology, Peter Munk Cardiac CentreUniversity Health NetworkTorontoCanada
- Institute of Health Policy, Management and EvaluationUniversity of TorontoCanada
- ICESUniversity of TorontoCanada
| | - Duminda N. Wijeysundera
- Institute of Health Policy, Management and EvaluationUniversity of TorontoCanada
- ICESUniversity of TorontoCanada
- Department of AnesthesiaSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Li Ka Shing Knowledge InstituteSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Charles de Mestral
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health TorontoUniversity of TorontoCanada
- Institute of Health Policy, Management and EvaluationUniversity of TorontoCanada
- ICESUniversity of TorontoCanada
- Li Ka Shing Knowledge InstituteSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Muhammad Mamdani
- Institute of Medical ScienceUniversity of TorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoCanada
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoCanada
- Institute of Health Policy, Management and EvaluationUniversity of TorontoCanada
- ICESUniversity of TorontoCanada
- Li Ka Shing Knowledge InstituteSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Leslie Dan Faculty of PharmacyUniversity of TorontoCanada
| | - Mohammed Al‐Omran
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health TorontoUniversity of TorontoCanada
- Institute of Medical ScienceUniversity of TorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoCanada
- College of MedicineAlfaisal UniversityRiyadhKingdom of Saudi Arabia
- Li Ka Shing Knowledge InstituteSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Department of SurgeryKing Faisal Specialist Hospital and Research CenterRiyadhKingdom of Saudi Arabia
| |
Collapse
|
23
|
Li B, Beaton D, Eisenberg N, Lee DS, Wijeysundera DN, Lindsay TF, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following carotid endarterectomy. J Vasc Surg 2023; 78:973-987.e6. [PMID: 37211142 DOI: 10.1016/j.jvs.2023.05.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 05/08/2023] [Accepted: 05/13/2023] [Indexed: 05/23/2023]
Abstract
OBJECTIVE Prediction of outcomes following carotid endarterectomy (CEA) remains challenging, with a lack of standardized tools to guide perioperative management. We used machine learning (ML) to develop automated algorithms that predict outcomes following CEA. METHODS The Vascular Quality Initiative (VQI) database was used to identify patients who underwent CEA between 2003 and 2022. We identified 71 potential predictor variables (features) from the index hospitalization (43 preoperative [demographic/clinical], 21 intraoperative [procedural], and 7 postoperative [in-hospital complications]). The primary outcome was stroke or death at 1 year following CEA. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, insurance status, symptom status, and urgency of surgery. RESULTS Overall, 166,369 patients underwent CEA during the study period. In total, 7749 patients (4.7%) had the primary outcome of stroke or death at 1 year. Patients with an outcome were older with more comorbidities, had poorer functional status, and demonstrated higher risk anatomic features. They were also more likely to undergo intraoperative surgical re-exploration and have in-hospital complications. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.90 (95% confidence interval [CI], 0.89-0.91). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. Our XGBoost models maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.90 (95% CI, 0.89-0.91) and 0.94 (95% CI, 0.93-0.95), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.15 (preoperative), 0.14 (intraoperative), and 0.11 (postoperative). Of the top 10 predictors, eight were preoperative features, including comorbidities, functional status, and previous procedures. Model performance remained robust on all subgroup analyses. CONCLUSIONS We developed ML models that accurately predict outcomes following CEA. Our algorithms perform better than logistic regression and existing tools, and therefore, have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Derek Beaton
- Data Science and Advanced Analytics Department, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Thomas F Lindsay
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science and Advanced Analytics Department, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
24
|
Shooshani T, Pooladzandi O, Nguyen A, Shipley JH, Harris MH, Hovis GEA, Barrios C. Field Measures Are All You Need: Predicting Need for Surgery in Elderly Ground-Level Fall Patients via Machine Learning. Am Surg 2023; 89:4095-4100. [PMID: 37218170 DOI: 10.1177/00031348231177917] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
BACKGROUND As ground-level falls (GLFs) are a significant cause of mortality in elderly patients, field triage plays an essential role in patient outcomes. This research investigates how machine learning algorithms can supplement traditional t-tests to recognize statistically significant patterns in medical data and to aid clinical guidelines. METHODS This is a retrospective study using data from 715 GLF patients over 75 years old. We first calculated P-values for each recorded factor to determine the factor's significance in contributing to a need for surgery (P < .05 is significant). We then utilized the XGBoost machine learning method to rank contributing factors. We applied SHapley Additive exPlanations (SHAP) values to interpret the feature importance and provide clinical guidance via decision trees. RESULTS The three most significant P-values when comparing patients with and without surgery are as follows: Glasgow Coma Scale (GCS) (P < .001), no comorbidities (P < .001), and transfer-in (P = .019). The XGBoost algorithm determined that GCS and systolic blood pressure contribute most strongly. The prediction accuracy of these XGBoost results based on the test/train split was 90.3%. DISCUSSION When compared to P-values, XGBoost provides more robust, detailed results regarding the factors that suggest a need for surgery. This demonstrates the clinical applicability of machine learning algorithms. Paramedics can use resulting decision trees to inform medical decision-making in real time. XGBoost's generalizability power increases with more data and can be tuned to prospectively assist individual hospitals.
Collapse
Affiliation(s)
- Tara Shooshani
- University of California, Irvine School of Medicine, Irvine, CA, USA
| | | | - Andrew Nguyen
- University of California, Los Angeles, Los Angeles, CA, USA
| | | | - Mark H Harris
- University of California, Irvine School of Medicine, Irvine, CA, USA
| | | | - Cristobal Barrios
- University of California, Irvine School of Medicine, Irvine, CA, USA
| |
Collapse
|
25
|
Yang Z, Cui X, Song Z. Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis. BMC Infect Dis 2023; 23:635. [PMID: 37759175 PMCID: PMC10523763 DOI: 10.1186/s12879-023-08614-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2023] [Accepted: 09/15/2023] [Indexed: 09/29/2023] Open
Abstract
BACKGROUND Sepsis is a life-threatening condition caused by an abnormal response of the body to infection and imposes a significant health and economic burden worldwide due to its high mortality rate. Early recognition of sepsis is crucial for effective treatment. This study aimed to systematically evaluate the performance of various machine learning models in predicting the onset of sepsis. METHODS We conducted a comprehensive search of the Cochrane Library, PubMed, Embase, and Web of Science databases, covering studies from database inception to November 14, 2022. We used the PROBAST tool to assess the risk of bias. We calculated the predictive performance for sepsis onset using the C-index and accuracy. We followed the PRISMA guidelines for this study. RESULTS We included 23 eligible studies with a total of 4,314,145 patients and 26 different machine learning models. The most frequently used models in the studies were random forest (n = 9), extreme gradient boost (n = 7), and logistic regression (n = 6) models. The random forest (test set n = 9, acc = 0.911) and extreme gradient boost (test set n = 7, acc = 0.957) models were the most accurate based on our analysis of the predictive performance. In terms of the C-index outcome, the random forest (n = 6, acc = 0.79) and extreme gradient boost (n = 7, acc = 0.83) models showed the highest performance. CONCLUSION Machine learning has proven to be an effective tool for predicting sepsis at an early stage. However, to obtain more accurate results, additional machine learning methods are needed. In our research, we discovered that the XGBoost and random forest models exhibited the best predictive performance and were most frequently utilized for predicting the onset of sepsis. TRIAL REGISTRATION CRD42022384015.
Collapse
Affiliation(s)
- Zhenyu Yang
- Kunming Medical University, Kunming, Yunnan, China
| | - Xiaoju Cui
- Chengyang District People's Hospital, Qingdao, Shandong, China
| | - Zhe Song
- Qinghai University, Xining, Qinghai, China.
| |
Collapse
|
26
|
Saigal K, Patel AB, Lucke-Wold B. Artificial Intelligence and Neurosurgery: Tracking Antiplatelet Response Patterns for Endovascular Intervention. MEDICINA (KAUNAS, LITHUANIA) 2023; 59:1714. [PMID: 37893432 PMCID: PMC10608122 DOI: 10.3390/medicina59101714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 09/22/2023] [Accepted: 09/22/2023] [Indexed: 10/29/2023]
Abstract
Platelets play a critical role in blood clotting and the development of arterial blockages. Antiplatelet therapy is vital for preventing recurring events in conditions like coronary artery disease and strokes. However, there is a lack of comprehensive guidelines for using antiplatelet agents in elective neurosurgery. Continuing therapy during surgery poses a bleeding risk, while discontinuing it before surgery increases the risk of thrombosis. Discontinuation is recommended in neurosurgical settings but carries an elevated risk of ischemic events. Conversely, maintaining antithrombotic therapy may increase bleeding and the need for transfusions, leading to a poor prognosis. Artificial intelligence (AI) holds promise in making difficult decisions regarding antiplatelet therapy. This paper discusses current clinical guidelines and supported regimens for antiplatelet therapy in neurosurgery. It also explores methodologies like P2Y12 reaction units (PRU) monitoring and thromboelastography (TEG) mapping for monitoring the use of antiplatelet regimens as well as their limitations. The paper explores the potential of AI to overcome such limitations associated with PRU monitoring and TEG mapping. It highlights various studies in the field of cardiovascular and neuroendovascular surgery which use AI prediction models to forecast adverse outcomes such as ischemia and bleeding, offering assistance in decision-making for antiplatelet therapy. In addition, the use of AI to improve patient adherence to antiplatelet regimens is also considered. Overall, this research aims to provide insights into the use of antiplatelet therapy and the role of AI in optimizing treatment plans in neurosurgical settings.
Collapse
Affiliation(s)
- Khushi Saigal
- College of Medicine, University of Florida, Gainesville, FL 32610, USA
| | - Anmol Bharat Patel
- College of Medicine, University of Miami—Miller School of Medicine, Miami, FL 33136, USA;
| | - Brandon Lucke-Wold
- Department of Neurosurgery, University of Florida, Gainesville, FL 32608, USA
| |
Collapse
|
27
|
Trakulpanitkit A, Tunthanathip T. Comparison of intracranial pressure prediction in hydrocephalus patients among linear, non-linear, and machine learning regression models in Thailand. Acute Crit Care 2023; 38:362-370. [PMID: 37652865 PMCID: PMC10497900 DOI: 10.4266/acc.2023.00094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Revised: 04/23/2023] [Accepted: 06/20/2023] [Indexed: 09/02/2023] Open
Abstract
BACKGROUND Hydrocephalus (HCP) is one of the most significant concerns in neurosurgical patients because it can cause increased intracranial pressure (ICP), resulting in mortality and morbidity. To date, machine learning (ML) has been helpful in predicting continuous outcomes. The primary objective of the present study was to identify the factors correlated with ICP, while the secondary objective was to compare the predictive performances among linear, non-linear, and ML regression models for ICP prediction. METHODS A total of 412 patients with various types of HCP who had undergone ventriculostomy was retrospectively included in the present study, and intraoperative ICP was recorded following ventricular catheter insertion. Several clinical factors and imaging parameters were analyzed for the relationship with ICP by linear correlation. The predictive performance of ICP was compared among linear, non-linear, and ML regression models. RESULTS Optic nerve sheath diameter (ONSD) had a moderately positive correlation with ICP (r=0.530, P<0.001), while several ventricular indexes were not statistically significant in correlation with ICP. For prediction of ICP, random forest (RF) and extreme gradient boosting (XGBoost) algorithms had low mean absolute error and root mean square error values and high R2 values compared to linear and non-linear regression when the predictive model included ONSD and ventricular indexes. CONCLUSIONS The XGBoost and RF algorithms are advantageous for predicting preoperative ICP and establishing prognoses for HCP patients. Furthermore, ML-based prediction could be used as a non-invasive method.
Collapse
Affiliation(s)
- Avika Trakulpanitkit
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, Prince of Songkla University, Songkhla, Thailand
| | - Thara Tunthanathip
- Division of Neurosurgery, Department of Surgery, Faculty of Medicine, Prince of Songkla University, Songkhla, Thailand
| |
Collapse
|
28
|
Assessing Metabolic Markers in Glioblastoma Using Machine Learning: A Systematic Review. Metabolites 2023; 13:metabo13020161. [PMID: 36837779 PMCID: PMC9958885 DOI: 10.3390/metabo13020161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Revised: 01/14/2023] [Accepted: 01/18/2023] [Indexed: 01/24/2023] Open
Abstract
Glioblastoma (GBM) is a common and deadly brain tumor with late diagnoses and poor prognoses. Machine learning (ML) is an emerging tool that can create highly accurate diagnostic and prognostic prediction models. This paper aimed to systematically search the literature on ML for GBM metabolism and assess recent advancements. A literature search was performed using predetermined search terms. Articles describing the use of an ML algorithm for GBM metabolism were included. Ten studies met the inclusion criteria for analysis: diagnostic (n = 3, 30%), prognostic (n = 6, 60%), or both (n = 1, 10%). Most studies analyzed data from multiple databases, while 50% (n = 5) included additional original samples. At least 2536 data samples were run through an ML algorithm. Twenty-seven ML algorithms were recorded with a mean of 2.8 algorithms per study. Algorithms were supervised (n = 24, 89%), unsupervised (n = 3, 11%), continuous (n = 19, 70%), or categorical (n = 8, 30%). The mean reported accuracy and AUC of ROC were 95.63% and 0.779, respectively. One hundred six metabolic markers were identified, but only EMP3 was reported in multiple studies. Many studies have identified potential biomarkers for GBM diagnosis and prognostication. These algorithms show promise; however, a consensus on even a handful of biomarkers has not yet been made.
Collapse
|