1
|
Li B, Aljabri B, Verma R, Beaton D, Hussain MA, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al‐Omran M. Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning. J Am Heart Assoc 2024; 13:e033194. [PMID: 38639373 PMCID: PMC11179886 DOI: 10.1161/jaha.123.033194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 03/01/2024] [Indexed: 04/20/2024]
Abstract
BACKGROUND Lower extremity endovascular revascularization for peripheral artery disease carries nonnegligible perioperative risks; however, outcome prediction tools remain limited. Using machine learning, we developed automated algorithms that predict 30-day outcomes following lower extremity endovascular revascularization. METHODS AND RESULTS The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity endovascular revascularization (angioplasty, stent, or atherectomy) for peripheral artery disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day postprocedural major adverse limb event (composite of major reintervention, untreated loss of patency, or major amputation) or death. Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Overall, 21 886 patients were included, and 30-day major adverse limb event/death occurred in 1964 (9.0%) individuals. The best performing model for predicting 30-day major adverse limb event/death was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.94). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.72 (95% CI, 0.70-0.74). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.09. The top 3 predictive features in our algorithm were (1) chronic limb-threatening ischemia, (2) tibial intervention, and (3) congestive heart failure. CONCLUSIONS Our machine learning models accurately predict 30-day outcomes following lower extremity endovascular revascularization using preoperative data with good discrimination and calibration. Prospective validation is warranted to assess for generalizability and external validity.
Collapse
Affiliation(s)
- Ben Li
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
| | - Badr Aljabri
- Department of SurgeryKing Saud UniversityRiyadhSaudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in IrelandUniversity of Medicine and Health SciencesDublinIreland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoTorontoCanada
| | - Mohamad A. Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women’s HospitalHarvard Medical SchoolBostonMAUSA
| | - Douglas S. Lee
- Division of Cardiology, Peter Munk Cardiac CentreUniversity Health NetworkTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
| | - Duminda N. Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Department of AnesthesiaSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Charles de Mestral
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Muhammad Mamdani
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Leslie Dan Faculty of PharmacyUniversity of TorontoTorontoCanada
| | - Mohammed Al‐Omran
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Department of SurgeryKing Faisal Specialist Hospital and Research CenterRiyadhSaudi Arabia
| |
Collapse
|
2
|
Li B, Eisenberg N, Beaton D, Lee DS, Aljabri B, Verma R, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using Machine Learning (XGBoost) to Predict Outcomes After Infrainguinal Bypass for Peripheral Artery Disease. Ann Surg 2024; 279:705-713. [PMID: 38116648 DOI: 10.1097/sla.0000000000006181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2023]
Abstract
OBJECTIVE To develop machine learning (ML) algorithms that predict outcomes after infrainguinal bypass. BACKGROUND Infrainguinal bypass for peripheral artery disease carries significant surgical risks; however, outcome prediction tools remain limited. METHODS The Vascular Quality Initiative database was used to identify patients who underwent infrainguinal bypass for peripheral artery disease between 2003 and 2023. We identified 97 potential predictor variables from the index hospitalization [68 preoperative (demographic/clinical), 13 intraoperative (procedural), and 16 postoperative (in-hospital course/complications)]. The primary outcome was 1-year major adverse limb event (composite of surgical revision, thrombectomy/thrombolysis, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained 6 ML models using preoperative features. The primary model evaluation metric was the area under the receiver operating characteristic curve (AUROC). The top-performing algorithm was further trained using intraoperative and postoperative features. Model robustness was evaluated using calibration plots and Brier scores. RESULTS Overall, 59,784 patients underwent infrainguinal bypass, and 15,942 (26.7%) developed 1-year major adverse limb event/death. The best preoperative prediction model was XGBoost, achieving an AUROC (95% CI) of 0.94 (0.93-0.95). In comparison, logistic regression had an AUROC (95% CI) of 0.61 (0.59-0.63). Our XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs (95% CI's) of 0.94 (0.93-0.95) and 0.96 (0.95-0.97), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.08 (preoperative), 0.07 (intraoperative), and 0.05 (postoperative). CONCLUSIONS ML models can accurately predict outcomes after infrainguinal bypass, outperforming logistic regression.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- ICES, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Kingdom of Saudi Arabia
| |
Collapse
|
3
|
Li B, Warren BE, Eisenberg N, Beaton D, Lee DS, Aljabri B, Verma R, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Machine Learning to Predict Outcomes of Endovascular Intervention for Patients With PAD. JAMA Netw Open 2024; 7:e242350. [PMID: 38483388 PMCID: PMC10940965 DOI: 10.1001/jamanetworkopen.2024.2350] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 01/19/2024] [Indexed: 03/17/2024] Open
Abstract
Importance Endovascular intervention for peripheral artery disease (PAD) carries nonnegligible perioperative risks; however, outcome prediction tools are limited. Objective To develop machine learning (ML) algorithms that can predict outcomes following endovascular intervention for PAD. Design, Setting, and Participants This prognostic study included patients who underwent endovascular intervention for PAD between January 1, 2004, and July 5, 2023, with 1 year of follow-up. Data were obtained from the Vascular Quality Initiative (VQI), a multicenter registry containing data from vascular surgeons and interventionalists at more than 1000 academic and community hospitals. From an initial cohort of 262 242 patients, 26 565 were excluded due to treatment for acute limb ischemia (n = 14 642) or aneurysmal disease (n = 3456), unreported symptom status (n = 4401) or procedure type (n = 2319), or concurrent bypass (n = 1747). Data were split into training (70%) and test (30%) sets. Exposures A total of 112 predictive features (75 preoperative [demographic and clinical], 24 intraoperative [procedural], and 13 postoperative [in-hospital course and complications]) from the index hospitalization were identified. Main Outcomes and Measures Using 10-fold cross-validation, 6 ML models were trained using preoperative features to predict 1-year major adverse limb event (MALE; composite of thrombectomy or thrombolysis, surgical reintervention, or major amputation) or death. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intraoperative and postoperative data. Results Overall, 235 677 patients who underwent endovascular intervention for PAD were included (mean [SD] age, 68.4 [11.1] years; 94 979 [40.3%] female) and 71 683 (30.4%) developed 1-year MALE or death. The best preoperative prediction model was extreme gradient boosting (XGBoost), achieving the following performance metrics: AUROC, 0.94 (95% CI, 0.93-0.95); accuracy, 0.86 (95% CI, 0.85-0.87); sensitivity, 0.87; specificity, 0.85; positive predictive value, 0.85; and negative predictive value, 0.87. In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). The XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs of 0.94 (95% CI, 0.93-0.95) and 0.98 (95% CI, 0.97-0.99), respectively. Conclusions and Relevance In this prognostic study, ML models were developed that accurately predicted outcomes following endovascular intervention for PAD, which performed better than logistic regression. These algorithms have potential for important utility in guiding perioperative risk-mitigation strategies to prevent adverse outcomes following endovascular intervention for PAD.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
| | - Blair E. Warren
- Division of Vascular and Interventional Radiology, Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Ontario, Canada
| | - Douglas S. Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Duminda N. Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Department of Anesthesia, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Ori D. Rotstein
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Division of General Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- ICES, University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular and Interventional Radiology, Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Vascular Surgery, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Ontario, Canada
- Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia
| |
Collapse
|
4
|
Li B, Eisenberg N, Beaton D, Lee DS, Aljabri B, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following suprainguinal bypass. J Vasc Surg 2024; 79:593-608.e8. [PMID: 37804954 DOI: 10.1016/j.jvs.2023.09.037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Revised: 09/20/2023] [Accepted: 09/24/2023] [Indexed: 10/09/2023]
Abstract
OBJECTIVE Suprainguinal bypass for peripheral artery disease (PAD) carries important surgical risks; however, outcome prediction tools remain limited. We developed machine learning (ML) algorithms that predict outcomes following suprainguinal bypass. METHODS The Vascular Quality Initiative database was used to identify patients who underwent suprainguinal bypass for PAD between 2003 and 2023. We identified 100 potential predictor variables from the index hospitalization (68 preoperative [demographic/clinical], 13 intraoperative [procedural], and 19 postoperative [in-hospital course/complications]). The primary outcomes were major adverse limb events (MALE; composite of untreated loss of patency, thrombectomy/thrombolysis, surgical revision, or major amputation) or death at 1 year following suprainguinal bypass. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The best performing algorithm was further trained using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, symptom status, procedure type, prior intervention for PAD, concurrent interventions, and urgency. RESULTS Overall, 16,832 patients underwent suprainguinal bypass, and 3136 (18.6%) developed 1-year MALE or death. Patients with 1-year MALE or death were older (mean age, 64.9 vs 63.5 years; P < .001) with more comorbidities, had poorer functional status (65.7% vs 80.9% independent at baseline; P < .001), and were more likely to have chronic limb-threatening ischemia (67.4% vs 47.6%; P < .001) than those without an outcome. Despite being at higher cardiovascular risk, they were less likely to receive acetylsalicylic acid or statins preoperatively and at discharge. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.92 (95% confidence interval [CI], 0.91-0.93). In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). Our XGBoost model maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.93 (95% CI, 0.92-0.94) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (preoperative), 0.11 (intraoperative), and 0.10 (postoperative). Of the top 10 predictors, nine were preoperative features including chronic limb-threatening ischemia, previous procedures, comorbidities, and functional status. Model performance remained robust on all subgroup analyses. CONCLUSIONS We developed ML models that accurately predict outcomes following suprainguinal bypass, performing better than logistic regression. Our algorithms have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes following suprainguinal bypass.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
5
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al-Omran M. Predicting outcomes following lower extremity open revascularization using machine learning. Sci Rep 2024; 14:2899. [PMID: 38316811 PMCID: PMC10844206 DOI: 10.1038/s41598-024-52944-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 01/25/2024] [Indexed: 02/07/2024] Open
Abstract
Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Institute of Medical Science, University of Toronto, Toronto, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, USA
| | - Jamal J Hoballah
- Division of Vascular and Endovascular Surgery, Department of Surgery, American University of Beirut Medical Center, Beirut, Lebanon
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Canada.
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada.
- Institute of Medical Science, University of Toronto, Toronto, Canada.
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada.
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia.
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
6
|
Tjardes T, Marche B, Imach S. Mangled extremity: limb salvage for reconstruction versus primary amputation. Curr Opin Crit Care 2023; 29:682-688. [PMID: 37909372 DOI: 10.1097/mcc.0000000000001108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2023]
Abstract
PURPOSE OF REVIEW While MESS has historically influenced limb salvage versus amputation decisions, its universal applicability remains uncertain. With trauma systems expanding and advancements in trauma care, the need for a nuanced understanding of limb salvage has become paramount. RECENT FINDINGS Recent literature reflects a shift in the management of mangled extremities. Vascular surgery, plastic surgery, and technological advancements have garnered attention. The MESS's efficacy in predicting amputation postvascular reconstruction has been questioned. Machine learning techniques have emerged as a means to predict peritraumatic amputation, incorporating a broader set of variables. Additionally, advancements in socket design, such as automated adjustments and bone-anchored prosthetics, show promise in enhancing prosthetic care. Surgical strategies to mitigate neuropathic pain, including targeted muscle reinnervation (TMR), are evolving and may offer relief for amputees. Predicting the long-term course of osteomyelitis following limb salvage is challenging, but it significantly influences patient quality of life. SUMMARY The review underscores the evolving landscape of limb salvage decision-making, emphasizing the need for personalized, patient-centered approaches. The Ganga Hospital Score (GHS) introduces a nuanced approach with a 'grey zone' for patients requiring individualized assessments. Future research may leverage artificial intelligence (AI) and predictive models to enhance decision support. Overall, the care of mangled extremities extends beyond a binary choice of limb salvage or amputation, necessitating a holistic understanding of patients' injury patterns, expectations, and abilities for optimal outcomes.
Collapse
Affiliation(s)
- Thorsten Tjardes
- Department of Trauma and Orthopedic Surgery, Cologne-Merheim Medical Center (CMMC), University of Witten/Herdecke, Cologne, Germany
| | | | | |
Collapse
|
7
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al-Omran M. Predicting outcomes following open revascularization for aortoiliac occlusive disease using machine learning. J Vasc Surg 2023; 78:1449-1460.e7. [PMID: 37454952 DOI: 10.1016/j.jvs.2023.07.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 06/12/2023] [Accepted: 07/08/2023] [Indexed: 07/18/2023]
Abstract
OBJECTIVE Open surgical treatment options for aortoiliac occlusive disease carry significant perioperative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following open aortoiliac revascularization. METHODS The National Surgical Quality Improvement Program (NSQIP) targeted vascular database was used to identify patients who underwent open aortoiliac revascularization for atherosclerotic disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. The 30-day secondary outcomes were individual components of the primary outcome, major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death), individual components of MACE, wound complication, bleeding, other morbidity, non-home discharge, and unplanned readmission. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. Variable importance scores were calculated to determine the top 10 predictive features. Performance was assessed on subgroups based on age, sex, race, ethnicity, symptom status, procedure type, and urgency. RESULTS Overall, 9649 patients were included. The primary outcome of 30-day MALE or death occurred in 1021 patients (10.6%). Our best performing prediction model for 30-day MALE or death was XGBoost, achieving an AUROC of 0.95 (95% confidence interval [CI], 0.94-0.96). In comparison, logistic regression had an AUROC of 0.79 (95% CI, 0.77-0.81). For 30-day secondary outcomes, XGBoost achieved AUROCs between 0.87 and 0.97 (untreated loss of patency [0.95], major reintervention [0.88], major amputation [0.96], death [0.97], MACE [0.95], myocardial infarction [0.88], stroke [0.93], wound complication [0.94], bleeding [0.87], other morbidity [0.96], non-home discharge [0.90], and unplanned readmission [0.91]). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.05. The strongest predictive feature in our algorithm was chronic limb-threatening ischemia. Model performance remained robust on all subgroup analyses of specific demographic/clinical populations. CONCLUSIONS Our ML models accurately predict 30-day outcomes following open aortoiliac revascularization using preoperative data, performing better than logistic regression. They have potential for important utility in guiding risk-mitigation strategies for patients being considered for open aortoiliac revascularization to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Department of Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon; College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, MA
| | - Jamal J Hoballah
- Division of Vascular and Endovascular Surgery, Department of Surgery, American University of Beirut Medical Center, Beirut, Lebanon
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Department of Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; Institute for Clinical Evaluative Sciences, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
8
|
Yao PF, Diao YD, McMullen EP, Manka M, Murphy J, Lin C. Predicting amputation using machine learning: A systematic review. PLoS One 2023; 18:e0293684. [PMID: 37934767 PMCID: PMC10629636 DOI: 10.1371/journal.pone.0293684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Accepted: 10/17/2023] [Indexed: 11/09/2023] Open
Abstract
Amputation is an irreversible, last-line treatment indicated for a multitude of medical problems. Delaying amputation in favor of limb-sparing treatment may lead to increased risk of morbidity and mortality. This systematic review aims to synthesize the literature on how ML is being applied to predict amputation as an outcome. OVID Embase, OVID Medline, ACM Digital Library, Scopus, Web of Science, and IEEE Xplore were searched from inception to March 5, 2023. 1376 studies were screened; 15 articles were included. In the diabetic population, models ranged from sub-optimal to excellent performance (AUC: 0.6-0.94). In trauma patients, models had strong to excellent performance (AUC: 0.88-0.95). In patients who received amputation secondary to other etiologies (e.g.: burns and peripheral vascular disease), models had similar performance (AUC: 0.81-1.0). Many studies were found to have a high PROBAST risk of bias, most often due to small sample sizes. In conclusion, multiple machine learning models have been successfully developed that have the potential to be superior to traditional modeling techniques and prospective clinical judgment in predicting amputation. Further research is needed to overcome the limitations of current studies and to bring applicability to a clinical setting.
Collapse
Affiliation(s)
- Patrick Fangping Yao
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Yi David Diao
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Eric P. McMullen
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Marlin Manka
- Department of Computer Science, University of Western Ontario, London, ON, Canada
| | - Jessica Murphy
- Division of Physical Medicine and Rehabilitation, McMaster University, Hamilton, ON, Canada
| | - Celina Lin
- Division of Physical Medicine and Rehabilitation, McMaster University, Hamilton, ON, Canada
- Division of Physical Medicine and Rehabilitation, Hamilton Health Sciences, Hamilton, ON, Canada
| |
Collapse
|
9
|
Huang RW, Tsai TY, Hsieh YH, Hsu CC, Chen SH, Lee CH, Lin YT, Kao HK, Lin CH. Reliability of Postoperative Free Flap Monitoring with a Novel Prediction Model Based on Supervised Machine Learning. Plast Reconstr Surg 2023; 152:943e-952e. [PMID: 36790782 DOI: 10.1097/prs.0000000000010307] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/16/2023]
Abstract
BACKGROUND Postoperative free flap monitoring is a critical part of reconstructive microsurgery. Postoperative clinical assessments rely heavily on specialty-trained staff. Therefore, in regions with limited specialist availability, the feasibility of performing microsurgery is restricted. This study aimed to apply artificial intelligence in postoperative free flap monitoring and validate the ability of machine learning in predicting and differentiating types of postoperative free flap circulation. METHODS Postoperative data from 176 patients who received free flap surgery were prospectively collected, including free flap photographs and clinical evaluation measures. Flap circulation outcome variables included normal, arterial insufficiency, and venous insufficiency. The Synthetic Minority Oversampling Technique plus Tomek Links (SMOTE-Tomek) was applied for data balance. Data were divided into 80%:20% for model training and validation. Shapley Additive Explanations were used for prediction interpretations of the model. RESULTS Of 805 total included flaps, 555 (69%) were normal, 97 (12%) had arterial insufficiency, and 153 (19%) had venous insufficiency. The most effective prediction model was developed based on random forest, with an accuracy of 98.4%. Temperature and color differences between the flap and the surrounding skin were the most significant contributing factors to predict a vascular compromised flap. CONCLUSIONS This study demonstrated the reliability of a machine-learning model in differentiating various types of postoperative flap circulation. This novel technique may reduce the burden of free flap monitoring and encourage the broader use of reconstructive microsurgery in regions with a limited number of staff specialists.
Collapse
Affiliation(s)
- Ren-Wen Huang
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Tzong-Yueh Tsai
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Yun-Huan Hsieh
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Chung-Chen Hsu
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Shih-Heng Chen
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Che-Hsiung Lee
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Yu-Te Lin
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Huang-Kai Kao
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| | - Cheng-Hung Lin
- From the Department of Plastic and Reconstructive Surgery, Chang Gung Memorial Hospital, Chang Gung Medical College and Chang Gung University
| |
Collapse
|
10
|
Zheng S, Chen X, Liu W, Li Z, Xiao M, Liu Y, Chen H. Association of loneliness and grey matter volume in the dorsolateral prefrontal cortex: the mediating role of interpersonal self-support traits. Brain Imaging Behav 2023; 17:481-493. [PMID: 37277604 DOI: 10.1007/s11682-023-00776-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/12/2023] [Indexed: 06/07/2023]
Abstract
As a social and public health concern, loneliness is associated with an abundance of negative life outcomes such as depressive symptomatology, mortality, and sleep disturbance. Nevertheless, the neural basis underlying loneliness remains elusive; in addition, previous neuroimaging studies about loneliness mainly focused on the elderly and were limited by small sample sizes. Here, utilizing the voxel-based morphometry (VBM) approach via structural magnetic resonance imaging, we investigated the association between brain GMV and loneliness in 462 young adults (67.7% females, age = 18.59 ± 1.14 years). Results from whole-brain VBM analyses revealed that individuals with higher loneliness tended to have greater GMV in the right dorsolateral prefrontal cortex (DLPFC), which was thought to be associated with emotional regulation deficits and executive dysfunction. Importantly, the GMV-based predictive models (a machine-learning method) demonstrated that the correlation between loneliness and GMV in the DLPFC was robust. Further, interpersonal self-support traits (ISS), a Chinese indigenous personality construct and pivotal personality factor for resisting negative life outcomes, mediated the relationship between the GMV in the right DLPFC and loneliness. Taken together, the present study reveals that the GMV in right DLPFC acts as an underlying neurostructural correlate of loneliness in the healthy brain, and further provides a brain-personality-symptom pathway for protection against loneliness in which GMV of DLPFC affects loneliness through ISS traits. Future intervention procedures aiming to decrease loneliness and enhance mental health levels among young adults should be developed through improving interpersonal relationships such as social skills training.
Collapse
Affiliation(s)
- Shuang Zheng
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China
| | - Ximei Chen
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China
| | - Weijun Liu
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China
| | - Ziang Li
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China
| | - Mingyue Xiao
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China
| | - Yong Liu
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China
| | - Hong Chen
- Key Laboratory of Cognition and Personality (SWU), Ministry of Education, Chongqing, 400715, China.
- Faculty of Psychology, Southwest University, No. 2, Tiansheng Road, Beibei District, Chongqing, 400715, China.
- Research Center of Psychology and Social Development, Chongqing, 400715, China.
| |
Collapse
|
11
|
Chen TLW, Buddhiraju A, Seo HH, Subih MA, Tuchinda P, Kwon YM. Internal and External Validation of the Generalizability of Machine Learning Algorithms in Predicting Non-home Discharge Disposition Following Primary Total Knee Joint Arthroplasty. J Arthroplasty 2023; 38:1973-1981. [PMID: 36764409 DOI: 10.1016/j.arth.2023.01.065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 01/24/2023] [Accepted: 01/31/2023] [Indexed: 02/12/2023] Open
Abstract
BACKGROUND Nonhome discharge disposition following primary total knee arthroplasty (TKA) is associated with a higher rate of complications and constitutes a socioeconomic burden on the health care system. While existing algorithms predicting nonhome discharge disposition varied in degrees of mathematical complexity and prediction power, their capacity to generalize predictions beyond the development dataset remains limited. Therefore, this study aimed to establish the machine learning model generalizability by performing internal and external validations using nation-scale and institutional cohorts, respectively. METHODS Four machine learning models were trained using the national cohort. Recursive feature elimination and hyper-parameter tuning were applied. Internal validation was achieved through five-fold cross-validation during model training. The trained models' performance was externally validated using the institutional cohort and assessed by discrimination, calibration, and clinical utility. RESULTS The national (424,354 patients) and institutional (10,196 patients) cohorts had non-home discharge rates of 19.4 and 36.4%, respectively. The areas under the receiver operating curve of the model predictions were 0.83 to 0.84 during internal validation and increased to 0.88 to 0.89 during external validation. Artificial neural network and histogram-based gradient boosting elicited the best performance with a mean area under the receiver operating curve of 0.89, calibration slope of 1.39, and Brier score of 0.14, which indicated that the two models were robust in distinguishing non-home discharge and well-calibrated with accurate predictions of the probabilities. The low inter-dataset similarity indicated reliable external validation. Length of stay, age, body mass index, and sex were the strongest predictors of discharge destination after primary TKA. CONCLUSION The machine learning models demonstrated excellent predictive performance during both internal and external validations, supporting their generalizability across different patient cohorts and potential applicability in the clinical workflow.
Collapse
Affiliation(s)
- Tony Lin-Wei Chen
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Anirudh Buddhiraju
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Henry Hojoon Seo
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Murad Abdullah Subih
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Pete Tuchinda
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Young-Min Kwon
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
12
|
Winarski A, Tambyraja AL. Looking Through the Crystal Ball of Data Science. Eur J Vasc Endovasc Surg 2023; 65:608. [PMID: 36682403 DOI: 10.1016/j.ejvs.2023.01.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 01/12/2023] [Indexed: 01/21/2023]
Affiliation(s)
- Allison Winarski
- Edinburgh Vascular Service, Royal Infirmary of Edinburgh, Edinburgh, UK
| | - Andrew L Tambyraja
- Edinburgh Vascular Service, Royal Infirmary of Edinburgh, Edinburgh, UK.
| |
Collapse
|
13
|
Buddhiraju A, Chen TLW, Subih MA, Seo HH, Esposito JG, Kwon YM. Validation and Generalizability of Machine Learning Models for the Prediction of Discharge Disposition Following Revision Total Knee Arthroplasty. J Arthroplasty 2023; 38:S253-S258. [PMID: 36849013 DOI: 10.1016/j.arth.2023.02.054] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/06/2022] [Revised: 02/16/2023] [Accepted: 02/20/2023] [Indexed: 03/01/2023] Open
Abstract
BACKGROUND Postoperative discharge to facilities account for over 33% of the $ 2.7 billion revision total knee arthroplasty (TKA)-associated annual expenditures and are associated with increased complications when compared to home discharges. Prior studies predicting discharge disposition using advanced machine learning (ML) have been limited due to a lack of generalizability and validation. This study aimed to establish ML model generalizability by externally validating its prediction for nonhome discharge following revision TKA using national and institutional databases. METHODS The national and institutional cohorts comprised 52,533 and 1,628 patients, respectively, with 20.6 and 19.4% nonhome discharge rates. Five ML models were trained and internally validated (five-fold cross-validation) on a large national dataset. Subsequently, external validation was performed on our institutional dataset. Model performance was assessed using discrimination, calibration, and clinical utility. Global predictor importance plots and local surrogate models were used for interpretation. RESULTS The strongest predictors of nonhome discharge were patient age, body mass index, and surgical indication. The area under the receiver operating characteristic curve increased from internal to external validation and ranged between 0.77 and 0.79. Artificial neural network was the best predictive model for identifying patients at risk for nonhome discharge (area under the receiver operating characteristic curve = 0.78), and also the most accurate (calibration slope = 0.93, intercept = 0.02, and Brier score = 0.12). CONCLUSION All five ML models demonstrated good-to-excellent discrimination, calibration, and clinical utility on external validation, with artificial neural network being the best model for predicting discharge disposition following revision TKA. Our findings establish the generalizability of ML models developed using data from a national database. The integration of these predictive models into clinical workflow may assist in optimizing discharge planning, bed management, and cost containment associated with revision TKA.
Collapse
Affiliation(s)
- Anirudh Buddhiraju
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Tony L-W Chen
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Murad A Subih
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Henry H Seo
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - John G Esposito
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Young-Min Kwon
- Bioengineering Laboratory, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
14
|
Peng HT, Siddiqui MM, Rhind SG, Zhang J, da Luz LT, Beckett A. Artificial intelligence and machine learning for hemorrhagic trauma care. Mil Med Res 2023; 10:6. [PMID: 36793066 PMCID: PMC9933281 DOI: 10.1186/s40779-023-00444-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Accepted: 02/01/2023] [Indexed: 02/17/2023] Open
Abstract
Artificial intelligence (AI), a branch of machine learning (ML) has been increasingly employed in the research of trauma in various aspects. Hemorrhage is the most common cause of trauma-related death. To better elucidate the current role of AI and contribute to future development of ML in trauma care, we conducted a review focused on the use of ML in the diagnosis or treatment strategy of traumatic hemorrhage. A literature search was carried out on PubMed and Google scholar. Titles and abstracts were screened and, if deemed appropriate, the full articles were reviewed. We included 89 studies in the review. These studies could be grouped into five areas: (1) prediction of outcomes; (2) risk assessment and injury severity for triage; (3) prediction of transfusions; (4) detection of hemorrhage; and (5) prediction of coagulopathy. Performance analysis of ML in comparison with current standards for trauma care showed that most studies demonstrated the benefits of ML models. However, most studies were retrospective, focused on prediction of mortality, and development of patient outcome scoring systems. Few studies performed model assessment via test datasets obtained from different sources. Prediction models for transfusions and coagulopathy have been developed, but none is in widespread use. AI-enabled ML-driven technology is becoming integral part of the whole course of trauma care. Comparison and application of ML algorithms using different datasets from initial training, testing and validation in prospective and randomized controlled trials are warranted for provision of decision support for individualized patient care as far forward as possible.
Collapse
Affiliation(s)
- Henry T Peng
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada.
| | - M Musaab Siddiqui
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada
| | - Shawn G Rhind
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada
| | - Jing Zhang
- Defence Research and Development Canada, Toronto Research Centre, Toronto, ON, M3K 2C9, Canada
| | | | - Andrew Beckett
- St. Michael's Hospital, Toronto, ON, M5B 1W8, Canada
- Royal Canadian Medical Services, Ottawa, K1A 0K2, Canada
| |
Collapse
|
15
|
Lee W, Schwartz N, Bansal A, Khor S, Hammarlund N, Basu A, Devine B. A Scoping Review of the Use of Machine Learning in Health Economics and Outcomes Research: Part 2-Data From Nonwearables. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:2053-2061. [PMID: 35989154 DOI: 10.1016/j.jval.2022.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 06/10/2022] [Accepted: 07/10/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVES Despite the increasing interest in applying machine learning (ML) methods in health economics and outcomes research (HEOR), stakeholders face uncertainties in when and how ML can be used. We reviewed the recent applications of ML in HEOR. METHODS We searched PubMed for studies published between January 2020 and March 2021 and randomly chose 20% of the identified studies for the sake of manageability. Studies that were in HEOR and applied an ML technique were included. Studies related to wearable devices were excluded. We abstracted information on the ML applications, data types, and ML methods and analyzed it using descriptive statistics. RESULTS We retrieved 805 articles, of which 161 (20%) were randomly chosen. Ninety-two of the random sample met the eligibility criteria. We found that ML was primarily used for predicting future events (86%) rather than current events (14%). The most common response variables were clinical events or disease incidence (42%) and treatment outcomes (22%). ML was less used to predict economic outcomes such as health resource utilization (16%) or costs (3%). Although electronic medical records (35%) were frequently used for model development, claims data were used less frequently (9%). Tree-based methods (eg, random forests and boosting) were the most commonly used ML methods (31%). CONCLUSIONS The use of ML techniques in HEOR is growing rapidly, but there remain opportunities to apply them to predict economic outcomes, especially using claims databases, which could inform the development of cost-effectiveness models.
Collapse
Affiliation(s)
- Woojung Lee
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA.
| | - Naomi Schwartz
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Aasthaa Bansal
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Sara Khor
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Noah Hammarlund
- Department of Health Services Research, Management & Policy, University of Florida, Gainesville, FL, USA
| | - Anirban Basu
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| | - Beth Devine
- The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA, USA
| |
Collapse
|
16
|
Moris D, Henao R, Hensman H, Stempora L, Chasse S, Schobel S, Dente CJ, Kirk AD, Elster E. Multidimensional machine learning models predicting outcomes after trauma. Surgery 2022; 172:1851-1859. [PMID: 36116976 DOI: 10.1016/j.surg.2022.08.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Revised: 08/01/2022] [Accepted: 08/04/2022] [Indexed: 01/07/2023]
Abstract
BACKGROUND An emerging body of literature supports the role of individualized prognostic tools to guide the management of patients after trauma. The aim of this study was to develop advanced modeling tools from multidimensional data sources, including immunological analytes and clinical and administrative data, to predict outcomes in trauma patients. METHODS This was a prospective study of trauma patients at Level 1 centers from 2015 to 2019. Clinical, flow cytometry, and serum cytokine data were collected within 48 hours of admission. Sparse logistic regression models were developed, jointly selecting predictors and estimating the risk of ventilator-associated pneumonia, acute kidney injury, complicated disposition (death, rehabilitation, or nursing facility), and return to the operating room. Model parameters (regularization controlling model sparsity) and performance estimation were obtained via nested leave-one-out cross-validation. RESULTS A total of 179 patients were included. The incidences of ventilator-associated pneumonia, acute kidney injury, complicated disposition, and return to the operating room were 17.7%, 28.8%, 22.5%, and 12.3%, respectively. Regarding extensive resource use, 30.7% of patients had prolonged intensive care unit stay, 73.2% had prolonged length of stay, and 23.5% had need for prolonged ventilatory support. The models were developed and cross-validated for ventilator-associated pneumonia, acute kidney injury, complicated dispositions, and return to the operating room, yielding predictive areas under the curve from 0.70 to 0.91. Each model derived its optimal predictive value by combining clinical, administrative, and immunological analyte data. CONCLUSION Clinical, immunological, and administrative data can be combined to predict post-traumatic outcomes and resource use. Multidimensional machine learning modeling can identify trauma patients with complicated clinical trajectories and high resource needs.
Collapse
Affiliation(s)
| | | | - Hannah Hensman
- DecisionQ, Arlington, VA; Surgical Critical Care Initiative, Department of Surgery, Uniformed Services University of the Health Sciences; Bethesda, MD
| | - Linda Stempora
- Medical Center, Duke University Durham, NC; Surgical Critical Care Initiative, Department of Surgery, Uniformed Services University of the Health Sciences; Bethesda, MD
| | - Scott Chasse
- Medical Center, Duke University Durham, NC; Surgical Critical Care Initiative, Department of Surgery, Uniformed Services University of the Health Sciences; Bethesda, MD
| | - Seth Schobel
- Surgical Critical Care Initiative, Department of Surgery, Uniformed Services University of the Health Sciences; Bethesda, MD; Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc, Bethesda, MD
| | | | - Allan D Kirk
- Medical Center, Duke University Durham, NC; Surgical Critical Care Initiative, Department of Surgery, Uniformed Services University of the Health Sciences; Bethesda, MD
| | - Eric Elster
- Surgical Critical Care Initiative, Department of Surgery, Uniformed Services University of the Health Sciences; Bethesda, MD; Walter Reed National Military Medical Center, Bethesda, MD
| |
Collapse
|
17
|
Perkins ZB, Kersey AJ, White JM, Lauria AL, Propper BW, Tai NRM, Rasmussen TE. Impact of Ischemia Duration on Lower Limb Salvage in Combat Casualties. Ann Surg 2022; 276:532-538. [PMID: 35972512 DOI: 10.1097/sla.0000000000005560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
INTRODUCTION The 6-hour threshold to revascularization of an ischemic limb is ubiquitous in the trauma literature, however, contemporary evidence suggests that this threshold should be less. This study aims to characterize the relationship between the duration of limb ischemia and successful limb salvage following lower extremity arterial trauma. METHODS This is a cohort study of the United States and UK military service members injured while serving in Iraq or Afghanistan between 2003 and 2013. Consecutive patients who sustained iliac, femoral, or popliteal artery injuries, and underwent surgery to attempt revascularization, were included. The association between limb outcome and the duration of limb ischemia was assessed using the Kaplan-Meier method. RESULTS One hundred twenty-two patients (129 limbs) who sustained iliac (2.3%), femoral (56.6%), and popliteal (41.1%) arterial injuries were included. Overall, 87 limbs (67.4%) were successfully salvaged. The probability of limb salvage was 86.0% when ischemia was ≤1 hour; 68.3% when between 1 and 3 hours; 56.3% when between 3 and 6 hours; and 6.7% when >6 hours ( P <0.0001). Shock more than doubled the risk of failed limb salvage [hazard ratio=2.42 (95% confidence interval: 1.27-4.62)]. CONCLUSIONS Limb salvage is critically dependent on the duration of ischemia with a 10% reduction in the probability of successful limb salvage for every hour delay to revascularization. The presence of shock significantly worsens this relationship. Military trauma systems should prioritize rapid hemorrhage control and early limb revascularization within 1 hour of injury.
Collapse
Affiliation(s)
- Zane B Perkins
- Centre for Trauma Sciences, Queen Mary University of London, London, UK
| | - Alexander J Kersey
- Battlefield Shock and Organ Support Laboratory, Department of Surgery, Uniformed Services University of Health Sciences, Bethesda, MD
- Walter Reed National Military Medical Center, Bethesda, MD
| | - Joseph M White
- Battlefield Shock and Organ Support Laboratory, Department of Surgery, Uniformed Services University of Health Sciences, Bethesda, MD
- Walter Reed National Military Medical Center, Bethesda, MD
| | - Alexis L Lauria
- Battlefield Shock and Organ Support Laboratory, Department of Surgery, Uniformed Services University of Health Sciences, Bethesda, MD
- Walter Reed National Military Medical Center, Bethesda, MD
| | | | - Nigel R M Tai
- Centre for Trauma Sciences, Queen Mary University of London, London, UK
- Academic Department of Military Surgery and Trauma, Royal Centre for Defence Medicine, Birmingham, UK
| | - Todd E Rasmussen
- Centre for Trauma Sciences, Queen Mary University of London, London, UK
- Mayo Clinic, Rochester, MN
| |
Collapse
|
18
|
Machine Learning in the Prediction of Trauma Outcomes: A Systematic Review. Ann Emerg Med 2022; 80:440-455. [PMID: 35842343 DOI: 10.1016/j.annemergmed.2022.05.011] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 03/20/2022] [Accepted: 05/04/2022] [Indexed: 11/23/2022]
Abstract
STUDY OBJECTIVE Machine learning models carry unique potential as decision-making aids and prediction tools for improving patient care. Traumatically injured patients provide a uniquely heterogeneous population with severe injuries that can be difficult to predict. Given the relative infancy of machine learning applications in medicine, this systematic review aimed to better understand the current state of machine learning development and implementation to help create a basis for future research. METHODS We conducted a systematic review from inception to May 2021, using Embase, MEDLINE through Ovid, Web of Science, Google Scholar, and relevant gray literature, for uses of machine learning in predicting the outcomes of trauma patients. The screening and data extraction were performed by 2 independent reviewers. RESULTS Of the 14,694 identified articles screened, 67 were included for data extraction. Artificial neural networks comprised the most commonly used model, and mortality was the most prevalent outcome of interest. In terms of machine learning model development, there was a lack of studies that employed external validation, feature selection methods, and performed formal calibration testing. Significant heterogeneity in reporting was also observed between the machine learning models employed, patient populations, performance metrics, and features employed. CONCLUSION This review highlights the heterogeneity in the development and reporting of machine learning models for the prediction of trauma outcomes. While these models present an area of opportunity as an ancillary to clinical decision-making, we recommend more standardization and rigorous guidelines for the development of future models.
Collapse
|
19
|
Li B, Feridooni T, Cuen-Ojeda C, Kishibe T, de Mestral C, Mamdani M, Al-Omran M. Machine learning in vascular surgery: a systematic review and critical appraisal. NPJ Digit Med 2022; 5:7. [PMID: 35046493 PMCID: PMC8770468 DOI: 10.1038/s41746-021-00552-y] [Citation(s) in RCA: 44] [Impact Index Per Article: 22.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2021] [Accepted: 12/13/2021] [Indexed: 12/18/2022] Open
Abstract
Machine learning (ML) is a rapidly advancing field with increasing utility in health care. We conducted a systematic review and critical appraisal of ML applications in vascular surgery. MEDLINE, Embase, and Cochrane CENTRAL were searched from inception to March 1, 2021. Study screening, data extraction, and quality assessment were performed by two independent reviewers, with a third author resolving discrepancies. All original studies reporting ML applications in vascular surgery were included. Publication trends, disease conditions, methodologies, and outcomes were summarized. Critical appraisal was conducted using the PROBAST risk-of-bias and TRIPOD reporting adherence tools. We included 212 studies from a pool of 2235 unique articles. ML techniques were used for diagnosis, prognosis, and image segmentation in carotid stenosis, aortic aneurysm/dissection, peripheral artery disease, diabetic foot ulcer, venous disease, and renal artery stenosis. The number of publications on ML in vascular surgery increased from 1 (1991-1996) to 118 (2016-2021). Most studies were retrospective and single center, with no randomized controlled trials. The median area under the receiver operating characteristic curve (AUROC) was 0.88 (range 0.61-1.00), with 79.5% [62/78] studies reporting AUROC ≥ 0.80. Out of 22 studies comparing ML techniques to existing prediction tools, clinicians, or traditional regression models, 20 performed better and 2 performed similarly. Overall, 94.8% (201/212) studies had high risk-of-bias and adherence to reporting standards was poor with a rate of 41.4%. Despite improvements over time, study quality and reporting remain inadequate. Future studies should consider standardized tools such as PROBAST and TRIPOD to improve study quality and clinical applicability.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, 149 College St, Toronto, ON, M5T 1P5, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, 1 King's College Circle, Toronto, ON, M5S 1A8, Canada
| | - Tiam Feridooni
- Department of Surgery, University of Toronto, 149 College St, Toronto, ON, M5T 1P5, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
| | - Cesar Cuen-Ojeda
- Department of Surgery, University of Toronto, 149 College St, Toronto, ON, M5T 1P5, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
| | - Teruko Kishibe
- Health Sciences Library, St. Michael's Hospital, Unity Health Toronto, 209 Victoria St, Toronto, ON, M5B 1T8, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria St, Toronto, ON, M5B 1T8, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, 149 College St, Toronto, ON, M5T 1P5, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria St, Toronto, ON, M5B 1T8, Canada
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON, M5T 3M7, Canada
| | - Muhammad Mamdani
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, 1 King's College Circle, Toronto, ON, M5S 1A8, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria St, Toronto, ON, M5B 1T8, Canada
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON, M5T 3M7, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, 144 College St, Toronto, ON, M5S 3M2, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, 149 College St, Toronto, ON, M5T 1P5, Canada.
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada.
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, 1 King's College Circle, Toronto, ON, M5S 1A8, Canada.
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria St, Toronto, ON, M5B 1T8, Canada.
- Institute of Medical Science, University of Toronto, 1 King's College Circle, Toronto, ON, M5S 1A8, Canada.
- Department of Surgery, King Saud University, ZIP 4545, Riyadh, 11451, Kingdom of Saudi Arabia.
| |
Collapse
|
20
|
Urrechaga E, Jabori S, Kang N, Kenel-Pierre S, Lopez A, Rattan R, Rey J, Bornak A. Traumatic Lower Extremity Vascular Injuries and Limb Salvage in a Civilian Urban Trauma Center. Ann Vasc Surg 2021; 82:30-40. [PMID: 34954038 DOI: 10.1016/j.avsg.2021.12.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Revised: 10/19/2021] [Accepted: 12/03/2021] [Indexed: 12/11/2022]
Abstract
BACKGROUND Data on management of traumatic lower extremity injuries comes largely from military experience and involves few civilian centers. This study reports on the experience of an urban trauma center and factors associated with limb loss. METHODS A retrospective review of lower extremity arterial injuries between 2013-2020 at an academic urban level 1 trauma center was completed. Patients with lower extremity revascularization were included in the final data analysis. Demographics, clinical variables, operative details, type of revascularization, as well as 30-day morbidity and postoperative outcomes were analyzed. The primary outcome of interest was 30-day limb loss. Secondary outcomes included postoperative complications and functional outcomes. RESULTS 75 patients were included in our analysis. 69 were male (92%), mean age 33 ±15 years, 50 patients had penetrating trauma (67%), mean injury severity score was 15±9. 30-day limb loss was reported in 8 (11%). Factors associated with limb loss included female sex (p=0.001), high body mass index (BMI) (p=0.001), blunt injury (p=0.001), associated fractures (p=0.005), significant soft tissue injury (p=0.007), delayed repair after shunt placement (p=0.003), bypass revascularization (p=0.001), initial revascularization failure (p=0.019), and wound complications (p<0.001). Fifty-five patients had at least one return to the operating room (ROR), including 24 patients (32%) for complications related to their revascularization. These included delayed compartment syndrome (n=7), revascularization failure (n=9), bleeding (n=3), and vascular surgical wound complications (n=5). Mean length of hospital stay (LOS) for the cohort was 24±20 days with 3±3 ROR, in contrast patients who ultimately required amputation had LOS of 57±21 days with 8±4 ROR. 57 patients (76%) followed in clinic for a median 36[14-110] days, with only 32 (43%) at > 30 days. 23 reported ambulation without assistance, 9 neuromotor deficit including 1 patient that had delayed amputation. CONCLUSION Patients with blunt trauma and associated fracture and/or extensive soft tissue injury are at risk of limb loss. These injuries are often associated with postoperative wound complications, requiring aggressive soft tissue care that substantially increases ROR and LOS; Expectations for limb salvage in these patients should be tempered when the other associated factors with limb loss mentioned above are also present. When limb salvage is achieved, regaining full limb function remains a challenge.
Collapse
Affiliation(s)
- Eva Urrechaga
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA
| | - Sinan Jabori
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA
| | - Naixin Kang
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA
| | - Stefan Kenel-Pierre
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA
| | - Alberto Lopez
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA
| | - Rishi Rattan
- Department of Surgery, Division of Trauma and Acute Care Surgery, University of Miami & Ryder Trauma Center- Jackson Memorial Hospital, Miami, FL, USA
| | - Jorge Rey
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA
| | - Arash Bornak
- Department of Surgery, Division of Vascular & Endovascular Surgery, University of Miami & Ryder Trauma Center-Jackson Memorial Hospital, Miami, FL, USA.
| |
Collapse
|
21
|
Magee GA, O'Banion LA. Reply. J Vasc Surg 2021; 75:1496. [PMID: 34921968 DOI: 10.1016/j.jvs.2021.11.068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Accepted: 11/23/2021] [Indexed: 10/19/2022]
Affiliation(s)
- Gregory A Magee
- Division of Vascular and Endovascular Surgery, Department of Surgery, University of Southern California, Los Angeles, Calif
| | - Leigh Ann O'Banion
- Division of Vascular Surgery, Department of Surgery, University of California, San Francisco, Fresno, Calif
| |
Collapse
|
22
|
Wohlgemut JM, Kyrimi E, Stoner RS, Pisirir E, Marsh W, Perkins ZB, Tai NRM. The outcome of a prediction algorithm should be a true patient state rather than an available surrogate. J Vasc Surg 2021; 75:1495-1496. [PMID: 34921966 DOI: 10.1016/j.jvs.2021.10.059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 10/30/2021] [Indexed: 10/19/2022]
Affiliation(s)
- Jared M Wohlgemut
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, United Kingdom; Royal London Hospital, Barts NHS Health Trust, London, United Kingdom
| | - Evangelia Kyrimi
- Department of Electrical Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | - Rebecca S Stoner
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, United Kingdom; Royal London Hospital, Barts NHS Health Trust, London, United Kingdom
| | - Erhan Pisirir
- Department of Electrical Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | - William Marsh
- Department of Electrical Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | - Zane B Perkins
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, United Kingdom; Royal London Hospital, Barts NHS Health Trust, London, United Kingdom
| | - Nigel R M Tai
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, United Kingdom; Royal London Hospital, Barts NHS Health Trust, London, United Kingdom; Royal Centre for Defence Medicine, Birmingham, United Kingdom
| |
Collapse
|
23
|
Ambiguity in extremity vascular trauma predicting tools. J Vasc Surg 2021; 74:1050-1051. [PMID: 34425950 DOI: 10.1016/j.jvs.2021.04.059] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Accepted: 04/16/2021] [Indexed: 11/23/2022]
|
24
|
Ratnayake A, Worlton TJ. Defining the Predictors in "Failed" Combat Trauma Lower Extremity Arterial Injury Revascularization. Ann Vasc Surg 2020; 70:e8-e10. [PMID: 32946994 DOI: 10.1016/j.avsg.2020.09.014] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 09/08/2020] [Indexed: 11/29/2022]
Affiliation(s)
- Amila Ratnayake
- Military Hospital Narahenpita, Colombo, Sri Lanka; Department of Surgery, Uniformed Services University of the Health Sciences, Bethesda, MD
| | - Tamara J Worlton
- Department of Surgery, Uniformed Services University of the Health Sciences, Bethesda, MD.
| |
Collapse
|