1
|
Li B, Eisenberg N, Beaton D, Lee DS, Al-Omran L, Wijeysundera DN, Hussain MA, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Predicting inferior vena cava filter complications using machine learning. J Vasc Surg Venous Lymphat Disord 2024; 12:101943. [PMID: 39084408 PMCID: PMC11523346 DOI: 10.1016/j.jvsv.2024.101943] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2024] [Revised: 06/03/2024] [Accepted: 06/26/2024] [Indexed: 08/02/2024]
Abstract
OBJECTIVE Inferior vena cava (IVC) filter placement is associated with important long-term complications. Predictive models for filter-related complications may help guide clinical decision-making but remain limited. We developed machine learning (ML) algorithms that predict 1-year IVC filter complications using preoperative data. METHODS The Vascular Quality Initiative database was used to identify patients who underwent IVC filter placement between 2013 and 2024. We identified 77 preoperative demographic and clinical features from the index hospitalization when the filter was placed. The primary outcome was 1-year filter-related complications (composite of filter thrombosis, migration, angulation, fracture, and embolization or fragmentation, vein perforation, new caval or iliac vein thrombosis, new pulmonary embolism, access site thrombosis, or failed retrieval). The data were divided into training (70%) and test (30%) sets. Six ML models were trained using preoperative features with 10-fold cross-validation (Extreme Gradient Boosting, random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was assessed using calibration plot and Brier score. Performance was evaluated across subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, planned duration of filter, landing site of filter, and presence of prior IVC filter placement. RESULTS Overall, 14,476 patients underwent IVC filter placement and 584 (4.0%) experienced 1-year filter-related complications. Patients with a primary outcome were younger (59.3 ± 16.7 years vs 63.8 ± 16.0 years; P < .001) and more likely to have thrombotic risk factors including thrombophilia, prior venous thromboembolism (VTE), and family history of VTE. The best prediction model was Extreme Gradient Boosting, achieving an AUROC of 0.93 (95% confidence interval, 0.92-0.94). In comparison, logistic regression had an AUROC of 0.63 (95% confidence interval, 0.61-0.65). Calibration plot showed good agreement between predicted/observed event probabilities with a Brier score of 0.07. The top 10 predictors of 1-year filter-related complications were (1) thrombophilia, (2) prior VTE, (3) antiphospholipid antibodies, (4) factor V Leiden mutation, (5) family history of VTE, (6) planned duration of IVC filter (temporary), (7) unable to maintain therapeutic anticoagulation, (8) malignancy, (9) recent or active bleeding, and (10) age. Model performance remained robust across all subgroups. CONCLUSIONS We developed ML models that can accurately predict 1-year IVC filter complications, performing better than logistic regression. These algorithms have potential to guide patient selection for filter placement, counselling, perioperative management, and follow-up to mitigate filter-related complications and improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Institute of Medical Science, University of Toronto, Toronto, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada
| | - Leen Al-Omran
- School of Medicine, Alfaisal University, Riyadh, Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, MA
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, Canada; Institute of Medical Science, University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada; Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; ICES, University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada; Division of Vascular and Interventional Radiology, University Health Network, Toronto, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Institute of Medical Science, University of Toronto, Toronto, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia.
| |
Collapse
|
2
|
Levy O, Shahar S. Artificial Intelligence for Climate Change Biology: From Data Collection to Predictions. Integr Comp Biol 2024; 64:953-974. [PMID: 39081076 DOI: 10.1093/icb/icae127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Revised: 07/19/2024] [Accepted: 07/18/2024] [Indexed: 09/28/2024] Open
Abstract
In the era of big data, ecological research is experiencing a transformative shift, yet big-data advancements in thermal ecology and the study of animal responses to climate conditions remain limited. This review discusses how big data analytics and artificial intelligence (AI) can significantly enhance our understanding of microclimates and animal behaviors under changing climatic conditions. We explore AI's potential to refine microclimate models and analyze data from advanced sensors and camera technologies, which capture detailed, high-resolution information. This integration can allow researchers to dissect complex ecological and physiological processes with unprecedented precision. We describe how AI can enhance microclimate modeling through improved bias correction and downscaling techniques, providing more accurate estimates of the conditions that animals face under various climate scenarios. Additionally, we explore AI's capabilities in tracking animal responses to these conditions, particularly through innovative classification models that utilize sensors such as accelerometers and acoustic loggers. For example, the widespread usage of camera traps can benefit from AI-driven image classification models to accurately identify thermoregulatory responses, such as shade usage and panting. AI is therefore instrumental in monitoring how animals interact with their environments, offering vital insights into their adaptive behaviors. Finally, we discuss how these advanced data-driven approaches can inform and enhance conservation strategies. In particular, detailed mapping of microhabitats essential for species survival under adverse conditions can guide the design of climate-resilient conservation and restoration programs that prioritize habitat features crucial for biodiversity resilience. In conclusion, the convergence of AI, big data, and ecological science heralds a new era of precision conservation, essential for addressing the global environmental challenges of the 21st century.
Collapse
Affiliation(s)
- Ofir Levy
- Tel Aviv University, Faculty of Life Sciences, School of Zoology, Tel Aviv 6997801, Israel
| | - Shimon Shahar
- Tel Aviv University, The AI and Data Science Center, Tel Aviv 6997801, Israel
| |
Collapse
|
3
|
Li B, Eisenberg N, Beaton D, Lee DS, Al-Omran L, Wijeysundera DN, Hussain MA, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using Machine Learning to Predict Outcomes Following Transfemoral Carotid Artery Stenting. J Am Heart Assoc 2024; 13:e035425. [PMID: 39189482 DOI: 10.1161/jaha.124.035425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/08/2024] [Accepted: 07/23/2024] [Indexed: 08/28/2024]
Abstract
BACKGROUND Transfemoral carotid artery stenting (TFCAS) carries important perioperative risks. Outcome prediction tools may help guide clinical decision-making but remain limited. We developed machine learning algorithms that predict 1-year stroke or death following TFCAS. METHODS AND RESULTS The VQI (Vascular Quality Initiative) database was used to identify patients who underwent TFCAS for carotid artery stenosis between 2005 and 2024. We identified 112 features from the index hospitalization (82 preoperative [demographic/clinical], 13 intraoperative [procedural], and 17 postoperative [in-hospital course/complications]). The primary outcome was 1-year postprocedural stroke or death. The data were divided into training (70%) and test (30%) sets. Six machine learning models were trained using preoperative features with 10-fold cross-validation. The primary model evaluation metric was area under the receiver operating characteristic curve. The algorithm with the best performance was further trained using intra- and postoperative features. Model robustness was assessed using calibration plots and Brier scores. Overall, 35 214 patients underwent TFCAS during the study period and 3257 (9.2%) developed 1-year stroke or death. The best preoperative prediction model was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.94 (95% CI, 0.93-0.95). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67). The extreme gradient boosting model maintained excellent performance at the intra- and postoperative stages, with area under the receiver operating characteristic curve values of 0.94 (95% CI, 0.93-0.95) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted/observed event probabilities with Brier scores of 0.11 (preoperative), 0.11 (intraoperative), and 0.09 (postoperative). CONCLUSIONS Machine learning can accurately predict 1-year stroke or death following TFCAS, performing better than logistic regression.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Institute of Medical Science, University of Toronto Ontario Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM) University of Toronto Ontario Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre University Health Network Toronto Ontario Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto University of Toronto Ontario Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre University Health Network Toronto Ontario Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
| | - Leen Al-Omran
- School of Medicine Alfaisal University Riyadh Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
- Department of Anesthesia St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health Brigham and Women's Hospital, Harvard Medical School Boston MA USA
| | - Ori D Rotstein
- Department of Surgery University of Toronto Ontario Canada
- Institute of Medical Science, University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Division of General Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
| | - Charles de Mestral
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto Ontario Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM) University of Toronto Ontario Canada
- Data Science & Advanced Analytics, Unity Health Toronto University of Toronto Ontario Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto Ontario Canada
- ICES, University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Leslie Dan Faculty of Pharmacy University of Toronto Ontario Canada
| | - Graham Roche-Nagle
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery, Peter Munk Cardiac Centre University Health Network Toronto Ontario Canada
- Division of Vascular and Interventional Radiology University Health Network Toronto Ontario Canada
| | - Mohammed Al-Omran
- Department of Surgery University of Toronto Ontario Canada
- Division of Vascular Surgery St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Institute of Medical Science, University of Toronto Ontario Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM) University of Toronto Ontario Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto Toronto Ontario Canada
- Department of Surgery King Faisal Specialist Hospital and Research Center Riyadh Saudi Arabia
| |
Collapse
|
4
|
Cheng WHG, Dong W, Tse ETY, Wong CKH, Chin WY, Bedford LE, Fong DYT, Ko WWK, Chao DVK, Tan KCB, Lam CLK. External validation of the Hong Kong Chinese non-laboratory risk models and scoring algorithm for case finding of prediabetes and diabetes mellitus in primary care. J Diabetes Investig 2024; 15:1317-1325. [PMID: 39212338 PMCID: PMC11363091 DOI: 10.1111/jdi.14256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 05/20/2024] [Accepted: 06/06/2024] [Indexed: 09/04/2024] Open
Abstract
AIMS/INTRODUCTION Two Hong Kong Chinese non-laboratory-based prediabetes/diabetes mellitus (pre-DM/DM) risk models were developed using logistic regression (LR) and machine learning, respectively. We aimed to evaluate the models' validity in case finding of pre-DM/DM in a Chinese primary care (PC) population. We also evaluated the validity of a risk-scoring algorithm derived from the LR model. MATERIALS AND METHODS This was a cross-sectional external validation study on Chinese adults, without a prior DM diagnosis, who were recruited from public/private PC clinics in Hong Kong. A total of 1,237 participants completed a questionnaire on the models' predictors. Of that, 919 underwent blood glucose testing. The primary outcome was the models' and the algorithm's sensitivity in finding pre-DM/DM cases. The secondary outcomes were the models' and the algorithm's specificity, positive/negative predictive values, discrimination and calibration. RESULTS The models' sensitivity were 0.70 (machine learning) and 0.72 (LR). Both showed good external discrimination (area under the receiver operating characteristic curve: machine learning 0.744, LR 0.739). The risks estimated by the models were lower than the observed incidence, indicating poor calibration. Both models were more effective among participants with lower pretest probabilities; that is, age 18-44 years. The algorithm's sensitivity was 0.77 at the cut-off score of ≥16 out of 41. CONCLUSION This study showed the validity of the models and the algorithm for finding pre-DM/DM cases in a Chinese PC population in Hong Kong. They can facilitate more cost-effective identification of high-risk individuals for blood testing to diagnose pre-DM/DM in PC. Further studies should recalibrate the models for more precise risk estimation in PC populations.
Collapse
Affiliation(s)
- Will HG Cheng
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
| | - Weinan Dong
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
| | - Emily TY Tse
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
- Department of Family MedicineThe University of Hong Kong‐Shenzhen HospitalShenzhenChina
| | - Carlos KH Wong
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
- Department of Pharmacology and Pharmacy, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
- Laboratory of Data Discovery for Health (D24H)Hong Kong Science and Technology ParkSha TinHong Kong
| | - Weng Y Chin
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
| | - Laura E Bedford
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
| | - Daniel YT Fong
- School of Nursing, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
| | - Welchie WK Ko
- Family Medicine and Primary Healthcare Department, Queen Mary Hospital, Hong Kong West ClusterHospital AuthorityHong KongHong Kong
| | - David VK Chao
- Department of Family Medicine & Primary Health Care, United Christian Hospital, Kowloon East ClusterHospital AuthorityHong KongHong Kong
- Department of Family Medicine & Primary Health Care, Tseung Kwan O Hospital, Kowloon East ClusterHospital AuthorityHong KongHong Kong
| | - Kathryn CB Tan
- Department of Medicine, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
| | - Cindy LK Lam
- Department of Family Medicine and Primary Care, School of Clinical Medicine, Li Ka Shing Faculty of MedicineThe University of Hong KongHong KongHong Kong
- Department of Family MedicineThe University of Hong Kong‐Shenzhen HospitalShenzhenChina
| |
Collapse
|
5
|
Bhak Y, Ahn TK, Peterson TA, Han HW, Nam SM. Machine Learning Models for Low Back Pain Detection and Factor Identification: Insights From a 6-Year Nationwide Survey. THE JOURNAL OF PAIN 2024; 25:104497. [PMID: 38342191 DOI: 10.1016/j.jpain.2024.02.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 02/03/2024] [Accepted: 02/06/2024] [Indexed: 02/13/2024]
Abstract
This study aimed to enhance performance, identify additional predictors, and improve the interpretability of biopsychosocial machine learning models for low back pain (LBP). Using survey data from a 6-year nationwide study involving 17,609 adults aged ≥50 years (Korea National Health and Nutrition Examination Survey), we explored 119 factors to detect LBP in individuals who reported experiencing LBP for at least 30 days within the previous 3 months. Our primary model, model 1, employed eXtreme Gradient Boosting (XGBoost) and selected primary factors (PFs) based on their feature importance scores. To extend this, we introduced additional factors, such as lumbar X-ray findings, physical activity, sitting time, and nutrient intake levels, which were available only during specific survey periods, into models 2 to 4. Model performance was evaluated using the area under the curve, with predicted probabilities explained by SHapley Additive exPlanations. Eleven PFs were identified, and model 1 exhibited an enhanced area under the curve .8 (.77-.84, 95% confidence interval). The factors had varying impacts across individuals, underscoring the need for personalized assessment. Hip and knee joint pain were the most significant PFs. High levels of physical activity were found to have a negative association with LBP, whereas a high intake of omega-6 was found to have a positive association. Notably, we identified factor clusters, including hip joint pain and female sex, potentially linked to osteoarthritis. In summary, this study successfully developed effective XGBoost models for LBP detection, thereby providing valuable insight into LBP-related factors. Comprehensive LBP management, particularly in women with osteoarthritis, is crucial given the presence of multiple factors. PERSPECTIVE: This article introduces XGBoost models designed to detect LBP and explores the multifactorial aspects of LBP through the application of SHapley Additive exPlanations and network analysis on the 4 developed models. The utilization of this analytical system has the potential to aid in devising personalized management strategies to address LBP.
Collapse
Affiliation(s)
- YoungMin Bhak
- Department of Biomedical Engineering, College of Information and Biotechnology, Ulsan National Institute of Science and Technology (UNIST), Ulsan, Republic of Korea
| | - Tae-Keun Ahn
- Department of Orthopedic Surgery, CHA Bundang Medical Center, CHA University, Seongnam, Republic of Korea
| | - Thomas A Peterson
- UCSF REACH Informatics Core, Department of Orthopaedic Surgery, Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, California
| | - Hyun Wook Han
- Department of Biomedical Informatics, CHA University, Seongnam, Republic of Korea; Institute for Biomedical Informatics, CHA University, Seongnam, Republic of Korea
| | - Sang Min Nam
- Department of Biomedical Informatics, CHA University, Seongnam, Republic of Korea; Institute for Biomedical Informatics, CHA University, Seongnam, Republic of Korea; Daechi Yonsei Eye Clinics, Seoul, Republic of Korea
| |
Collapse
|
6
|
Hwang UJ, Kwon OY, Kim JH, Yang S. Machine learning models for classifying non-specific neck pain using craniocervical posture and movement. Musculoskelet Sci Pract 2024; 71:102945. [PMID: 38527390 DOI: 10.1016/j.msksp.2024.102945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 03/11/2024] [Accepted: 03/17/2024] [Indexed: 03/27/2024]
Abstract
OBJECTIVE Physical therapists and clinicians commonly confirm craniocervical posture (CCP), cervical retraction, and craniocervical flexion as screening tests because they contribute to non-specific neck pain (NSNP). We compared the predictive performance of statistical machine learning (ML) models for classifying individuals with and without NSNP using datasets containing CCP and cervical kinematics during pro- and retraction (CKdPR). DESIGN Exploratory, cross-sectional design. SETTING AND PARTICIPANTS In total, 773 public service office workers (PSOWs) were screened for eligibility (NSNP, 441; without NSNP, 332). METHODS We set up five datasets (CCP, cervical kinematics during the protraction, cervical kinematics during the retraction, CKdPR and combination of the CCP and CKdPR). Four ML algorithms-random forest, logistic regression, Extreme Gradient boosting, and support vector machine-were trained. MAIN OUTCOME MEASURES Model performance were assessed using area under the curve (AUC), accuracy, precision, recall and F1-score. To interpret the predictions, we used Feature permutation importance and SHapley Additive explanation values. RESULTS The random forest model in the CKdPR dataset classified PSOWs with and without NSNP and achieved the best AUC among the five datasets using the test data (AUC, 0.892 [good]; F1, 0.832). The random forest model in the CCP dataset had the worst AUC among the five datasets using the test data [AUC, 0.738 (fair); F1, 0.715]. CONCLUSION ML performance was higher for the CKdPR dataset than for the CCP dataset, suggesting that ML algorithms are more suitable than classical statistical methods for developing robust models for classifying PSOWs with and without NSNP.
Collapse
Affiliation(s)
- Ui-Jae Hwang
- Department of Physical Therapy, College of Health Science, Laboratory of KEMA AI Research (KAIR), Yonsei University, Wonju, 26426, Republic of Korea.
| | - Oh-Yun Kwon
- Department of Physical Therapy, College of Health Science, Laboratory of Kinetic Ergocise Based on Movement Analysis, Yonsei University, Wonju, 26426, Republic of Korea.
| | - Jun-Hee Kim
- Department of Physical Therapy, College of Health Science, Laboratory of KEMA AI Research (KAIR), Yonsei University, Wonju, 26426, Republic of Korea.
| | - Sejung Yang
- Department of Precision Medicine, Yonsei University Wonju College of Medicine, Wonju, 26426, Republic of Korea; Department of Medical Informatics and Biostatistics, Graduate School, Yonsei University, Republic of Korea.
| |
Collapse
|
7
|
Moustafa IM, Ozsahin DU, Mustapha MT, Ahbouch A, Oakley PA, Harrison DE. Utilizing machine learning to predict post-treatment outcomes in chronic non-specific neck pain patients undergoing cervical extension traction. Sci Rep 2024; 14:11781. [PMID: 38783089 PMCID: PMC11116459 DOI: 10.1038/s41598-024-62812-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Accepted: 05/21/2024] [Indexed: 05/25/2024] Open
Abstract
This study explored the application of machine learning in predicting post-treatment outcomes for chronic neck pain patients undergoing a multimodal program featuring cervical extension traction (CET). Pre-treatment demographic and clinical variables were used to develop predictive models capable of anticipating modifications in cervical lordotic angle (CLA), pain and disability of 570 patients treated between 2014 and 2020. Linear regression models used pre-treatment variables of age, body mass index, CLA, anterior head translation, disability index, pain score, treatment frequency, duration and compliance. These models used the sci-kit-learn machine learning library within Python for implementing linear regression algorithms. The linear regression models demonstrated high precision and accuracy, and effectively explained 30-55% of the variability in post-treatment outcomes, the highest for the CLA. This pioneering study integrates machine learning into spinal rehabilitation. The developed models offer valuable information to customize interventions, set realistic expectations, and optimize treatment strategies based on individual patient characteristics as treated conservatively with rehabilitation programs using CET as part of multimodal care.
Collapse
Affiliation(s)
- Ibrahim M Moustafa
- Department of Physiotherapy, College of Health Sciences, University of Sharjah, 27272, Sharjah, United Arab Emirates
- Neuromusculoskeletal Rehabilitation Research Group, RIMHS-Research Institute of Medical and Health Sciences, University of Sharjah, 27272, Sharjah, United Arab Emirates
- Faculty of Physical Therapy, Cairo University, Giza, 12613, Egypt
| | - Dilber Uzun Ozsahin
- Department of Medical Diagnostic Imaging, College of Health Science, University of Sharjah, Sharjah, United Arab Emirates
- Operational Research Centre in Healthcare, Near East University, TRNC Mersin 10, 99138, Nicosia, Turkey
- Research Institute for Medical and Health Sciences, University of Sharjah, Sharjah, United Arab Emirates
| | - Mubarak Taiwo Mustapha
- Operational Research Centre in Healthcare, Near East University, TRNC Mersin 10, 99138, Nicosia, Turkey
- Research Institute for Medical and Health Sciences, University of Sharjah, Sharjah, United Arab Emirates
- Department of Biomedical Engineering, Near East University, Nicosia, Mersin 10, Turkey
| | - Amal Ahbouch
- Department of Physiotherapy, College of Health Sciences, University of Sharjah, 27272, Sharjah, United Arab Emirates
- Neuromusculoskeletal Rehabilitation Research Group, RIMHS-Research Institute of Medical and Health Sciences, University of Sharjah, 27272, Sharjah, United Arab Emirates
| | - Paul A Oakley
- CBP Nonprofit (a Spine Research Foundation), Eagle, ID, 83616, USA
- Private Practice, Newmarket, ON, L3Y 8Y8, Canada
- Kinesiology and Health Science, York University, Toronto, ON, M3J 1P3, Canada
| | - Deed E Harrison
- CBP Nonprofit (a Spine Research Foundation), Eagle, ID, 83616, USA.
| |
Collapse
|
8
|
Li B, Aljabri B, Verma R, Beaton D, Hussain MA, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al‐Omran M. Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning. J Am Heart Assoc 2024; 13:e033194. [PMID: 38639373 PMCID: PMC11179886 DOI: 10.1161/jaha.123.033194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 03/01/2024] [Indexed: 04/20/2024]
Abstract
BACKGROUND Lower extremity endovascular revascularization for peripheral artery disease carries nonnegligible perioperative risks; however, outcome prediction tools remain limited. Using machine learning, we developed automated algorithms that predict 30-day outcomes following lower extremity endovascular revascularization. METHODS AND RESULTS The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity endovascular revascularization (angioplasty, stent, or atherectomy) for peripheral artery disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day postprocedural major adverse limb event (composite of major reintervention, untreated loss of patency, or major amputation) or death. Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Overall, 21 886 patients were included, and 30-day major adverse limb event/death occurred in 1964 (9.0%) individuals. The best performing model for predicting 30-day major adverse limb event/death was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.94). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.72 (95% CI, 0.70-0.74). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.09. The top 3 predictive features in our algorithm were (1) chronic limb-threatening ischemia, (2) tibial intervention, and (3) congestive heart failure. CONCLUSIONS Our machine learning models accurately predict 30-day outcomes following lower extremity endovascular revascularization using preoperative data with good discrimination and calibration. Prospective validation is warranted to assess for generalizability and external validity.
Collapse
Affiliation(s)
- Ben Li
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
| | - Badr Aljabri
- Department of SurgeryKing Saud UniversityRiyadhSaudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in IrelandUniversity of Medicine and Health SciencesDublinIreland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoTorontoCanada
| | - Mohamad A. Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women’s HospitalHarvard Medical SchoolBostonMAUSA
| | - Douglas S. Lee
- Division of Cardiology, Peter Munk Cardiac CentreUniversity Health NetworkTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
| | - Duminda N. Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Department of AnesthesiaSt. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Charles de Mestral
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
| | - Muhammad Mamdani
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
- Data Science & Advanced Analytics, Unity Health TorontoUniversity of TorontoTorontoCanada
- Institute of Health Policy, Management and Evaluation, University of TorontoTorontoCanada
- ICES, University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Leslie Dan Faculty of PharmacyUniversity of TorontoTorontoCanada
| | - Mohammed Al‐Omran
- Department of SurgeryUniversity of TorontoCanada
- Division of Vascular SurgerySt. Michael’s Hospital, Unity Health Toronto, University of TorontoTorontoCanada
- Institute of Medical Science, University of TorontoTorontoCanada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T‐CAIREM)University of TorontoTorontoCanada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health TorontoTorontoCanada
- Department of SurgeryKing Faisal Specialist Hospital and Research CenterRiyadhSaudi Arabia
| |
Collapse
|
9
|
Li B, Shaikh F, Zamzam A, Syed MH, Abdin R, Qadura M. A machine learning algorithm for peripheral artery disease prognosis using biomarker data. iScience 2024; 27:109081. [PMID: 38361633 PMCID: PMC10867451 DOI: 10.1016/j.isci.2024.109081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 01/11/2024] [Accepted: 01/26/2024] [Indexed: 02/17/2024] Open
Abstract
Peripheral artery disease (PAD) biomarkers have been studied in isolation; however, an algorithm that considers a protein panel to inform PAD prognosis may improve predictive accuracy. Biomarker-based prediction models were developed and evaluated using a model development (n = 270) and prospective validation cohort (n = 277). Plasma concentrations of 37 proteins were measured at baseline and the patients were followed for 2 years. The primary outcome was 2-year major adverse limb event (MALE; composite of vascular intervention or major amputation). Of the 37 proteins tested, 6 were differentially expressed in patients with vs. without PAD (ADAMTS13, ICAM-1, ANGPTL3, Alpha 1-microglobulin, GDF15, and endostatin). Using 10-fold cross-validation, we developed a random forest machine learning model that accurately predicts 2-year MALE in a prospective validation cohort of PAD patients using a 6-protein panel (AUROC 0.84). This algorithm can support PAD risk stratification, informing clinical decisions on further vascular evaluation and management.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Farah Shaikh
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Abdelrahman Zamzam
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Muzammil H. Syed
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Rawand Abdin
- Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Mohammad Qadura
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Division of Vascular Surgery, St. Michael’s Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada
- Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
10
|
Li B, Eisenberg N, Beaton D, Lee DS, Aljabri B, Wijeysundera DN, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following suprainguinal bypass. J Vasc Surg 2024; 79:593-608.e8. [PMID: 37804954 DOI: 10.1016/j.jvs.2023.09.037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Revised: 09/20/2023] [Accepted: 09/24/2023] [Indexed: 10/09/2023]
Abstract
OBJECTIVE Suprainguinal bypass for peripheral artery disease (PAD) carries important surgical risks; however, outcome prediction tools remain limited. We developed machine learning (ML) algorithms that predict outcomes following suprainguinal bypass. METHODS The Vascular Quality Initiative database was used to identify patients who underwent suprainguinal bypass for PAD between 2003 and 2023. We identified 100 potential predictor variables from the index hospitalization (68 preoperative [demographic/clinical], 13 intraoperative [procedural], and 19 postoperative [in-hospital course/complications]). The primary outcomes were major adverse limb events (MALE; composite of untreated loss of patency, thrombectomy/thrombolysis, surgical revision, or major amputation) or death at 1 year following suprainguinal bypass. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The best performing algorithm was further trained using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, symptom status, procedure type, prior intervention for PAD, concurrent interventions, and urgency. RESULTS Overall, 16,832 patients underwent suprainguinal bypass, and 3136 (18.6%) developed 1-year MALE or death. Patients with 1-year MALE or death were older (mean age, 64.9 vs 63.5 years; P < .001) with more comorbidities, had poorer functional status (65.7% vs 80.9% independent at baseline; P < .001), and were more likely to have chronic limb-threatening ischemia (67.4% vs 47.6%; P < .001) than those without an outcome. Despite being at higher cardiovascular risk, they were less likely to receive acetylsalicylic acid or statins preoperatively and at discharge. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.92 (95% confidence interval [CI], 0.91-0.93). In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). Our XGBoost model maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.93 (95% CI, 0.92-0.94) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (preoperative), 0.11 (intraoperative), and 0.10 (postoperative). Of the top 10 predictors, nine were preoperative features including chronic limb-threatening ischemia, previous procedures, comorbidities, and functional status. Model performance remained robust on all subgroup analyses. CONCLUSIONS We developed ML models that accurately predict outcomes following suprainguinal bypass, performing better than logistic regression. Our algorithms have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes following suprainguinal bypass.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
11
|
Jenkinson AC, Dassios T, Greenough A. Artificial intelligence in the NICU to predict extubation success in prematurely born infants. J Perinat Med 2024; 52:119-125. [PMID: 38059494 DOI: 10.1515/jpm-2023-0454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Accepted: 11/11/2023] [Indexed: 12/08/2023]
Abstract
OBJECTIVES Mechanical ventilation in prematurely born infants, particularly if prolonged, can cause long term complications including bronchopulmonary dysplasia. Timely extubation then is essential, yet predicting its success remains challenging. Artificial intelligence (AI) may provide a potential solution. CONTENT A narrative review was undertaken to explore AI's role in predicting extubation success in prematurely born infants. Across the 11 studies analysed, the range of reported area under the receiver operator characteristic curve (AUC) for the selected prediction models was between 0.7 and 0.87. Only two studies implemented an external validation procedure. Comparison to the results of clinical predictors was made in two studies. One group reported a logistic regression model that outperformed clinical predictors on decision tree analysis, while another group reported clinical predictors outperformed their artificial neural network model (AUCs: ANN 0.68 vs. clinical predictors 0.86). Amongst the studies there was an heterogenous selection of variables for inclusion in prediction models, as well as variations in definitions of extubation failure. SUMMARY Although there is potential for AI to enhance extubation success, no model's performance has yet surpassed that of clinical predictors. OUTLOOK Future studies should incorporate external validation to increase the applicability of the models to clinical settings.
Collapse
Affiliation(s)
- Allan C Jenkinson
- Department of Women and Children's Health, School of Life Course Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
| | - Theodore Dassios
- Department of Women and Children's Health, School of Life Course Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
- Neonatal Intensive Care Centre, King's College Hospital NHS Foundation Trust, London, UK
| | - Anne Greenough
- Department of Women and Children's Health, School of Life Course Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
- Neonatal Intensive Care Centre, King's College Hospital NHS Foundation Trust, London, UK
| |
Collapse
|
12
|
Li B, Verma R, Beaton D, Tamim H, Hussain MA, Hoballah JJ, Lee DS, Wijeysundera DN, de Mestral C, Mamdani M, Al-Omran M. Predicting outcomes following lower extremity open revascularization using machine learning. Sci Rep 2024; 14:2899. [PMID: 38316811 PMCID: PMC10844206 DOI: 10.1038/s41598-024-52944-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 01/25/2024] [Indexed: 02/07/2024] Open
Abstract
Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Institute of Medical Science, University of Toronto, Toronto, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
| | - Hani Tamim
- Faculty of Medicine, Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia
| | - Mohamad A Hussain
- Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, USA
| | - Jamal J Hoballah
- Division of Vascular and Endovascular Surgery, Department of Surgery, American University of Beirut Medical Center, Beirut, Lebanon
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, Canada
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada
- Data Science & Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
- ICES, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, Canada.
- Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, University of Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada.
- Institute of Medical Science, University of Toronto, Toronto, Canada.
- Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, Canada.
- College of Medicine, Alfaisal University, Riyadh, Kingdom of Saudi Arabia.
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.
- Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
13
|
Joseph J, Niemczak C, Lichtenstein J, Kobrina A, Magohe A, Leigh S, Ealer C, Fellows A, Reike C, Massawe E, Gui J, Buckey JC. Central auditory test performance predicts future neurocognitive function in children living with and without HIV. Sci Rep 2024; 14:2712. [PMID: 38302516 PMCID: PMC10834399 DOI: 10.1038/s41598-024-52380-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 01/18/2024] [Indexed: 02/03/2024] Open
Abstract
Tests of the brain's ability to process complex sounds (central auditory tests) correlate with overall measures of neurocognitive performance. In the low- middle-income countries where resources to conduct detailed cognitive testing is limited, tests that assess the central auditory system may provide a novel and useful way to track neurocognitive performance. This could be particularly useful for children living with HIV (CLWH). To evaluate this, we administered central auditory tests to CLWH and children living without HIV and examined whether central auditory tests given early in a child's life could predict later neurocognitive performance. We used a machine learning technique to incorporate factors known to affect performance on neurocognitive tests, such as education. The results show that central auditory tests are useful predictors of neurocognitive performance and perform as well or in some cases better than factors such as education. Central auditory tests may offer an objective way to track neurocognitive performance in CLWH.
Collapse
Affiliation(s)
- Jeff Joseph
- Department of Quantitative Biomedical Sciences, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Christopher Niemczak
- Department of Quantitative Biomedical Sciences, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
- Department of Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
| | - Jonathan Lichtenstein
- Department of Psychiatry, Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
- The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Anastasiya Kobrina
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Albert Magohe
- Muhimbili University of Health and Allied Sciences, Dar es Salaam, Tanzania
| | - Samantha Leigh
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Christin Ealer
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Abigail Fellows
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Catherine Reike
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Enica Massawe
- Muhimbili University of Health and Allied Sciences, Dar es Salaam, Tanzania
| | - Jiang Gui
- Department of Biomedical Data Science, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
| | - Jay C Buckey
- Department of Quantitative Biomedical Sciences, Geisel School of Medicine at Dartmouth, Hanover, NH, USA.
- Space Medicine Innovations Laboratory, Geisel School of Medicine at Dartmouth, Hanover, NH, USA.
- Department of Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA.
| |
Collapse
|
14
|
Zang X, Feng L, Qin W, Wang W, Zang X. Using machine learning methods to analyze the association between urinary polycyclic aromatic hydrocarbons and chronic bowel disorders in American adults. CHEMOSPHERE 2024; 346:140602. [PMID: 37931709 DOI: 10.1016/j.chemosphere.2023.140602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 10/25/2023] [Accepted: 10/31/2023] [Indexed: 11/08/2023]
Abstract
The etiology of chronic bowel disorders is multifaceted, with environmental exposure to harmful substances potentially playing a significant role in their pathogenesis. However, research on the correlation between polycyclic aromatic hydrocarbons (PAHs) and chronic bowel disorders remains limited. Using data from the National Health and Nutrition Examination Survey (NHANES) conducted in 2009-2010, we investigated the relationship between 9 PAHs and chronic diarrhea and constipation in U.S. adults. We employed unsupervised methods such as clustering and Principal Component Analysis (PCA) to identify participants with similar exposure patterns. Additionally, we used supervised learning techniques, namely weighted quantile sum (WQS) and Bayesian kernel machine (BKMR) regressions, to assess the association between PAHs and the occurrence of chronic diarrhea and chronic constipation. PCA identified three principal components in the unsupervised analysis, explaining 86.5% of the total PAH variability. The first component displayed a mild association with chronic diarrhea, but no correlation with chronic constipation. Participants were divided into three clusters via K-means clustering, based on PAH concentrations. Clusters with higher PAH exposure demonstrated an increased odds ratio for chronic diarrhea, but no meaningful connection with chronic constipation. In the supervised analysis, the WQS regression underscored a positive relationship between the PAH mixture and chronic diarrhea, with three PAHs significantly impacting the mixture effect. The mixture index showed no correlation with chronic constipation. BKMR analysis illustrated a positive trend in the impact of four specific PAHs on chronic diarrhea, given other metabolites were fixed at their 50th percentiles. Our results suggest a clear association between higher PAH exposure and an increased risk of chronic diarrhea, but not chronic constipation. It also underscores the potential role of specific PAHs in contributing to the risk of chronic diarrhea.
Collapse
Affiliation(s)
- Xiaodong Zang
- Department of Pediatrics, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230001, China
| | - Liandong Feng
- Hubei Provincial Key Laboratory of Occurrence and Intervention of Rheumatic Diseases, Minda Hospital of Hubei Minzu University, Enshi, 445000, China
| | - Wengang Qin
- Department of Pediatrics, Provincial Hospital Affiliated to Anhui Medical University, Hefei, Anhui, 230001, China
| | - Weilin Wang
- Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230001, China
| | - Xiaowei Zang
- College of Safety Science and Engineering, Nanjing Tech University, Nanjing, 211816, China.
| |
Collapse
|
15
|
Hwang UJ, Kwon OY, Kim JH, Gwak GT. Classification of chronic ankle instability using machine learning technique based on ankle kinematics during heel rise in delivery workers. Digit Health 2024; 10:20552076241235116. [PMID: 38419804 PMCID: PMC10901058 DOI: 10.1177/20552076241235116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 02/07/2024] [Indexed: 03/02/2024] Open
Abstract
Objective Ankle injuries in delivery workers (DWs) are often caused by trips, and high recurrence rates of ankle sprains are related to chronic ankle instability (CAI). Heel rise requires joint angles and moments similar to those of the terminal stance phase of walking that the foot supinates. Thus, our study aimed to develop, determine, and compare the predictive performance of statistical machine learning models to classify DWs with and without CAI using ankle kinematics during heel rise. Methods In total, 203 DWs were screened for eligibility. Seven predictors were included in our study (age, work duration, body mass index, calcaneal stance position angle [CSPA] in the initial and terminal positions during heel rise, calcaneal movement during heel rise [CMHR], and plantar flexion angle during heel rise). Six machine learning algorithms, including logistic regression, decision tree, AdaBoost, Extreme Gradient boosting machines, random forest, and support vector machine, were trained. Results The random forest model (area under the curve [AUC], 0.967 [excellent]; F1, 0.889; accuracy, 0.925) confirmed the best predictive performance in the test datasets among the six machine learning models. For Shapley Additive Explanations, old age, low CMHR, high CSPA in the initial position, high PFA, long work duration, low CSPA in the terminal position, and high body mass index were the most important predictors of CAI in the random forest model. Conclusion Ankle kinematics during heel rise can be considered in the classification of DWs with and without CAI.
Collapse
Affiliation(s)
- Ui-jae Hwang
- Department of Physical Therapy, College of Health Science, Laboratory of KEMA AI Research (KAIR), Yonsei University, Wonju, South Korea
| | - Oh-yun Kwon
- Department of Physical Therapy, College of Health Science, Laboratory of Kinetic Ergocise Based on Movement Analysis, Yonsei University, Wonju, South Korea
| | - Jun-hee Kim
- Department of Physical Therapy, College of Health Science, Laboratory of KEMA AI Research (KAIR), Yonsei University, Wonju, South Korea
| | - Gyeong-tae Gwak
- Department of Physical Therapy, College of Health Science, Laboratory of KEMA AI Research (KAIR), Yonsei University, Wonju, South Korea
| |
Collapse
|
16
|
Dominke C, Fischer AM, Grimmer T, Diehl-Schmid J, Jahn T. CERAD-NAB and flexible battery based neuropsychological differentiation of Alzheimer's dementia and depression using machine learning approaches. NEUROPSYCHOLOGY, DEVELOPMENT, AND COGNITION. SECTION B, AGING, NEUROPSYCHOLOGY AND COGNITION 2024; 31:221-248. [PMID: 36320158 DOI: 10.1080/13825585.2022.2138255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Accepted: 10/14/2022] [Indexed: 11/06/2022]
Abstract
Depression (DEP) and dementia of the Alzheimer's type (DAT) represent the most common neuropsychiatric disorders in elderly patients. Accurate differential diagnosis is indispensable to ensure appropriate treatment. However, DEP can yet mimic cognitive symptoms of DAT and patients with DAT often also present with depressive symptoms, impeding correct diagnosis. Machine learning (ML) approaches could eventually improve this discrimination using neuropsychological test data, but evidence is still missing. We therefore employed Support Vector Machine (SVM), Naïve Bayes (NB), Random Forest (RF) and conventional Logistic Regression (LR) to retrospectively predict the diagnoses of 189 elderly patients (68 DEP and 121 DAT) based on either the well-established Consortium to Establish a Registry for Alzheimer's Disease neuropsychological assessment battery (CERAD-NAB) or a flexible battery approach (FLEXBAT). The best performing combination consisted of FLEXBAT and NB, correctly classifying 87.0% of patients as either DAT or DEP. However, all accuracies were similar across algorithms and test batteries (83.0% - 87.0%). Accordingly, our study is the first to show that common ML algorithms with their default parameters can accurately differentiate between patients clinically diagnosed with DAT or DEP using neuropsychological test data, but do not necessarily outperform conventional LR.
Collapse
Affiliation(s)
- Clara Dominke
- Division Clinical Neuropsychology, Department of Psychology, Ludwig-Maximilians-University, Munich, Germany
| | - Alina Maria Fischer
- School of Medicine, Department of Psychiatry and Psychotherapy, Technical University of Munich, Munich, Germany
| | - Timo Grimmer
- School of Medicine, Department of Psychiatry and Psychotherapy, Technical University of Munich, Munich, Germany
| | - Janine Diehl-Schmid
- School of Medicine, Department of Psychiatry and Psychotherapy, Technical University of Munich, Munich, Germany
- Centre for Geriatric Medicine, Kbo-Inn-Salzach-Klinikum, Wasserburg am Inn, Germany
| | - Thomas Jahn
- Division Clinical Neuropsychology, Department of Psychology, Ludwig-Maximilians-University, Munich, Germany
- School of Medicine, Department of Psychiatry and Psychotherapy, Technical University of Munich, Munich, Germany
| |
Collapse
|
17
|
Climent-Peris VJ, Martí-Bonmatí L, Rodríguez-Ortega A, Doménech-Fernández J. Predictive value of texture analysis on lumbar MRI in patients with chronic low back pain. EUROPEAN SPINE JOURNAL : OFFICIAL PUBLICATION OF THE EUROPEAN SPINE SOCIETY, THE EUROPEAN SPINAL DEFORMITY SOCIETY, AND THE EUROPEAN SECTION OF THE CERVICAL SPINE RESEARCH SOCIETY 2023; 32:4428-4436. [PMID: 37715790 DOI: 10.1007/s00586-023-07936-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2023] [Revised: 08/02/2023] [Accepted: 08/30/2023] [Indexed: 09/18/2023]
Abstract
PURPOSE The aim of this study was to determine whether MRI texture analysis could predict the prognosis of patients with non-specific chronic low back pain. METHODS A prospective observational study was conducted on 100 patients with non-specific chronic low back pain, who underwent a conventional MRI, followed by rehabilitation treatment, and revisited after 6 months. Sociodemographic variables, numeric pain scale (NPS) value, and the degree of disability as measured by the Roland-Morris disability questionnaire (RMDQ), were collected. The MRI analysis included segmentation of regions of interest (vertebral endplates and intervertebral disks from L3-L4 to L5-S1, paravertebral musculature at the L4-L5 space) to extract texture variables (PyRadiomics software). The classification random forest algorithm was applied to identify individuals who would improve less than 30% in the NPS or would score more than 4 in the RMDQ at the end of the follow-up. Sensitivity, specificity, and the area under the ROC curve were calculated. RESULTS The final series included 94 patients. The predictive model for classifying patients whose pain did not improve by 30% or more offered a sensitivity of 0.86, specificity 0.57, and area under the ROC curve 0.71. The predictive model for classifying patients with a RMDQ score 4 or more offered a sensitivity of 0.83, specificity of 0.20, and area under the ROC curve of 0.52. CONCLUSION The texture analysis of lumbar MRI could help identify patients who are more likely to improve their non-specific chronic low back pain through rehabilitation programs, allowing a personalized therapeutic plan to be established.
Collapse
Affiliation(s)
| | - Luís Martí-Bonmatí
- Medical Imaging Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain
| | | | | |
Collapse
|
18
|
Spielman LA, Maruta J, Ghajar J. Dual statistical models link baseline visual attention measure to risk for significant symptomatic concussion in sports. Concussion 2023; 8:CNC112. [PMID: 38855758 PMCID: PMC10945612 DOI: 10.2217/cnc-2023-0002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 12/01/2023] [Indexed: 06/11/2024] Open
Abstract
Aim Athletic pre-season testing can establish functional baseline for comparison following concussion. Whether impacts of future concussions may be foretold by such testing is little known. Materials & methods Two sets of models for a significant burden of concussion were generated: a traditional approach using a series of logistic regressions, and a penalized regression approach using elastic net. Results 3091 youth and adult athletes were baseline-assessed. 90 subsequently experienced concussion and 35 were still experiencing a significant burden of concussion when tested within two weeks. Both models associated prior history of head injury and visual attention-related metrics with a significant burden of concussion. Conclusion Pre-season testing of visual attention may identify athletes who are at risk for significant sports-related concussion.
Collapse
Affiliation(s)
- Lisa A Spielman
- Department of Rehabilitation & Human Performance, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Jun Maruta
- Department of Rehabilitation & Human Performance, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
- Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | | |
Collapse
|
19
|
Li B, Aljabri B, Verma R, Beaton D, Eisenberg N, Lee DS, Wijeysundera DN, Forbes TL, Rotstein OD, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following open abdominal aortic aneurysm repair. J Vasc Surg 2023; 78:1426-1438.e6. [PMID: 37634621 DOI: 10.1016/j.jvs.2023.08.121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 08/15/2023] [Accepted: 08/19/2023] [Indexed: 08/29/2023]
Abstract
OBJECTIVE Prediction of outcomes following open abdominal aortic aneurysm (AAA) repair remains challenging with a lack of widely used tools to guide perioperative management. We developed machine learning (ML) algorithms that predict outcomes following open AAA repair. METHODS The Vascular Quality Initiative (VQI) database was used to identify patients who underwent elective open AAA repair between 2003 and 2023. Input features included 52 preoperative demographic/clinical variables. All available preoperative variables from VQI were used to maximize predictive performance. The primary outcome was in-hospital major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death). Secondary outcomes were individual components of the primary outcome, other in-hospital complications, and 1-year mortality and any reintervention. We split our data into training (70%) and test (30%) sets. Using 10-fold cross-validation, six ML models were trained using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. The top 10 predictive features in our final model were determined based on variable importance scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median area deprivation index, proximal clamp site, prior aortic surgery, and concomitant procedures. RESULTS Overall, 12,027 patients were included. The primary outcome of in-hospital MACE occurred in 630 patients (5.2%). Compared with patients without a primary outcome, those who developed in-hospital MACE were older with more comorbidities, demonstrated poorer functional status, had more complex aneurysms, and were more likely to require concomitant procedures. Our best performing prediction model for in-hospital MACE was XGBoost, achieving an AUROC of 0.93 (95% confidence interval, 0.92-0.94). Comparatively, logistic regression had an AUROC of 0.71 (95% confidence interval, 0.70-0.73). For secondary outcomes, XGBoost achieved AUROCs between 0.84 and 0.94. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.05. These findings highlight the excellent predictive performance of the XGBoost model. The top three predictive features in our algorithm for in-hospital MACE following open AAA repair were: (1) coronary artery disease; (2) American Society of Anesthesiologists classification; and (3) proximal clamp site. Model performance remained robust on all subgroup analyses. CONCLUSIONS Open AAA repair outcomes can be accurately predicted using preoperative data with our ML models, which perform better than logistic regression. Our automated algorithms can help guide risk-mitigation strategies for patients being considered for open AAA repair to improve outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Badr Aljabri
- Department of Surgery, King Saud University, Riyadh, Kingdom of Saudi Arabia
| | - Raj Verma
- School of Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, Dublin, Ireland
| | - Derek Beaton
- Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Thomas L Forbes
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Ori D Rotstein
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Division of General Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science and Advanced Analytics, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
20
|
Scebba F, Salvadori S, Cateni S, Mantellini P, Carozzi F, Bisanzi S, Sani C, Robotti M, Barravecchia I, Martella F, Colla V, Angeloni D. Top-Down Proteomics of Human Saliva, Analyzed with Logistic Regression and Machine Learning Methods, Reveal Molecular Signatures of Ovarian Cancer. Int J Mol Sci 2023; 24:15716. [PMID: 37958700 PMCID: PMC10648137 DOI: 10.3390/ijms242115716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 10/11/2023] [Accepted: 10/16/2023] [Indexed: 11/15/2023] Open
Abstract
Ovarian cancer (OC) is the most lethal of all gynecological cancers. Due to vague symptoms, OC is mostly detected at advanced stages, with a 5-year survival rate (SR) of only 30%; diagnosis at stage I increases the 5-year SR to 90%, suggesting that early diagnosis is essential to cure OC. Currently, the clinical need for an early, reliable diagnostic test for OC screening remains unmet; indeed, screening is not even recommended for healthy women with no familial history of OC for fear of post-screening adverse events. Salivary diagnostics is considered a major resource for diagnostics of the future. In this work, we searched for OC biomarkers (BMs) by comparing saliva samples of patients with various stages of OC, breast cancer (BC) patients, and healthy subjects using an unbiased, high-throughput proteomics approach. We analyzed the results using both logistic regression (LR) and machine learning (ML) for pattern analysis and variable selection to highlight molecular signatures for OC and BC diagnosis and possibly re-classification. Here, we show that saliva is an informative test fluid for an unbiased proteomic search of candidate BMs for identifying OC patients. Although we were not able to fully exploit the potential of ML methods due to the small sample size of our study, LR and ML provided patterns of candidate BMs that are now available for further validation analysis in the relevant population and for biochemical identification.
Collapse
Affiliation(s)
- Francesca Scebba
- Health Science Interdisciplinary Center, Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy;
| | - Stefano Salvadori
- Institute of Clinical Physiology, National Research Council, Via G. Moruzzi, 1, 56124 Pisa, Italy;
| | - Silvia Cateni
- Center for Information and Communication Technologies for Complex Industrial Systems and Processes (ICT-COISP), Telecommunications, Computer Engineering, and Photonics Institute (TeCIP), Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy; (S.C.); (V.C.)
| | - Paola Mantellini
- Istituto per lo Studio, la Prevenzione e la Rete Oncologica (ISPRO), Via Cosimo il Vecchio, 2, 50139 Firenze, Italy; (P.M.); (F.C.); (S.B.); (C.S.)
| | - Francesca Carozzi
- Istituto per lo Studio, la Prevenzione e la Rete Oncologica (ISPRO), Via Cosimo il Vecchio, 2, 50139 Firenze, Italy; (P.M.); (F.C.); (S.B.); (C.S.)
| | - Simonetta Bisanzi
- Istituto per lo Studio, la Prevenzione e la Rete Oncologica (ISPRO), Via Cosimo il Vecchio, 2, 50139 Firenze, Italy; (P.M.); (F.C.); (S.B.); (C.S.)
| | - Cristina Sani
- Istituto per lo Studio, la Prevenzione e la Rete Oncologica (ISPRO), Via Cosimo il Vecchio, 2, 50139 Firenze, Italy; (P.M.); (F.C.); (S.B.); (C.S.)
| | - Marzia Robotti
- Ph.D. School in Translational Medicine, Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy;
| | - Ivana Barravecchia
- The Institute of Biorobotics, Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy;
| | - Francesca Martella
- Breast Unit and SOC Oncologia Medica Firenze—Dipartimento Oncologico, Azienda Usl Toscana Centro, Ospedale Santa Maria Annunziata, Via dell’Antella, 58, 50012 Firenze, Italy;
| | - Valentina Colla
- Center for Information and Communication Technologies for Complex Industrial Systems and Processes (ICT-COISP), Telecommunications, Computer Engineering, and Photonics Institute (TeCIP), Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy; (S.C.); (V.C.)
| | - Debora Angeloni
- Health Science Interdisciplinary Center, Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy;
- Ph.D. School in Translational Medicine, Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy;
- The Institute of Biorobotics, Scuola Superiore Sant’Anna, Via G. Moruzzi, 1, 56124 Pisa, Italy;
| |
Collapse
|
21
|
Ratna MB, Bhattacharya S, McLernon DJ. External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment. Hum Reprod 2023; 38:1998-2010. [PMID: 37632223 PMCID: PMC10546080 DOI: 10.1093/humrep/dead165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 07/28/2023] [Indexed: 08/27/2023] Open
Abstract
STUDY QUESTION Can two prediction models developed using data from 1999 to 2009 accurately predict the cumulative probability of live birth per woman over multiple complete cycles of IVF in an updated UK cohort? SUMMARY ANSWER After being updated, the models were able to estimate individualized chances of cumulative live birth over multiple complete cycles of IVF with greater accuracy. WHAT IS KNOWN ALREADY The McLernon models were the first to predict cumulative live birth over multiple complete cycles of IVF. They were converted into an online calculator called OPIS (Outcome Prediction In Subfertility) which has 3000 users per month on average. A previous study externally validated the McLernon models using a Dutch prospective cohort containing data from 2011 to 2014. With changes in IVF practice over time, it is important that the McLernon models are externally validated on a more recent cohort of patients to ensure that predictions remain accurate. STUDY DESIGN, SIZE, DURATION A population-based cohort of 91 035 women undergoing IVF in the UK between January 2010 and December 2016 was used for external validation. Data on frozen embryo transfers associated with these complete IVF cycles conducted from 1 January 2017 to 31 December 2017 were also collected. PARTICIPANTS/MATERIALS, SETTING, METHODS Data on IVF treatments were obtained from the Human Fertilisation and Embryology Authority (HFEA). The predictive performances of the McLernon models were evaluated in terms of discrimination and calibration. Discrimination was assessed using the c-statistic and calibration was assessed using calibration-in-the-large, calibration slope, and calibration plots. Where any model demonstrated poor calibration in the validation cohort, the models were updated using intercept recalibration, logistic recalibration, or model revision to improve model performance. MAIN RESULTS AND THE ROLE OF CHANCE Following exclusions, 91 035 women who underwent 144 734 complete cycles were included. The validation cohort had a similar distribution age profile to women in the development cohort. Live birth rates over all complete cycles of IVF per woman were higher in the validation cohort. After calibration assessment, both models required updating. The coefficients of the pre-treatment model were revised, and the updated model showed reasonable discrimination (c-statistic: 0.67, 95% CI: 0.66 to 0.68). After logistic recalibration, the post-treatment model showed good discrimination (c-statistic: 0.75, 95% CI: 0.74 to 0.76). As an example, in the updated pre-treatment model, a 32-year-old woman with 2 years of primary infertility has a 42% chance of having a live birth in the first complete ICSI cycle and a 77% chance over three complete cycles. In a couple with 2 years of primary male factor infertility where a 30-year-old woman has 15 oocytes collected in the first cycle, a single fresh blastocyst embryo transferred in the first cycle and spare embryos cryopreserved, the estimated chance of live birth provided by the post-treatment model is 46% in the first complete ICSI cycle and 81% over three complete cycles. LIMITATIONS, REASONS FOR CAUTION Two predictors from the original models, duration of infertility and previous pregnancy, which were not available in the recent HFEA dataset, were imputed using data from the older cohort used to develop the models. The HFEA dataset does not contain some other potentially important predictors, e.g. BMI, ethnicity, race, smoking and alcohol intake in women, as well as measures of ovarian reserve such as antral follicle count. WIDER IMPLICATIONS OF THE FINDINGS Both updated models show improved predictive ability and provide estimates which are more reflective of current practice and patient case mix. The updated OPIS tool can be used by clinicians to help shape couples' expectations by informing them of their individualized chances of live birth over a sequence of multiple complete cycles of IVF. STUDY FUNDING/COMPETING INTEREST(S) This study was supported by an Elphinstone scholarship scheme at the University of Aberdeen and Aberdeen Fertility Centre, University of Aberdeen. S.B. has a commitment of research funding from Merck. D.J.M. and M.B.R. declare support for the present manuscript from Elphinstone scholarship scheme at the University of Aberdeen and Assisted Reproduction Unit at Aberdeen Fertility Centre, University of Aberdeen. D.J.M. declares grants received by University of Aberdeen from NHS Grampian, The Meikle Foundation, and Chief Scientist Office in the past 3 years. D.J.M. declares receiving an honorarium for lectures from Merck. D.J.M. is Associate Editor of Human Reproduction Open and Statistical Advisor for Reproductive BioMed Online. S.B. declares royalties from Cambridge University Press for a book. S.B. declares receiving an honorarium for lectures from Merck, Organon, Ferring, Obstetric and Gynaecological Society of Singapore, and Taiwanese Society for Reproductive Medicine. S.B. has received support from Merck, ESHRE, and Ferring for attending meetings as speaker and is on the METAFOR and CAPRE Trials Data Monitoring Committee. TRIAL REGISTRATION NUMBER N/A.
Collapse
Affiliation(s)
- Mariam B Ratna
- Institute of Applied Health Sciences, School of Medicine, Medical Sciences & Nutrition, University of Aberdeen, Aberdeen, UK
- Clinical Trials Unit, Warwick Medical School, University of Warwick, Warwick, UK
| | | | - David J McLernon
- Institute of Applied Health Sciences, School of Medicine, Medical Sciences & Nutrition, University of Aberdeen, Aberdeen, UK
| |
Collapse
|
22
|
Li B, Beaton D, Eisenberg N, Lee DS, Wijeysundera DN, Lindsay TF, de Mestral C, Mamdani M, Roche-Nagle G, Al-Omran M. Using machine learning to predict outcomes following carotid endarterectomy. J Vasc Surg 2023; 78:973-987.e6. [PMID: 37211142 DOI: 10.1016/j.jvs.2023.05.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 05/08/2023] [Accepted: 05/13/2023] [Indexed: 05/23/2023]
Abstract
OBJECTIVE Prediction of outcomes following carotid endarterectomy (CEA) remains challenging, with a lack of standardized tools to guide perioperative management. We used machine learning (ML) to develop automated algorithms that predict outcomes following CEA. METHODS The Vascular Quality Initiative (VQI) database was used to identify patients who underwent CEA between 2003 and 2022. We identified 71 potential predictor variables (features) from the index hospitalization (43 preoperative [demographic/clinical], 21 intraoperative [procedural], and 7 postoperative [in-hospital complications]). The primary outcome was stroke or death at 1 year following CEA. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, insurance status, symptom status, and urgency of surgery. RESULTS Overall, 166,369 patients underwent CEA during the study period. In total, 7749 patients (4.7%) had the primary outcome of stroke or death at 1 year. Patients with an outcome were older with more comorbidities, had poorer functional status, and demonstrated higher risk anatomic features. They were also more likely to undergo intraoperative surgical re-exploration and have in-hospital complications. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.90 (95% confidence interval [CI], 0.89-0.91). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. Our XGBoost models maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.90 (95% CI, 0.89-0.91) and 0.94 (95% CI, 0.93-0.95), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.15 (preoperative), 0.14 (intraoperative), and 0.11 (postoperative). Of the top 10 predictors, eight were preoperative features, including comorbidities, functional status, and previous procedures. Model performance remained robust on all subgroup analyses. CONCLUSIONS We developed ML models that accurately predict outcomes following CEA. Our algorithms perform better than logistic regression and existing tools, and therefore, have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes.
Collapse
Affiliation(s)
- Ben Li
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada
| | - Derek Beaton
- Data Science and Advanced Analytics Department, Unity Health Toronto, University of Toronto, Toronto, ON, Canada
| | - Naomi Eisenberg
- Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Douglas S Lee
- Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada
| | - Duminda N Wijeysundera
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Department of Anesthesia, St Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Thomas F Lindsay
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Charles de Mestral
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Data Science and Advanced Analytics Department, Unity Health Toronto, University of Toronto, Toronto, ON, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada; ICES, University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Graham Roche-Nagle
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
| | - Mohammed Al-Omran
- Department of Surgery, University of Toronto, Toronto, ON, Canada; Division of Vascular Surgery, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Institute of Medical Science, University of Toronto, Toronto, ON, Canada; Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM), University of Toronto, Toronto, ON, Canada; Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Surgery, King Faisal Specialist Hospital and Research Center, Riyadh, Kingdom of Saudi Arabia.
| |
Collapse
|
23
|
Liew BXW, Kovacs FM, Rügamer D, Royuela A. Automatic Variable Selection Algorithms in Prognostic Factor Research in Neck Pain. J Clin Med 2023; 12:6232. [PMID: 37834877 PMCID: PMC10573798 DOI: 10.3390/jcm12196232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 09/21/2023] [Accepted: 09/26/2023] [Indexed: 10/15/2023] Open
Abstract
This study aims to compare the variable selection strategies of different machine learning (ML) and statistical algorithms in the prognosis of neck pain (NP) recovery. A total of 3001 participants with NP were included. Three dichotomous outcomes of an improvement in NP, arm pain (AP), and disability at 3 months follow-up were used. Twenty-five variables (twenty-eight parameters) were included as predictors. There were more parameters than variables, as some categorical variables had >2 levels. Eight modelling techniques were compared: stepwise regression based on unadjusted p values (stepP), on adjusted p values (stepPAdj), on Akaike information criterion (stepAIC), best subset regression (BestSubset) least absolute shrinkage and selection operator [LASSO], Minimax concave penalty (MCP), model-based boosting (mboost), and multivariate adaptive regression splines (MuARS). The algorithm that selected the fewest predictors was stepPAdj (number of predictors, p = 4 to 8). MuARS was the algorithm with the second fewest predictors selected (p = 9 to 14). The predictor selected by all algorithms with the largest coefficient magnitude was "having undergone a neuroreflexotherapy intervention" for NP (β = from 1.987 to 2.296) and AP (β = from 2.639 to 3.554), and "Imaging findings: spinal stenosis" (β = from -1.331 to -1.763) for disability. Stepwise regression based on adjusted p-values resulted in the sparsest models, which enhanced clinical interpretability. MuARS appears to provide the optimal balance between model sparsity whilst retaining high predictive performance across outcomes. Different algorithms produced similar performances but resulted in a different number of variables selected. Rather than relying on any single algorithm, confidence in the variable selection may be increased by using multiple algorithms.
Collapse
Affiliation(s)
- Bernard X. W. Liew
- School of Sport, Rehabilitation and Exercise Sciences, University of Essex, Colchester CO4 3SQ, Essex, UK
| | - Francisco M. Kovacs
- Unidad de la Espalda Kovacs, HLA-Moncloa University Hospital, 28008 Madrid, Spain;
| | - David Rügamer
- Department of Statistics, Ludwig-Maximilians-Universität München, 80539 Munich, Germany;
| | - Ana Royuela
- Biostatistics Unit, Hospital Puerta de Hierro, Instituto Investigación Sanitaria Puerta de Hierro-Segovia de Arana, Consorcio de Investigación Biomédica en Red de Epidemiología y Salud Pública, Red Española de Investigadores en Dolencias de la Espalda, 28222 Madrid, Spain;
| |
Collapse
|
24
|
Hoffman M, Lanza J, Simon SJ, Schoeller L, Fang C, Coden G, Hollenbeck B. Risk Factors for Surgical Site Complications After Outpatient Lumbar Spine Surgery. Surg Infect (Larchmt) 2023. [PMID: 37437125 DOI: 10.1089/sur.2023.040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/14/2023] Open
Abstract
Background: With the recent increase of minor lumbar spine surgeries being performed in the outpatient setting, there is a need for information on factors that contribute to post-operative complications for these surgeries. Patients and Methods: This was a prospective observational study examining risk factors for self-reported post-operative drainage in patients who underwent lumbar spine surgery. Patient surveys and the hospital's electronic medical records were used to collect data on patient demographic, patient lifestyle, and surgical variables. Univariable and multivariable analyses in addition to a random forest classifier were performed. Results: A total of 146 patients were enrolled in the study with 111 patients included in the final analysis. The average age and body mass index (BMI) of these patients was 66 and 27.8, respectively. None of the 146 patients in this study developed surgical site infection. Older age, no steroid use, no pet ownership, and spine surgery involving two or more levels were all found to be risk factors for wound drainage. Conclusions: This study evaluated lifestyle, environmental, and traditional risk factors for surgical site drainage that have not been explored cohesively related to outpatient orthopedic surgery. Consistent with the existing literature, outpatient spine surgery involving two or more levels was most strongly associated with surgical site drainage after surgery.
Collapse
Affiliation(s)
- Megan Hoffman
- Division of Research, New England Baptist Hospital, Boston Massachusetts, USA
| | - Jennifer Lanza
- Division of Research, New England Baptist Hospital, Boston Massachusetts, USA
| | - Samantha J Simon
- Division of Research, New England Baptist Hospital, Boston Massachusetts, USA
| | - Lauren Schoeller
- Division of Research, New England Baptist Hospital, Boston Massachusetts, USA
| | - Christopher Fang
- Division of Research, New England Baptist Hospital, Boston Massachusetts, USA
| | - Gloria Coden
- Division of Research, New England Baptist Hospital, Boston Massachusetts, USA
| | - Brian Hollenbeck
- Division of Infectious Diseases, New England Baptist Hospital, Boston Massachusetts, USA
| |
Collapse
|
25
|
Dosis A, Helliwell J, Syversen A, Tiernan J, Zhang Z, Jayne D. Estimating postoperative mortality in colorectal surgery- a systematic review of risk prediction models. Int J Colorectal Dis 2023; 38:155. [PMID: 37261539 DOI: 10.1007/s00384-023-04455-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/25/2023] [Indexed: 06/02/2023]
Abstract
PURPOSE Risk prediction models are frequently used to support decision-making in colorectal surgery but can be inaccurate. Machine learning (ML) is becoming increasingly popular, and its application may increase predictive accuracy. We compared conventional risk prediction models for postoperative mortality (based on regression analysis) with ML models to determine the benefit of the latter approach. METHODS The study was registered in PROSPERO(CRD42022364753). Following the PRISMA guidelines, a systematic search of three databases (MEDLINE, EMBASE, WoS) was conducted (from 1/1/2000 to 29/09/2022). Studies were included if they reported the development of a risk model to estimate short-term postoperative mortality for patients undergoing colorectal surgery. Discrimination and calibration performance metrics were compared. Studies were evaluated against CHARMS and TRIPOD criteria. RESULTS 3,052 articles were screened, and 45 studies were included. The total sample size was 1,356,058 patients. Six studies used ML techniques for model development. Most studies (n = 42) reported the area under the receiver operating characteristic curve (AUROC) as a measure of discrimination. There was no significant difference in the mean AUROC values between regression models (0.833 s.d. ± 0.52) and ML (0.846 s.d. ± 0.55), p = 0.539. Calibration statistics, which measure the agreement between predicted estimates and observed outcomes, were less consistent. Risk of bias assessment found most concerns in the data handling and analysis domains of eligible studies. CONCLUSIONS Our study showed comparable predictive performance between regression and ML methods in colorectal surgery. Integration of ML in colorectal risk prediction is promising but further refinement of the models is required to support routine clinical adoption.
Collapse
Affiliation(s)
| | | | | | - Jim Tiernan
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | | |
Collapse
|
26
|
Liu Y, Ko CY, Hall BL, Cohen ME. American College of Surgeons NSQIP Risk Calculator Accuracy Using a Machine Learning Algorithm Compared with Regression. J Am Coll Surg 2023; 236:1024-1030. [PMID: 36728295 DOI: 10.1097/xcs.0000000000000556] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
BACKGROUND The American College of Surgeons NSQIP risk calculator (RC) uses regression to make predictions for fourteen 30-day surgical outcomes. While this approach provides accurate (discrimination and calibration) risk estimates, they might be improved by machine learning (ML). To investigate this possibility, accuracy for regression-based risk estimates were compared to estimates from an extreme gradient boosting (XGB)-ML algorithm. STUDY DESIGN A cohort of 5,020,713 million NSQIP patient records was randomly divided into 80% for model construction and 20% for validation. Risk predictions using regression and XGB-ML were made for 13 RC binary 30-day surgical complications and one continuous outcome (length of stay [LOS]). For the binary outcomes, discrimination was evaluated using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC), and calibration was evaluated using Hosmer-Lemeshow statistics. Mean squared error and a calibration curve analog were evaluated for the continuous LOS outcome. RESULTS For every binary outcome, discrimination (AUROC and AUPRC) was slightly greater for XGB-ML than for regression (mean [across the outcomes] AUROC was 0.8299 vs 0.8251, and mean AUPRC was 0.1558 vs 0.1476, for XGB-ML and regression, respectively). For each outcome, miscalibration was greater (larger Hosmer-Lemeshow values) with regression; there was statistically significant miscalibration for all regression-based estimates, but only for 4 of 13 when XGB-ML was used. For LOS, mean squared error was lower for XGB-ML. CONCLUSIONS XGB-ML provided more accurate risk estimates than regression in terms of discrimination and calibration. Differences in calibration between regression and XGB-ML were of substantial magnitude and support transitioning the RC to XGB-ML.
Collapse
Affiliation(s)
- Yaoming Liu
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
| | - Clifford Y Ko
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
- the Department of Surgery, University of California Los Angeles David Geffen School of Medicine and the VA Greater Los Angeles Healthcare System, Los Angeles, CA (Ko)
| | - Bruce L Hall
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
- the Department of Surgery, Washington University in St. Louis; Center for Health Policy and the Olin Business School at Washington University in St Louis; John Cochran Veterans Affairs Medical Center; and BJC Healthcare, St Louis, MO (Hall)
| | - Mark E Cohen
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
| |
Collapse
|
27
|
Eysenbach G, Chao HJ, Chiang YC, Chen HY. Explainable Machine Learning Techniques To Predict Amiodarone-Induced Thyroid Dysfunction Risk: Multicenter, Retrospective Study With External Validation. J Med Internet Res 2023; 25:e43734. [PMID: 36749620 PMCID: PMC9944157 DOI: 10.2196/43734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Revised: 12/25/2022] [Accepted: 01/16/2023] [Indexed: 02/08/2023] Open
Abstract
BACKGROUND Machine learning offers new solutions for predicting life-threatening, unpredictable amiodarone-induced thyroid dysfunction. Traditional regression approaches for adverse-effect prediction without time-series consideration of features have yielded suboptimal predictions. Machine learning algorithms with multiple data sets at different time points may generate better performance in predicting adverse effects. OBJECTIVE We aimed to develop and validate machine learning models for forecasting individualized amiodarone-induced thyroid dysfunction risk and to optimize a machine learning-based risk stratification scheme with a resampling method and readjustment of the clinically derived decision thresholds. METHODS This study developed machine learning models using multicenter, delinked electronic health records. It included patients receiving amiodarone from January 2013 to December 2017. The training set was composed of data from Taipei Medical University Hospital and Wan Fang Hospital, while data from Taipei Medical University Shuang Ho Hospital were used as the external test set. The study collected stationary features at baseline and dynamic features at the first, second, third, sixth, ninth, 12th, 15th, 18th, and 21st months after amiodarone initiation. We used 16 machine learning models, including extreme gradient boosting, adaptive boosting, k-nearest neighbor, and logistic regression models, along with an original resampling method and 3 other resampling methods, including oversampling with the borderline-synthesized minority oversampling technique, undersampling-edited nearest neighbor, and over- and undersampling hybrid methods. The model performance was compared based on accuracy; Precision, recall, F1-score, geometric mean, area under the curve of the receiver operating characteristic curve (AUROC), and the area under the precision-recall curve (AUPRC). Feature importance was determined by the best model. The decision threshold was readjusted to identify the best cutoff value and a Kaplan-Meier survival analysis was performed. RESULTS The training set contained 4075 patients from Taipei Medical University Hospital and Wan Fang Hospital, of whom 583 (14.3%) developed amiodarone-induced thyroid dysfunction, while the external test set included 2422 patients from Taipei Medical University Shuang Ho Hospital, of whom 275 (11.4%) developed amiodarone-induced thyroid dysfunction. The extreme gradient boosting oversampling machine learning model demonstrated the best predictive outcomes among all 16 models. The accuracy; Precision, recall, F1-score, G-mean, AUPRC, and AUROC were 0.923, 0.632, 0.756, 0.688, 0.845, 0.751, and 0.934, respectively. After readjusting the cutoff, the best value was 0.627, and the F1-score reached 0.699. The best threshold was able to classify 286 of 2422 patients (11.8%) as high-risk subjects, among which 275 were true-positive patients in the testing set. A shorter treatment duration; higher levels of thyroid-stimulating hormone and high-density lipoprotein cholesterol; and lower levels of free thyroxin, alkaline phosphatase, and low-density lipoprotein were the most important features. CONCLUSIONS Machine learning models combined with resampling methods can predict amiodarone-induced thyroid dysfunction and serve as a support tool for individualized risk prediction and clinical decision support.
Collapse
Affiliation(s)
| | - Horng-Jiun Chao
- Department of Clinical Pharmacy, School of Pharmacy, Taipei Medical University, Taipei, Taiwan
| | - Yi-Chun Chiang
- Department of Clinical Pharmacy, School of Pharmacy, Taipei Medical University, Taipei, Taiwan.,Department of Pharmacy, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
| | - Hsiang-Yin Chen
- Department of Clinical Pharmacy, School of Pharmacy, Taipei Medical University, Taipei, Taiwan.,Department of Pharmacy, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
| |
Collapse
|
28
|
Lin C, Pan LF, He ZQ, Hsu S. Early prediction of 30- and 14-day all-cause unplanned readmissions. Health Informatics J 2023; 29:14604582231164694. [PMID: 36913624 DOI: 10.1177/14604582231164694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/15/2023]
Abstract
BACKGROUND An unplanned readmission is a dual metric for both the cost and quality of medical care. METHODS We employed the random forest (RF) method to build a prediction model using a large dataset from patients' electronic health records (EHRs) from a medical center in Taiwan. The discrimination abilities between the RF and regression-based models were compared using the areas under the ROC curves (AUROC). RESULTS When compared with standardized risk prediction tools, the RF constructed using data readily available at admission had a marginally yet significantly better ability to identify high-risk readmissions within 30 and 14 days without compromising sensitivity and specificity. The most important predictor for 30-day readmissions was directly related to the representing factors of index hospitalization, whereas for 14-day readmissions the most important predictor was associated with a higher chronic illness burden. CONCLUSIONS Identifying dominant risk factors based on index admission and different readmission time intervals is crucial for healthcare planning.
Collapse
Affiliation(s)
- Chaohsin Lin
- Department of Risk Management and Insurance, 517768National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan
| | - Li-Fei Pan
- Department of General Affairs Administration, 38024Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan
| | - Zuo-Quan He
- Department of Risk Management and Insurance, 517768National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan
| | - Shuofen Hsu
- Department of Risk Management and Insurance, 517768National Kaohsiung University of Science and Technology, Kaohsiung, Taiwan
| |
Collapse
|
29
|
Hasegawa S, Mizokami F, Kameya Y, Hayakawa Y, Watanabe T, Matsui Y. Machine learning versus binomial logistic regression analysis for fall risk based on SPPB scores in older adult outpatients. Digit Health 2023; 9:20552076231219438. [PMID: 38107982 PMCID: PMC10722919 DOI: 10.1177/20552076231219438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/21/2023] [Indexed: 12/19/2023] Open
Abstract
Objective To compare the performance of the diagnostic model for fall risk based on the short physical performance battery (SPPB) developed using commercial machine learning software (MLS) and binomial logistic regression analysis (BLRA). Methods We enrolled 797 out of 850 outpatients who visited the clinic between March 2016 and November 2021. Patients were categorized into the development (n = 642) and validation (n = 155) datasets. Age, sex, number of comorbidities, number of medications, body mass index (BMI), calf circumference (left-right average), handgrip strength (left-right average), total SPPB score, and history of falls were determined. We defined fall risk by an SPPB score of ≤6 in men and ≤9 in women. The main metrics used for evaluating the machine learning model and BLRA were the area under the curve (AUC), accuracy, precision, recall (sensitivity), specificity, and F-measure. The commercial MLS automatically calculates the parameter range of the highest contribution. Results The participants included 797 outpatients (mean age, 76.3 years; interquartile range, 73.0-81.0; 288 men). The metrics of the current diagnostic model in the commercial MLS were as follows: AUC = 0.78, accuracy = 0.74, precision = 0.46, recall (sensitivity) = 0.81, specificity = 0.71, F-measure = 0.59. The metrics of the current diagnostic model in the BLRA were as follows: AUC = 0.77, accuracy = 0.75, precision = 0.47, recall (sensitivity) = 0.67, specificity = 0.77, F-measure = 0.55. The risk factors for falls in older adult outpatients were handgrip strength, female sex, experience of falls, BMI, and calf circumference in the commercial MLS. Conclusions The diagnostic model for fall risk based on SPPB scores constructed using commercial MLS is noninferior to BLRA.
Collapse
Affiliation(s)
- Sho Hasegawa
- Department of Pharmacotherapeutics and Informatics, Fujita Health University, Toyoake, Japan
- Department of Education and Innovation, Training for Pharmacy, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Fumihiro Mizokami
- Department of Education and Innovation, Training for Pharmacy, National Center for Geriatrics and Gerontology, Obu, Japan
- Department of Pharmacy, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Yoshitaka Kameya
- Department of Information Engineering, Meijo University, Nagoya, Japan
| | - Yuji Hayakawa
- Department of Education and Innovation, Training for Pharmacy, National Center for Geriatrics and Gerontology, Obu, Japan
- Department of Pharmacy, National Hospital Organization, Nagoya Medical Center, Nagoya, Japan
| | - Tsuyoshi Watanabe
- Department of Orthopaedic Surgery, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Yasumoto Matsui
- Center for Frailty and Locomotive Syndrome, National Center for Geriatrics and Gerontology, Obu, Japan
| |
Collapse
|
30
|
Machine Learning Approach for Metabolic Syndrome Diagnosis Using Explainable Data-Augmentation-Based Classification. Diagnostics (Basel) 2022; 12:diagnostics12123117. [PMID: 36553124 PMCID: PMC9777696 DOI: 10.3390/diagnostics12123117] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Revised: 12/02/2022] [Accepted: 12/03/2022] [Indexed: 12/14/2022] Open
Abstract
Metabolic syndrome (MetS) is a cluster of risk factors including hypertension, hyperglycemia, dyslipidemia, and abdominal obesity. Metabolism-related risk factors include diabetes and heart disease. MetS is also linked to numerous cancers and chronic kidney disease. All of these variables raise medical costs. Developing a prediction model that can quickly identify persons at high risk of MetS and offer them a treatment plan is crucial. Early prediction of metabolic syndrome will highly impact the quality of life of patients as it gives them a chance for making a change to the bad habit and preventing a serious illness in the future. In this paper, we aimed to assess the performance of various algorithms of machine learning in order to decrease the cost of predictive diagnoses of metabolic syndrome. We employed ten machine learning algorithms along with different metaheuristics for feature selection. Moreover, we examined the effects of data augmentation in the prediction accuracy. The statistics show that the augmentation of data after applying feature selection on the data highly improves the performance of the classifiers.
Collapse
|
31
|
Shingshetty L, Maheshwari A, McLernon DJ, Bhattacharya S. Should we adopt a prognosis-based approach to unexplained infertility? Hum Reprod Open 2022; 2022:hoac046. [PMID: 36382011 PMCID: PMC9662706 DOI: 10.1093/hropen/hoac046] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 09/09/2022] [Indexed: 08/27/2023] Open
Abstract
The treatment of unexplained infertility is a contentious topic that continues to attract a great deal of interest amongst clinicians, patients and policy makers. The inability to identify an underlying pathology makes it difficult to devise effective treatments for this condition. Couples with unexplained infertility can conceive on their own and any proposed intervention needs to offer a better chance of having a baby. Over the years, several prognostic and prediction models based on routinely collected clinical data have been developed, but these are not widely used by clinicians and patients. In this opinion paper, we propose a prognosis-based approach such that a decision to access treatment is based on the estimated chances of natural and treatment-related conception, which, in the same couple, can change over time. This approach avoids treating all couples as a homogeneous group and minimizes unnecessary treatment whilst ensuring access to those who need it early.
Collapse
Affiliation(s)
- Laxmi Shingshetty
- Aberdeen Centre for Reproductive Medicine, NHS Grampian, Aberdeen, UK
| | - Abha Maheshwari
- Aberdeen Centre for Reproductive Medicine, NHS Grampian, Aberdeen, UK
| | - David J McLernon
- Medical Statistics Team, School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, UK
| | | |
Collapse
|