1
|
Kelly BS, Mathur P, Vaca SD, Duignan J, Power S, Lee EH, Huang Y, Prolo LM, Yeom KW, Lawlor A, Killeen RP, Thornton J. iSPAN: Explainable prediction of outcomes post thrombectomy with Machine Learning. Eur J Radiol 2024; 173:111357. [PMID: 38401408 DOI: 10.1016/j.ejrad.2024.111357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2023] [Revised: 01/03/2024] [Accepted: 02/02/2024] [Indexed: 02/26/2024]
Abstract
PURPOSE This study aimed to develop and evaluate a machine learning model and a novel clinical score for predicting outcomes in stroke patients undergoing endovascular thrombectomy. MATERIALS AND METHODS This retrospective study included all patients aged over 18 years with an anterior circulation stroke treated at a thrombectomy centre from 2010 to 2020 with external validation. The primary outcome was day 90 mRS ≥3. Existing clinical scores (SPAN and PRE) and Machine Learning (ML) models were compared. A novel clinical score (iSPAN) was derived by adding an optimised weighting of the most important ML features to the SPAN. RESULTS 812 patients were initially included (397 female, average age 73), 63 for external validation. The best performing clinical score and ML model were SPAN and XGB (sensitivity, specificity and accuracy 0.290, 0.967, 0.628 and 0.693, 0.783, 0.738 respectively). A significant difference was found overall and our XGB model was more accurate than SPAN (p < 0.0018). The most important features were Age, mTICI and total number of passes. The addition of 11 points for mTICI of ≤2B and 3 points for ≥3 passes to the SPAN achieved the best accuracy and was used to create the iSPAN. iSPAN was not significantly less accurate than our XGB model (p > 0.5). In the external validation set, iSPAN and SPAN achieved sensitivity, specificity, and accuracy of (0.735, 0.862, 0.79) and (0.471, 0.897, 0.67) respectively. CONCLUSION iSPAN incorporates machine-derived features to achieve better predictions compared to existing clinical scores. It is not inferior to our XGB model and is externally generalisable.
Collapse
Affiliation(s)
- Brendan S Kelly
- St Vincent's University Hospital, Dublin, Ireland; Insight Centre for Data Analytics, UCD, Dublin, Ireland; Wellcome Trust - HRB, Irish Clinical Academic Training, Dublin, Ireland; School of Medicine, University College Dublin, Dublin, Ireland; Lucille Packard Children's Hospital at Stanford, Stanford, CA, USA.
| | | | - Silvia D Vaca
- Lucille Packard Children's Hospital at Stanford, Stanford, CA, USA
| | - John Duignan
- Department of Radiology, Beaumont Hospital Dublin, Ireland
| | - Sarah Power
- Department of Neurointerventional Radiology, Beaumont Hospital Dublin, Ireland
| | - Edward H Lee
- Lucille Packard Children's Hospital at Stanford, Stanford, CA, USA
| | - Yuhao Huang
- Lucille Packard Children's Hospital at Stanford, Stanford, CA, USA
| | - Laura M Prolo
- Lucille Packard Children's Hospital at Stanford, Stanford, CA, USA
| | - Kristen W Yeom
- Lucille Packard Children's Hospital at Stanford, Stanford, CA, USA
| | | | | | - John Thornton
- Department of Neurointerventional Radiology, Beaumont Hospital Dublin, Ireland; School of Medicine, Royal College of Surgeons in Ireland, Ireland
| |
Collapse
|
2
|
Kelly BS, Mathur P, McGuinness G, Dillon H, Lee EH, Yeom KW, Lawlor A, Killeen RP. A Radiomic "Warning Sign" of Progression on Brain MRI in Individuals with MS. AJNR Am J Neuroradiol 2024; 45:236-243. [PMID: 38216299 DOI: 10.3174/ajnr.a8104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 11/08/2023] [Indexed: 01/14/2024]
Abstract
BACKGROUND AND PURPOSE MS is a chronic progressive, idiopathic, demyelinating disorder whose diagnosis is contingent on the interpretation of MR imaging. New MR imaging lesions are an early biomarker of disease progression. We aimed to evaluate a machine learning model based on radiomics features in predicting progression on MR imaging of the brain in individuals with MS. MATERIALS AND METHODS This retrospective cohort study with external validation on open-access data obtained full ethics approval. Longitudinal MR imaging data for patients with MS were collected and processed for machine learning. Radiomics features were extracted at the future location of a new lesion in the patients' prior MR imaging ("prelesion"). Additionally, "control" samples were obtained from the normal-appearing white matter for each participant. Machine learning models for binary classification were trained and tested and then evaluated the external data of the model. RESULTS The total number of participants was 167. Of the 147 in the training/test set, 102 were women and 45 were men. The average age was 42 (range, 21-74 years). The best-performing radiomics-based model was XGBoost, with accuracy, precision, recall, and F1-score of 0.91, 0.91, 0.91, and 0.91 on the test set, and 0.74, 0.74, 0.74, and 0.70 on the external validation set. The 5 most important radiomics features to the XGBoost model were associated with the overall heterogeneity and low gray-level emphasis of the segmented regions. Probability maps were produced to illustrate potential future clinical applications. CONCLUSIONS Our machine learning model based on radiomics features successfully differentiated prelesions from normal-appearing white matter. This outcome suggests that radiomics features from normal-appearing white matter could serve as an imaging biomarker for progression of MS on MR imaging.
Collapse
Affiliation(s)
- Brendan S Kelly
- From the Department of Radiology (B.S.K., G.M., H.D., R.P.K.), St. Vincent's University Hospital, Dublin, Ireland
- Insight Centre for Data Analytics (B.S.K., P.M., A.L.), University College Dublin, Dublin, Ireland
- Wellcome Trust and Health Research Board (B.S.K.), Irish Clinical Academic Training, Dublin, Ireland
- School of Medicine (B.S.K.), University College Dublin, Dublin, Ireland
| | - Prateek Mathur
- Insight Centre for Data Analytics (B.S.K., P.M., A.L.), University College Dublin, Dublin, Ireland
| | - Gerard McGuinness
- From the Department of Radiology (B.S.K., G.M., H.D., R.P.K.), St. Vincent's University Hospital, Dublin, Ireland
| | - Henry Dillon
- From the Department of Radiology (B.S.K., G.M., H.D., R.P.K.), St. Vincent's University Hospital, Dublin, Ireland
| | - Edward H Lee
- Lucille Packard Children's Hospital at Stanford (E.H.L., K.W.Y.), Stanford, California
| | - Kristen W Yeom
- Lucille Packard Children's Hospital at Stanford (E.H.L., K.W.Y.), Stanford, California
| | - Aonghus Lawlor
- Insight Centre for Data Analytics (B.S.K., P.M., A.L.), University College Dublin, Dublin, Ireland
| | - Ronan P Killeen
- From the Department of Radiology (B.S.K., G.M., H.D., R.P.K.), St. Vincent's University Hospital, Dublin, Ireland
| |
Collapse
|