Predicting 6-Month Unfavorable Outcome of Acute Ischemic Stroke Using Machine Learning

Predicting 3-month poor functional outcomes of acute ischemic stroke in young patients using machine learning

BACKGROUND

Prediction of short-term outcomes in young patients with acute ischemic stroke (AIS) may assist in making therapy decisions. Machine learning (ML) is increasingly used in healthcare due to its high accuracy. This study aims to use a ML-based predictive model for poor 3-month functional outcomes in young AIS patients and to compare the predictive performance of ML models with the logistic regression model.

METHODS

We enrolled AIS patients aged between 18 and 50 years from the Third Chinese National Stroke Registry (CNSR-III), collected between 2015 and 2018. A modified Rankin Scale (mRS) ≥ 3 was a poor functional outcome at 3 months. Four ML tree models were developed: The extreme Gradient Boosting (XGBoost), Light Gradient Boosted Machine (lightGBM), Random Forest (RF), and The Gradient Boosting Decision Trees (GBDT), compared with logistic regression. We assess the model performance based on both discrimination and calibration.

RESULTS

A total of 2268 young patients with a mean age of 44.3 ± 5.5 years were included. Among them, (9%) had poor functional outcomes. The mRS at admission, living alone conditions, and high National Institutes of Health Stroke Scale (NIHSS) at discharge remained independent predictors of poor 3-month outcomes. The best AUC in the test group was XGBoost (AUC = 0.801), followed by GBDT, RF, and lightGBM (AUCs of 0.795, 0, 794, and 0.792, respectively). The XGBoost, RF, and lightGBM models were significantly better than logistic regression (P < 0.05).

CONCLUSIONS

ML outperformed logistic regression, where XGBoost the boost was the best model for predicting poor functional outcomes in young AIS patients. It is important to consider living alone conditions with high severity scores to improve stroke prognosis.

Accurately predicting the risk of unfavorable outcomes after endovascular coil therapy in patients with aneurysmal subarachnoid hemorrhage: an interpretable machine learning model

BACKGROUND

Despite endovascular coiling as a valid modality in treatment of aneurysmal subarachnoid hemorrhage (aSAH), there is a risk of poor prognosis. However, the clinical utility of previously proposed early prediction tools remains limited. We aimed to develop a clinically generalizable machine learning (ML) models for accurately predicting unfavorable outcomes in aSAH patients after endovascular coiling.

METHODS

Functional outcomes at 6 months after endovascular coiling were assessed via the modified Rankin Scale (mRS) and unfavorable outcomes were defined as mRS 3-6. Five ML algorithms (logistic regression, random forest, support vector machine, deep neural network, and extreme gradient boosting) were used for model development. The area under precision-recall curve (AUPRC) and receiver operating characteristic curve (AUROC) was used as main indices of model evaluation. SHapley Additive exPlanations (SHAP) method was applied to interpret the best-performing ML model.

RESULTS

A total of 371 patients were eventually included into this study, and 85.4% of them had favorable outcomes. Among the five models, the DNN model had a better performance with AUPRC of 0.645 (AUROC of 0.905). Postoperative GCS score, size of aneurysm, and age were the top three powerful predictors. The further analysis of five random cases presented the good interpretability of the DNN model.

CONCLUSION

Interpretable clinical prediction models based on different ML algorithms have been successfully constructed and validated, which would serve as reliable tools in optimizing the treatment decision-making of aSAH. Our DNN model had better performance to predict the unfavorable outcomes at 6 months in aSAH patients compared with Yan's nomogram model.

Systematic Review of Machine Learning Applied to the Secondary Prevention of Ischemic Stroke

Ischemic stroke is a serious disease posing significant threats to human health and life, with the highest absolute and relative risks of a poor prognosis following the first occurrence, and more than 90% of strokes are attributable to modifiable risk factors. Currently, machine learning (ML) is widely used for the prediction of ischemic stroke outcomes. By identifying risk factors, predicting the risk of poor prognosis and thus developing personalized treatment plans, it effectively reduces the probability of poor prognosis, leading to more effective secondary prevention. This review includes 41 studies since 2018 that used ML algorithms to build prognostic prediction models for ischemic stroke, transient ischemic attack (TIA), and acute ischemic stroke (AIS). We analyzed in detail the risk factors used in these studies, the sources and processing methods of the required data, the model building and validation, and their application in different prediction time windows. The results indicate that among the included studies, the top five risk factors in terms of frequency were cardiovascular diseases, age, sex, national institutes of health stroke scale (NIHSS) score, and diabetes. Furthermore, 64% of the studies used single-center data, 65% of studies using imbalanced data did not perform data balancing, 88% of the studies did not utilize external validation datasets for model validation, and 72% of the studies did not provide explanations for their models. Addressing these issues is crucial for enhancing the credibility and effectiveness of the research, consequently improving the development and implementation of secondary prevention measures.

Machine learning prediction of motor function in chronic stroke patients: a systematic review and meta-analysis

Background

Recent studies have reported that machine learning (ML), with a relatively strong capacity for processing non-linear data and adaptive ability, could improve the accuracy and efficiency of prediction. The article summarizes the published studies on ML models that predict motor function 3-6 months post-stroke.

Methods

A systematic literature search was conducted in PubMed, Embase, Cochorane and Web of Science as of April 3, 2023 for studies on ML prediction of motor function in stroke patients. The quality of the literature was assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST). A random-effects model was preferred for meta-analysis using R4.2.0 because of the different variables and parameters.

Results

A total of 44 studies were included in this meta-analysis, involving 72,368 patients and 136 models. Models were categorized into subgroups according to the predicted outcome Modified Rankin Scale cut-off value and whether they were constructed based on radiomics. C-statistics, sensitivity, and specificity were calculated. The random-effects model showed that the C-statistics of all models were 0.81 (95% CI: 0.79; 0.83) in the training set and 0.82 (95% CI: 0.80; 0.85) in the validation set. According to different Modified Rankin Scale cut-off values, C-statistics of ML models predicting Modified Rankin Scale>2(used most widely) in stroke patients were 0.81 (95% CI: 0.78; 0.84) in the training set, and 0.84 (95% CI: 0.81; 0.87) in the validation set. C-statistics of radiomics-based ML models in the training set and validation set were 0.81 (95% CI: 0.78; 0.84) and 0.87 (95% CI: 0.83; 0.90), respectively.

Conclusion

ML can be used as an assessment tool for predicting the motor function in patients with 3-6 months of post-stroke. Additionally, the study found that ML models with radiomics as a predictive variable were also demonstrated to have good predictive capabilities. This systematic review provides valuable guidance for the future optimization of ML prediction systems that predict poor motor outcomes in stroke patients.

Systematic review registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022335260, identifier: CRD42022335260.

Artificial Intelligence for Clinical Decision Support in Acute Ischemic Stroke: A Systematic Review

BACKGROUND

Established randomized trial-based parameters for acute ischemic stroke group patients into generic treatment groups, leading to attempts using various artificial intelligence (AI) methods to directly correlate patient characteristics to outcomes and thereby provide decision support to stroke clinicians. We review AI-based clinical decision support systems in the development stage, specifically regarding methodological robustness and constraints for clinical implementation.

METHODS

Our systematic review included full-text English language publications proposing a clinical decision support system using AI techniques for direct decision support in acute ischemic stroke cases in adult patients. We (1) describe data and outcomes used in those systems, (2) estimate the systems' benefits compared with traditional stroke diagnosis and treatment, and (3) reported concordance with reporting standards for AI in healthcare.

RESULTS

One hundred twenty-one studies met our inclusion criteria. Sixty-five were included for full extraction. In our sample, utilized data sources, methods, and reporting practices were highly heterogeneous.

CONCLUSIONS

Our results suggest significant validity threats, dissonance in reporting practices, and challenges to clinical translation. We outline practical recommendations for the successful implementation of AI research in acute ischemic stroke treatment and diagnosis.

Improvement of predictive accuracies of functional outcomes after subacute stroke inpatient rehabilitation by machine learning models

OBJECTIVES

Stepwise linear regression (SLR) is the most common approach to predicting activities of daily living at discharge with the Functional Independence Measure (FIM) in stroke patients, but noisy nonlinear clinical data decrease the predictive accuracies of SLR. Machine learning is gaining attention in the medical field for such nonlinear data. Previous studies reported that machine learning models, regression tree (RT), ensemble learning (EL), artificial neural networks (ANNs), support vector regression (SVR), and Gaussian process regression (GPR), are robust to such data and increase predictive accuracies. This study aimed to compare the predictive accuracies of SLR and these machine learning models for FIM scores in stroke patients.

METHODS

Subacute stroke patients (N = 1,046) who underwent inpatient rehabilitation participated in this study. Only patients' background characteristics and FIM scores at admission were used to build each predictive model of SLR, RT, EL, ANN, SVR, and GPR with 10-fold cross-validation. The coefficient of determination (R2) and root mean square error (RMSE) values were compared between the actual and predicted discharge FIM scores and FIM gain.

RESULTS

Machine learning models (R2 of RT = 0.75, EL = 0.78, ANN = 0.81, SVR = 0.80, GPR = 0.81) outperformed SLR (0.70) to predict discharge FIM motor scores. The predictive accuracies of machine learning methods for FIM total gain (R2 of RT = 0.48, EL = 0.51, ANN = 0.50, SVR = 0.51, GPR = 0.54) were also better than of SLR (0.22).

CONCLUSIONS

This study suggested that the machine learning models outperformed SLR for predicting FIM prognosis. The machine learning models used only patients' background characteristics and FIM scores at admission and more accurately predicted FIM gain than previous studies. ANN, SVR, and GPR outperformed RT and EL. GPR could have the best predictive accuracy for FIM prognosis.

Deep learning-based personalised outcome prediction after acute ischaemic stroke

BACKGROUND

Whether deep learning models using clinical data and brain imaging can predict the long-term risk of major adverse cerebro/cardiovascular events (MACE) after acute ischaemic stroke (AIS) at the individual level has not yet been studied.

METHODS

A total of 8590 patients with AIS admitted within 5 days of symptom onset were enrolled. The primary outcome was the occurrence of MACEs (a composite of stroke, acute myocardial infarction or death) over 12 months. The performance of deep learning models (DeepSurv and Deep-Survival-Machines (DeepSM)) and traditional survival models (Cox proportional hazards (CoxPH) and random survival forest (RSF)) were compared using the time-dependent concordance index ([Formula: see text] index).

RESULTS

Given the top 1 to all 60 clinical factors according to feature importance, CoxPH and RSF yielded [Formula: see text] index of 0.7236-0.8222 and 0.7279-0.8335, respectively. Adding image features improved the performance of deep learning models and traditional models assisted by deep learning models. DeepSurv and DeepSM yielded the best [Formula: see text] index of 0.8496 and 0.8531 when images were added to all 39 relevant clinical factors, respectively. In feature importance, brain image was consistently ranked highly. Deep learning models automatically extracted the image features directly from personalised brain images and predicted the risk and date of future MACEs at the individual level.

CONCLUSIONS

Deep learning models using clinical data and brain images could improve the prediction of MACEs and provide personalised outcome prediction for patients with AIS. Deep learning models will allow us to develop more accurate and tailored prognostic prediction systems that outperform traditional models.

Machine learning-based prediction of clinical outcomes after first-ever ischemic stroke

Background

Accurate prediction of clinical outcomes in individual patients following acute stroke is vital for healthcare providers to optimize treatment strategies and plan further patient care. Here, we use advanced machine learning (ML) techniques to systematically compare the prediction of functional recovery, cognitive function, depression, and mortality of first-ever ischemic stroke patients and to identify the leading prognostic factors.

Methods

We predicted clinical outcomes for 307 patients (151 females, 156 males; 68 ± 14 years) from the PROSpective Cohort with Incident Stroke Berlin study using 43 baseline features. Outcomes included modified Rankin Scale (mRS), Barthel Index (BI), Mini-Mental State Examination (MMSE), Modified Telephone Interview for Cognitive Status (TICS-M), Center for Epidemiologic Studies Depression Scale (CES-D) and survival. The ML models included a Support Vector Machine with a linear kernel and a radial basis function kernel as well as a Gradient Boosting Classifier based on repeated 5-fold nested cross-validation. The leading prognostic features were identified using Shapley additive explanations.

Results

The ML models achieved significant prediction performance for mRS at patient discharge and after 1 year, BI and MMSE at patient discharge, TICS-M after 1 and 3 years and CES-D after 1 year. Additionally, we showed that National Institutes of Health Stroke Scale (NIHSS) was the top predictor for most functional recovery outcomes as well as education for cognitive function and depression.

Conclusion

Our machine learning analysis successfully demonstrated the ability to predict clinical outcomes after first-ever ischemic stroke and identified the leading prognostic factors that contribute to this prediction.

Association of cognitive reserve with stroke outcome: a protocol for a systematic review

INTRODUCTION

The concept of cognitive reserve (CR) was introduced to account for individual differences in the clinical manifestation of neurodegenerative diseases. Though several mechanisms and risk factors are shared between neurodegeneration and stroke, the effect of CR on poststroke functional outcome has been poorly addressed. This systematic review aims to synthesise the available research evidence on the association of CR with stroke outcome, in order to implement the understanding of interindividual variability in stroke outcome and to improve its prediction.

METHODS AND ANALYSIS

Cochrane Library, Embase, PubMed, Web of Science and reference lists of relevant literature will be searched for publications on CR proxies (eg, education, years of education, occupational attainment, premorbid intelligence) and stroke outcome, published between 1 January 1980 and 10 March 2022. Two reviewers will independently perform the study selection, data extraction and quality assessment. Disagreements between reviewers will be resolved by a third independent reviewer. The Quality In Prognosis Studies tool will be used to assess the quality of each included study. The primary outcome will be functional outcome after stroke assessed with modified Rankin Scale, activities of daily living (eg, Barthel Index), National Institute of Health Stroke Scale, dichotomised as favourable versus not favourable as well as reported as continuous or ordinal variables. Qualitative and quantitative findings will be summarised and, if possible, data will be synthesised using appropriate meta-analytical methods. The quality of evidence will be assessed using the Grading of Recommendations Assessment, Development and Evaluation framework.

ETHICS AND DISSEMINATION

No ethical approval is required as it is a protocol for a systematic review and the data used will be extracted from published studies. The findings from this systematic review will be disseminated in a peer-reviewed scientific journal and presented at conferences. The data will be made freely available.

PROSPERO REGISTRATION NUMBER

CRD42021256175.

Precision medicine in stroke: towards personalized outcome predictions using artificial intelligence

Stroke ranks among the leading causes for morbidity and mortality worldwide. New and continuously improving treatment options such as thrombolysis and thrombectomy have revolutionized acute stroke treatment in recent years. Following modern rhythms, the next revolution might well be the strategic use of the steadily increasing amounts of patient-related data for generating models enabling individualized outcome predictions. Milestones have already been achieved in several health care domains, as big data and artificial intelligence have entered everyday life. The aim of this review is to synoptically illustrate and discuss how artificial intelligence approaches may help to compute single-patient predictions in stroke outcome research in the acute, subacute and chronic stage. We will present approaches considering demographic, clinical and electrophysiological data, as well as data originating from various imaging modalities and combinations thereof. We will outline their advantages, disadvantages, their potential pitfalls and the promises they hold with a special focus on a clinical audience. Throughout the review we will highlight methodological aspects of novel machine-learning approaches as they are particularly crucial to realize precision medicine. We will finally provide an outlook on how artificial intelligence approaches might contribute to enhancing favourable outcomes after stroke.

Early Prediction of Functional Outcomes After Acute Ischemic Stroke Using Unstructured Clinical Text: Retrospective Cohort Study

Background

Several prognostic scores have been proposed to predict functional outcomes after an acute ischemic stroke (AIS). Most of these scores are based on structured information and have been used to develop prediction models via the logistic regression method. With the increased use of electronic health records and the progress in computational power, data-driven predictive modeling by using machine learning techniques is gaining popularity in clinical decision-making.

Objective

We aimed to investigate whether machine learning models created by using unstructured text could improve the prediction of functional outcomes at an early stage after AIS.

Methods

We identified all consecutive patients who were hospitalized for the first time for AIS from October 2007 to December 2019 by using a hospital stroke registry. The study population was randomly split into a training (n=2885) and test set (n=962). Free text in histories of present illness and computed tomography reports was transformed into input variables via natural language processing. Models were trained by using the extreme gradient boosting technique to predict a poor functional outcome at 90 days poststroke. Model performance on the test set was evaluated by using the area under the receiver operating characteristic curve (AUC).

Results

The AUCs of text-only models ranged from 0.768 to 0.807 and were comparable to that of the model using National Institutes of Health Stroke Scale (NIHSS) scores (0.811). Models using both patient age and text achieved AUCs of 0.823 and 0.825, which were similar to those of the model containing age and NIHSS scores (0.841); the model containing preadmission comorbidities, level of consciousness, age, and neurological deficit (PLAN) scores (0.837); and the model containing Acute Stroke Registry and Analysis of Lausanne (ASTRAL) scores (0.840). Adding variables from clinical text improved the predictive performance of the model containing age and NIHSS scores, the model containing PLAN scores, and the model containing ASTRAL scores (the AUC increased from 0.841 to 0.861, from 0.837 to 0.856, and from 0.840 to 0.860, respectively).

Conclusions

Unstructured clinical text can be used to improve the performance of existing models for predicting poststroke functional outcomes. However, considering the different terminologies that are used across health systems, each individual health system may consider using the proposed methods to develop and validate its own models.

Deep Learning for Outcome Prediction in Neurosurgery: A Systematic Review of Design, Reporting, and Reproducibility

Deep learning (DL) is a powerful machine learning technique that has increasingly been used to predict surgical outcomes. However, the large quantity of data required and lack of model interpretability represent substantial barriers to the validity and reproducibility of DL models. The objective of this study was to systematically review the characteristics of DL studies involving neurosurgical outcome prediction and to assess their bias and reporting quality. Literature search using the PubMed, Scopus, and Embase databases identified 1949 records of which 35 studies were included. Of these, 32 (91%) developed and validated a DL model while 3 (9%) validated a pre-existing model. The most commonly represented subspecialty areas were oncology (16 of 35, 46%), spine (8 of 35, 23%), and vascular (6 of 35, 17%). Risk of bias was low in 18 studies (51%), unclear in 5 (14%), and high in 12 (34%), most commonly because of data quality deficiencies. Adherence to transparent reporting of a multivariable prediction model for individual prognosis or diagnosis reporting standards was low, with a median of 12 transparent reporting of a multivariable prediction model for individual prognosis or diagnosis items (39%) per study not reported. Model transparency was severely limited because code was provided in only 3 studies (9%) and final models in 2 (6%). With the exception of public databases, no study data sets were readily available. No studies described DL models as ready for clinical use. The use of DL for neurosurgical outcome prediction remains nascent. Lack of appropriate data sets poses a major concern for bias. Although studies have demonstrated promising results, greater transparency in model development and reporting is needed to facilitate reproducibility and validation.

Natural Language Processing Enhances Prediction of Functional Outcome After Acute Ischemic Stroke

Background

Conventional prognostic scores usually require predefined clinical variables to predict outcome. The advancement of natural language processing has made it feasible to derive meaning from unstructured data. We aimed to test whether using unstructured text in electronic health records can improve the prediction of functional outcome after acute ischemic stroke.

Methods and Results

Patients hospitalized for acute ischemic stroke were identified from 2 hospital stroke registries (3847 and 2668 patients, respectively). Prediction models developed using the first cohort were externally validated using the second cohort, and vice versa. Free text in the history of present illness and computed tomography reports was used to build machine learning models using natural language processing to predict poor functional outcome at 90 days poststroke. Four conventional prognostic models were used as baseline models. The area under the receiver operating characteristic curves of the model using history of present illness in the internal and external validation sets were 0.820 and 0.792, respectively, which were comparable to the National Institutes of Health Stroke Scale score (0.811 and 0.807). The model using computed tomography reports achieved area under the receiver operating characteristic curves of 0.758 and 0.658. Adding information from clinical text significantly improved the predictive performance of each baseline model in terms of area under the receiver operating characteristic curves, net reclassification improvement, and integrated discrimination improvement indices (all P<0.001). Swapping the study cohorts led to similar results.

Conclusions

By using natural language processing, unstructured text in electronic health records can provide an alternative tool for stroke prognostication, and even enhance the performance of existing prognostic scores.