1
|
Scrutinio D, Ricciardi C, Donisi L, Losavio E, Battista P, Guida P, Cesarelli M, Pagano G, D'Addio G. Machine learning to predict mortality after rehabilitation among patients with severe stroke. Sci Rep 2020; 10:20127. [PMID: 33208913 PMCID: PMC7674405 DOI: 10.1038/s41598-020-77243-3] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Accepted: 11/02/2020] [Indexed: 12/23/2022] Open
Abstract
Stroke is among the leading causes of death and disability worldwide. Approximately 20–25% of stroke survivors present severe disability, which is associated with increased mortality risk. Prognostication is inherent in the process of clinical decision-making. Machine learning (ML) methods have gained increasing popularity in the setting of biomedical research. The aim of this study was twofold: assessing the performance of ML tree-based algorithms for predicting three-year mortality model in 1207 stroke patients with severe disability who completed rehabilitation and comparing the performance of ML algorithms to that of a standard logistic regression. The logistic regression model achieved an area under the Receiver Operating Characteristics curve (AUC) of 0.745 and was well calibrated. At the optimal risk threshold, the model had an accuracy of 75.7%, a positive predictive value (PPV) of 33.9%, and a negative predictive value (NPV) of 91.0%. The ML algorithm outperformed the logistic regression model through the implementation of synthetic minority oversampling technique and the Random Forests, achieving an AUC of 0.928 and an accuracy of 86.3%. The PPV was 84.6% and the NPV 87.5%. This study introduced a step forward in the creation of standardisable tools for predicting health outcomes in individuals affected by stroke.
Collapse
Affiliation(s)
| | - Carlo Ricciardi
- Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy. .,Department of Advanced Biomedical Sciences, University Hospital of Naples "Federico II", Naples, Italy.
| | - Leandro Donisi
- Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy.,Department of Advanced Biomedical Sciences, University Hospital of Naples "Federico II", Naples, Italy
| | | | | | - Pietro Guida
- Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy
| | - Mario Cesarelli
- Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy.,Department of Electrical Engineering and Information Technology, University of Naples "Federico II", Naples, Italy
| | - Gaetano Pagano
- Istituti Clinici Scientifici Maugeri IRCCS, Pavia, Italy
| | | |
Collapse
|
2
|
Du W, Zhao X, Wang Y, Zhang G, Fang J, Pan Y, Liu L, Dong K, Liu G, Wang Y. The PLAN score can predict poor outcomes of intracerebral hemorrhage. ANNALS OF TRANSLATIONAL MEDICINE 2020; 8:14. [PMID: 32055605 DOI: 10.21037/atm.2019.11.88] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/17/2023]
Abstract
Background For patients hospitalized after acute ischemic stroke (AIS), the preadmission comorbidities, level of consciousness (LOC), age and neurologic deficit (PLAN) score can help to identify those who may have a poor outcome. Implementing the PLAN score in other types of stroke may also have predictive value. Our study aimed to evaluate the PLAN score's prognostic accuracy in predicting 1-year mortality and severe disability after intracerebral hemorrhage (ICH). Methods We analyzed data found in the China National Stroke Registry (CNSR) of 2,453 hospitalized patients in 132 urban Chinese hospitals, diagnosed with ICH from September 2007 to August 2008. The outcomes analysis included 30-day mortality, modified Rankin Scale score (mRS) of 5-6 at discharge, and 1-year mortality. Univariate and multivariate analysis was performed, and we calculated consistency statistics (C statistic). We evaluated the PLAN score performance using area under the curve (AUC) calculations. Results We found that the 30-day mortality was 12.6%, the frequency of a mRS 5-6 at discharge was 20.6%, and 1-year mortality was 21.9%. The PLAN score had good predictive value in 30-day mortality (C statistic, 0.82), death or severe dependence at discharge (0.84), and 1-year mortality (0.82). Conclusions In patients hospitalized for ICH, the 30-day mortality, death or severe dependence at discharge and 1-year mortality can be predicted by the PLAN score. Similarly to patients hospitalized after AIS, the PLAN score can help to identify patients likely to have poor outcomes following hospitalization for ICH.
Collapse
Affiliation(s)
- Wanliang Du
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Xingquan Zhao
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China.,Beijing Key Laboratory of Translational Medicine for Cerebrovascular Disease, Beijing 100070, China
| | - Yilong Wang
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China.,Center of Stroke, Beijing Institute for Brain Disorders, Beijing 100070, China
| | - Guitao Zhang
- China National Clinical Research Center for Neurological Diseases, Beijing 100070, China
| | - Jiming Fang
- Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
| | - Yuesong Pan
- China National Clinical Research Center for Neurological Diseases, Beijing 100070, China
| | - Liping Liu
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Kehui Dong
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Gaifen Liu
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China.,China National Clinical Research Center for Neurological Diseases, Beijing 100070, China
| | - Yongjun Wang
- Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China.,Beijing Key Laboratory of Translational Medicine for Cerebrovascular Disease, Beijing 100070, China.,Center of Stroke, Beijing Institute for Brain Disorders, Beijing 100070, China.,China National Clinical Research Center for Neurological Diseases, Beijing 100070, China.,Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing 100070, China
| | | |
Collapse
|
3
|
Thompson MP, Luo Z, Gardiner J, Burke JF, Nickles A, Reeves MJ. Impact of Missing Stroke Severity Data on the Accuracy of Hospital Ischemic Stroke Mortality Profiling. Circ Cardiovasc Qual Outcomes 2019; 11:e004951. [PMID: 30354572 DOI: 10.1161/circoutcomes.118.004951] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
BACKGROUND The Centers for Medicare and Medicaid Services have proposed 30-day ischemic stroke risk-standardized mortality rates that include adjustment for stroke severity using the National Institute of Health Stroke Scale (NIHSS), which is often undocumented. We used simulations to quantify the effect of missing NIHSS data on the accuracy of hospital-level ischemic stroke risk-standardized mortality rate profiling for 100 hypothetical hospitals with different case volumes. METHODS AND RESULTS We generated simulated data sets of patients with NIHSS scores and other predictors of 30-day mortality based on empirical analysis of data from 7654 patients with ischemic stroke in the Michigan Stroke Registry. We assigned and rank-ordered a true (known) 30-day mortality rate to each hospital in the simulated data sets of N=100 hospitals with either low (100 patients with stroke), medium (300), or high (500) case volumes. We then estimated and rank-ordered 30-day risk-standardized mortality rates for the N=100 hospitals in each simulated data set using hierarchical logistic regression models. In each data set, we systematically varied the rate of missing NIHSS data and whether missing NIHSS data was independent (missing completely at random) or dependent (missing not at random) on the NIHSS score. With no missing NIHSS data, the Spearman correlation between the true hospital performance rank order assigned during data set generation and the estimated 30-day risk-standardized mortality rate rank order was 0.72, 0.88, and 0.91 in low, medium, and high volume hospitals, respectively. However, this fell to as low as 0.50, 0.71, and 0.79 as missing NIHSS data reached 70%. CONCLUSIONS Missing NIHSS data had substantial detrimental effects on the accuracy of profiling of ischemic stroke mortality, especially in lower volume hospitals. Even with complete NIHSS documentation, significant limitations in ischemic stroke mortality profiling remain.
Collapse
Affiliation(s)
- Michael P Thompson
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.).,Department of Cardiac Surgery, University of Michigan Medical School, Ann Arbor, MI (M.P.T.)
| | - Zhehui Luo
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.)
| | - Joseph Gardiner
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.)
| | - James F Burke
- Department of Neurology, University of Michigan Medical School, Ann Arbor, MI (J.F.B.)
| | | | - Mathew J Reeves
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.)
| |
Collapse
|
4
|
Drozdowska BA, Singh S, Quinn TJ. Thinking About the Future: A Review of Prognostic Scales Used in Acute Stroke. Front Neurol 2019; 10:274. [PMID: 30949127 PMCID: PMC6437031 DOI: 10.3389/fneur.2019.00274] [Citation(s) in RCA: 51] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Accepted: 03/01/2019] [Indexed: 11/25/2022] Open
Abstract
Background: There are many prognostic scales that aim to predict functional outcome following acute stroke. Despite considerable research interest, these scales have had limited impact in routine clinical practice. This may be due to perceived problems with internal validity (quality of research), as well as external validity (generalizability of results). We set out to collate information on exemplar stroke prognosis scales, giving particular attention to the scale content, derivation, and validation. Methods: We performed a focused literature search, designed to return high profile scales that use baseline clinical data to predict mortality or disability. We described prognostic utility and collated information on the content, development and validation of the tools. We critically appraised chosen scales based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies (CHARMS). Results: We chose 10 primary scales that met our inclusion criteria, six of which had revised/modified versions. Most primary scales used 5 input variables (range: 4–13), with substantial overlap in the variables included. All scales included age, eight included a measure of stroke severity, while five scales incorporated pre-stroke level of function (often using modified Rankin Scale), comorbidities and classification of stroke type. Through our critical appraisal, we found issues relating to excluding patients with missing data from derivation studies, and basing the selection of model variable on significance in univariable analysis (in both cases noted for six studies). We identified separate external validation studies for all primary scales but one, with a total of 60 validation studies. Conclusions: Most acute stroke prognosis scales use similar variables to predict long-term outcomes and most have reasonable prognostic accuracy. While not all published scales followed best practice in development, most have been subsequently validated. Lack of clinical uptake may relate more to practical application of scales rather than validity. Impact studies are now necessary to investigate clinical usefulness of existing scales.
Collapse
Affiliation(s)
- Bogna A Drozdowska
- Institute of Cardiovascular and Medical Sciences, College of Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow, United Kingdom
| | - Sarjit Singh
- Institute of Cardiovascular and Medical Sciences, College of Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow, United Kingdom
| | - Terence J Quinn
- Institute of Cardiovascular and Medical Sciences, College of Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow, United Kingdom
| |
Collapse
|
5
|
Chu X, Yang Y, Zhang F, Ye R, Chu W. Validation of iScore and PLAN Score for Death in Thrombectomy in Acute Stroke Due to Anterior Circulation Large Artery Occlusion. J Stroke Cerebrovasc Dis 2018; 27:3261-3265. [PMID: 30087077 DOI: 10.1016/j.jstrokecerebrovasdis.2018.07.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2018] [Revised: 07/03/2018] [Accepted: 07/07/2018] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND To validate iScore and PLAN score in acute anterior circulation large vessel occlusion stroke patients undergoing thrombectomy. METHODS iScore and PLAN score were calculated for consecutive acute ischemic stroke undergoing thrombectomy were included and death at 1 month and death at 3 months were recorded. The area under the receiver operating characteristic curve was used to assess the discrimination ability of the scales for death. RESULTS Two hundred and twenty-nine patients were included, 25.3% (58 of 229) of patient died at 1 month after thrombectomy and 25.8% (59 of 229) of them died at 3 months after thrombectomy. The receiver operator curve analysis found that iScore (area under the curve [AUC] = .76, 95% confidence interval [CI] .69-.83) was numerically better than PLAN score (AUC = .73, 95% CI .66-.81) for predicting death at day 90. The cut-off for iScore is 193, with sensitivity 64%, specificity 79%, positive predictive value 75% and negative predictive value 69%. CONCLUSIONS The iScore scale is a valid predictive tool for death in anterior circulation large vessel occlusions undergoing thrombectomy.
Collapse
Affiliation(s)
- Xiaopeng Chu
- Department of Neurology, JiangDu People's Hospital of Yangzhou, Yangzhou, Jiangsu Province, China
| | - Yongxin Yang
- Department of Neurology, JiangDu People's Hospital of Yangzhou, Yangzhou, Jiangsu Province, China
| | - Fushan Zhang
- Department of Neurology, JiangDu People's Hospital of Yangzhou, Yangzhou, Jiangsu Province, China
| | - Ruidiong Ye
- Department of Neurology, Jinling Hospital, Medical School of Nanjing University, Nanjing, Jiangsu Province, China
| | - Wenzheng Chu
- Department of Neurology, Yantai Yuhuangding Hospital, Yantai, Shandong Province, China.
| |
Collapse
|
6
|
The Prognostic Value of the THRIVE Score, the iScore Score and the ASTRAL Score in Chinese Patients With Acute Ischemic Stroke. J Stroke Cerebrovasc Dis 2018; 27:2877-2886. [PMID: 30077603 DOI: 10.1016/j.jstrokecerebrovasdis.2018.06.011] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2018] [Revised: 06/01/2018] [Accepted: 06/14/2018] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Acute ischemic stroke (AIS) has a higher morbidity and mortality rate. Many prediction tools have been developed to predict the risk of poor outcomes in patients after AIS, such as the THRIVE score, the iScore score, and the ASTRAL score. However, the predictive value of above 3 prediction tools in Chinese patients with AIS need to be further verified. So, this study aimed to determine the ability of the THRIVE score, the iScore score, and the ASTRAL score in predicting clinical poor outcomes in Chinese patients with AIS at 1 year. METHODS A total of 772 patients with AIS were included in this study. The baseline data of all patients were collected. The THRIVE score, the iScore score, and the ASTRAL score were calculated. All patients were followed up at 1 year. The poor outcome was defined as death, moderate/severe disabilities (modified Rankin scale, mRS > 2), most severe disability (mRS ≥ 5). Model discrimination was quantified by calculating the area under the receiver operating characteristic curve (AUC). The calibration was assessed using Hosmer-Lemeshow goodness-of-fit test and Pearson correlation coefficient. RESULTS We identified 576 (74.6%) patients with good prognosis and 196 (25.4%) patients with poor prognosis. AUC values of THRIVE score in predicting 1-year poor prognosis was lower than the iScore score and the ASTRAL scores (P < .05). The chi-square values of Hosmer-Lemeshow for the 3 prediction tools were 2.114, 4.877, 5.838 (all P < .05), respectively. There was a high correlation between the observed and the expected poor prognosis (Pearson correlation coefficient, .985, .693, and .620; all P < .05). AUC values of THRIVE score in predicting 1-year mortality and severe disability were lower than the iScore scores (all P < .05). CONCLUSIONS The iScore score and the ASTRAL score reliably predict 1-year poor outcomes in Chinese patients with AIS, and the iScore score can accurately predict 1-year mortality and severe disability in Chinese AIS patients.
Collapse
|
7
|
Joundi RA, Martino R, Saposnik G, Giannakeas V, Fang J, Kapral MK. Predictors and Outcomes of Dysphagia Screening After Acute Ischemic Stroke. Stroke 2017; 48:900-906. [DOI: 10.1161/strokeaha.116.015332] [Citation(s) in RCA: 68] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2016] [Revised: 12/23/2016] [Accepted: 01/20/2017] [Indexed: 12/21/2022]
Abstract
Background and Purpose—
Guidelines advocate screening all acute stroke patients for dysphagia. However, limited data are available regarding how many and which patients are screened and how failing a swallowing screen affects patient outcomes. We sought to evaluate predictors of receiving dysphagia screening after acute ischemic stroke and outcomes after failing a screening test.
Methods—
We used the Ontario Stroke Registry from April 1, 2010, to March 31, 2013, to identify patients hospitalized with acute ischemic stroke and determine predictors of documented dysphagia screening and outcomes after failing the screening test, including pneumonia, disability, and death.
Results—
Among 7171 patients, 6677 patients were eligible to receive dysphagia screening within 72 hours, yet 1280 (19.2%) patients did not undergo documented screening. Patients with mild strokes were significantly less likely than those with more severe strokes to have documented screening (adjusted odds ratio, 0.51; 95% confidence interval [CI], 0.41–0.64). Failing dysphagia screening was associated with poor outcomes, including pneumonia (adjusted odds ratio, 4.71; 95% CI, 3.43–6.47), severe disability (adjusted odds ratio, 5.19; 95% CI, 4.48–6.02), discharge to long-term care (adjusted odds ratio, 2.79; 95% CI, 2.11–3.79), and 1-year mortality (adjusted hazard ratio, 2.42; 95% CI, 2.09–2.80). Associations were maintained in patients with mild strokes.
Conclusions—
One in 5 patients with acute ischemic stroke did not have documented dysphagia screening, and patients with mild strokes were substantially less likely to have documented screening. Failing dysphagia screening was associated with poor outcomes, including in patients with mild strokes, highlighting the importance of dysphagia screening for all patients with acute ischemic stroke.
Collapse
Affiliation(s)
- Raed A. Joundi
- From the Division of Neurology, Department of Medicine (R.A.J.), Department of Speech-Language Pathology, Graduate Department of Rehabilitation Science (R.M.), Institute of Health Policy, Management and Evaluation (G.S., M.K.K.), and Department of Medicine, Division of General Internal Medicine (M.K.K.), University of Toronto, Ontario, Canada; Stroke Outcomes Research Unit, Division of Neurology, Department of Medicine, St Michael’s Hospital (G.S.); Health Care and Outcomes Research, Krembil
| | - Rosemary Martino
- From the Division of Neurology, Department of Medicine (R.A.J.), Department of Speech-Language Pathology, Graduate Department of Rehabilitation Science (R.M.), Institute of Health Policy, Management and Evaluation (G.S., M.K.K.), and Department of Medicine, Division of General Internal Medicine (M.K.K.), University of Toronto, Ontario, Canada; Stroke Outcomes Research Unit, Division of Neurology, Department of Medicine, St Michael’s Hospital (G.S.); Health Care and Outcomes Research, Krembil
| | - Gustavo Saposnik
- From the Division of Neurology, Department of Medicine (R.A.J.), Department of Speech-Language Pathology, Graduate Department of Rehabilitation Science (R.M.), Institute of Health Policy, Management and Evaluation (G.S., M.K.K.), and Department of Medicine, Division of General Internal Medicine (M.K.K.), University of Toronto, Ontario, Canada; Stroke Outcomes Research Unit, Division of Neurology, Department of Medicine, St Michael’s Hospital (G.S.); Health Care and Outcomes Research, Krembil
| | - Vasily Giannakeas
- From the Division of Neurology, Department of Medicine (R.A.J.), Department of Speech-Language Pathology, Graduate Department of Rehabilitation Science (R.M.), Institute of Health Policy, Management and Evaluation (G.S., M.K.K.), and Department of Medicine, Division of General Internal Medicine (M.K.K.), University of Toronto, Ontario, Canada; Stroke Outcomes Research Unit, Division of Neurology, Department of Medicine, St Michael’s Hospital (G.S.); Health Care and Outcomes Research, Krembil
| | - Jiming Fang
- From the Division of Neurology, Department of Medicine (R.A.J.), Department of Speech-Language Pathology, Graduate Department of Rehabilitation Science (R.M.), Institute of Health Policy, Management and Evaluation (G.S., M.K.K.), and Department of Medicine, Division of General Internal Medicine (M.K.K.), University of Toronto, Ontario, Canada; Stroke Outcomes Research Unit, Division of Neurology, Department of Medicine, St Michael’s Hospital (G.S.); Health Care and Outcomes Research, Krembil
| | - Moira K. Kapral
- From the Division of Neurology, Department of Medicine (R.A.J.), Department of Speech-Language Pathology, Graduate Department of Rehabilitation Science (R.M.), Institute of Health Policy, Management and Evaluation (G.S., M.K.K.), and Department of Medicine, Division of General Internal Medicine (M.K.K.), University of Toronto, Ontario, Canada; Stroke Outcomes Research Unit, Division of Neurology, Department of Medicine, St Michael’s Hospital (G.S.); Health Care and Outcomes Research, Krembil
| |
Collapse
|