Validation of automated Alberta Stroke Program Early CT Score (ASPECTS) software for detection of early ischemic changes on non-contrast brain CT scans

Deep learning-based automatic ASPECTS calculation can improve diagnosis efficiency in patients with acute ischemic stroke: a multicenter study

OBJECTIVES

The Alberta Stroke Program Early CT Score (ASPECTS), a systematic method for assessing ischemic changes in acute ischemic stroke using non-contrast computed tomography (NCCT), is often interpreted relying on expert experience and can vary between readers. This study aimed to develop a clinically applicable automatic ASPECTS system employing deep learning (DL).

METHODS

This study enrolled 1987 NCCT scans that were retrospectively collected from four centers between January 2017 and October 2021. A DL-based system for automated ASPECTS assessment was trained on a development cohort (N = 1767) and validated on an independent test cohort (N = 220). The consensus of experienced physicians was regarded as a reference standard. The validity and reliability of the proposed system were assessed against physicians' readings. A real-world prospective application study with 13,399 patients was used for system validation in clinical contexts.

RESULTS

The DL-based system achieved an area under the receiver operating characteristic curve (AUC) of 84.97% and an intraclass correlation coefficient (ICC) of 0.84 for overall-level analysis on the test cohort. The system's diagnostic sensitivity was 94.61% for patients with dichotomized ASPECTS at a threshold of ≥ 6, with substantial agreement (ICC = 0.65) with expert ratings. Combining the system with physicians improved AUC from 67.43 to 89.76%, reducing diagnosis time from 130.6 ± 66.3 s to 33.3 ± 8.3 s (p < 0.001). During the application in clinical contexts, 94.0% (12,591) of scans successfully processed by the system were utilized by clinicians, and 96% of physicians acknowledged significant improvement in work efficiency.

CONCLUSION

The proposed DL-based system could accurately and rapidly determine ASPECTS, which might facilitate clinical workflow for early intervention.

CLINICAL RELEVANCE STATEMENT

The deep learning-based automated ASPECTS evaluation system can accurately and rapidly determine ASPECTS for early intervention in clinical workflows, reducing processing time for physicians by 74.8%, but still requires validation by physicians when in clinical applications.

KEY POINTS

The deep learning-based system for ASPECTS quantification has been shown to be non-inferior to expert-rated ASPECTS. This system improved the consistency of ASPECTS evaluation and reduced processing time to 33.3 seconds per scan. 94.0% of scans successfully processed by the system were utilized by clinicians during the prospective clinical application.

Automated ASPECTS calculation may equal the performance of experienced clinicians: a machine learning study based on a large cohort

OBJECTIVES

The Alberta Stroke Program Early CT Score (ASPECTS) is a semi-quantitative method to evaluate the severity of early ischemic change on non-contrast computed tomography (NCCT) in patients with acute ischemic stroke (AIS). In this work, we propose an automated ASPECTS method based on large cohort of data and machine learning.

METHODS

For this study, we collected 3626 NCCT cases from multiple centers and annotated directly on this dataset by neurologists. Based on image analysis and machine learning methods, we constructed a two-stage machine learning model. The validity and reliability of this automated ASPECTS method were tested on an independent external validation set of 300 cases. Statistical analyses on the total ASPECTS, dichotomized ASPECTS, and region-level ASPECTS were presented.

RESULTS

On an independent external validation set of 300 cases, for the total ASPECTS results, the intraclass correlation coefficient between automated ASPECTS and expert-rated was 0.842. The agreement between ASPECTS threshold of ≥ 6 versus < 6 using a dichotomized method was moderate (κ = 0.438, 0.391-0.477), and the detection rate (sensitivity) was 86.5% for patients with ASPECTS threshold of ≥ 6. Compared with the results of previous studies, our method achieved a slight lead in sensitivity (67.8%) and AUC (0.845), with comparable accuracy (78.9%) and specificity (81.2%).

CONCLUSION

The proposed automated ASPECTS method driven by a large cohort of NCCT images performed equally well compared with expert-rated ASPECTS. This work further demonstrates the validity and reliability of automated ASPECTS evaluation method.

CLINICAL RELEVANCE STATEMENT

The automated ASPECTS method proposed by this study may help AIS patients to receive rapid intervention, but should not be used as a stand-alone diagnostic basis.

KEY POINTS

NCCT-based manual ASPECTS scores were poorly consistent. Machine learning can automate the ASPECTS scoring process. Machine learning model design based on large cohort data can effectively improve the consistency of ASPECTS scores.

Clinical evaluation of a deep-learning model for automatic scoring of the Alberta stroke program early CT score on non-contrast CT

BACKGROUND

Automated measurement of the Alberta Stroke Program Early Computed Tomography Score (ASPECTS) can support clinical decision making. Based on a deep learning algorithm, we developed an automated ASPECTS scoring system (Heuron ASPECTS) and validated its performance in a prespecified clinical trial.

METHODS

For model training, we used non-contrast computed tomography images of 487 patients with acute ischemic stroke (AIS). For the clinical trial, 326 patients (87 with AIS, 56 with other acute brain diseases, and 183 with no brain disease) were enrolled. The results of Heuron ASPECTS were compared with the consensus generated by two stroke experts using the Bland-Altman agreement. A mean difference of less than 0.35 and a maximum allowed difference of less than 3.8 were considered the primary outcome target. The sensitivity and specificity of the model for the 10 regions of interest and dichotomized ASPECTS were calculated.

RESULTS

The Bland-Altman agreement had a mean difference of 0.03 [95% confidence interval (CI): -0.08 to 0.14], and the upper and lower limits of agreement were 2.80 [95% CI: 2.62 to 2.99] and -2.74 [95% CI: -2.92 to -2.55], respectively. For ASPECTS calculation, sensitivity and specificity to detect the early ischemic change for 10 ASPECTS regions were 62.78% [95% CI: 58.50 to 67.07] and 96.63% [95% CI: 96.18 to 97.09], respectively. Furthermore, in a dichotomized analysis (ASPECTS >4 vs. ≤4), the sensitivity and specificity were 94.01% [95% CI: 91.26 to 96.77] and 61.90% [95% CI: 47.22 to 76.59], respectively.

CONCLUSIONS

The current trial results show that Heuron ASPECTS reliably measures the ASPECTS for use in clinical practice.

Artificial intelligence-driven ASPECTS for the detection of early stroke changes in non-contrast CT: a systematic review and meta-analysis

BACKGROUND

Recent advances in machine learning have enabled development of the automated Alberta Stroke Program Early CT Score (ASPECTS) prediction algorithms using non-contrast enhanced computed tomography (NCCT) scans. The applicability of automated ASPECTS in daily clinical practice is yet to be established. The objective of this meta-analysis was to directly compare the performance of automated and manual ASPECTS predictions in recognizing early stroke changes on NCCT.

METHODS

The MEDLINE, Scopus, and Cochrane databases were searched. The last database search was performed on March 10, 2022. Studies reporting the diagnostic performance and validity of automated ASPECTS software compared with expert readers were included. The outcomes were the interobserver reliability of outputs between ASPECTS versus expert readings, experts versus reference standard, and ASPECTS versus reference standard by means of pooled Fisher's Z transformation of the interclass correlation coefficients (ICCs).

RESULTS

Eleven studies were included in the meta-analysis, involving 1976 patients. The meta-analyses showed good interobserver reliability between experts (ICC 0.72 (95% CI 0.63 to 0.79); p<0.001), moderate reliability in the correlation between automated and expert readings (ICC 0.54 (95% CI 0.40 to 0.67); p<0.001), good reliability between the total expert readings and the reference standard (ICC 0.62 (95% CI 0.52 to 0.71); p<0.001), and good reliability between the automated predictions and the reference standard (ICC 0.72 (95% CI 0.61 to 0.80); p<0.001).

CONCLUSIONS

Artificial intelligence-driven ASPECTS software has comparable or better performance than physicians in terms of recognizing early stroke changes on NCCT.

Falsely normal CT perfusion ischemic core readings are common and often associated with deep infarcts

BACKGROUND

Proper identification of infarct extent is crucial for thrombectomy and prognostication. We sought to study the frequency and topographic aspects of those cases in which CT perfusion (CTP) misses a core lesion that is present on initial non-contrast CT (NCCT).

METHODS

A review was carried out of a prospectively collected database of endovascular patients with anterior circulation large vessel occlusion strokes from January 2014 to November 2018. Patients with an e-ASPECTS <10 and adequate CTP maps were included. Total missed ischemic core (TMC) was defined as a CTP core lesion (relative cerebral blood flow <30%) <1 mL with a visualized hypodensity on NCCT.

RESULTS

In total, 629 patients were analyzed of which 161 (25.6%) had a TMC. On univariate analysis, TMC was associated with isolated deep middle cerebral artery (MCA) strokes (77.6% vs 56.6%, p<0.001), lower National Institutes of Health Stroke Scale (NIHSS) score (9 (15-20) vs 17 (13-21), p=0.007) and longer times to treatment (452 (288-652) min vs 355 (236-655) min, p=0.03). After adjusting for identifiable confounders, isolated deep MCA stroke was an independent predictor of TMC (OR 2.49 (95% CI 1.63 to 3.8), p<0.001). There were no differences between patients presenting with a TMC and those not with good outcomes (modified Rankin Scale 0-2) (50.8% vs 47.6%, p=0.53) or 90-day mortality (23% vs 17.6%, p=0.17). However, TMC was associated with lower rates of any parenchymal hematomas (5.2% vs 14.6%, p=0.02; aOR 0.11 (95% CI 0.01 to 0.91), p=0.04) and smaller final infarct volumes (20.5 (11.3-42.9) mL vs 47.5 (20.3-85) mL, p<0.001).

CONCLUSIONS

CTP may completely fail to detect ischemic core in as many as 25% of cases, especially in isolated deep MCA strokes. Technical refinements of the post-processing algorithms are therefore warranted. TMC infarcts may have a lower risk of reperfusion hemorrhage, potentially due to greater preservation of the neurovascular unit structure in face of delayed recovery of cerebral blood flow.

Automated quantification of atrophy and acute ischemic volume for outcome prediction in endovascular thrombectomy

Background

Short- and long-term outcomes from endovascular thrombectomy (EVT) for large vessel occlusion stroke remain variable. Numerous relevant predictors have been identified, including severity of neurological deficits, age, and imaging features. The latter is typically defined as acute changes (most commonly Alberta Stroke Programme Early CT Score, ASPECTS, at presentation), but there is little information on the impact of imaging assessment of premorbid brain health as a determinant of outcome.

Aims

To examine the impact of automated measures of stroke severity and underlying brain frailty on short- and long-term outcomes in acute stroke treated with EVT.

Methods

In 215 patients with anterior circulation stroke, who subsequently underwent EVT, automated analysis of presenting non-contrast CT scans was used to determine acute ischemic volume (AIV) and e-ASPECTS as markers of stroke severity, and cerebral atrophy as a marker of brain frailty. Univariate and multivariate logistic regression were used to identify significant predictors of NIHSS improvement, modified Rankin scale (mRS) at 90 and 30 days, mortality at 90 days and symptomatic intracranial hemorrhage (sICH) following successful EVT.

Results

For long-term outcome, atrophy and presenting NIHSS were significant predictors of mRS 0-2 and death at 90 days, whereas age did not reach significance in multivariate analysis. Conversely, for short-term NIHSS improvement, AIV and age were significant predictors, unlike presenting NIHSS. The interaction between age and NIHSS was similar to the interaction of AIV and atrophy for mRS 0-2 at 90 days.

Conclusion

Combinations of automated software-based imaging analysis and clinical data can be useful for predicting short-term neurological outcome and may improve long-term prognostication in EVT. These results provide a basis for future development of predictive tools built into decision-aiding software in stroke.

The Utility of Automated ASPECTS in Acute Ischemic Stroke for Intravenous Recombinant Tissue Plasminogen Activator (IV-rtPA) Therapy

PURPOSE

This study aimed to investigate the accuracy and clinical significance of an artificial intelligence (AI)-based automated Alberta Stroke Program Early Computed Tomography (ASPECT) scoring software of head CT for the indication of intravenous recombinant tissue plasminogen activator (rt-PA) therapy.

METHODS

This study included two populations of acute ischemic stroke: one comprised patients who had undergone head CT within 48 h of presentation (Population #1, n = 448), while the other included patients within 4.5 h from onset (Population #2, n = 132). The primary endpoint was the concordance rate of ASPECTS of the neurologists and AI software against the benchmark score. The secondary endpoints were to validate the accuracy of the neurologist and AI software in assessing the ability to rule out extensive infarction (ASPECTS of 0-5) in population #2.

RESULTS

The reading accuracy of AI software was comparable to that of the board-certified vascular neurologists. The detection rate of cardiogenic cerebral embolism was better than that of atherothrombotic cerebral infarction. By excluding extensive infarction, AI-software showed a higher specificity and equivalent sensitivity compared to those of experts.

CONCLUSIONS

The AI software for ASPECTS showed convincing agreement with expert evaluation and would be supportive in determining the indications of intravenous rt-PA therapy.

Editorial comment on "Automated quantitative lesion water uptake in acute stroke is a predictor of malignant cerebral edema"

KEY POINTS

• In patients with acute ischemic stroke, automated measurements of net water uptake in baseline CTA an NCCT scans can be used as predictor for developing malignant cerebral edema. • A creative approach can lead to broader usability of automated tools.

Automated quantitative lesion water uptake in acute stroke is a predictor of malignant cerebral edema

OBJECTIVES

Net water uptake (NWU) has been shown to have a linear relationship with brain edema. Based on an automated-Alberta Stroke Program Early Computed Tomography Score (ASPECTS) technique, we automatically derived NWU from baseline multimodal computed tomography (CT), namely ASPECTS-NWU. We aimed to determine if ASPECTS-NWU can predict the development of malignant cerebral edema (MCE).

METHODS

One hundred and forty-six patients with large-vessel occlusion were retrospectively enrolled. Quantitative NWU based on automated-ASPECTS was measured both on nonenhanced CT (NECT) and CT angiography (CTA), namely NECT-ASPECT-NWU and CTA-ASPECTS-NWU. The correlation between ASPECTS-NWU and cerebral edema (CED) grades was calculated using Spearman rank correlation. Univariate logistic regression was used to assess the effect of radiological and clinical features on MCE, and a multivariable model with significant factors from the univariate regression analysis was built. Receiver operating characteristic (ROC) was obtained and area under curve (AUC) was compared.

RESULTS

CTA-ASPECTS-NWU had a moderate positive correlation with CED grades (r = 0.62; 95% confidence interval [CI], 0.51-0.71; p < 0.001). The CTA-ASPECTS-NWU performed better than the NECT-ASPECTS-NWU with AUC: 0.88 vs. 0.71 (p < 0.001). Multivariable logistic regression model integrating CTA-ASPECTS-NWU, collateral score, and age showed the CTA-ASPECTS-NWU was an independent predictor of MCE with an AUC of 0.94 (95% CI: 0.90-0.98; p < 0.001).

CONCLUSIONS

This study demonstrates that ASPECTS-NWU is a quantitative predictor of MCE after large-vessel occlusion of the middle cerebral artery territory. The multivariable logistic regression model may enhance the identification of patients with MCE needing anti-edematous treatment.

KEY POINTS

• The automated-ASPECTS technique can automatically detect the affected regions with early ischemic changes and NWU could be manually calculated. • The CTA-ASPECTS-NWU performs better than the NECT-ASPECTS-NWU on predicting the development of MCE. • The multivariable logistic regression model may enhance the identification of patients with MCE needing anti-edematous treatment.