External Validation of e-ASPECTS Software for Interpreting Brain CT in Stroke

Evaluating artificial intelligence software for delineating hemorrhage extent on CT brain imaging in stroke: AI delineation of ICH on CT

BACKGROUND

The extent and distribution of intracranial hemorrhage (ICH) directly affects clinical management. Artificial intelligence (AI) software can detect and may delineate ICH extent on brain CT. We evaluated e-ASPECTS software (Brainomix Ltd.) performance for ICH delineation.

METHODS

We qualitatively assessed software delineation of ICH on CT using patients from six stroke trials. We assessed hemorrhage delineation in five compartments: lobar, deep, posterior fossa, intraventricular, extra-axial. We categorized delineation as excellent, good, moderate, or poor. We assessed quality of software delineation with number of affected compartments in univariate analysis (Kruskall-Wallis test) and ICH location using logistic regression (dependent variable: dichotomous delineation categories 'excellent-good' versus 'moderate-poor'), and report odds ratios (OR) and 95 % confidence intervals (95 %CI).

RESULTS

From 651 patients with ICH (median age 75 years, 53 % male), we included 628 with assessable CTs. Software delineation of ICH extent was 'excellent' in 189/628 (30 %), 'good' in 255/628 (41 %), 'moderate' in 127/628 (20 %), and 'poor' in 57/628 cases (9 %). The quality of software delineation of ICH was better when fewer compartments were affected (Z = 3.61-6.27; p = 0.0063). Software delineation of ICH extent was more likely to be 'excellent-good' quality when lobar alone (OR = 1.56, 95 %CI = 0.97-2.53) but 'moderate-poor' with any intraventricular (OR = 0.56, 95 %CI = 0.39-0.81, p = 0.002) or any extra-axial (OR = 0.41, 95 %CI = 0.27-0.62, p<0.001) extension.

CONCLUSIONS

Delineation of ICH extent on stroke CT scans by AI software was excellent or good in 71 % of cases but was more likely to over- or under-estimate extent when ICH was either more extensive, intraventricular, or extra-axial.

Real-world evaluation of Brainomix e-Stroke software

BACKGROUND AND PURPOSE

Brainomix e-Stroke is an artificial intelligence-based decision support tool that aids the interpretation of CT imaging in the context of acute stroke. While e-Stroke has the potential to improve the speed and accuracy of diagnosis, real-world validation is essential. The aim of this study was to prospectively evaluate the performance of Brainomix e-Stroke in an unselected cohort of patients with suspected acute ischaemic stroke.

METHODS

The study cohort included all patients admitted to the University College London Hospital Hyperacute Stroke Unit between October 2021 and April 2022. For e-ASPECTS and e-CTA, the ground truth was determined by a neuroradiologist with access to all clinical and imaging data. For e-CTP, the values of the core infarct and ischaemic penumbra were compared with those derived from syngo.via, an alternate software used at our institution.

RESULTS

1163 studies were performed in 551 patients admitted during the study period. Of these, 1130 (97.2%) were successfully processed by e-Stroke in an average of 4 min. For identifying acute middle cerebral artery territory ischaemia, e-ASPECTS had an accuracy of 77.0% and was more specific (83.5%) than sensitive (58.6%). The accuracy for identifying hyperdense thrombus was lower (69.1%), which was mainly due to many false positives (positive predictive value of 22.9%). Identification of acute haemorrhage was highly accurate (97.8%) with a sensitivity of 100% and a specificity of 97.6%; false positives were typically caused by areas of calcification. The accuracy of e-CTA for large vessel occlusions was 91.5%. The core infarct and ischaemic penumbra volumes provided by e-CTP strongly correlated with those provided by syngo.via (ρ=0.804-0.979).

CONCLUSION

Brainomix e-Stroke software provides rapid and reliable analysis of CT imaging in the acute stroke setting although, in line with the manufacturer's guidance, it should be used as an adjunct to expert interpretation rather than a standalone decision-making tool.

Most Promising Approaches to Improve Stroke Outcomes: The Stroke Treatment Academic Industry Roundtable XII Workshop

The Stroke Treatment Academic Industry Roundtable XII included a workshop to discuss the most promising approaches to improve outcome from acute stroke. The workshop brought together representatives from academia, industry, and government representatives. The discussion examined approaches in 4 epochs: pre-reperfusion, reperfusion, post-reperfusion, and access to acute stroke interventions. The participants identified areas of priority for developing new and existing treatments and approaches to improve stroke outcomes. Although many advances in acute stroke therapy have been achieved, more work is necessary for reperfusion therapies to benefit the most possible patients. Prioritization of promising approaches should help guide the use of resources and investigator efforts.

Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study

Background

The Alberta Stroke Program Early CT Score (ASPECTS) is used to quantify the extent of injury to the brain following acute ischemic stroke (AIS) and to inform treatment decisions. The e-ASPECTS software uses artificial intelligence methods to automatically process non-contrast CT (NCCT) brain scans from patients with AIS affecting the middle cerebral artery (MCA) territory and generate an ASPECTS. This study aimed to evaluate the impact of e-ASPECTS (Brainomix, Oxford, UK) on the performance of US physicians compared to a consensus ground truth.

Methods

The study used a multi-reader, multi-case design. A total of 10 US board-certified physicians (neurologists and neuroradiologists) scored 54 NCCT brain scans of patients with AIS affecting the MCA territory. Each reader scored each scan on two occasions: once with and once without reference to the e-ASPECTS software, in random order. Agreement with a reference standard (expert consensus read with reference to follow-up imaging) was evaluated with and without software support.

Results

A comparison of the area under the curve (AUC) for each reader showed a significant improvement from 0.81 to 0.83 (p = 0.028) with the support of the e-ASPECTS tool. The agreement of reader ASPECTS scoring with the reference standard was improved with e-ASPECTS compared to unassisted reading of scans: Cohen's kappa improved from 0.60 to 0.65, and the case-based weighted Kappa improved from 0.70 to 0.81.

Conclusion

Decision support with the e-ASPECTS software significantly improves the accuracy of ASPECTS scoring, even by expert US neurologists and neuroradiologists.

Performance of deep learning-based autodetection of arterial stenosis on head and neck CT angiography: an independent external validation study

PURPOSE

To externally validate the performance of automated stenosis detection on head and neck CT angiography (CTA) and investigate the impact factors using an independent bi-center dataset with digital subtraction angiography (DSA) as the ground truth.

MATERIAL AND METHODS

Patients who underwent head and neck CTA and DSA between January 2019 and December 2021 were retrospectively included. The degree of stenosis was automatically evaluated using CerebralDoc based on CTA. The performance of CerebralDoc across levels (per-patient, per-region, per-vessel, and per-segment) and thresholds (≥ 50%, ≥ 70%, and = 100%) was evaluated. Logistic regression was performed to identify independent factors associated with false negative results.

RESULTS

296 patients were analyzed. Specificity across levels and thresholds was high, exceeding 92%. The area under the curve ranged from poor (0.615, 95% CI: 0.544, 0.686; at the region-based analysis for stenosis ≥ 70%) to excellent (0.945, 95% CI: 0.905, 0.985; at the patient-based analysis for stenosis ≥ 50%). Sensitivity ranged from 0.714 (95% CI: 0.675, 0.750) at the segment-based analysis for stenosis ≥ 70% to 0.895 (95% CI: 0.849, 0.919) at the patient-based analysis for stenosis ≥ 50%. The multiple logistic regression analysis revealed that false negative results were primarily more likely to specific stenosis locations (particularly the M2 segment and skull base segment of the internal carotid artery) and occlusion.

CONCLUSIONS

CerebralDoc has the potential to automated stenosis detection on head and neck CTA, but further efforts are needed to optimize its performance.

Performance of Automated ASPECTS Software and Value as a Computer-Aided Detection Tool

BACKGROUND AND PURPOSE

ASPECTS quantifies early ischemic changes in anterior circulation stroke on NCCT but has interrater variability. We examined the agreement of conventional and automated ASPECTS and studied the value of computer-aided detection.

MATERIALS AND METHODS

We retrospectively collected imaging data from consecutive patients with acute ischemic stroke with large-vessel occlusion undergoing thrombectomy. Five raters scored conventional ASPECTS on baseline NCCTs, which were also processed by RAPID software. Conventional and automated ASPECTS were compared with a consensus criterion standard. We determined the agreement over the full ASPECTS range as well as dichotomized, reflecting thrombectomy eligibility according to the guidelines (ASPECTS 0-5 versus 6-10). Raters subsequently scored ASPECTS on the same NCCTs with assistance of the automated ASPECTS outputs, and agreement was obtained.

RESULTS

For the total of 175 cases, agreement among raters individually and the criterion standard varied from fair to good (weighted κ = between 0.38 and 0.76) and was moderate (weighted κ = 0.59) for the automated ASPECTS. The agreement of all raters individually versus the criterion standard improved with software assistance, as did the interrater agreement (overall Fleiss κ = 0.15-0.23; P < .001 and .39 to .55; P = .01 for the dichotomized ASPECTS).

CONCLUSIONS

Automated ASPECTS had agreement with the criterion standard similar to that of conventional ASPECTS. However, including automated ASPECTS during the evaluation of NCCT in acute stroke improved the agreement with the criterion standard and improved interrater agreement, which could, therefore, result in more uniform scoring in clinical practice.

Accuracy of artificial intelligence software for CT angiography in stroke

OBJECTIVE

Software developed using artificial intelligence may automatically identify arterial occlusion and provide collateral vessel scoring on CT angiography (CTA) performed acutely for ischemic stroke. We aimed to assess the diagnostic accuracy of e-CTA by Brainomix™ Ltd by large-scale independent testing using expert reading as the reference standard.

METHODS

We identified a large clinically representative sample of baseline CTA from 6 studies that recruited patients with acute stroke symptoms involving any arterial territory. We compared e-CTA results with masked expert interpretation of the same scans for the presence and location of laterality-matched arterial occlusion and/or abnormal collateral score combined into a single measure of arterial abnormality. We tested the diagnostic accuracy of e-CTA for identifying any arterial abnormality (and in a sensitivity analysis compliant with the manufacturer's guidance that software only be used to assess the anterior circulation).

RESULTS

We include CTA from 668 patients (50% female; median: age 71 years, NIHSS 9, 2.3 h from stroke onset). Experts identified arterial occlusion in 365 patients (55%); most (343, 94%) involved the anterior circulation. Software successfully processed 545/668 (82%) CTAs. The sensitivity, specificity and diagnostic accuracy of e-CTA for detecting arterial abnormality were each 72% (95% CI = 66-77%). Diagnostic accuracy was non-significantly improved in a sensitivity analysis excluding occlusions from outside the anterior circulation (76%, 95% CI = 72-80%).

INTERPRETATION

Compared to experts, the diagnostic accuracy of e-CTA for identifying acute arterial abnormality was 72-76%. Users of e-CTA should be competent in CTA interpretation to ensure all potential thrombectomy candidates are identified.

What's the Future of Vascular Neurology?

The field of vascular neurology has made tremendous advances over the last several decades, with major shifts in diagnosis, treatment, prevention, and rehabilitation of patients with stroke. Furthermore, the individuals who are providing the care represent a different cohort than those who were caring for stroke patients 30 years ago, with the increasing need for rapid decision-making for acute interventions and a larger workforce being needed to provide the many complicated aspects of care of stroke patients. Understanding the history of the field is critical before one can speculate about its future directions. In summarizing some of the past massive shifts in the past few decades, this review will discuss future opportunities and future challenges and will introduce the rest of this special issue focusing on vascular neurology in a post-thrombectomy era. Although thrombolysis and thrombectomy remain a major part of ischemic stroke management and care, in the coming years, there will likely be further modifications in how we provide the care, who provides it, how we train those individuals who provide it, where it is provided, and what data inform early management decisions.