Detection of ischemic changes on baseline multimodal computed tomography: expert reading vs. Brainomix and RAPID software

Deep Learning-Based ASPECTS Algorithm Enhances Reader Performance and Reduces Interpretation Time

BACKGROUND AND PURPOSE

ASPECTS is a long-standing and well-documented selection criterion for acute ischemic stroke treatment; however, the interpretation of ASPECTS is a challenging and time-consuming task for physicians with notable interobserver variabilities. We conducted a multireader, multicase study in which readers assessed ASPECTS without and with the support of a deep learning (DL)-based algorithm to analyze the impact of the software on clinicians' performance and interpretation time.

MATERIALS AND METHODS

A total of 200 NCCT scans from 5 clinical sites (27 scanner models, 4 different vendors) were retrospectively collected. The reference standard was established through the consensus of 3 expert neuroradiologists who had access to baseline CTA and CTP data. Subsequently, 8 additional clinicians (4 typical ASPECTS readers and 4 senior neuroradiologists) analyzed the NCCT scans without and with the assistance of CINA-ASPECTS (Avicenna.AI), a DL-based, FDA-cleared, and CE-marked algorithm designed to compute ASPECTS automatically. Differences were evaluated in both performance and interpretation time between the assisted and unassisted assessments.

RESULTS

With software aid, readers demonstrated increased region-based accuracy from 72.4% to 76.5% (P < .05) and increased receiver operating characteristic area under the curve (ROC AUC) from 0.749 to 0.788 (P < .05). Notably, all readers exhibited an improved ROC AUC when utilizing the software. Moreover, the use of the algorithm improved the score-based interobserver reliability and correlation coefficient of ASPECTS evaluation by 0.222 and 0.087 (P < .0001), respectively. Additionally, the readers' mean time spent analyzing a case was significantly reduced by 6% (P < .05) when aided by the algorithm.

CONCLUSIONS

With the assistance of the algorithm, readers' analyses were not only more accurate but also faster. Additionally, the overall ASPECTS evaluation exhibited greater consistency, fewer variabilities, and higher precision compared with the reference standard. This novel tool has the potential to enhance patient selection for appropriate treatment by enabling physicians to deliver accurate and timely diagnoses of acute ischemic stroke.

Software with artificial intelligence-derived algorithms for analysing CT brain scans in people with a suspected acute stroke: a systematic review and cost-effectiveness analysis

Background

Artificial intelligence-derived software technologies have been developed that are intended to facilitate the review of computed tomography brain scans in patients with suspected stroke.

Objectives

To evaluate the clinical and cost-effectiveness of using artificial intelligence-derived software to support review of computed tomography brain scans in acute stroke in the National Health Service setting.

Methods

Twenty-five databases were searched to July 2021. The review process included measures to minimise error and bias. Results were summarised by research question, artificial intelligence-derived software technology and study type. The health economic analysis focused on the addition of artificial intelligence-derived software-assisted review of computed tomography angiography brain scans for guiding mechanical thrombectomy treatment decisions for people with an ischaemic stroke. The de novo model (developed in R Shiny, R Foundation for Statistical Computing, Vienna, Austria) consisted of a decision tree (short-term) and a state transition model (long-term) to calculate the mean expected costs and quality-adjusted life-years for people with ischaemic stroke and suspected large-vessel occlusion comparing artificial intelligence-derived software-assisted review to usual care.

Results

A total of 22 studies (30 publications) were included in the review; 18/22 studies concerned artificial intelligence-derived software for the interpretation of computed tomography angiography to detect large-vessel occlusion. No study evaluated an artificial intelligence-derived software technology used as specified in the inclusion criteria for this assessment. For artificial intelligence-derived software technology alone, sensitivity and specificity estimates for proximal anterior circulation large-vessel occlusion were 95.4% (95% confidence interval 92.7% to 97.1%) and 79.4% (95% confidence interval 75.8% to 82.6%) for Rapid (iSchemaView, Menlo Park, CA, USA) computed tomography angiography, 91.2% (95% confidence interval 77.0% to 97.0%) and 85.0 (95% confidence interval 64.0% to 94.8%) for Viz LVO (Viz.ai, Inc., San Fransisco, VA, USA) large-vessel occlusion, 83.8% (95% confidence interval 77.3% to 88.7%) and 95.7% (95% confidence interval 91.0% to 98.0%) for Brainomix (Brainomix Ltd, Oxford, UK) e-computed tomography angiography and 98.1% (95% confidence interval 94.5% to 99.3%) and 98.2% (95% confidence interval 95.5% to 99.3%) for Avicenna CINA (Avicenna AI, La Ciotat, France) large-vessel occlusion, based on one study each. These studies were not considered appropriate to inform cost-effectiveness modelling but formed the basis by which the accuracy of artificial intelligence plus human reader could be elicited by expert opinion. Probabilistic analyses based on the expert elicitation to inform the sensitivity of the diagnostic pathway indicated that the addition of artificial intelligence to detect large-vessel occlusion is potentially more effective (quality-adjusted life-year gain of 0.003), more costly (increased costs of £8.61) and cost-effective for willingness-to-pay thresholds of £3380 per quality-adjusted life-year and higher.

Limitations and conclusions

The available evidence is not suitable to determine the clinical effectiveness of using artificial intelligence-derived software to support the review of computed tomography brain scans in acute stroke. The economic analyses did not provide evidence to prefer the artificial intelligence-derived software strategy over current clinical practice. However, results indicated that if the addition of artificial intelligence-derived software-assisted review for guiding mechanical thrombectomy treatment decisions increased the sensitivity of the diagnostic pathway (i.e. reduced the proportion of undetected large-vessel occlusions), this may be considered cost-effective.

Future work

Large, preferably multicentre, studies are needed (for all artificial intelligence-derived software technologies) that evaluate these technologies as they would be implemented in clinical practice.

Study registration

This study is registered as PROSPERO CRD42021269609.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR133836) and is published in full in Health Technology Assessment; Vol. 28, No. 11. See the NIHR Funding and Awards website for further award information.

Automated advanced imaging in acute ischemic stroke. Certainties and uncertainties

The purpose of this is study was to review pearls and pitfalls of advanced imaging, such as computed tomography perfusion and diffusion-weighed imaging and perfusion-weighted imaging in the selection of acute ischemic stroke (AIS) patients suitable for endovascular treatment (EVT) in the late time window (6-24 h from symptom onset). Advanced imaging can quantify infarct core and ischemic penumbra using specific threshold values and provides optimal selection parameters, collectively called target mismatch. More precisely, target mismatch criteria consist of core volume and/or penumbra volume and mismatch ratio (the ratio between total hypoperfusion and core volumes) with precise cut-off values. The parameters of target mismatch are automatically calculated with dedicated software packages that allow a quick and standardized interpretation of advanced imaging. However, this approach has several limitations leading to a misclassification of core and penumbra volumes. In fact, automatic software platforms are affected by technical artifacts and are not interchangeable due to a remarkable vendor-dependent variability, resulting in different estimate of target mismatch parameters. In addition, advanced imaging is not completely accurate in detecting infarct core, that can be under- or overestimated. Finally, the selection of candidates for EVT remains currently suboptimal due to the high rates of futile reperfusion and overselection caused by the use of very stringent inclusion criteria. For these reasons, some investigators recently proposed to replace advanced with conventional imaging in the selection for EVT, after the demonstration that non-contrast CT ASPECTS and computed tomography angiography collateral evaluation are not inferior to advanced images in predicting outcome in AIS patients treated with EVT. However, other authors confirmed that CTP and PWI/DWI postprocessed images are superior to conventional imaging in establishing the eligibility of patients for EVT. Therefore, the routine application of automatic assessment of advanced imaging remains a matter of debate. Recent findings suggest that the combination of conventional and advanced imaging might improving our selection criteria.

Machine-learning algorithm in acute stroke: real-world experience

AIM

To assess the clinical performance of a commercially available machine learning (ML) algorithm in acute stroke.

MATERIALS AND METHODS

CT and CT angiography (CTA) studies of 104 consecutive patients (43 females, age range 19-93, median age 62) performed for suspected acute stroke at a single tertiary institution with real-time ML software analysis (RAPID™ ASPECTS and CTA) were included. Studies were retrospectively reviewed independently by two neuroradiologists in a blinded manner.

RESULTS

The cohort included 24 acute infarcts and 16 large vessel occlusions (LVO). RAPID™ ASPECTS interpretation demonstrated high sensitivity (87.5%) and NPV (87.5%) but very poor specificity (30.9%) and PPV (30.9%) for detection of acute ischaemic parenchymal changes. There was a high percentage of false positives (51.1%). In cases of proven LVO, RAPID™ ASPECTS showed good correlation with neuroradiologists' blinded independent interpretation, Pearson correlation coefficient = 0.96 (both readers), 0.63 (RAPID™ vs reader 1), 0.69 (RAPID™ vs reader 2). RAPID™ CTA interpretation demonstrated high sensitivity (92.3%), specificity (85.3%), and negative predictive (NPV) (98.5%) with moderate positive predictive value (PPV) (52.2%) for detection of LVO (N=13). False positives accounted for 12.5% of cases, of which 27.3% were attributed to arterial stenosis.

CONCLUSION

RAPID™ CTA was robust and reliable in detection of LVO. Although demonstrating high sensitivity and NPV, RAPID™ ASPECTS interpretation was associated with a high number of false positives, which decreased clinicians' confidence in the algorithm. However, in cases of proven LVO, RAPID™ ASPECTS performed well and had good correlation with neuroradiologists' blinded interpretation.

External Validation of e-ASPECTS Software for Interpreting Brain CT in Stroke

OBJECTIVE

The purpose of this study was to test e-ASPECTS software in patients with stroke. Marketed as a decision-support tool, e-ASPECTS may detect features of ischemia or hemorrhage on computed tomography (CT) imaging and quantify ischemic extent using Alberta Stroke Program Early CT Score (ASPECTS).

METHODS

Using CT from 9 stroke studies, we compared software with masked experts. As per indications for software use, we assessed e-ASPECTS results for patients with/without middle cerebral artery (MCA) ischemia but no other cause of stroke. In an analysis outside the intended use of the software, we enriched our dataset with non-MCA ischemia, hemorrhage, and mimics to simulate a representative "front door" hospital population. With final diagnosis as the reference standard, we tested the diagnostic accuracy of e-ASPECTS for identifying stroke features (ischemia, hyperattenuated arteries, and hemorrhage) in the representative population.

RESULTS

We included 4,100 patients (51% women, median age = 78 years, National Institutes of Health Stroke Scale [NIHSS] = 10, onset to scan = 2.5 hours). Final diagnosis was ischemia (78%), hemorrhage (14%), or mimic (8%). From 3,035 CTs with expert-rated ASPECTS, most (2084/3035, 69%) e-ASPECTS results were within one point of experts. In the representative population, the diagnostic accuracy of e-ASPECTS was 71% (95% confidence interval [CI] = 70-72%) for detecting ischemic features, 85% (83-86%) for hemorrhage. Software identified more false positive ischemia (12% vs 2%) and hemorrhage (14% vs <1%) than experts.

INTERPRETATION

On independent testing, e-ASPECTS provided moderate agreement with experts and overcalled stroke features. Therefore, future prospective trials testing impacts of artificial intelligence (AI) software on patient care and outcome are required before widespread implementation of stroke decision-support software. ANN NEUROL 2022;92:943-957.

Clinical Imaging of the Penumbra in Ischemic Stroke: From the Concept to the Era of Mechanical Thrombectomy

The ischemic penumbra is defined as the severely hypoperfused, functionally impaired, at-risk but not yet infarcted tissue that will be progressively recruited into the infarct core. Early reperfusion aims to save the ischemic penumbra by preventing infarct core expansion and is the mainstay of acute ischemic stroke therapy. Intravenous thrombolysis and mechanical thrombectomy for selected patients with large vessel occlusion has been shown to improve functional outcome. Given the varying speed of infarct core progression among individuals, a therapeutic window tailored to each patient has recently been proposed. Recent studies have demonstrated that reperfusion therapies are beneficial in patients with a persistent ischemic penumbra, beyond conventional time windows. As a result, mapping the penumbra has become crucial in emergency settings for guiding personalized therapy. The penumbra was first characterized as an area with a reduced cerebral blood flow, increased oxygen extraction fraction and preserved cerebral metabolic rate of oxygen using positron emission tomography (PET) with radiolabeled O₂. Because this imaging method is not feasible in an acute clinical setting, the magnetic resonance imaging (MRI) mismatch between perfusion-weighted imaging and diffusion-weighted imaging, as well as computed tomography perfusion have been proposed as surrogate markers to identify the penumbra in acute ischemic stroke patients. Transversal studies comparing PET and MRI or using longitudinal assessment of a limited sample of patients have been used to define perfusion thresholds. However, in the era of mechanical thrombectomy, these thresholds are debatable. Using various MRI methods, the original penumbra definition has recently gained a significant interest. The aim of this review is to provide an overview of the evolution of the ischemic penumbra imaging methods, including their respective strengths and limitations, as well as to map the current intellectual structure of the field using bibliometric analysis and explore future directions.

How to Improve the Management of Acute Ischemic Stroke by Modern Technologies, Artificial Intelligence, and New Treatment Methods

Stroke remains one of the leading causes of death and disability in Europe. The European Stroke Action Plan (ESAP) defines four main targets for the years 2018 to 2030. The COVID-19 pandemic forced the use of innovative technologies and created pressure to improve internet networks. Moreover, 5G internet network will be helpful for the transfer and collecting of extremely big databases. Nowadays, the speed of internet connection is a limiting factor for robotic systems, which can be controlled and commanded potentially from various places in the world. Innovative technologies can be implemented for acute stroke patient management soon. Artificial intelligence (AI) and robotics are used increasingly often without the exception of medicine. Their implementation can be achieved in every level of stroke care. In this article, all steps of stroke health care processes are discussed in terms of how to improve them (including prehospital diagnosis, consultation, transfer of the patient, diagnosis, techniques of the treatment as well as rehabilitation and usage of AI). New ethical problems have also been discovered. Everything must be aligned to the concept of “time is brain”.