Automated Large Artery Occlusion Detection in Stroke: A Single-Center Validation Study of an Artificial Intelligence Algorithm

Viz LVO versus Rapid LVO in detection of large vessel occlusion on CT angiography for acute stroke

BACKGROUND

Endovascular thrombectomy improves outcomes and reduces mortality for large vessel occlusion (LVO) and is time-sensitive. Computer automation may aid in the early detection of LVOs, but false values may lead to alarm desensitization. We compared Viz LVO and Rapid LVO for automated LVO detection.

METHODS

Data were retrospectively extracted from Rapid LVO and Viz LVO running concurrently from January 2022 to January 2023 on CT angiography (CTA) images compared with a radiologist interpretation. We calculated diagnostic accuracy measures and performed a McNemar test to look for a difference between the algorithms' errors. We collected demographic data, comorbidities, ejection fraction (EF), and imaging features and performed a multiple logistic regression to determine if any of these variables predicted the incorrect classification of LVO on CTA.

RESULTS

360 participants were included, with 47 large vessel occlusions. Viz LVO and Rapid LVO had a specificity of 0.96 and 0.85, a sensitivity of 0.87 and 0.87, a positive predictive value of 0.75 and 0.46, and a negative predictive value of 0.98 and 0.97, respectively. A McNemar test on correct and incorrect classifications showed a statistically significant difference between the two algorithms' errors (P=0.00000031). A multiple logistic regression showed that low EF (Viz P=0.00125, Rapid P=0.0286) and Modified Woodcock Score >1 (Viz P=0.000198, Rapid P=0.000000975) were significant predictors of incorrect classification.

CONCLUSION

Rapid LVO produced a significantly larger number of false positive values that may contribute to alarm desensitization, leading to missed alarms or delayed responses. EF and intracranial atherosclerosis were significant predictors of incorrect predictions.

The Influence of the Novel Computer-Aided Triage System Based on Artificial Intelligence on Endovascular Therapy in Patients with Large Vascular Occlusions: A Meta-Analysis

OBJECTIVE

The integration of artificial intelligence (AI) with modern healthcare has become increasingly prominent. The purpose of this study is to explore the impact of the novel computer-aided triage system based on artificial intelligence (AI-CTS) on endovascular therapy (EVT) in patients with large vascular occlusions (LVO). This study marks the first comprehensive systematic review and meta-analysis on the subject.

METHODS

A comprehensive study was performed on PubMed, Medline, Embase, Cochrane Library, and Chinese databases from their establishment to September 2023, in accordance with PRISMA recommendations. RevMan 5.4 software was used for summative analysis. The outcomes included door-to-groin (DTG) time, time from CT scan initiation to EVT, time from CT scan to reperfusion, and 90-day modified Rankin Scale (mRS).

RESULTS

A total of 7 studies involving 752 participants were included in the meta-analysis. The pooled results demonstrated that patients in the post-AI group had less time of DTG [SMD, 0.54; 95% CI, 0.40-0.69; P < 0.00001] and CT scan to EVT [SMD, 0.57; 95% CI, 0.42-0.73; P < 0.00001], as well as less time of CTA to recanalization [SMD, 0.63; 95% CI, 0.36-0.90; P < 0.00001]. There was no significant difference between the 2 groups in terms of the mRS at 90 days [OR, 0.66; 95% CI, 0.43-1.01; P = 0.06].

CONCLUSIONS

The combination of AI-CTS and EVT has improved the therapy process for LVO patients. However, the improvement in mRS at 90 days was not significant; further research is warranted.

Real-World Performance of Large Vessel Occlusion Artificial Intelligence-Based Computer-Aided Triage and Notification Algorithms-What the Stroke Team Needs to Know

PURPOSE

To evaluate the real-world performance of two FDA-approved artificial intelligence (AI)-based computer-aided triage and notification (CADt) detection devices and compare them with the manufacturer-reported performance testing in the instructions for use.

MATERIALS AND METHODS

Clinical performance of two FDA-cleared CADt large-vessel occlusion (LVO) devices was retrospectively evaluated at two separate stroke centers. Consecutive "code stroke" CT angiography examinations were included and assessed for patient demographics, scanner manufacturer, presence or absence of CADt result, CADt result, and LVO in the internal carotid artery (ICA), horizontal middle cerebral artery (MCA) segment (M1), Sylvian MCA segments after the bifurcation (M2), precommunicating part of cerebral artery, postcommunicating part of the cerebral artery, vertebral artery, basilar artery vessel segments. The original radiology report served as the reference standard, and a study radiologist extracted the above data elements from the imaging examination and radiology report.

RESULTS

At hospital A, the CADt algorithm manufacturer reports assessment of intracranial ICA and MCA with sensitivity of 97% and specificity of 95.6%. Real-world performance of 704 cases included 79 in which no CADt result was available. Sensitivity and specificity in ICA and M1 segments were 85.3% and 91.9%. Sensitivity decreased to 68.5% when M2 segments were included and to 59.9% when all proximal vessel segments were included. At hospital B the CADt algorithm manufacturer reports sensitivity of 87.8% and specificity of 89.6%, without specifying the vessel segments. Real-world performance of 642 cases included 20 cases in which no CADt result was available. Sensitivity and specificity in ICA and M1 segments were 90.7% and 97.9%. Sensitivity decreased to 76.4% when M2 segments were included and to 59.4% when all proximal vessel segments are included.

DISCUSSION

Real-world testing of two CADt LVO detection algorithms identified gaps in the detection and communication of potentially treatable LVOs when considering vessels beyond the intracranial ICA and M1 segments and in cases with absent and uninterpretable data.

FDA-approved machine learning algorithms in neuroradiology: A systematic review of the current evidence for approval

Over the past decade, machine learning (ML) and artificial intelligence (AI) have become increasingly prevalent in the medical field. In the United States, the Food and Drug Administration (FDA) is responsible for regulating AI algorithms as "medical devices" to ensure patient safety. However, recent work has shown that the FDA approval process may be deficient. In this study, we evaluate the evidence supporting FDA-approved neuroalgorithms, the subset of machine learning algorithms with applications in the central nervous system (CNS), through a systematic review of the primary literature. Articles covering the 53 FDA-approved algorithms with applications in the CNS published in PubMed, EMBASE, Google Scholar and Scopus between database inception and January 25, 2022 were queried. Initial searches identified 1505 studies, of which 92 articles met the criteria for extraction and inclusion. Studies were identified for 26 of the 53 neuroalgorithms, of which 10 algorithms had only a single peer-reviewed publication. Performance metrics were available for 15 algorithms, external validation studies were available for 24 algorithms, and studies exploring the use of algorithms in clinical practice were available for 7 algorithms. Papers studying the clinical utility of these algorithms focused on three domains: workflow efficiency, cost savings, and clinical outcomes. Our analysis suggests that there is a meaningful gap between the FDA approval of machine learning algorithms and their clinical utilization. There appears to be room for process improvement by implementation of the following recommendations: the provision of compelling evidence that algorithms perform as intended, mandating minimum sample sizes, reporting of a predefined set of performance metrics for all algorithms and clinical application of algorithms prior to widespread use. This work will serve as a baseline for future research into the ideal regulatory framework for AI applications worldwide.

Deep-learning based detection of vessel occlusions on CT-angiography in patients with suspected acute ischemic stroke

Swift diagnosis and treatment play a decisive role in the clinical outcome of patients with acute ischemic stroke (AIS), and computer-aided diagnosis (CAD) systems can accelerate the underlying diagnostic processes. Here, we developed an artificial neural network (ANN) which allows automated detection of abnormal vessel findings without any a-priori restrictions and in <2 minutes. Pseudo-prospective external validation was performed in consecutive patients with suspected AIS from 4 different hospitals during a 6-month timeframe and demonstrated high sensitivity (≥87%) and negative predictive value (≥93%). Benchmarking against two CE- and FDA-approved software solutions showed significantly higher performance for our ANN with improvements of 25-45% for sensitivity and 4-11% for NPV (p ≤ 0.003 each). We provide an imaging platform ( https://stroke.neuroAI-HD.org ) for online processing of medical imaging data with the developed ANN, including provisions for data crowdsourcing, which will allow continuous refinements and serve as a blueprint to build robust and generalizable AI algorithms.

Artificial Intelligence in Neuroradiology: A Review of Current Topics and Competition Challenges

There is an expanding body of literature that describes the application of deep learning and other machine learning and artificial intelligence methods with potential relevance to neuroradiology practice. In this article, we performed a literature review to identify recent developments on the topics of artificial intelligence in neuroradiology, with particular emphasis on large datasets and large-scale algorithm assessments, such as those used in imaging AI competition challenges. Numerous applications relevant to ischemic stroke, intracranial hemorrhage, brain tumors, demyelinating disease, and neurodegenerative/neurocognitive disorders were discussed. The potential applications of these methods to spinal fractures, scoliosis grading, head and neck oncology, and vascular imaging were also reviewed. The AI applications examined perform a variety of tasks, including localization, segmentation, longitudinal monitoring, diagnostic classification, and prognostication. While research on this topic is ongoing, several applications have been cleared for clinical use and have the potential to augment the accuracy or efficiency of neuroradiologists.

Duration and accuracy of automated stroke CT workflow with AI-supported intracranial large vessel occlusion detection

The Automation Platform (AP) is a software platform to support the workflow of radiologists and includes a stroke CT package with integrated artificial intelligence (AI) based tools. The aim of this study was to evaluate the diagnostic performance of the AP for the detection of intracranial large vessel occlusions (LVO) on conventional CT angiography (CTA), and the duration of CT processing in a cohort of acute stroke patients. The diagnostic performance for intracranial LVO detection on CTA by the AP was evaluated in a retrospective cohort of 100 acute stroke patients and compared to the diagnostic performance of five radiologists with different levels of experience. The reference standard was set by an independent neuroradiologist, with access to the readings of the different radiologists, clinical data, and follow-up. The data processing time of the AP for ICH detection on non-contrast CT, LVO detection on CTA, and the processing of CTP maps was assessed in a subset 60 patients of the retrospective cohort. This was compared to 13 radiologists, who were prospectively timed for the processing and reading of 21 stroke CTs. The AP showed shorter processing time of CTA (mean 60 versus 395 s) and CTP (mean 196 versus 243-349 s) as compared to radiologists, but showed lower sensitivity for LVO detection (sensitivity 77% of the AP vs mean sensitivity 87% of radiologists). If the AP would have been used as a stand-alone system, 1 ICA occlusion, 2 M1 occlusions and 8 M2 occlusions would have been missed, which would be eligible for mechanical thrombectomy. In conclusion, the AP showed shorter processing time of CTA and CTP as compared with radiologists, which illustrates the potential of the AP to speed-up the diagnostic work-up. However, its performance for LVO detection was lower as compared with radiologists, especially for M2 vessel occlusions.

Artificial Intelligence and Machine Learning in the Diagnosis and Management of Stroke: A Narrative Review of United States Food and Drug Administration-Approved Technologies

Stroke is an emergency in which delays in treatment can lead to significant loss of neurological function and be fatal. Technologies that increase the speed and accuracy of stroke diagnosis or assist in post-stroke rehabilitation can improve patient outcomes. No resource exists that comprehensively assesses artificial intelligence/machine learning (AI/ML)-enabled technologies indicated for the management of ischemic and hemorrhagic stroke. We queried a United States Food and Drug Administration (FDA) database, along with PubMed and private company websites, to identify the recent literature assessing the clinical performance of FDA-approved AI/ML-enabled technologies. The FDA has approved 22 AI/ML-enabled technologies that triage brain imaging for more immediate diagnosis or promote post-stroke neurological/functional recovery. Technologies that assist with diagnosis predominantly use convolutional neural networks to identify abnormal brain images (e.g., CT perfusion). These technologies perform comparably to neuroradiologists, improve clinical workflows (e.g., time from scan acquisition to reading), and improve patient outcomes (e.g., days spent in the neurological ICU). Two devices are indicated for post-stroke rehabilitation by leveraging neuromodulation techniques. Multiple FDA-approved technologies exist that can help clinicians better diagnose and manage stroke. This review summarizes the most up-to-date literature regarding the functionality, performance, and utility of these technologies so clinicians can make informed decisions when using them in practice.

Classification of nucleic acid amplification on ISFET arrays using spectrogram-based neural networks

The COVID-19 pandemic has highlighted a significant research gap in the field of molecular diagnostics. This has brought forth the need for AI-based edge solutions that can provide quick diagnostic results whilst maintaining data privacy, security and high standards of sensitivity and specificity. This paper presents a novel proof-of-concept method to detect nucleic acid amplification using ISFET sensors and deep learning. This enables the detection of DNA and RNA on a low-cost and portable lab-on-chip platform for identifying infectious diseases and cancer biomarkers. We show that by using spectrograms to transform the signal to the time-frequency domain, image processing techniques can be applied to achieve the reliable classification of the detected chemical signals. Transformation to spectrograms is beneficial as it makes the data compatible with 2D convolutional neural networks and helps gain significant performance improvement over neural networks trained on the time domain data. The trained network achieves an accuracy of 84% with a size of 30kB making it suitable for deployment on edge devices. This facilitates a new wave of intelligent lab-on-chip platforms that combine microfluidics, CMOS-based chemical sensing arrays and AI-based edge solutions for more intelligent and rapid molecular diagnostics.

Automated detection of intracranial large vessel occlusions using Viz.ai software: Experience in a large, integrated stroke network

BACKGROUND AND PURPOSE

Endovascular thrombectomy is an evidence-based treatment for large vessel occlusion (LVO) stroke. Commercially available artificial intelligence has been designed to detect the presence of an LVO on computed tomography angiogram (CTA). We compared Viz.ai-LVO (San Francisco, CA, USA) to CTA interpretation by board-certified neuroradiologists (NRs) in a large, integrated stroke network.

METHODS

From January 2021 to December 2021, we compared Viz.ai detection of an internal carotid artery (ICA) or middle cerebral artery first segment (MCA-M1) occlusion to the gold standard of CTA interpretation by board-certified NRs for all code stroke CTAs. On a monthly basis, sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) were calculated. Trend analyses were conducted to evaluate for any improvement of LVO detection by the software over time.

RESULTS

3851 patients met study inclusion criteria, of whom 220 (5.7%) had an ICA or MCA-M1 occlusion per NR. Sensitivity and specificity were 78.2% (95% CI 72%-83%) and 97% (95% CI 96%-98%), respectively. PPV was 61% (95% CI 55%-67%), NPV 99% (95% CI 98%-99%), and accuracy was 95.9% (95% CI 95.3%-96.5%). Neither specificity or sensitivity improved over time in the trend analysis.

CONCLUSIONS

Viz.ai-LVO has high specificity and moderately high sensitivity to detect an ICA or proximal MCA occlusion. The software has the potential to streamline code stroke workflows and may be particularly impactful when emergency access to NRs or vascular neurologists is limited.