Deep Learning-Based Software Improves Clinicians' Detection Sensitivity of Aneurysms on Brain TOF-MRA

Reproducibility and across-site transferability of an improved deep learning approach for aneurysm detection and segmentation in time-of-flight MR-angiograms

This study aimed to (1) replicate a deep-learning-based model for cerebral aneurysm segmentation in TOF-MRAs, (2) improve the approach by testing various fully automatic pre-processing pipelines, and (3) rigorously validate the model's transferability on independent, external test-datasets. A convolutional neural network was trained on 235 TOF-MRAs acquired on local scanners from a single vendor to segment intracranial aneurysms. Different pre-processing pipelines including bias field correction, resampling, cropping and intensity-normalization were compared regarding their effect on model performance. The models were tested on independent, external same-vendor and other-vendor test-datasets, each comprised of 70 TOF-MRAs, including patients with and without aneurysms. The best-performing model achieved excellent results on the external same-vendor test-dataset, surpassing the results of the previous publication with an improved sensitivity (0.97 vs. ~ 0.86), a higher Dice score coefficient (DSC, 0.60 ± 0.25 vs. 0.53 ± 0.31), and an improved false-positive rate (0.87 ± 1.35 vs. ~ 2.7 FPs/case). The model further showed excellent performance in the external other-vendor test-datasets (DSC 0.65 ± 0.26; sensitivity 0.92, 0.96 ± 2.38 FPs/case). Specificity was 0.38 and 0.53, respectively. Raising the voxel-size from 0.5 × 0.5×0.5 mm to 1 × 1×1 mm reduced the false-positive rate seven-fold. This study successfully replicated core principles of a previous approach for detecting and segmenting cerebral aneurysms in TOF-MRAs with a robust, fully automatable pre-processing pipeline. The model demonstrated robust transferability on two independent external datasets using TOF-MRAs from the same scanner vendor as the training dataset and from other vendors. These findings are very encouraging regarding the clinical application of such an approach.

Impact of an AI software on the diagnostic performance and reading time for the detection of cerebral aneurysms on time of flight MR-angiography

PURPOSE

To evaluate the impact of an AI-based software trained to detect cerebral aneurysms on TOF-MRA on the diagnostic performance and reading times across readers with varying experience levels.

METHODS

One hundred eighty-six MRI studies were reviewed by six readers to detect cerebral aneurysms. Initially, readings were assisted by the CNN-based software mdbrain. After 6 weeks, a second reading was conducted without software assistance. The results were compared to the consensus reading of two neuroradiological specialists and sensitivity (lesion and patient level), specificity (patient level), and false positives per case were calculated for the group of all readers, for the subgroup of physicians, and for each individual reader. Also, reading times for each reader were measured.

RESULTS

The dataset contained 54 aneurysms. The readers had no experience (three medical students), 2 years experience (resident in neuroradiology), 6 years experience (radiologist), and 12 years (neuroradiologist). Significant improvements of overall specificity and the overall number of false positives per case were observed in the reading with AI support. For the physicians, we found significant improvements of sensitivity on lesion and patient level and false positives per case. Four readers experienced reduced reading times with the software, while two encountered increased times.

CONCLUSION

In the reading with the AI-based software, we observed significant improvements in terms of specificity and false positives per case for the group of all readers and significant improvements of sensitivity and false positives per case for the physicians. Further studies are needed to investigate the effects of the AI-based software in a prospective setting.

Role of artificial intelligence and machine learning in the diagnosis of cerebrovascular disease

Introduction

Cerebrovascular diseases are known to cause significant morbidity and mortality to the general population. In patients with cerebrovascular disease, prompt clinical evaluation and radiographic interpretation are both essential in optimizing clinical management and in triaging patients for critical and potentially life-saving neurosurgical interventions. With recent advancements in the domains of artificial intelligence (AI) and machine learning (ML), many AI and ML algorithms have been developed to further optimize the diagnosis and subsequent management of cerebrovascular disease. Despite such advances, further studies are needed to substantively evaluate both the diagnostic accuracy and feasibility of these techniques for their application in clinical practice. This review aims to analyze the current use of AI and MI algorithms in the diagnosis of, and clinical decision making for cerebrovascular disease, and to discuss both the feasibility and future applications of utilizing such algorithms.

Methods

We review the use of AI and ML algorithms to assist clinicians in the diagnosis and management of ischemic stroke, hemorrhagic stroke, intracranial aneurysms, and arteriovenous malformations (AVMs). After identifying the most widely used algorithms, we provide a detailed analysis of the accuracy and effectiveness of these algorithms in practice.

Results

The incorporation of AI and ML algorithms for cerebrovascular patients has demonstrated improvements in time to detection of intracranial pathologies such as intracerebral hemorrhage (ICH) and infarcts. For ischemic and hemorrhagic strokes, commercial AI software platforms such as RapidAI and Viz.AI have bene implemented into routine clinical practice at many stroke centers to expedite the detection of infarcts and ICH, respectively. Such algorithms and neural networks have also been analyzed for use in prognostication for such cerebrovascular pathologies. These include predicting outcomes for ischemic stroke patients, hematoma expansion, risk of aneurysm rupture, bleeding of AVMs, and in predicting outcomes following interventions such as risk of occlusion for various endovascular devices. Preliminary analyses have yielded promising sensitivities when AI and ML are used in concert with imaging modalities and a multidisciplinary team of health care providers.

Conclusion

The implementation of AI and ML algorithms to supplement clinical practice has conferred a high degree of accuracy, efficiency, and expedited detection in the clinical and radiographic evaluation and management of ischemic and hemorrhagic strokes, AVMs, and aneurysms. Such algorithms have been explored for further purposes of prognostication for these conditions, with promising preliminary results. Further studies should evaluate the longitudinal implementation of such techniques into hospital networks and residency programs to supplement clinical practice, and the extent to which these techniques improve patient care and clinical outcomes in the long-term.

The automatic evaluation of steno-occlusive changes in time-of-flight magnetic resonance angiography of moyamoya patients using a 3D coordinate attention residual network

Background

Moyamoya disease (MMD) is a rare cerebrovascular occlusive disease with progressive stenosis of the terminal portion of internal cerebral artery (ICA) and its main branches, which can cause complications, such as high risks of disability and increased mortality. Accurate and timely diagnosis may be difficult for physicians who are unfamiliar to MMD. Therefore, this study aims to achieve a preoperative deep-learning-based evaluation of MMD by detecting steno-occlusive changes in the middle cerebral artery or distal ICA areas.

Methods

A fine-tuned deep learning model was developed using a three-dimensional (3D) coordinate attention residual network (3D CA-ResNet). This study enrolled 50 preoperative patients with MMD and 50 controls, and the corresponding time of flight magnetic resonance angiography (TOF-MRA) imaging data were acquired. The 3D CA-ResNet was trained based on sub-volumes and tested using patch-based and subject-based methods. The performance of the 3D CA-ResNet, as evaluated by the area under the curve (AUC) of receiving-operator characteristic, was compared with that of three other conventional 3D networks.

Results

With the resulting network, the patch-based test achieved an AUC value of 0.94 for the 3D CA-ResNet in 480 patches from 10 test patients and 10 test controls, which is significantly higher than the results of the others. The 3D CA-ResNet correctly classified the MMD patients and normal healthy controls, and the vascular lesion distribution in subjects with the disease was investigated by generating a stenosis probability map and 3D vascular structure segmentation.

Conclusions

The results demonstrated the reliability of the proposed 3D CA-ResNet in detecting stenotic areas on TOF-MRA imaging, and it outperformed three other models in identifying vascular steno-occlusive changes in patients with MMD.

Dense, deep learning-based intracranial aneurysm detection on TOF MRI using two-stage regularized U-Net

BACKGROUND AND PURPOSE

The prevalence of unruptured intracranial aneurysms in the general population is high and aneurysms are usually asymptomatic. Their diagnosis is often fortuitous on MRI and might be difficult and time consuming for the radiologist. The purpose of this study was to develop a deep learning neural network tool for automated segmentation of intracranial arteries and automated detection of intracranial aneurysms from 3D time-of-flight magnetic resonance angiography (TOF-MRA).

MATERIALS AND METHODS

3D TOF-MRA with aneurysms were retrospectively extracted. All were confirmed with angiography. The data were divided into two sets: a training set of 24 examinations and a test set of 25 examinations. Manual annotations of intracranial blood vessels and aneurysms were performed by neuroradiologists. A double convolutional neuronal network based on the U-Net architecture with regularization was used to increase performance despite a small amount of training data. The performance was evaluated for the test set. Subgroup analyses according to size and location of aneurysms were performed.

RESULTS

The average processing time was 15 min. Overall, the sensitivity and the positive predictive value of the proposed algorithm were 78% (21 of 27; 95% CI: 62-94) and 62% (21 of 34; 95%CI: 46-78) respectively, with 0.5 FP/case. Despite gradual improvement in sensitivity regarding aneurysm size, there was no significant difference of sensitivity detection between subgroups of size and location.

CONCLUSIONS

This developed tool based on a double CNN with regularization trained with small dataset, enables accurate intracranial arteries segmentation as well as effective aneurysm detection on 3D TOF MRA.

Automated Detection of Cerebral Aneurysms on TOF-MRA Using a Deep Learning Approach: An External Validation Study

BACKGROUND AND PURPOSE

Cerebral aneurysms yield the risk of rupture, severe disability and death. Thus, early detection of cerebral aneurysms is crucial to ensure timely treatment, if necessary. AI-based software tools are expected to enhance radiologists' performance in detecting pathologies like cerebral aneurysms in the future. Our aim was to evaluate the diagnostic performance of an artificial intelligence-based software designed to detect intracranial aneurysms on TOF-MRA.

MATERIALS AND METHODS

One hundred ninety-one MR imaging data sets were analyzed using the software mdbrain for the presence of intracranial aneurysms on TOF-MRA obtained using two 3T MR imaging scanners or a 1.5T MR imaging scanner according to our clinical standard protocol. The results were compared with the reading of an experienced radiologist as a criterion standard to measure the sensitivity, specificity, positive and negative predictive values, and accuracy of the software. Additionally, detection rates depending on size, morphology, and location of the aneurysms were evaluated.

RESULTS

Fifty-four aneurysms were detected by the expert reader. The overall sensitivity of the software for the detection of cerebral aneurysms was 72.6%, the specificity was 87.2%, and the accuracy was 82.6%. The positive predictive value was 67.9%, and the negative predictive value was 88.5%. We observed a sensitivity of 100% for saccular aneurysms of >5 mm without signs of thrombosis and low detection rates for fusiform or thrombosed aneurysms of 33.3% and 16.7%, respectively. Of 8 aneurysms that were not included in the initial written reports but were detected by the expert reader, retrospectively, 4 were detected by the software.

CONCLUSIONS

Our data suggest that the software can assist radiologists in reporting TOF-MRA. The software was highly reliable in detecting saccular aneurysms, while for fusiform or thrombosed aneurysms, further improvements are needed. Further studies are necessary to investigate the impact of the software on detection rates, interrater reliability, and reading times.