MF-AV-Net: an open-source deep learning network with multimodal fusion options for artery-vein segmentation in OCT angiography

Response of Diabetic Macular Edema to Anti-VEGF Medications Correlates with Improvement in Macular Vessel Architecture Measured with OCT Angiography

Purpose

Improvements in best-corrected visual acuity (BCVA) and central subfield thickness (CST) have been well documented after intravitreal injection of anti-VEGF medications in diabetic macular edema (DME); however, their effect on the vasculature of the macula in diabetic retinopathy (DR) remains poorly understood. Our aim was to explore the effect of intravitreal injection of anti-VEGF on parameters of retinal vascular microstructure in DR with OCT angiography (OCTA).

Design

Retrospective study of adult patients with DME that were treated with anti-VEGF intravitreal injections at the University of Illinois at Chicago between 2017 and 2022.

Participants

Forty-one eyes from 30 patients with nonproliferative or proliferative DR with a mean age of 58.83 ± 11.71 years, mean number of intravitreal injections of 2.8 ± 1.4, and mean follow-up of 6.5 ± 1.7 months.

Methods

ImageJ was employed to measure parameters of retinal vascular microstructure in OCTA images, which included perfusion density, vessel-length density (VLD), vessel diameter, and foveal avascular zone (FAZ) characteristics (area, perimeter, and circularity). Student t tests and analysis of variance were used to determine statistical significance.

Main Outcome Measures

A primary analysis was performed comparing the mean of each parameter of all patients as a single group at the beginning and end of the study period. A subgroup analysis was then performed after stratifying patients based on visual improvement, change in CST, prior injection history, and number of injections.

Results

Eyes demonstrated statistical improvement in BCVA logarithm of the minimum angle of resolution score and CST after anti-VEGF treatment. Primary analysis showed a reduction in the vessel diameter of the superficial and deep retinal vasculature, as well as an increase in the circularity of the FAZ within the superficial retinal vasculature after anti-VEGF treatment. Subgroup analysis revealed that eyes with improvement in BCVA exhibited reduced vessel diameter in the superficial retinal vasculature and that eyes with the largest decrease in CST displayed increased perfusion density and VLD in the deep retinal vasculature.

Conclusions

Intravitreal injection of anti-VEGF agents to treat DME improved parameters of retinal vascular microstructure on OCTA over a period of 3 to 9 months, and this effect was most pronounced in eyes that experienced improvement in BCVA and CST.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Differential artery-vein analysis improves the OCTA classification of diabetic retinopathy

This study investigates the impact of differential artery-vein (AV) analysis in optical coherence tomography angiography (OCTA) on machine learning classification of diabetic retinopathy (DR). Leveraging deep learning for arterial-venous area (AVA) segmentation, six quantitative features, including perfusion intensity density (PID), blood vessel density (BVD), vessel area flux (VAF), blood vessel caliber (BVC), blood vessel tortuosity (BVT), and vessel perimeter index (VPI) features, were derived from OCTA images before and after AV differentiation. A support vector machine (SVM) classifier was utilized to assess both binary and multiclass classifications of control, diabetic patients without DR (NoDR), mild DR, moderate DR, and severe DR groups. Initially, one-region features, i.e., quantitative features extracted from the entire OCTA, were evaluated for DR classification. Differential AV analysis improved classification accuracies from 78.86% to 87.63% and from 79.62% to 85.66% for binary and multiclass classifications, respectively. Additionally, three-region features derived from the entire image, parafovea, and perifovea, were incorporated for DR classification. Differential AV analysis further enhanced classification accuracies from 84.43% to 93.33% and from 83.40% to 89.25% for binary and multiclass classifications, respectively. These findings highlight the potential of differential AV analysis in augmenting disease diagnosis and treatment assessment using OCTA.

OCTA-500: A retinal dataset for optical coherence tomography angiography study

Optical coherence tomography angiography (OCTA) is a novel imaging modality that has been widely utilized in ophthalmology and neuroscience studies to observe retinal vessels and microvascular systems. However, publicly available OCTA datasets remain scarce. In this paper, we introduce the largest and most comprehensive OCTA dataset dubbed OCTA-500, which contains OCTA imaging under two fields of view (FOVs) from 500 subjects. The dataset provides rich images and annotations including two modalities (OCT/OCTA volumes), six types of projections, four types of text labels (age/gender/eye/disease) and seven types of segmentation labels (large vessel/capillary/artery/vein/2D FAZ/3D FAZ/retinal layers). Then, we propose a multi-object segmentation task called CAVF, which integrates capillary segmentation, artery segmentation, vein segmentation, and FAZ segmentation under a unified framework. In addition, we optimize the 3D-to-2D image projection network (IPN) to IPN-V2 to serve as one of the segmentation baselines. Experimental results demonstrate that IPN-V2 achieves an about 10% mIoU improvement over IPN on CAVF task. Finally, we further study the impact of several dataset characteristics: the training set size, the model input (OCT/OCTA, 3D volume/2D projection), the baseline networks, and the diseases. The dataset and code are publicly available at: https://ieee-dataport.org/open-access/octa-500.

Visualization and quantization of 3D retinal vessels in OCTA images

Optical coherence tomography angiography (OCTA) has been increasingly used in the analysis of ophthalmic diseases in recent years. Automatic vessel segmentation in 2D OCTA projection images is commonly used in clinical practice. However, OCTA provides a 3D volume of the retinal blood vessels with rich spatial distribution information, and it is incomplete to segment retinal vessels only in 2D projection images. Here, considering that it is difficult to manually label 3D vessels, we introduce a 3D vessel segmentation and reconstruction method for OCTA images with only 2D vessel labels. We implemented 3D vessel segmentation in the OCTA volume using a specially trained 2D vessel segmentation model. The 3D vessel segmentation results are further used to calculate 3D vessel parameters and perform 3D reconstruction. The experimental results on the public dataset OCTA-500 demonstrate that 3D vessel parameters have higher sensitivity to vascular alteration than 2D vessel parameters, which makes it meaningful for clinical analysis. The 3D vessel reconstruction provides vascular visualization in different retinal layers that can be used to monitor the development of retinal diseases. Finally, we also illustrate the use of 3D reconstruction results to determine the relationship between the location of arteries and veins.

OCT-angiography based artificial intelligence-inferred fluorescein angiography for leakage detection in retina [Invited]

Optical coherence tomography angiography (OCTA) covers most functions of fluorescein angiography (FA) when imaging the retina but lacks the ability to depict vascular leakage. Based on OCTA, we developed artificial intelligence-inferred-FA (AI-FA) to delineate leakage in eyes with diabetic retinopathy (DR). Training data of 19,648 still FA images were prepared from FA-photo and videos of 43 DR eyes. AI-FA images were generated using a convolutional neural network. AI-FA images achieved a structural similarity index of 0.91 with corresponding real FA images in DR. The AI-FA generated from OCTA correctly depicted vascular occlusion and associated leakage with enough quality, enabling precise DR diagnosis and treatment planning. A combination of OCT, OCTA, and AI-FA yields more information than real FA with reduced acquisition time without risk of allergic reactions.

Optimizing the OCTA layer fusion option for deep learning classification of diabetic retinopathy

The purpose of this study is to evaluate layer fusion options for deep learning classification of optical coherence tomography (OCT) angiography (OCTA) images. A convolutional neural network (CNN) end-to-end classifier was utilized to classify OCTA images from healthy control subjects and diabetic patients with no retinopathy (NoDR) and non-proliferative diabetic retinopathy (NPDR). For each eye, three en-face OCTA images were acquired from the superficial capillary plexus (SCP), deep capillary plexus (DCP), and choriocapillaris (CC) layers. The performances of the CNN classifier with individual layer inputs and multi-layer fusion architectures, including early-fusion, intermediate-fusion, and late-fusion, were quantitatively compared. For individual layer inputs, the superficial OCTA was observed to have the best performance, with 87.25% accuracy, 78.26% sensitivity, and 90.10% specificity, to differentiate control, NoDR, and NPDR. For multi-layer fusion options, the best option is the intermediate-fusion architecture, which achieved 92.65% accuracy, 87.01% sensitivity, and 94.37% specificity. To interpret the deep learning performance, the Gradient-weighted Class Activation Mapping (Grad-CAM) was utilized to identify spatial characteristics for OCTA classification. Comparative analysis indicates that the layer data fusion options can affect the performance of deep learning classification, and the intermediate-fusion approach is optimal for OCTA classification of DR.

Deep learning for artery-vein classification in optical coherence tomography angiography

Major retinopathies can differentially impact the arteries and veins. Traditional fundus photography provides limited resolution for visualizing retinal vascular details. Optical coherence tomography (OCT) can provide improved resolution for retinal imaging. However, it cannot discern capillary-level structures due to the limited image contrast. As a functional extension of OCT modality, optical coherence tomography angiography (OCTA) is a non-invasive, label-free method for enhanced contrast visualization of retinal vasculatures at the capillary level. Recently differential artery-vein (AV) analysis in OCTA has been demonstrated to improve the sensitivity for staging of retinopathies. Therefore, AV classification is an essential step for disease detection and diagnosis. However, current methods for AV classification in OCTA have employed multiple imagers, that is, fundus photography and OCT, and complex algorithms, thereby making it difficult for clinical deployment. On the contrary, deep learning (DL) algorithms may be able to reduce computational complexity and automate AV classification. In this article, we summarize traditional AV classification methods, recent DL methods for AV classification in OCTA, and discuss methods for interpretability in DL models.

An open-source deep learning network AVA-Net for arterial-venous area segmentation in optical coherence tomography angiography

BACKGROUND

Differential artery-vein (AV) analysis in optical coherence tomography angiography (OCTA) holds promise for the early detection of eye diseases. However, currently available methods for AV analysis are limited for binary processing of retinal vasculature in OCTA, without quantitative information of vascular perfusion intensity. This study is to develop and validate a method for quantitative AV analysis of vascular perfusion intensity.

METHOD

A deep learning network AVA-Net has been developed for automated AV area (AVA) segmentation in OCTA. Seven new OCTA features, including arterial area (AA), venous area (VA), AVA ratio (AVAR), total perfusion intensity density (T-PID), arterial PID (A-PID), venous PID (V-PID), and arterial-venous PID ratio (AV-PIDR), were extracted and tested for early detection of diabetic retinopathy (DR). Each of these seven features was evaluated for quantitative evaluation of OCTA images from healthy controls, diabetic patients without DR (NoDR), and mild DR.

RESULTS

It was observed that the area features, i.e., AA, VA and AVAR, can reveal significant differences between the control and mild DR. Vascular perfusion parameters, including T-PID and A-PID, can differentiate mild DR from control group. AV-PIDR can disclose significant differences among all three groups, i.e., control, NoDR, and mild DR. According to Bonferroni correction, the combination of A-PID and AV-PIDR can reveal significant differences in all three groups.

CONCLUSIONS

AVA-Net, which is available on GitHub for open access, enables quantitative AV analysis of AV area and vascular perfusion intensity. Comparative analysis revealed AV-PIDR as the most sensitive feature for OCTA detection of early DR. Ensemble AV feature analysis, e.g., the combination of A-PID and AV-PIDR, can further improve the performance for early DR assessment.