Automatic Segmentation of Polypoidal Choroidal Vasculopathy from Indocyanine Green Angiography Using Spatial and Temporal Patterns

Joint multimodal deep learning-based automatic segmentation of ICGA and OCT images for assessment of PCV biomarkers

Purpose

To develop a fully-automatic hybrid algorithm to jointly segment and quantify biomarkers of polypoidal choroidal vasculopathy (PCV) on indocyanine green angiography (ICGA) and spectral domain-OCT (SD-OCT) images.

Design

Evaluation of diagnostic test or technology.

Participants

Seventy-two participants with PCV enrolled in clinical studies at Singapore National Eye Center.

Methods

The dataset consisted of 2-dimensional (2-D) ICGA and 3-dimensional (3-D) SD-OCT images which were spatially registered and manually segmented by clinicians. A deep learning-based hybrid algorithm called PCV-Net was developed for automatic joint segmentation of biomarkers. The PCV-Net consisted of a 2-D segmentation branch for ICGA and 3-D segmentation branch for SD-OCT. We developed fusion attention modules to connect the 2-D and 3-D branches for effective use of the spatial correspondence between the imaging modalities by sharing learned features. We also used self-supervised pretraining and ensembling to further enhance the performance of the algorithm without the need for additional datasets. We compared the proposed PCV-Net to several alternative model variants.

Main Outcome Measures

The PCV-Net was evaluated based on the Dice similarity coefficient (DSC) of the segmentations and the Pearson's correlation and absolute difference of the clinical measurements obtained from the segmentations. Manual grading was used as the gold standard.

Results

The PCV-Net showed good performance compared to manual grading and alternative model variants based on both quantitative and qualitative analyses. Compared to the baseline variant, PCV-Net improved the DSC by 0.04 to 0.43 across the different biomarkers, increased the correlations, and decreased the absolute differences of clinical measurements of interest. Specifically, the largest average (mean ± standard error) DSC improvement was for intraretinal fluid, from 0.02 ± 0.00 (baseline variant) to 0.45 ± 0.06 (PCV-Net). In general, improving trends were observed across the model variants as more technical specifications were added, demonstrating the importance of each aspect of the proposed method.

Conclusion

The PCV-Net has the potential to aid clinicians in disease assessment and research to improve clinical understanding and management of PCV.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Diagnosis of Polypoidal Choroidal Vasculopathy From Fluorescein Angiography Using Deep Learning

Purpose

To differentiate polypoidal choroidal vasculopathy (PCV) from choroidal neovascularization (CNV) and to determine the extent of PCV from fluorescein angiography (FA) using attention-based deep learning networks.

Methods

We build two deep learning networks for diagnosis of PCV using FA, one for detection and one for segmentation. Attention-gated convolutional neural network (AG-CNN) differentiates PCV from other types of wet age-related macular degeneration. Gradient-weighted class activation map (Grad-CAM) is generated to highlight important regions in the image for making the prediction, which offers explainability of the network. Attention-gated recurrent neural network (AG-PCVNet) for spatiotemporal prediction is applied for segmentation of PCV.

Results

AG-CNN is validated with a dataset containing 167 FA sequences of PCV and 70 FA sequences of CNV. AG-CNN achieves a classification accuracy of 82.80% at image-level, and 86.21% at patient-level for PCV. Grad-CAM shows that regions contributing to decision-making have on average 21.91% agreement with pathological regions identified by experts. AG-PCVNet is validated with 56 PCV sequences from the EVEREST-I study and achieves a balanced accuracy of 81.132% and dice score of 0.54.

Conclusions

The developed software provides a means of performing detection and segmentation of PCV on FA images for the first time. This study is a promising step in changing the diagnostic procedure of PCV and therefore improving the detection rate of PCV using FA alone.

Translational Relevance

The developed deep learning system enables early diagnosis of PCV using FA to assist the physician in choosing the best treatment for optimal visual prognosis.

[Uniform classification of the pachychoroid spectrum disorders]

Pachychoroid spectrum disorders are characterized by a thickening of the choroid. The spectrum includes pachychoroid pigment epitheliopathy (PPE), central serous chorioretinopathy (CCS), pachychoroid neovasculopathy (PNV), polypoidal choroidal vasculopathy (PCV)/aneurysmal type 1 choroidal neovascularization (ACNV-1), focal choroidal excavation (FCE) and peripapillary pachychoroid syndrome (PPS). If the choroid is thickened and there is a pathological alteration in the retinal pigment epithelium, the diagnosis is PPE; if the thickened choroid is accompanied by subretinal fluid, the diagnosis is CCS; if choroidal neovascularization is present, the diagnosis is PNV; if accompanied by aneurysms, the diagnosis is ACNV‑1. The PPE, CCS, PNV and ACNV‑1 were formerly regarded as independent disease entities but can be classified into four forms of a single disease family, pachychoroid macular disease.

Temporally Coherent Illumination Normalization for Indocyanine Green Video Angiography

Indocyanine green (ICG) angiography is an imaging method for doctors to observe choroidal abnormalities in human eyes. The ICG angiograms typically exhibit inhomogeneous illumination, which poses serious difficulties for the development of computer-aided diagnostic tools. In this paper, we propose a novel illumination normalization method to alleviate the inhomogeneous illumination in ICG video angiograms. In particular, we first align the viewpoint of the input ICG video angiogram using an image registration method. Then, we acquire temporal information using time-dependent intrinsic image and compute the corresponding illumination image. Finally, we correct inhomogeneous illumination from the illumination image by estimating contrast and luminosity distortion. We have conducted extensive evaluation using ICG video angiograms of 60 patients. Two video quality assessment methods are utilized to evaluate the performance of our proposed illumination normalization method. The results show that our proposed method can help improve the visual quality of ICG video angiogram. Visual evaluation by a human expert also confirms that our method yields better illumination normalization results.