Multi-path cascaded U-net for vessel segmentation from fundus fluorescein angiography sequential images

Applications of Deep Learning: Automated Assessment of Vascular Tortuosity in Mouse Models of Oxygen-Induced Retinopathy

Objective

To develop a generative adversarial network (GAN) to segment major blood vessels from retinal flat-mount images from oxygen-induced retinopathy (OIR) and demonstrate the utility of these GAN-generated vessel segmentations in quantifying vascular tortuosity.

Design

Development and validation of GAN.

Subjects

Three datasets containing 1084, 50, and 20 flat-mount mice retina images with various stains used and ages at sacrifice acquired from previously published manuscripts.

Methods

Four graders manually segmented major blood vessels from flat-mount images of retinas from OIR mice. Pix2Pix, a high-resolution GAN, was trained on 984 pairs of raw flat-mount images and manual vessel segmentations and then tested on 100 and 50 image pairs from a held-out and external test set, respectively. GAN-generated and manual vessel segmentations were then used as an input into a previously published algorithm (iROP-Assist) to generate a vascular cumulative tortuosity index (CTI) for 20 image pairs containing mouse eyes treated with aflibercept versus control.

Main Outcome Measures

Mean dice coefficients were used to compare segmentation accuracy between the GAN-generated and manually annotated segmentation maps. For the image pairs treated with aflibercept versus control, mean CTIs were also calculated for both GAN-generated and manual vessel maps. Statistical significance was evaluated using Wilcoxon signed-rank tests (P ≤ 0.05 threshold for significance).

Results

The dice coefficient for the GAN-generated versus manual vessel segmentations was 0.75 ± 0.27 and 0.77 ± 0.17 for the held-out test set and external test set, respectively. The mean CTI generated from the GAN-generated and manual vessel segmentations was 1.12 ± 0.07 versus 1.03 ± 0.02 (P = 0.003) and 1.06 ± 0.04 versus 1.01 ± 0.01 (P < 0.001), respectively, for eyes treated with aflibercept versus control, demonstrating that vascular tortuosity was rescued by aflibercept when quantified by GAN-generated and manual vessel segmentations.

Conclusions

GANs can be used to accurately generate vessel map segmentations from flat-mount images. These vessel maps may be used to evaluate novel metrics of vascular tortuosity in OIR, such as CTI, and have the potential to accelerate research in treatments for ischemic retinopathies.

Financial Disclosures

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Automatic interpretation and clinical evaluation for fundus fluorescein angiography images of diabetic retinopathy patients by deep learning

BACKGROUND/AIMS

Fundus fluorescein angiography (FFA) is an important technique to evaluate diabetic retinopathy (DR) and other retinal diseases. The interpretation of FFA images is complex and time-consuming, and the ability of diagnosis is uneven among different ophthalmologists. The aim of the study is to develop a clinically usable multilevel classification deep learning model for FFA images, including prediagnosis assessment and lesion classification.

METHODS

A total of 15 599 FFA images of 1558 eyes from 845 patients diagnosed with DR were collected and annotated. Three convolutional neural network (CNN) models were trained to generate the label of image quality, location, laterality of eye, phase and five lesions. Performance of the models was evaluated by accuracy, F-1 score, the area under the curve and human-machine comparison. The images with false positive and false negative results were analysed in detail.

RESULTS

Compared with LeNet-5 and VGG16, ResNet18 got the best result, achieving an accuracy of 80.79%-93.34% for prediagnosis assessment and an accuracy of 63.67%-88.88% for lesion detection. The human-machine comparison showed that the CNN had similar accuracy with junior ophthalmologists. The false positive and false negative analysis indicated a direction of improvement.

CONCLUSION

This is the first study to do automated standardised labelling on FFA images. Our model is able to be applied in clinical practice, and will make great contributions to the development of intelligent diagnosis of FFA images.

An artificial intelligence system for the whole process from diagnosis to treatment suggestion of ischemic retinal diseases

Ischemic retinal diseases (IRDs) are a series of common blinding diseases that depend on accurate fundus fluorescein angiography (FFA) image interpretation for diagnosis and treatment. An artificial intelligence system (Ai-Doctor) was developed to interpret FFA images. Ai-Doctor performed well in image phase identification (area under the curve [AUC], 0.991-0.999, range), diabetic retinopathy (DR) and branch retinal vein occlusion (BRVO) diagnosis (AUC, 0.979-0.992), and non-perfusion area segmentation (Dice similarity coefficient [DSC], 89.7%-90.1%) and quantification. The segmentation model was expanded to unencountered IRDs (central RVO and retinal vasculitis), with DSCs of 89.2% and 83.6%, respectively. A clinically applicable ischemia index (CAII) was proposed to evaluate ischemic degree; patients with CAII values exceeding 0.17 in BRVO and 0.08 in DR may be associated with increased possibility for laser therapy. Ai-Doctor is expected to achieve accurate FFA image interpretation for IRDs, potentially reducing the reliance on retinal specialists.

Automatic segmentation of neurovascular bundle on mri using deep learning based topological modulated network

PURPOSE

Radiation damage on neurovascular bundles (NVBs) may be the cause of sexual dysfunction after radiotherapy for prostate cancer. However, it is challenging to delineate NVBs as organ-at-risks from planning CTs during radiotherapy. Recently, the integration of MR into radiotherapy made NVBs contour delineating possible. In this study, we aim to develop an MRI-based deep learning method for automatic NVB segmentation.

METHODS

The proposed method, named topological modulated network, consists of three subnetworks, that is, a focal modulation, a hierarchical block and a topological fully convolutional network (FCN). The focal modulation is used to derive the location and bounds of left and right NVBs', namely the candidate volume-of-interests (VOIs). The hierarchical block aims to highlight the NVB boundaries information on derived feature map. The topological FCN then segments the NVBs inside the VOIs by considering the topological consistency nature of the vascular delineating. Based on the location information of candidate VOIs, the segmentations of NVBs can then be brought back to the input MRI's coordinate system.

RESULTS

A five-fold cross-validation study was performed on 60 patient cases to evaluate the performance of the proposed method. The segmented results were compared with manual contours. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95 ) are (left NVB) 0.81 ± 0.10, 1.49 ± 0.88 mm, and (right NVB) 0.80 ± 0.15, 1.54 ± 1.22 mm, respectively.

CONCLUSION

We proposed a novel deep learning-based segmentation method for NVBs on pelvic MR images. The good segmentation agreement of our method with the manually drawn ground truth contours supports the feasibility of the proposed method, which can be potentially used to spare NVBs during proton and photon radiotherapy and thereby improve the quality of life for prostate cancer patients.

Discovery of Aptamers and the Acceleration of the Development of Targeting Research in Ophthalmology

Aptamers are widely applied to diagnosis and therapy because of their targeting. However, the current progress of research into aptamers for the treatment of eye disorders has not been well-documented. The current literature on aptamers was reviewed in this study. Aptamer-related drugs and biochemical sensors have been evaluated for several eye disorders within the past decade; S58 targeting TGF-β receptor II and pegaptanib targeting vascular endothelial growth factor (VEGF) are used to prevent fibrosis after glaucoma filtration surgery. Anti-brain-derived neurotrophic factor aptamer has been used to diagnose glaucoma. The first approved aptamer drug (pegaptanib) has been used to inhibit angiogenesis in age-related macular degeneration (AMD) and diabetic retinopathy (DR), and its efficacy and safety have been demonstrated in clinical trials. Aptamers, including E10030, RBM-007, AS1411, and avacincaptad pegol, targeting other angiogenesis-related biomarkers have also been discovered and subjected to clinical trials. Aptamers, such as C promoter binding factor 1, CD44, and advanced end products in AMD and DR, targeting other signal pathway proteins have also been discovered for therapy, and biochemical sensors for early diagnosis have been developed based on aptamers targeting VEGF, connective tissue growth factor, and lipocalin 1. Aptamers used for early detection and treatment of ocular tumors were derived from other disease biomarkers, such as CD71, nucleolin, and high mobility group A. In this review, the development and application of aptamers in eye disorders in recent years are systematically discussed, which may inspire a new link between aptamers and eye disorders. The aptamer development trajectory also facilitates the discovery of the pathogenesis and therapeutic strategies for various eye disorders.

Assessment of a Novel Semi-Automated Algorithm for the Quantification of the Parafoveal Capillary Network

Introduction

We present a novel semi-automated computerized method for the detection and quantification of parafoveal capillary network (PCN) in fluorescein angiography (FA) images.

Material and Methods

An algorithm detecting the superficial parafoveal capillary bed in high-resolution grayscale FA images and creating a one-pixel-wide PCN skeleton was developed using MatLab software. In addition to PCN detection, capillary density and branch point density in two circular areas centered on the center of the foveal avascular zone of 500μm and 750μm radius was calculated by the algorithm. Three consecutive FA images with distinguishable PCN from 56 eyes from 56 subjects were used for analysis. Both manual and semi-automated detection of the PCN and branch points was performed and compared. Three different intensity thresholds were used for the PCN detection to optimize the method defined as mean(I)+0.05*SD(I), mean(I) and mean(I)-0.05*SD(I), where I is the grayscale intensity of each image and SD the standard deviation. Limits of agreement (LoA), intraclass correlation coefficient (ICC) and Pearson's correlation coefficient (r) were calculated.

Results

Using mean(I)-0.05*SD(I) as threshold the average difference in PCN density between semi-automated and manual method was 0.197 (0.316) deg^-1 at 500μm radius and 0.409 (0.562) deg^-1 at 750μm radius. The LoA were -0.421 to 0.817 and -0.693 to 1.510 deg^-1, respectively. The average difference of branch point density between semi-automated and manual method was zero for both areas; LoA were -0.001 to 0.002 and -0.001 to 0.001 branch points/degrees², respectively. The other two intensity thresholds provided wider LoA for both metrics. The semi-automated algorithm showed great repeatability (ICC>0.91 in the 500μm radius and ICC>0.84 in the 750μm radius) for both metrics.

Conclusion

This semi-automated algorithm seems to provide readings in agreement with those of manual capillary tracing in FA. Larger prospective studies are needed to confirm the utility of the algorithm in clinical practice.

Retinal Microvascular Signs in Pre- and Early-Stage Diabetic Retinopathy Detected Using Wide-Field Swept-Source Optical Coherence Tomographic Angiography

Purpose Using a wide-field, high-resolution swept-source optical coherence tomographic angiography (OCTA), this study investigated microvascular abnormalities in patients with pre- and early-stage diabetic retinopathy. Methods 38 eyes of 20 people with diabetes mellitus (DM) type 2 without diabetic retinopathy (DR) and 39 eyes of 21 people with DR were enrolled in this observational and cross-sectional cohort study, and a refractive error-matched group consisting of 42 eyes of 21 non-diabetic subjects of similar age were set as the control. Each participant underwent a wide-field swept-source OCTA. On OCTA scans (1.2 cm × 1.2 cm), the mean central macular thickness (CMT), the vessel density of the inner retina, superficial capillary plexus (SCP), and deep capillary plexus (DCP) were independently measured in the whole area (1.2 cm diameter) via concentric rings with varying radii (0–0.3, 0.3–0.6, 0.6–0.9, and 0.9–1.2 cm). Results Patients whose eyes had pre-and early-stage DR showed significantly decreased vessel density in the inner retina, SCP, DCP and CMT (early-stage DR) compared with the control. In addition, compared with the average values upon wide-field OCTA, the decreases were even more pronounced for concentric rings with a radius of 0.9–1.2 cm in terms of the inner retina, SCP, DCP and CMT. Conclusions Widefield OCTA allows for a more thorough assessment of retinal changes in patients with pre- and early-stage DR.; retinal microvascular abnormalities were observed in both groups. In addition, the decreases in retinal vessel density were more significant in the peripheral concentric ring with a radius of 0.9–1.2 cm. The application of novel and wide-field OCTA could potentially help to detect earlier diabetic microvascular abnormalities.