Automated diagnosis and segmentation of choroidal neovascularization in OCT angiography using deep learning

Optical Coherence Tomography Angiography: A 2023 Focused Update on Age-Related Macular Degeneration

Optical coherence tomography angiography (OCTA) has extensively enhanced our comprehension of eye microcirculation and of its associated diseases. In this narrative review, we explored the key concepts behind OCTA, as well as the most recent evidence in the pathophysiology of age-related macular degeneration (AMD) made possible by OCTA. These recommendations were updated since the publication in 2020, and are targeted for 2023. Importantly, as a future perspective in OCTA technology, we will discuss how artificial intelligence has been applied to OCTA, with a particular emphasis on its application to AMD study.

Applications of deep learning for detecting ophthalmic diseases with ultrawide-field fundus images

AIM

To summarize the application of deep learning in detecting ophthalmic disease with ultrawide-field fundus images and analyze the advantages, limitations, and possible solutions common to all tasks.

METHODS

We searched three academic databases, including PubMed, Web of Science, and Ovid, with the date of August 2022. We matched and screened according to the target keywords and publication year and retrieved a total of 4358 research papers according to the keywords, of which 23 studies were retrieved on applying deep learning in diagnosing ophthalmic disease with ultrawide-field images.

RESULTS

Deep learning in ultrawide-field images can detect various ophthalmic diseases and achieve great performance, including diabetic retinopathy, glaucoma, age-related macular degeneration, retinal vein occlusions, retinal detachment, and other peripheral retinal diseases. Compared to fundus images, the ultrawide-field fundus scanning laser ophthalmoscopy enables the capture of the ocular fundus up to 200° in a single exposure, which can observe more areas of the retina.

CONCLUSION

The combination of ultrawide-field fundus images and artificial intelligence will achieve great performance in diagnosing multiple ophthalmic diseases in the future.

Detection of systemic cardiovascular illnesses and cardiometabolic risk factors with machine learning and optical coherence tomography angiography: a pilot study

BACKGROUND/OBJECTIVES

Optical coherence tomography angiography (OCTA) has been found to identify changes in the retinal microvasculature of people with various cardiometabolic factors. Machine learning has previously been applied within ophthalmic imaging but has not yet been applied to these risk factors. The study aims to assess the feasibility of predicting the presence or absence of cardiovascular conditions and their associated risk factors using machine learning and OCTA.

METHODS

Cross-sectional study. Demographic and co-morbidity data was collected for each participant undergoing 3 × 3 mm, 6 × 6 mm and 8 × 8 mm OCTA scanning using the Carl Zeiss CIRRUS HD-OCT model 5000. The data was then pre-processed and randomly split into training and testing datasets (75%/25% split) before being applied to two models (Convolutional Neural Network and MoblieNetV2). Once developed on the training dataset, their performance was assessed on the unseen test dataset.

RESULTS

Two hundred forty-seven participants were included. Both models performed best in predicting the presence of hyperlipidaemia in 3 × 3 mm scans with an AUC of 0.74 and 0.81, and accuracy of 0.79 for CNN and MobileNetV2 respectively. Modest performance was achieved in the identification of diabetes mellitus, hypertension and congestive heart failure in 3 × 3 mm scans (all with AUC and accuracy >0.5). There was no significant recognition for 6 × 6 and 8 × 8 mm for any cardiometabolic risk factor.

CONCLUSION

This study demonstrates the strength of ML to identify the presence cardiometabolic factors, in particular hyperlipidaemia, in high-resolution 3 × 3 mm OCTA scans. Early detection of risk factors prior to a clinically significant event, will assist in preventing adverse outcomes for people.

A Complete Review of Automatic Detection, Segmentation, and Quantification of Neovascularization in Optical Coherence Tomography Angiography Images

Optical coherence tomography (OCT) is revolutionizing the way we assess eye complications such as diabetic retinopathy (DR) and age-related macular degeneration (AMD). With its ability to provide layer-by-layer information on the retina, OCT enables the early detection of abnormalities emerging underneath the retinal surface. The latest advancement in this field, OCT angiography (OCTA), takes this to the next level by providing detailed vascular information without requiring dye injections. One of the most significant indicators of DR and AMD is neovascularization, the abnormal growth of unhealthy vessels. In this work, the techniques and algorithms used for the automatic detection, classification, and segmentation of neovascularization in OCTA images are explored. From image processing to machine learning and deep learning, works related to automated image analysis of neovascularization are summarized from different points of view. The problems and future work of each method are also discussed.

Artificial intelligence in retinal disease: clinical application, challenges, and future directions

Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.

OCT angiography and its retinal biomarkers [Invited]

Optical coherence tomography angiography (OCTA) is a high-resolution, depth-resolved imaging modality with important applications in ophthalmic practice. An extension of structural OCT, OCTA enables non-invasive, high-contrast imaging of retinal and choroidal vasculature that are amenable to quantification. As such, OCTA offers the capability to identify and characterize biomarkers important for clinical practice and therapeutic research. Here, we review new methods for analyzing biomarkers and discuss new insights provided by OCTA.

Two-step hierarchical neural network for classification of dry age-related macular degeneration using optical coherence tomography images

Purpose

The aim of this study is to apply deep learning techniques for the development and validation of a system that categorizes various phases of dry age-related macular degeneration (AMD), including nascent geographic atrophy (nGA), through the analysis of optical coherence tomography (OCT) images.

Methods

A total of 3,401 OCT macular images obtained from 338 patients admitted to Shenyang Aier Eye Hospital in 2019-2021 were collected for the development of the classification model. We adopted a convolutional neural network (CNN) model and introduced hierarchical structure along with image enhancement techniques to train a two-step CNN model to detect and classify normal and three phases of dry AMD: atrophy-associated drusen regression, nGA, and geographic atrophy (GA). Five-fold cross-validation was used to evaluate the performance of the multi-label classification model.

Results

Experimental results obtained from five-fold cross-validation with different dry AMD classification models show that the proposed two-step hierarchical model with image enhancement achieves the best classification performance, with a f1-score of 91.32% and a kappa coefficients of 96.09% compared to the state-of-the-art models. The results obtained from the ablation study demonstrate that the proposed method not only improves accuracy across all categories in comparison to a traditional flat CNN model, but also substantially enhances the classification performance of nGA, with an improvement from 66.79 to 81.65%.

Conclusion

This study introduces a novel two-step hierarchical deep learning approach in categorizing dry AMD progression phases, and demonstrates its efficacy. The high classification performance suggests its potential for guiding individualized treatment plans for patients with macular degeneration.

Annotation-efficient learning for OCT segmentation

Deep learning has been successfully applied to OCT segmentation. However, for data from different manufacturers and imaging protocols, and for different regions of interest (ROIs), it requires laborious and time-consuming data annotation and training, which is undesirable in many scenarios, such as surgical navigation and multi-center clinical trials. Here we propose an annotation-efficient learning method for OCT segmentation that could significantly reduce annotation costs. Leveraging self-supervised generative learning, we train a Transformer-based model to learn the OCT imagery. Then we connect the trained Transformer-based encoder to a CNN-based decoder, to learn the dense pixel-wise prediction in OCT segmentation. These training phases use open-access data and thus incur no annotation costs, and the pre-trained model can be adapted to different data and ROIs without re-training. Based on the greedy approximation for the k-center problem, we also introduce an algorithm for the selective annotation of the target data. We verified our method on publicly-available and private OCT datasets. Compared to the widely-used U-Net model with 100% training data, our method only requires ∼10% of the data for achieving the same segmentation accuracy, and it speeds the training up to ∼3.5 times. Furthermore, our proposed method outperforms other potential strategies that could improve annotation efficiency. We think this emphasis on learning efficiency may help improve the intelligence and application penetration of OCT-based technologies.

Signal attenuation-compensated projection-resolved OCT angiography

Projection artifacts are a significant limitation of optical coherence tomographic angiography (OCTA). Existing techniques to suppress these artifacts are sensitive to image quality, becoming less reliable on low-quality images. In this study, we propose a novel signal attenuation-compensated projection-resolved OCTA (sacPR-OCTA) algorithm. In addition to removing projection artifacts, our method compensates for shadows beneath large vessels. The proposed sacPR-OCTA algorithm improves vascular continuity, reduces the similarity of vascular patterns in different plexuses, and removes more residual artifacts compared to existing methods. In addition, the sacPR-OCTA algorithm better preserves flow signal in choroidal neovascular lesions and shadow-affected areas. Because sacPR-OCTA processes the data along normalized A-lines, it provides a general solution for removing projection artifacts agnostic to the platform.

CNV-Net: Segmentation, Classification and Activity Score Measurement of Choroidal Neovascularization (CNV) Using Optical Coherence Tomography Angiography (OCTA)

This paper aims to present an artificial intelligence-based algorithm for the automated segmentation of Choroidal Neovascularization (CNV) areas and to identify the presence or absence of CNV activity criteria (branching, peripheral arcade, dark halo, shape, loop and anastomoses) in OCTA images. Methods: This retrospective and cross-sectional study includes 130 OCTA images from 101 patients with treatment-naïve CNV. At baseline, OCTA volumes of 6 × 6 mm2 were obtained to develop an AI-based algorithm to evaluate the CNV activity based on five activity criteria, including tiny branching vessels, anastomoses and loops, peripheral arcades, and perilesional hypointense halos. The proposed algorithm comprises two steps. The first block includes the pre-processing and segmentation of CNVs in OCTA images using a modified U-Net network. The second block consists of five binary classification networks, each implemented with various models from scratch, and using transfer learning from pre-trained networks. Results: The proposed segmentation network yielded an averaged Dice coefficient of 0.86. The individual classifiers corresponding to the five activity criteria (branch, peripheral arcade, dark halo, shape, loop, and anastomoses) showed accuracies of 0.84, 0.81, 0.86, 0.85, and 0.82, respectively. The AI-based algorithm potentially allows the reliable detection and segmentation of CNV from OCTA alone, without the need for imaging with contrast agents. The evaluation of the activity criteria in CNV lesions obtains acceptable results, and this algorithm could enable the objective, repeatable assessment of CNV features.

Effectual accuracy of OCT image retinal segmentation with the aid of speckle noise reduction and boundary edge detection strategy

Optical coherence tomography (OCT) has shown to be a valuable imaging tool in the field of ophthalmology, and it is becoming increasingly relevant in the field of neurology. Several OCT image segmentation methods have been developed previously to segment retinal images, however sophisticated speckle noises with low-intensity restrictions, complex retinal tissues, and inaccurate retinal layer structure remain a challenge to perform effective retinal segmentation. Hence, in this research, complicated speckle noises are removed by using a novel far-flung ratio algorithm in which preprocessing has been done to treat the speckle noise thereby highly decreasing the speckle noise through new similarity and statistical measures. Additionally, a novel haphazard walk and inter-frame flattening algorithms have been presented to tackle the weak object boundaries in OCT images. These algorithms are effective at detecting edges and estimating minimal weighted paths to better diverge, which reduces the time complexity. In addition, the segmentation of OCT images is made simpler by using a novel N-ret layer segmentation approach that executes simultaneous segmentation of various surfaces, ensures unambiguous segmentation across neighbouring layers, and improves segmentation accuracy by using two grey scale values to construct data. Consequently, the novel work outperformed the OCT image segmentation with 98.5% of accuracy.

Deep Learning in Optical Coherence Tomography Angiography: Current Progress, Challenges, and Future Directions

Optical coherence tomography angiography (OCT-A) provides depth-resolved visualization of the retinal microvasculature without intravenous dye injection. It facilitates investigations of various retinal vascular diseases and glaucoma by assessment of qualitative and quantitative microvascular changes in the different retinal layers and radial peripapillary layer non-invasively, individually, and efficiently. Deep learning (DL), a subset of artificial intelligence (AI) based on deep neural networks, has been applied in OCT-A image analysis in recent years and achieved good performance for different tasks, such as image quality control, segmentation, and classification. DL technologies have further facilitated the potential implementation of OCT-A in eye clinics in an automated and efficient manner and enhanced its clinical values for detecting and evaluating various vascular retinopathies. Nevertheless, the deployment of this combination in real-world clinics is still in the "proof-of-concept" stage due to several limitations, such as small training sample size, lack of standardized data preprocessing, insufficient testing in external datasets, and absence of standardized results interpretation. In this review, we introduce the existing applications of DL in OCT-A, summarize the potential challenges of the clinical deployment, and discuss future research directions.

Applications of Artificial Intelligence in Optical Coherence Tomography Angiography Imaging

Optical coherence tomography angiography (OCTA) and artificial intelligence (AI) are two emerging fields that complement each other. OCTA enables the noninvasive, in vivo, 3D visualization of retinal blood flow with a micrometer resolution, which has been impossible with other imaging modalities. As it does not need dye-based injections, it is also a safer procedure for patients. AI has excited great interest in many fields of daily life, by enabling automatic processing of huge amounts of data with a performance that greatly surpasses previous algorithms. It has been used in many breakthrough studies in recent years, such as the finding that AlphaGo can beat humans in the strategic board game of Go. This paper will give a short introduction into both fields and will then explore the manifold applications of AI in OCTA imaging that have been presented in the recent years. These range from signal generation over signal enhancement to interpretation tasks like segmentation and classification. In all these areas, AI-based algorithms have achieved state-of-the-art performance that has the potential to improve standard care in ophthalmology when integrated into the daily clinical routine.

Integrating a pressure sensor with an OCT handheld probe to facilitate imaging of microvascular information in skin tissue beds

Optical coherence tomography (OCT) and OCT angiography (OCTA) have been increasingly applied in skin imaging applications in dermatology, where the imaging is often performed with the OCT probe in contact with the skin surface. However, this contact mode imaging can introduce uncontrollable mechanical stress applied to the skin, inevitably complicating the interpretation of OCT/OCTA imaging results. There remains a need for a strategy for assessing local pressure applied on the skin during imaging acquisition. This study reports a handheld scanning probe integrated with built-in pressure sensors, allowing the operator to control the mechanical stress applied to the skin in real-time. With real time feedback information, the operator can easily determine whether the pressure applied to the skin would affect the imaging quality so as to obtain repeatable and reliable OCTA images for a more accurate investigation of skin conditions. Using this probe, imaging of palm skin was used in this study to demonstrate how the OCTA imaging would have been affected by different mechanical pressures ranging from 0 to 69 kPa. The results showed that OCTA imaging is relatively stable when the pressure is less than 11 kPa, and within this range, the change of vascular area density calculated from the OCTA imaging is below 0.13%. In addition, the probe was used to augment the OCT monitoring of blood flow changes during a reactive hyperemia experiment, in which the operator could properly control the amount of pressure applied to the skin surface and achieve full release after compression stimulation.

Application of Deep Learning for Automated Detection of Polypoidal Choroidal Vasculopathy in Spectral Domain Optical Coherence Tomography

Objective

To develop an automated polypoidal choroidal vasculopathy (PCV) screening model to distinguish PCV from wet age-related macular degeneration (wet AMD).

Methods

A retrospective review of spectral domain optical coherence tomography (SD-OCT) images was undertaken. The included SD-OCT images were classified into two distinct categories (PCV or wet AMD) prior to the development of the PCV screening model. The automated detection of PCV using the developed model was compared with the results of gold-standard fundus fluorescein angiography and indocyanine green (FFA + ICG) angiography. A framework of SHapley Additive exPlanations was used to interpret the results from the model.

Results

A total of 2334 SD-OCT images were enrolled for training purposes, and an additional 1171 SD-OCT images were used for external validation. The ResNet attention model yielded superior performance with average area under the curve values of 0.8 and 0.81 for the training and external validation data sets, respectively. The sensitivity/specificity calculated at a patient level was 100%/60% and 85%/71% for the training and external validation data sets, respectively.

Conclusions

A conventional FFA + ICG investigation to differentiate PCV from wet AMD requires intense health care resources and adversely affects patients. A deep learning algorithm is proposed to automatically distinguish PCV from wet AMD. The developed algorithm exhibited promising performance for further development into an alternative PCV screening tool. Enhancement of the model's performance with additional data is needed prior to implementation of this diagnostic tool in real-world clinical practice. The invisibility of disease signs within SD-OCT images is the main limitation of the proposed model.

Translational Relevance

Basic research of deep learning algorithms was applied to differentiate PCV from wet AMD based on OCT images, benefiting a diagnosis process and minimizing a risk of ICG angiogram.

Recent developments on computer aided systems for diagnosis of diabetic retinopathy: a review

Diabetes is a long-term condition in which the pancreas quits producing insulin or the body's insulin isn't utilised properly. One of the signs of diabetes is Diabetic Retinopathy. Diabetic retinopathy is the most prevalent type of diabetes, if remains unaddressed, diabetic retinopathy can affect all diabetics and become very serious, raising the chances of blindness. It is a chronic systemic condition that affects up to 80% of patients for more than ten years. Many researchers believe that if diabetes individuals are diagnosed early enough, they can be rescued from the condition in 90% of cases. Diabetes damages the capillaries, which are microscopic blood vessels in the retina. On images, blood vessel damage is usually noticeable. Therefore, in this study, several traditional, as well as deep learning-based approaches, are reviewed for the classification and detection of this particular diabetic-based eye disease known as diabetic retinopathy, and also the advantage of one approach over the other is also described. Along with the approaches, the dataset and the evaluation metrics useful for DR detection and classification are also discussed. The main finding of this study is to aware researchers about the different challenges occurs while detecting diabetic retinopathy using computer vision, deep learning techniques. Therefore, a purpose of this review paper is to sum up all the major aspects while detecting DR like lesion identification, classification and segmentation, security attacks on the deep learning models, proper categorization of datasets and evaluation metrics. As deep learning models are quite expensive and more prone to security attacks thus, in future it is advisable to develop a refined, reliable and robust model which overcomes all these aspects which are commonly found while designing deep learning models.

Segmentation of macular neovascularization and leakage in fluorescein angiography images in neovascular age-related macular degeneration using deep learning

BACKGROUND/OBJECTIVES

We aim to develop an objective fully automated Artificial intelligence (AI) algorithm for MNV lesion size and leakage area segmentation on fluorescein angiography (FA) in patients with neovascular age-related macular degeneration (nAMD).

SUBJECTS/METHODS

Two FA image datasets collected form large prospective multicentre trials consisting of 4710 images from 513 patients and 4558 images from 514 patients were used to develop and evaluate a deep learning-based algorithm to detect CNV lesion size and leakage area automatically. Manual segmentation of was performed by certified FA graders of the Vienna Reading Center. Precision, Recall and F1 score between AI predictions and manual annotations were computed. In addition, two masked retina experts conducted a clinical-applicability evaluation, comparing the quality of AI based and manual segmentations.

RESULTS

For CNV lesion size and leakage area segmentation, we obtained F1 scores of 0.73 and 0.65, respectively. Expert review resulted in a slight preference for the automated segmentations in both datasets. The quality of automated segmentations was slightly more often judged as good compared to manual annotations.

CONCLUSIONS

CNV lesion size and leakage area can be segmented by our automated model at human-level performance, its output being well-accepted during clinical applicability testing. The results provide proof-of-concept that an automated deep learning approach can improve efficacy of objective biomarker analysis in FA images and will be well-suited for clinical application.

The Development and Clinical Application of Innovative Optical Ophthalmic Imaging Techniques

The field of ophthalmic imaging has grown substantially over the last years. Massive improvements in image processing and computer hardware have allowed the emergence of multiple imaging techniques of the eye that can transform patient care. The purpose of this review is to describe the most recent advances in eye imaging and explain how new technologies and imaging methods can be utilized in a clinical setting. The introduction of optical coherence tomography (OCT) was a revolution in eye imaging and has since become the standard of care for a plethora of conditions. Its most recent iterations, OCT angiography, and visible light OCT, as well as imaging modalities, such as fluorescent lifetime imaging ophthalmoscopy, would allow a more thorough evaluation of patients and provide additional information on disease processes. Toward that goal, the application of adaptive optics (AO) and full-field scanning to a variety of eye imaging techniques has further allowed the histologic study of single cells in the retina and anterior segment. Toward the goal of remote eye care and more accessible eye imaging, methods such as handheld OCT devices and imaging through smartphones, have emerged. Finally, incorporating artificial intelligence (AI) in eye images has the potential to become a new milestone for eye imaging while also contributing in social aspects of eye care.

From Data to Deployment: The Collaborative Community on Ophthalmic Imaging Roadmap for Artificial Intelligence in Age-Related Macular Degeneration

OBJECTIVE

Health care systems worldwide are challenged to provide adequate care for the 200 million individuals with age-related macular degeneration (AMD). Artificial intelligence (AI) has the potential to make a significant, positive impact on the diagnosis and management of patients with AMD; however, the development of effective AI devices for clinical care faces numerous considerations and challenges, a fact evidenced by a current absence of Food and Drug Administration (FDA)-approved AI devices for AMD.

PURPOSE

To delineate the state of AI for AMD, including current data, standards, achievements, and challenges.

METHODS

Members of the Collaborative Community on Ophthalmic Imaging Working Group for AI in AMD attended an inaugural meeting on September 7, 2020, to discuss the topic. Subsequently, they undertook a comprehensive review of the medical literature relevant to the topic. Members engaged in meetings and discussion through December 2021 to synthesize the information and arrive at a consensus.

RESULTS

Existing infrastructure for robust AI development for AMD includes several large, labeled data sets of color fundus photography and OCT images; however, image data often do not contain the metadata necessary for the development of reliable, valid, and generalizable models. Data sharing for AMD model development is made difficult by restrictions on data privacy and security, although potential solutions are under investigation. Computing resources may be adequate for current applications, but knowledge of machine learning development may be scarce in many clinical ophthalmology settings. Despite these challenges, researchers have produced promising AI models for AMD for screening, diagnosis, prediction, and monitoring. Future goals include defining benchmarks to facilitate regulatory authorization and subsequent clinical setting generalization.

CONCLUSIONS

Delivering an FDA-authorized, AI-based device for clinical care in AMD involves numerous considerations, including the identification of an appropriate clinical application; acquisition and development of a large, high-quality data set; development of the AI architecture; training and validation of the model; and functional interactions between the model output and clinical end user. The research efforts undertaken to date represent starting points for the medical devices that eventually will benefit providers, health care systems, and patients.

A multimodal deep learning system to distinguish late stages of AMD and to compare expert vs. AI ocular biomarkers

Within the next 1.5 decades, 1 in 7 U.S. adults is anticipated to suffer from age-related macular degeneration (AMD), a degenerative retinal disease which leads to blindness if untreated. Optical coherence tomography angiography (OCTA) has become a prime technique for AMD diagnosis, specifically for late-stage neovascular (NV) AMD. Such technologies generate massive amounts of data, challenging to parse by experts alone, transforming artificial intelligence into a valuable partner. We describe a deep learning (DL) approach which achieves multi-class detection of non-AMD vs. non-neovascular (NNV) AMD vs. NV AMD from a combination of OCTA, OCT structure, 2D b-scan flow images, and high definition (HD) 5-line b-scan cubes; DL also detects ocular biomarkers indicative of AMD risk. Multimodal data were used as input to 2D-3D Convolutional Neural Networks (CNNs). Both for CNNs and experts, choroidal neovascularization and geographic atrophy were found to be important biomarkers for AMD. CNNs predict biomarkers with accuracy up to 90.2% (positive-predictive-value up to 75.8%). Just as experts rely on multimodal data to diagnose AMD, CNNs also performed best when trained on multiple inputs combined. Detection of AMD and its biomarkers from OCTA data via CNNs has tremendous potential to expedite screening of early and late-stage AMD patients.

DW-Net: Dynamic Multi-Hierarchical Weighting Segmentation Network for Joint Segmentation of Retina Layers With Choroid Neovascularization

Choroid neovascularization (CNV) is one of the blinding factors. The early detection and quantitative measurement of CNV are crucial for the establishment of subsequent treatment. Recently, many deep learning-based methods have been proposed for CNV segmentation. However, CNV is difficult to be segmented due to the complex structure of the surrounding retina. In this paper, we propose a novel dynamic multi-hierarchical weighting segmentation network (DW-Net) for the simultaneous segmentation of retinal layers and CNV. Specifically, the proposed network is composed of a residual aggregation encoder path for the selection of informative feature, a multi-hierarchical weighting connection for the fusion of detailed information and abstract information, and a dynamic decoder path. Comprehensive experimental results show that our proposed DW-Net achieves better performance than other state-of-the-art methods.

An Open-Source Deep Learning Network for Reconstruction of High-Resolution OCT Angiograms of Retinal Intermediate and Deep Capillary Plexuses

Purpose

We propose a deep learning-based image reconstruction algorithm to produce high-resolution optical coherence tomographic angiograms (OCTA) of the intermediate capillary plexus (ICP) and deep capillary plexus (DCP).

Methods

In this study, 6-mm × 6-mm macular scans with a 400 × 400 A-line sampling density and 3-mm × 3-mm scans with a 304 × 304 A-line sampling density were acquired on one or both eyes of 180 participants (including 230 eyes with diabetic retinopathy and 44 healthy controls) using a 70-kHz commercial OCT system (RTVue-XR; Optovue, Inc., Fremont, California, USA). Projection-resolved OCTA algorithm removed projection artifacts in voxel. ICP and DCP angiograms were generated by maximum projection of the OCTA signal within the respective plexus. We proposed a deep learning-based method, which receives inputs from registered 3-mm × 3-mm ICP and DCP angiograms with proper sampling density as the ground truth reference to reconstruct 6-mm × 6-mm high-resolution ICP and DCP en face OCTA. We applied the same network on 3-mm × 3-mm angiograms to enhance these images further. We evaluated the reconstructed 3-mm × 3-mm and 6-mm × 6-mm angiograms based on vascular connectivity, Weber contrast, false flow signal (flow signal erroneously generated from background), and the noise intensity in the foveal avascular zone.

Results

Compared to the originals, the Deep Capillary Angiogram Reconstruction Network (DCARnet)-enhanced 6-mm × 6-mm angiograms had significantly reduced noise intensity (ICP, 7.38 ± 25.22, P < 0.001; DCP, 11.20 ± 22.52, P < 0.001), improved vascular connectivity (ICP, 0.95 ± 0.01, P < 0.001; DCP, 0.96 ± 0.01, P < 0.001), and enhanced Weber contrast (ICP, 4.25 ± 0.10, P < 0.001; DCP, 3.84 ± 0.84, P < 0.001), without generating false flow signal when noise intensity lower than 650. The DCARnet-enhanced 3-mm × 3-mm angiograms also reduced noise, improved connectivity, and enhanced Weber contrast in 3-mm × 3-mm ICP and DCP angiograms from 101 eyes. In addition, DCARnet preserved the appearance of the dilated vessels in the reconstructed angiograms in diabetic eyes.

Conclusions

DCARnet can enhance 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiogram image quality without introducing artifacts.

Translational Relevance

The enhanced 6-mm × 6-mm angiograms may be easier for clinicians to interpret qualitatively.

Artificial intelligence in OCT angiography

Optical coherence tomographic angiography (OCTA) is a non-invasive imaging modality that provides three-dimensional, information-rich vascular images. With numerous studies demonstrating unique capabilities in biomarker quantification, diagnosis, and monitoring, OCTA technology has seen rapid adoption in research and clinical settings. The value of OCTA imaging is significantly enhanced by image analysis tools that provide rapid and accurate quantification of vascular features and pathology. Today, the most powerful image analysis methods are based on artificial intelligence (AI). While AI encompasses a large variety of techniques, machine-learning-based, and especially deep-learning-based, image analysis provides accurate measurements in a variety of contexts, including different diseases and regions of the eye. Here, we discuss the principles of both OCTA and AI that make their combination capable of answering new questions. We also review contemporary applications of AI in OCTA, which include accurate detection of pathologies such as choroidal neovascularization, precise quantification of retinal perfusion, and reliable disease diagnosis.

Artificial intelligence for improving sickle cell retinopathy diagnosis and management

Sickle cell retinopathy is often initially asymptomatic even in proliferative stages, but can progress to cause vision loss due to vitreous haemorrhages or tractional retinal detachments. Challenges with access and adherence to screening dilated fundus examinations, particularly in medically underserved areas where the burden of sickle cell disease is highest, highlight the need for novel approaches to screening for patients with vision-threatening sickle cell retinopathy. This article reviews the existing literature on and suggests future research directions for coupling artificial intelligence with multimodal retinal imaging to expand access to automated, accurate, imaging-based screening for sickle cell retinopathy. Given the variability in retinal specialist practice patterns with regards to monitoring and treatment of sickle cell retinopathy, we also discuss recent progress toward development of machine learning models that can quantitatively track disease progression over time. These artificial intelligence-based applications have great potential for informing evidence-based and resource-efficient clinical diagnosis and management of sickle cell retinopathy.

An end-to-end network for segmenting the vasculature of three retinal capillary plexuses from OCT angiographic volumes

The segmentation of en face retinal capillary angiograms from volumetric optical coherence tomographic angiography (OCTA) usually relies on retinal layer segmentation, which is time-consuming and error-prone. In this study, we developed a deep-learning-based method to segment vessels in the superficial vascular plexus (SVP), intermediate capillary plexus (ICP), and deep capillary plexus (DCP) directly from volumetric OCTA data. The method contains a three-dimensional convolutional neural network (CNN) for extracting distinct retinal layers, a custom projection module to generate three vascular plexuses from OCTA data, and three parallel CNNs to segment vasculature. Experimental results on OCTA data from rat eyes demonstrated the feasibility of the proposed method. This end-to-end network has the potential to simplify OCTA data processing on retinal vasculature segmentation. The main contribution of this study is that we propose a custom projection module to connect retinal layer segmentation and vasculature segmentation modules and automatically convert data from three to two dimensions, thus establishing an end-to-end method to segment three retinal capillary plexuses from volumetric OCTA without any human intervention.

Machine learning in optical coherence tomography angiography

Optical coherence tomography angiography (OCTA) offers a noninvasive label-free solution for imaging retinal vasculatures at the capillary level resolution. In principle, improved resolution implies a better chance to reveal subtle microvascular distortions associated with eye diseases that are asymptomatic in early stages. However, massive screening requires experienced clinicians to manually examine retinal images, which may result in human error and hinder objective screening. Recently, quantitative OCTA features have been developed to standardize and document retinal vascular changes. The feasibility of using quantitative OCTA features for machine learning classification of different retinopathies has been demonstrated. Deep learning-based applications have also been explored for automatic OCTA image analysis and disease classification. In this article, we summarize recent developments of quantitative OCTA features, machine learning image analysis, and classification.

AI-based monitoring of retinal fluid in disease activity and under therapy

Retinal fluid as the major biomarker in exudative macular disease is accurately visualized by high-resolution three-dimensional optical coherence tomography (OCT), which is used world-wide as a diagnostic gold standard largely replacing clinical examination. Artificial intelligence (AI) with its capability to objectively identify, localize and quantify fluid introduces fully automated tools into OCT imaging for personalized disease management. Deep learning performance has already proven superior to human experts, including physicians and certified readers, in terms of accuracy and speed. Reproducible measurement of retinal fluid relies on precise AI-based segmentation methods that assign a label to each OCT voxel denoting its fluid type such as intraretinal fluid (IRF) and subretinal fluid (SRF) or pigment epithelial detachment (PED) and its location within the central 1-, 3- and 6-mm macular area. Such reliable analysis is most relevant to reflect differences in pathophysiological mechanisms and impacts on retinal function, and the dynamics of fluid resolution during therapy with different regimens and substances. Yet, an in-depth understanding of the mode of action of supervised and unsupervised learning, the functionality of a convolutional neural net (CNN) and various network architectures is needed. Greater insight regarding adequate methods for performance, validation assessment, and device- and scanning-pattern-dependent variations is necessary to empower ophthalmologists to become qualified AI users. Fluid/function correlation can lead to a better definition of valid fluid variables relevant for optimal outcomes on an individual and a population level. AI-based fluid analysis opens the way for precision medicine in real-world practice of the leading retinal diseases of modern times.

Biomarkers in Early Response to Brolucizumab on Pigment Epithelium Detachment Associated with Exudative Age-Related Macular Degeneration

Background: The purpose of this study was to describe early changes in the morphology of pigment epithelium detachments (PED) after an intravitreal injection of Brolucizumab into eyes with macular neovascularization secondary to exudative age-related macular degeneration (e-AMD). Method: We included twelve eyes of 12 patients with PED secondary to e-AMD which were not responding to prior anti-VEGF treatments. An ophthalmic examination and an assessment of PED-horizontal maximal diameter (PED-HMD), PED-maximum high (PED-MH) and macular neovascularization (MNV) flow area (MNV-FA) by the means of structural optical coherence tomography (OCT) and OCT Angiography (OCT-A) were performed at baseline, as well as 1, 7, 14 and 30 days after the injection. Results: The mean age of the population of study was 78.4 (SD ± 4.8). The mean number of previous Ranibizumab or Aflibercept injections was 13 (SD ± 8). At the last follow-up visit, the PED-HMD did not significantly change (p = 0.16; F(DF:1.94, 20,85) = 1.9), the PED-MH showed a significant reduction [p = 0.01; F(DF:1.31, 14.13) = 6.84.] and the MNV-FA did not significantly differ (p = 0.1; F(1.97, 21.67) = 2.54) from baseline. No signs of ocular inflammation were observed during follow-up. Conclusions: A single Brolucizumab injection was able to determine the short-term effects on PEDs’ anatomical features of eyes with an unresponsive e-AMD.

Tissue self-attention network for the segmentation of optical coherence tomography images on the esophagus

Automatic segmentation of layered tissue is the key to esophageal optical coherence tomography (OCT) image processing. With the advent of deep learning techniques, frameworks based on a fully convolutional network are proved to be effective in classifying pixels on images. However, due to speckle noise and unfavorable imaging conditions, the esophageal tissue relevant to the diagnosis is not always easy to identify. An effective approach to address this problem is extracting more powerful feature maps, which have similar expressions for pixels in the same tissue and show discriminability from those from different tissues. In this study, we proposed a novel framework, called the tissue self-attention network (TSA-Net), which introduces the self-attention mechanism for esophageal OCT image segmentation. The self-attention module in the network is able to capture long-range context dependencies from the image and analyzes the input image in a global view, which helps to cluster pixels in the same tissue and reveal differences of different layers, thus achieving more powerful feature maps for segmentation. Experiments have visually illustrated the effectiveness of the self-attention map, and its advantages over other deep networks were also discussed.

Guidelines on Optical Coherence Tomography Angiography Imaging: 2020 Focused Update

Optical coherence tomography angiography (OCTA) has significantly expanded our knowledge of the ocular vasculature. In this review, we provide a discussion of the fundamental principles of OCTA and the application of this imaging modality to study the retinal and choroidal vessels. These guidelines are focused on 2020, and include updates since the 2019 publication. Importantly, we will comment on recent findings on OCTA technology with a special focus on the three-dimensional (3D) OCTA visualization.

IA-net: informative attention convolutional neural network for choroidal neovascularization segmentation in OCT images

Choroidal neovascularization (CNV) is a characteristic feature of wet age-related macular degeneration (AMD). Quantification of CNV is useful to clinicians in the diagnosis and treatment of CNV disease. Before quantification, CNV lesion should be delineated by automatic CNV segmentation technology. Recently, deep learning methods have achieved significant success for medical image segmentation. However, some CNVs are small objects which are hard to discriminate, resulting in performance degradation. In addition, it's difficult to train an effective network for accurate segmentation due to the complicated characteristics of CNV in OCT images. In order to tackle these two challenges, this paper proposed a novel Informative Attention Convolutional Neural Network (IA-net) for automatic CNV segmentation in OCT images. Considering that the attention mechanism has the ability to enhance the discriminative power of the interesting regions in the feature maps, the attention enhancement block is developed by introducing the additional attention constraint. It has the ability to force the model to pay high attention on CNV in the learned feature maps, improving the discriminative ability of the learned CNV features, which is useful to improve the segmentation performance on small CNV. For accurate pixel classification, the novel informative loss is proposed with the incorporation of an informative attention map. It can focus training on a set of informative samples that are difficult to be predicted. Therefore, the trained model has the ability to learn enough information to classify these informative samples, further improving the performance. The experimental results on our database demonstrate that the proposed method outperforms traditional CNV segmentation methods.

Automated Segmentation of Retinal Fluid Volumes From Structural and Angiographic Optical Coherence Tomography Using Deep Learning

Purpose

We proposed a deep convolutional neural network (CNN), named Retinal Fluid Segmentation Network (ReF-Net), to segment retinal fluid in diabetic macular edema (DME) in optical coherence tomography (OCT) volumes.

Methods

The 3- × 3-mm OCT scans were acquired on one eye by a 70-kHz OCT commercial AngioVue system (RTVue-XR; Optovue, Inc., Fremont, CA, USA) from 51 participants in a clinical diabetic retinopathy (DR) study (45 with retinal edema and six healthy controls, age 61.3 ± 10.1 (mean ± SD), 33% female, and all DR cases were diagnosed as severe NPDR or PDR). A CNN with U-Net-like architecture was constructed to detect and segment the retinal fluid. Cross-sectional OCT and angiography (OCTA) scans were used for training and testing ReF-Net. The effect of including OCTA data for retinal fluid segmentation was investigated in this study. Volumetric retinal fluid can be constructed using the output of ReF-Net. Area-under-receiver-operating-characteristic-curve, intersection-over-union (IoU), and F1-score were calculated to evaluate the performance of ReF-Net.

Results

ReF-Net shows high accuracy (F1 = 0.864 ± 0.084) in retinal fluid segmentation. The performance can be further improved (F1 = 0.892 ± 0.038) by including information from both OCTA and structural OCT. ReF-Net also shows strong robustness to shadow artifacts. Volumetric retinal fluid can provide more comprehensive information than the two-dimensional (2D) area, whether cross-sectional or en face projections.

Conclusions

A deep-learning-based method can accurately segment retinal fluid volumetrically on OCT/OCTA scans with strong robustness to shadow artifacts. OCTA data can improve retinal fluid segmentation. Volumetric representations of retinal fluid are superior to 2D projections.

Translational Relevance

Using a deep learning method to segment retinal fluid volumetrically has the potential to improve the diagnostic accuracy of diabetic macular edema by OCT systems.

Artificial intelligence for retinopathy of prematurity

PURPOSE OF REVIEW

In this article, we review the current state of artificial intelligence applications in retinopathy of prematurity (ROP) and provide insight on challenges as well as strategies for bringing these algorithms to the bedside.

RECENT FINDINGS

In the past few years, there has been a dramatic shift from machine learning approaches based on feature extraction to 'deep' convolutional neural networks for artificial intelligence applications. Several artificial intelligence for ROP approaches have demonstrated adequate proof-of-concept performance in research studies. The next steps are to determine whether these algorithms are robust to variable clinical and technical parameters in practice. Integration of artificial intelligence into ROP screening and treatment is limited by generalizability of the algorithms to maintain performance on unseen data and integration of artificial intelligence technology into new or existing clinical workflows.

SUMMARY

Real-world implementation of artificial intelligence for ROP diagnosis will require massive efforts targeted at developing standards for data acquisition, true external validation, and demonstration of feasibility. We must now focus on ethical, technical, clinical, regulatory, and financial considerations to bring this technology to the infant bedside to realize the promise offered by this technology to reduce preventable blindness from ROP.

Real-time retinal layer segmentation of OCT volumes with GPU accelerated inferencing using a compressed, low-latency neural network

Segmentation of retinal layers in optical coherence tomography (OCT) is an essential step in OCT image analysis for screening, diagnosis, and assessment of retinal disease progression. Real-time segmentation together with high-speed OCT volume acquisition allows rendering of en face OCT of arbitrary retinal layers, which can be used to increase the yield rate of high-quality scans, provide real-time feedback during image-guided surgeries, and compensate aberrations in adaptive optics (AO) OCT without using wavefront sensors. We demonstrate here unprecedented real-time OCT segmentation of eight retinal layer boundaries achieved by 3 levels of optimization: 1) a modified, low complexity, neural network structure, 2) an innovative scheme of neural network compression with TensorRT, and 3) specialized GPU hardware to accelerate computation. Inferencing with the compressed network U-NetRT took 3.5 ms, improving by 21 times the speed of conventional U-Net inference without reducing the accuracy. The latency of the entire pipeline from data acquisition to inferencing was only 41 ms, enabled by parallelized batch processing. The system and method allow real-time updating of en face OCT and OCTA visualizations of arbitrary retinal layers and plexuses in continuous mode scanning. To the best our knowledge, our work is the first demonstration of an ophthalmic imager with embedded artificial intelligence (AI) providing real-time feedback.

Reconstruction of high-resolution 6×6-mm OCT angiograms using deep learning

Typical optical coherence tomographic angiography (OCTA) acquisition areas on commercial devices are 3×3- or 6×6-mm. Compared to 3×3-mm angiograms with proper sampling density, 6×6-mm angiograms have significantly lower scan quality, with reduced signal-to-noise ratio and worse shadow artifacts due to undersampling. Here, we propose a deep-learning-based high-resolution angiogram reconstruction network (HARNet) to generate enhanced 6×6-mm superficial vascular complex (SVC) angiograms. The network was trained on data from 3×3-mm and 6×6-mm angiograms from the same eyes. The reconstructed 6×6-mm angiograms have significantly lower noise intensity, stronger contrast and better vascular connectivity than the original images. The algorithm did not generate false flow signal at the noise level presented by the original angiograms. The image enhancement produced by our algorithm may improve biomarker measurements and qualitative clinical assessment of 6×6-mm OCTA.

Adversarial convolutional network for esophageal tissue segmentation on OCT images

Automatic segmentation is important for esophageal OCT image processing, which is able to provide tissue characteristics such as shape and thickness for disease diagnosis. Existing automatical segmentation methods based on deep convolutional networks may not generate accurate segmentation results due to limited training set and various layer shapes. This study proposed a novel adversarial convolutional network (ACN) to segment esophageal OCT images using a convolutional network trained by adversarial learning. The proposed framework includes a generator and a discriminator, both with U-Net alike fully convolutional architecture. The discriminator is a hybrid network that discriminates whether the generated results are real and implements pixel classification at the same time. Leveraging on the adversarial training, the discriminator becomes more powerful. In addition, the adversarial loss is able to encode high order relationships of pixels, thus eliminating the requirements of post-processing. Experiments on segmenting esophageal OCT images from guinea pigs confirmed that the ACN outperforms several deep learning frameworks in pixel classification accuracy and improves the segmentation result. The potential clinical application of ACN for detecting eosinophilic esophagitis (EoE), an esophageal disease, is also presented in the experiment.

Adversarial convolutional network for esophageal tissue segmentation on OCT images

Automatic segmentation is important for esophageal OCT image processing, which is able to provide tissue characteristics such as shape and thickness for disease diagnosis. Existing automatical segmentation methods based on deep convolutional networks may not generate accurate segmentation results due to limited training set and various layer shapes. This study proposed a novel adversarial convolutional network (ACN) to segment esophageal OCT images using a convolutional network trained by adversarial learning. The proposed framework includes a generator and a discriminator, both with U-Net alike fully convolutional architecture. The discriminator is a hybrid network that discriminates whether the generated results are real and implements pixel classification at the same time. Leveraging on the adversarial training, the discriminator becomes more powerful. In addition, the adversarial loss is able to encode high order relationships of pixels, thus eliminating the requirements of post-processing. Experiments on segmenting esophageal OCT images from guinea pigs confirmed that the ACN outperforms several deep learning frameworks in pixel classification accuracy and improves the segmentation result. The potential clinical application of ACN for detecting eosinophilic esophagitis (EoE), an esophageal disease, is also presented in the experiment.

Artifacts and artifact removal in optical coherence tomographic angiography

Optical coherence tomographic angiography (OCTA) enables rapid imaging of retinal vasculature in three dimensions. While the technique has provided quantification of healthy vessels as well as pathology in several diseases, it is not unusual for OCTA data to contain artifacts that may influence measurement outcomes or defy image interpretation. In this review, we discuss the sources of several OCTA artifacts-including projection, motion, and signal reduction-as well as strategies for their removal. Artifact compensation can improve the accuracy of OCTA measurements, and the most effective use of the technology will incorporate hardware and software that can perform such correction.