Quantification of Key Retinal Features in Early and Late Age-Related Macular Degeneration Using Deep Learning

Deep Neural Networks for Automated Outer Plexiform Layer Subsidence Detection on Retinal OCT of Patients With Intermediate AMD

Purpose

The subsidence of the outer plexiform layer (OPL) is an important imaging biomarker on optical coherence tomography (OCT) associated with early outer retinal atrophy and a risk factor for progression to geographic atrophy in patients with intermediate age-related macular degeneration (AMD). Deep neural networks (DNNs) for OCT can support automated detection and localization of this biomarker.

Methods

The method predicts potential OPL subsidence locations on retinal OCTs. A detection module (DM) infers bounding boxes around subsidences with a likelihood score, and a classification module (CM) assesses subsidence presence at the B-scan level. Overlapping boxes between B-scans are combined and scored by the product of the DM and CM predictions. The volume-wise score is the maximum prediction across all B-scans. One development and one independent external data set were used with 140 and 26 patients with AMD, respectively.

Results

The system detected more than 85% of OPL subsidences with less than one false-positive (FP)/scan. The average area under the curve was 0.94 ± 0.03 for volume-level detection. Similar or better performance was achieved on the independent external data set.

Conclusions

DNN systems can efficiently perform automated retinal layer subsidence detection in retinal OCT images. In particular, the proposed DNN system detects OPL subsidence with high sensitivity and a very limited number of FP detections.

Translational Relevance

DNNs enable objective identification of early signs associated with high risk of progression to the atrophic late stage of AMD, ideally suited for screening and assessing the efficacy of the interventions aiming to slow disease progression.

Anomaly guided segmentation: Introducing semantic context for lesion segmentation in retinal OCT using weak context supervision from anomaly detection

Automated lesion detection in retinal optical coherence tomography (OCT) scans has shown promise for several clinical applications, including diagnosis, monitoring and guidance of treatment decisions. However, segmentation models still struggle to achieve the desired results for some complex lesions or datasets that commonly occur in real-world, e.g. due to variability of lesion phenotypes, image quality or disease appearance. While several techniques have been proposed to improve them, one line of research that has not yet been investigated is the incorporation of additional semantic context through the application of anomaly detection models. In this study we experimentally show that incorporating weak anomaly labels to standard segmentation models consistently improves lesion segmentation results. This can be done relatively easy by detecting anomalies with a separate model and then adding these output masks as an extra class for training the segmentation model. This provides additional semantic context without requiring extra manual labels. We empirically validated this strategy using two in-house and two publicly available retinal OCT datasets for multiple lesion targets, demonstrating the potential of this generic anomaly guided segmentation approach to be used as an extra tool for improving lesion detection models.

Deep-learning automated quantification of longitudinal OCT scans demonstrates reduced RPE loss rate, preservation of intact macular area and predictive value of isolated photoreceptor degeneration in geographic atrophy patients receiving C3 inhibition treatment

OBJECTIVE

To evaluate the role of automated optical coherence tomography (OCT) segmentation, using a validated deep-learning model, for assessing the effect of C3 inhibition on the area of geographic atrophy (GA); the constituent features of GA on OCT (photoreceptor degeneration (PRD), retinal pigment epithelium (RPE) loss and hypertransmission); and the area of unaffected healthy macula.To identify OCT predictive biomarkers for GA growth.

METHODS

Post hoc analysis of the FILLY trial using a deep-learning model for spectral domain OCT (SD-OCT) autosegmentation. 246 patients were randomised 1:1:1 into pegcetacoplan monthly (PM), pegcetacoplan every other month (PEOM) and sham treatment (pooled) for 12 months of treatment and 6 months of therapy-free monitoring. Only participants with Heidelberg SD-OCT were included (n=197, single eye per participant).The primary efficacy endpoint was the square root transformed change in area of GA as complete RPE and outer retinal atrophy (cRORA) in each treatment arm at 12 months, with secondary endpoints including RPE loss, hypertransmission, PRD and intact macular area.

RESULTS

Eyes treated PM showed significantly slower mean change of cRORA progression at 12 and 18 months (0.151 and 0.277 mm, p=0.0039; 0.251 and 0.396 mm, p=0.039, respectively) and RPE loss (0.147 and 0.287 mm, p=0.0008; 0.242 and 0.410 mm, p=0.00809). PEOM showed significantly slower mean change of RPE loss compared with sham at 12 months (p=0.0313). Intact macular areas were preserved in PM compared with sham at 12 and 18 months (p=0.0095 and p=0.044). PRD in isolation and intact macula areas was predictive of reduced cRORA growth at 12 months (coefficient 0.0195, p=0.01 and 0.00752, p=0.02, respectively) CONCLUSION: The OCT evidence suggests that pegcetacoplan slows progression of cRORA overall and RPE loss specifically while protecting the remaining photoreceptors and slowing the progression of healthy retina to iRORA.

Progress in clinical research and applications of retinal vessel quantification technology based on fundus imaging

Retinal blood vessels are the only directly observed blood vessels in the body; changes in them can help effective assess the occurrence and development of ocular and systemic diseases. The specificity and efficiency of retinal vessel quantification technology has improved with the advancement of retinal imaging technologies and artificial intelligence (AI) algorithms; it has garnered attention in clinical research and applications for the diagnosis and treatment of common eye and related systemic diseases. A few articles have reviewed this topic; however, a summary of recent research progress in the field is still needed. This article aimed to provide a comprehensive review of the research and applications of retinal vessel quantification technology in ocular and systemic diseases, which could update clinicians and researchers on the recent progress in this field.

Human selection bias drives the linear nature of the more ground truth effect in explainable deep learning optical coherence tomography image segmentation

Supervised deep learning (DL) algorithms are highly dependent on training data for which human graders are assigned, for example, for optical coherence tomography (OCT) image annotation. Despite the tremendous success of DL, due to human judgment, these ground truth labels can be inaccurate and/or ambiguous and cause a human selection bias. We therefore investigated the impact of the size of the ground truth and variable numbers of graders on the predictive performance of the same DL architecture and repeated each experiment three times. The largest training dataset delivered a prediction performance close to that of human experts. All DL systems utilized were highly consistent. Nevertheless, the DL under-performers could not achieve any further autonomous improvement even after repeated training. Furthermore, a quantifiable linear relationship between ground truth ambiguity and the beneficial effect of having a larger amount of ground truth data was detected and marked as the more-ground-truth effect.

Deep-learning based automated quantification of critical optical coherence tomography features in neovascular age-related macular degeneration

PURPOSE

To validate a deep learning algorithm for automated intraretinal fluid (IRF), subretinal fluid (SRF) and neovascular pigment epithelium detachment (nPED) segmentations in neovascular age-related macular degeneration (nAMD).

METHODS

In this IRB-approved study, optical coherence tomography (OCT) data from 50 patients (50 eyes) with exudative nAMD were retrospectively analysed. Two models, A1 and A2, were created based on gradings from two masked readers, R1 and R2. Area under the curve (AUC) values gauged detection performance, and quantification between readers and models was evaluated using Dice and correlation (R2) coefficients.

RESULTS

The deep learning-based algorithms had high accuracies for all fluid types between all models and readers: per B-scan IRF AUCs were 0.953, 0.932, 0.990, 0.942 for comparisons A1-R1, A1-R2, A2-R1 and A2-R2, respectively; SRF AUCs were 0.984, 0.974, 0.987, 0.979; and nPED AUCs were 0.963, 0.969, 0.961 and 0.966. Similarly, the R2 coefficients for IRF were 0.973, 0.974, 0.889 and 0.973; SRF were 0.928, 0.964, 0.965 and 0.998; and nPED were 0.908, 0.952, 0.839 and 0.905. The Dice coefficients for IRF averaged 0.702, 0.667, 0.649 and 0.631; for SRF were 0.699, 0.651, 0.692 and 0.701; and for nPED were 0.636, 0.703, 0.719 and 0.775. In an inter-observer comparison between manual readers R1 and R2, the R2 coefficient was 0.968 for IRF, 0.960 for SRF, and 0.906 for nPED, with Dice coefficients of 0.692, 0.660 and 0.784 for the same features.

CONCLUSIONS

Our deep learning-based method applied on nAMD can segment critical OCT features with performance akin to manual grading.

Research on an artificial intelligence-based myopic maculopathy grading method using EfficientNet

PURPOSE

We aimed to develop an artificial intelligence-based myopic maculopathy grading method using EfficientNet to overcome the delayed grading and diagnosis of different myopic maculopathy degrees.

METHODS

The cooperative hospital provided 4642 healthy and myopic maculopathy color fundus photographs, comprising the four degrees of myopic maculopathy and healthy fundi. The myopic maculopathy grading models were trained using EfficientNet-B0 to EfficientNet-B7 models. The diagnostic results were compared with those of the VGG16 and ResNet50 classification models. The leading evaluation indicators were sensitivity, specificity, F1 score, area under the receiver operating characteristic (ROC) curve area under curve (AUC), 95% confidence interval, kappa value, and accuracy. The ROC curves of the ten grading models were also compared.

RESULTS

We used 1199 color fundus photographs to evaluate the myopic maculopathy grading models. The size of the EfficientNet-B0 myopic maculopathy grading model was 15.6 MB, and it had the highest kappa value (88.32%) and accuracy (83.58%). The model's sensitivities to diagnose tessellated fundus (TF), diffuse chorioretinal atrophy (DCA), patchy chorioretinal atrophy (PCA), and macular atrophy (MA) were 96.86%, 75.98%, 64.67%, and 88.75%, respectively. The specificity was above 93%, and the AUCs were 0.992, 0.960, 0.964, and 0.989, respectively.

CONCLUSION

The EfficientNet models were used to design grading diagnostic models for myopic maculopathy. Based on the collected fundus images, the models could diagnose a healthy fundus and four types of myopic maculopathy. The models might help ophthalmologists to make preliminary diagnoses of different degrees of myopic maculopathy.

Recent Advances in Imaging Macular Atrophy for Late-Stage Age-Related Macular Degeneration

Age-related macular degeneration (AMD) is a leading cause of blindness worldwide. In late-stage AMD, geographic atrophy (GA) of dry AMD or choroidal neovascularization (CNV) of neovascular AMD eventually results in macular atrophy (MA), leading to significant visual loss. Despite the development of innovative therapies, there are currently no established effective treatments for MA. As a result, early detection of MA is critical in identifying later central macular involvement throughout time. Accurate and early diagnosis is achieved through a combination of clinical examination and imaging techniques. Our review of the literature depicts advances in retinal imaging to identify biomarkers of progression and risk factors for late AMD. Imaging methods like fundus photography; dye-based angiography; fundus autofluorescence (FAF); near-infrared reflectance (NIR); optical coherence tomography (OCT); and optical coherence tomography angiography (OCTA) can be used to detect and monitor the progression of retinal atrophy. These evolving diverse imaging modalities optimize detection of pathologic anatomy and measurement of visual function; they may also contribute to the understanding of underlying mechanistic pathways, particularly the underlying MA changes in late AMD.

Machine Learning to Predict Response to Ranibizumab in Neovascular Age-Related Macular Degeneration

Purpose

Neovascular age-related macular degeneration (nAMD) shows variable treatment response to intravitreal anti-VEGF. This analysis compared the potential of different artificial intelligence (AI)-based machine learning models using OCT and clinical variables to accurately predict at baseline the best-corrected visual acuity (BCVA) at 9 months in response to ranibizumab in patients with nAMD.

Design

Retrospective analysis.

Participants

Baseline and imaging data from patients with subfoveal choroidal neovascularization secondary to age-related macular dengeration.

Methods

Baseline data from 502 study eyes from the HARBOR (NCT00891735) prospective clinical trial (monthly ranibizumab 0.5 and 2.0 mg arms) were pooled; 432 baseline OCT volume scans were included in the analysis. Seven models, based on baseline quantitative OCT features (Least absolute shrinkage and selection operator [Lasso] OCT minimum [min], Lasso OCT 1 standard error [SE]); on quantitative OCT features and clinical variables at baseline (Lasso min, Lasso 1SE, CatBoost, RF [random forest]); or on baseline OCT images only (deep learning [DL] model), were systematically compared with a benchmark linear model of baseline age and BCVA. Quantitative OCT features were derived by a DL segmentation model on the volume images, including retinal layer volumes and thicknesses, and retinal fluid biomarkers, including statistics on fluid volume and distribution.

Main Outcome Measures

Prognostic ability of the models was evaluated using coefficient of determination (R²) and median absolute error (MAE; letters).

Results

In the first cross-validation split, mean R² (MAE) of the Lasso min, Lasso 1SE, CatBoost, and RF models was 0.46 (7.87), 0.42 (8.43), 0.45 (7.75), and 0.43 (7.60), respectively. These models ranked higher than or similar to the benchmark model (mean R², 0.41; mean MAE, 8.20 letters) and better than OCT-only models (mean R²: Lasso OCT min, 0.20; Lasso OCT 1SE, 0.16; DL, 0.34). The Lasso min model was selected for detailed analysis; mean R² (MAE) of the Lasso min and benchmark models for 1000 repeated cross-validation splits were 0.46 (7.7) and 0.42 (8.0), respectively.

Conclusions

Machine learning models based on AI-segmented OCT features and clinical variables at baseline may predict future response to ranibizumab treatment in patients with nAMD. However, further developments will be needed to realize the clinical utility of such AI-based tools.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Predicting OCT biological marker localization from weak annotations

Recent developments in deep learning have shown success in accurately predicting the location of biological markers in Optical Coherence Tomography (OCT) volumes of patients with Age-Related Macular Degeneration (AMD) and Diabetic Retinopathy (DR). We propose a method that automatically locates biological markers to the Early Treatment Diabetic Retinopathy Study (ETDRS) rings, only requiring B-scan-level presence annotations. We trained a neural network using 22,723 OCT B-Scans of 460 eyes (433 patients) with AMD and DR, annotated with slice-level labels for Intraretinal Fluid (IRF) and Subretinal Fluid (SRF). The neural network outputs were mapped into the corresponding ETDRS rings. We incorporated the class annotations and domain knowledge into a loss function to constrain the output with biologically plausible solutions. The method was tested on a set of OCT volumes with 322 eyes (189 patients) with Diabetic Macular Edema, with slice-level SRF and IRF presence annotations for the ETDRS rings. Our method accurately predicted the presence of IRF and SRF in each ETDRS ring, outperforming previous baselines even in the most challenging scenarios. Our model was also successfully applied to en-face marker segmentation and showed consistency within C-scans, despite not incorporating volume information in the training process. We achieved a correlation coefficient of 0.946 for the prediction of the IRF area.

Deep learning in optical coherence tomography: Where are the gaps?

Optical coherence tomography (OCT) is a non-invasive optical imaging modality, which provides rapid, high-resolution and cross-sectional morphology of macular area and optic nerve head for diagnosis and managing of different eye diseases. However, interpreting OCT images requires experts in both OCT images and eye diseases since many factors such as artefacts and concomitant diseases can affect the accuracy of quantitative measurements made by post-processing algorithms. Currently, there is a growing interest in applying deep learning (DL) methods to analyse OCT images automatically. This review summarises the trends in DL-based OCT image analysis in ophthalmology, discusses the current gaps, and provides potential research directions. DL in OCT analysis shows promising performance in several tasks: (1) layers and features segmentation and quantification; (2) disease classification; (3) disease progression and prognosis; and (4) referral triage level prediction. Different studies and trends in the development of DL-based OCT image analysis are described and the following challenges are identified and described: (1) public OCT data are scarce and scattered; (2) models show performance discrepancies in real-world settings; (3) models lack of transparency; (4) there is a lack of societal acceptance and regulatory standards; and (5) OCT is still not widely available in underprivileged areas. More work is needed to tackle the challenges and gaps, before DL is further applied in OCT image analysis for clinical use.

Artificial intelligence in retinal disease: clinical application, challenges, and future directions

Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.

OCT-based deep-learning models for the identification of retinal key signs

A new system based on binary Deep Learning (DL) convolutional neural networks has been developed to recognize specific retinal abnormality signs on Optical Coherence Tomography (OCT) images useful for clinical practice. Images from the local hospital database were retrospectively selected from 2017 to 2022. Images were labeled by two retinal specialists and included central fovea cross-section OCTs. Nine models were developed using the Visual Geometry Group 16 architecture to distinguish healthy versus abnormal retinas and to identify eight different retinal abnormality signs. A total of 21,500 OCT images were screened, and 10,770 central fovea cross-section OCTs were included in the study. The system achieved high accuracy in identifying healthy retinas and specific pathological signs, ranging from 93 to 99%. Accurately detecting abnormal retinal signs from OCT images is crucial for patient care. This study aimed to identify specific signs related to retinal pathologies, aiding ophthalmologists in diagnosis. The high-accuracy system identified healthy retinas and pathological signs, making it a useful diagnostic aid. Labelled OCT images remain a challenge, but our approach reduces dataset creation time and shows DL models' potential to improve ocular pathology diagnosis and clinical decision-making.

Deep learning to detect macular atrophy in wet age-related macular degeneration using optical coherence tomography

Here, we have developed a deep learning method to fully automatically detect and quantify six main clinically relevant atrophic features associated with macular atrophy (MA) using optical coherence tomography (OCT) analysis of patients with wet age-related macular degeneration (AMD). The development of MA in patients with AMD results in irreversible blindness, and there is currently no effective method of early diagnosis of this condition, despite the recent development of unique treatments. Using OCT dataset of a total of 2211 B-scans from 45 volumetric scans of 8 patients, a convolutional neural network using one-against-all strategy was trained to present all six atrophic features followed by a validation to evaluate the performance of the models. The model predictive performance has achieved a mean dice similarity coefficient score of 0.706 ± 0.039, a mean Precision score of 0.834 ± 0.048, and a mean Sensitivity score of 0.615 ± 0.051. These results show the unique potential of using artificially intelligence-aided methods for early detection and identification of the progression of MA in wet AMD, which can further support and assist clinical decisions.

Detection of macular atrophy in age-related macular degeneration aided by artificial intelligence

INTRODUCTION

Age-related macular degeneration (AMD) is a leading cause of irreversible visual impairment worldwide. The endpoint of AMD, both in its dry or wet form, is macular atrophy (MA) which is characterized by the permanent loss of the RPE and overlying photoreceptors either in dry AMD or in wet AMD. A recognized unmet need in AMD is the early detection of MA development.

AREAS COVERED

Artificial Intelligence (AI) has demonstrated great impact in detection of retinal diseases, especially with its robust ability to analyze big data afforded by ophthalmic imaging modalities, such as color fundus photography (CFP), fundus autofluorescence (FAF), near-infrared reflectance (NIR), and optical coherence tomography (OCT). Among these, OCT has been shown to have great promise in identifying early MA using the new criteria in 2018.

EXPERT OPINION

There are few studies in which AI-OCT methods have been used to identify MA; however, results are very promising when compared to other imaging modalities. In this paper, we review the development and advances of ophthalmic imaging modalities and their combination with AI technology to detect MA in AMD. In addition, we emphasize the application of AI-OCT as an objective, cost-effective tool for the early detection and monitoring of the progression of MA in AMD.

Fully automated grading system for the evaluation of punctate epithelial erosions using deep neural networks

PURPOSE

The goal was to develop a fully automated grading system for the evaluation of punctate epithelial erosions (PEEs) using deep neural networks.

METHODS

A fully automated system was developed to detect corneal position and grade staining severity given a corneal fluorescein staining image. The fully automated pipeline consists of the following three steps: a corneal segmentation model extracts corneal area; five image patches are cropped from the staining image based on the five subregions of extracted cornea; a staining grading model predicts a score for each image patch from 0 to 3, and automated grading score for the whole cornea is obtained from 0 to 15. Finally, the clinical grading scores annotated by three ophthalmologists were compared with automated grading scores.

RESULTS

For corneal segmentation, the segmentation model achieved an intersection over union of 0.937. For punctate staining grading, the grading model achieved a classification accuracy of 76.5% and an area under the receiver operating characteristic curve of 0.940 (95% CI 0.932 to 0.949). For the fully automated pipeline, Pearson's correlation coefficient between the clinical and automated grading scores was 0.908 (p<0.01). Bland-Altman analysis revealed 95% limits of agreement between the clinical and automated grading scores of between -4.125 and 3.720 (concordance correlation coefficient=0.904). The average time required for processing a single stained image during pipeline was 0.58 s.

CONCLUSION

A fully automated grading system was developed to evaluate PEEs. The grading results may serve as a reference for ophthalmologists in clinical trials and residency training procedures.

The Need for Artificial Intelligence Based Risk Factor Analysis for Age-Related Macular Degeneration: A Review

In epidemiology, a risk factor is a variable associated with increased disease risk. Understanding the role of risk factors is significant for developing a strategy to improve global health. There is strong evidence that risk factors like smoking, alcohol consumption, previous cataract surgery, age, high-density lipoprotein (HDL) cholesterol, BMI, female gender, and focal hyper-pigmentation are independently associated with age-related macular degeneration (AMD). Currently, in the literature, statistical techniques like logistic regression, multivariable logistic regression, etc., are being used to identify AMD risk factors by employing numerical/categorical data. However, artificial intelligence (AI) techniques have not been used so far in the literature for identifying risk factors for AMD. On the other hand, artificial intelligence (AI) based tools can anticipate when a person is at risk of developing chronic diseases like cancer, dementia, asthma, etc., in providing personalized care. AI-based techniques can employ numerical/categorical and/or image data thus resulting in multimodal data analysis, which provides the need for AI-based tools to be used for risk factor analysis in ophthalmology. This review summarizes the statistical techniques used to identify various risk factors and the higher benefits that AI techniques provide for AMD-related disease prediction. Additional studies are required to review different techniques for risk factor identification for other ophthalmic diseases like glaucoma, diabetic macular edema, retinopathy of prematurity, cataract, and diabetic retinopathy.

Comprehensive Review on the Use of Artificial Intelligence in Ophthalmology and Future Research Directions

BACKGROUND

Having several applications in medicine, and in ophthalmology in particular, artificial intelligence (AI) tools have been used to detect visual function deficits, thus playing a key role in diagnosing eye diseases and in predicting the evolution of these common and disabling diseases. AI tools, i.e., artificial neural networks (ANNs), are progressively involved in detecting and customized control of ophthalmic diseases. The studies that refer to the efficiency of AI in medicine and especially in ophthalmology were analyzed in this review.

MATERIALS AND METHODS

We conducted a comprehensive review in order to collect all accounts published between 2015 and 2022 that refer to these applications of AI in medicine and especially in ophthalmology. Neural networks have a major role in establishing the demand to initiate preliminary anti-glaucoma therapy to stop the advance of the disease.

RESULTS

Different surveys in the literature review show the remarkable benefit of these AI tools in ophthalmology in evaluating the visual field, optic nerve, and retinal nerve fiber layer, thus ensuring a higher precision in detecting advances in glaucoma and retinal shifts in diabetes. We thus identified 1762 applications of artificial intelligence in ophthalmology: review articles and research articles (301 pub med, 144 scopus, 445 web of science, 872 science direct). Of these, we analyzed 70 articles and review papers (diabetic retinopathy (N = 24), glaucoma (N = 24), DMLV (N = 15), other pathologies (N = 7)) after applying the inclusion and exclusion criteria.

CONCLUSION

In medicine, AI tools are used in surgery, radiology, gynecology, oncology, etc., in making a diagnosis, predicting the evolution of a disease, and assessing the prognosis in patients with oncological pathologies. In ophthalmology, AI potentially increases the patient's access to screening/clinical diagnosis and decreases healthcare costs, mainly when there is a high risk of disease or communities face financial shortages. AI/DL (deep learning) algorithms using both OCT and FO images will change image analysis techniques and methodologies. Optimizing these (combined) technologies will accelerate progress in this area.

A Deep Learning Framework for the Detection and Quantification of Reticular Pseudodrusen and Drusen on Optical Coherence Tomography

Purpose

The purpose of this study was to develop and validate a deep learning (DL) framework for the detection and quantification of reticular pseudodrusen (RPD) and drusen on optical coherence tomography (OCT) scans.

Methods

A DL framework was developed consisting of a classification model and an out-of-distribution (OOD) detection model for the identification of ungradable scans; a classification model to identify scans with drusen or RPD; and an image segmentation model to independently segment lesions as RPD or drusen. Data were obtained from 1284 participants in the UK Biobank (UKBB) with a self-reported diagnosis of age-related macular degeneration (AMD) and 250 UKBB controls. Drusen and RPD were manually delineated by five retina specialists. The main outcome measures were sensitivity, specificity, area under the receiver operating characteristic (ROC) curve (AUC), kappa, accuracy, intraclass correlation coefficient (ICC), and free-response receiver operating characteristic (FROC) curves.

Results

The classification models performed strongly at their respective tasks (0.95, 0.93, and 0.99 AUC, respectively, for the ungradable scans classifier, the OOD model, and the drusen and RPD classification models). The mean ICC for the drusen and RPD area versus graders was 0.74 and 0.61, respectively, compared with 0.69 and 0.68 for intergrader agreement. FROC curves showed that the model's sensitivity was close to human performance.

Conclusions

The models achieved high classification and segmentation performance, similar to human performance.

Translational Relevance

Application of this robust framework will further our understanding of RPD as a separate entity from drusen in both research and clinical settings.

Artificial intelligence and deep learning in ophthalmology: Current status and future perspectives

Background

The ophthalmology field was among the first to adopt artificial intelligence (AI) in medicine. The availability of digitized ocular images and substantial data have made deep learning (DL) a popular topic.

Main text

At the moment, AI in ophthalmology is mostly used to improve disease diagnosis and assist decision-making aiming at ophthalmic diseases like diabetic retinopathy (DR), glaucoma, age-related macular degeneration (AMD), cataract and other anterior segment diseases. However, most of the AI systems developed to date are still in the experimental stages, with only a few having achieved clinical applications. There are a number of reasons for this phenomenon, including security, privacy, poor pervasiveness, trust and explainability concerns.

Conclusions

This review summarizes AI applications in ophthalmology, highlighting significant clinical considerations for adopting AI techniques and discussing the potential challenges and future directions.

Prediction of visual function from automatically quantified optical coherence tomography biomarkers in patients with geographic atrophy using machine learning

Geographic atrophy (GA) is a vision-threatening manifestation of age-related macular degeneration (AMD), one of the leading causes of blindness globally. Objective, rapid, reliable, and scalable quantification of GA from optical coherence tomography (OCT) retinal scans is necessary for disease monitoring, prognostic research, and clinical endpoints for therapy development. Such automatically quantified biomarkers on OCT are likely to further elucidate structure-function correlation in GA and thus the pathophysiological mechanisms of disease development and progression. In this work, we aimed to predict visual function with machine-learning applied to automatically acquired quantitative imaging biomarkers in GA. A post-hoc analysis of data from a clinical trial and routine clinical care was conducted. A deep-learning automated segmentation model was applied on OCT scans from 476 eyes (325 patients) with GA. A separate machine learning prediction model (Random Forest) used the resultant quantitative OCT (qOCT) biomarkers to predict cross-sectional visual acuity under standard (VA) and low luminance (LLVA). The primary outcome was regression coefficient (r2) and mean absolute error (MAE) for cross-sectional VA and LLVA in Early Treatment Diabetic Retinopathy Study (ETDRS) letters. OCT parameters were predictive of VA (r2 0.40 MAE 11.7 ETDRS letters) and LLVA (r2 0.25 MAE 12.1). Normalised random forest feature importance, as a measure of the predictive value of the three constituent features of GA; retinal pigment epithelium (RPE)-loss, photoreceptor degeneration (PDR), hypertransmission and their locations, was reported both on voxel-level heatmaps and ETDRS-grid subfields. The foveal region (46.5%) and RPE-loss (31.1%) had greatest predictive importance for VA. For LLVA, however, non-foveal regions (74.5%) and PDR (38.9%) were most important. In conclusion, automated qOCT biomarkers demonstrate predictive significance for VA and LLVA in GA. LLVA is itself predictive of GA progression, implying that the predictive qOCT biomarkers provided by our model are also prognostic.

Automated Detection of Vascular Leakage in Fluorescein Angiography - A Proof of Concept

Purpose

The purpose of this paper was to develop a deep learning algorithm to detect retinal vascular leakage (leakage) in fluorescein angiography (FA) of patients with uveitis and use the trained algorithm to determine clinically notable leakage changes.

Methods

An algorithm was trained and tested to detect leakage on a set of 200 FA images (61 patients) and evaluated on a separate 50-image test set (21 patients). The ground truth was leakage segmentation by two clinicians. The Dice Similarity Coefficient (DSC) was used to measure concordance.

Results

During training, the algorithm achieved a best average DSC of 0.572 (95% confidence interval [CI] = 0.548–0.596). The trained algorithm achieved a DSC of 0.563 (95% CI = 0.543–0.582) when tested on an additional set of 50 images. The trained algorithm was then used to detect leakage on pairs of FA images from longitudinal patient visits. Longitudinal leakage follow-up showed a >2.21% change in the visible retina area covered by leakage (as detected by the algorithm) had a sensitivity and specificity of 90% (area under the curve [AUC] = 0.95) of detecting a clinically notable change compared to the gold standard, an expert clinician's assessment.

Conclusions

This deep learning algorithm showed modest concordance in identifying vascular leakage compared to ground truth but was able to aid in identifying vascular FA leakage changes over time.

Translational Relevance

This is a proof-of-concept study that vascular leakage can be detected in a more standardized way and that tools can be developed to help clinicians more objectively compare vascular leakage between FAs.

Automatic segmentation of intraocular lens, the retrolental space and Berger's space using deep learning

PURPOSE

To develop and validate a deep learning model to automatically segment three structures using an anterior segment optical coherence tomography (AS-OCT): The intraocular lens (IOL), the retrolental space (IOL to the posterior lens capsule) and Berger's space (BS; posterior capsule to the anterior hyaloid membrane).

METHODS

An artificial intelligence (AI) approach based on a deep learning model to automatically segment the IOL, the retrolental space, and BS in AS-OCT, was trained using annotations from an experienced clinician. The training, validation and test set consisted of 92 cross-sectional OCT slices, acquired in 47 visits from 41 eyes. Annotations from a second experienced clinician in the test set were additionally evaluated to conduct an inter-reader variability analysis.

RESULTS

The AI model achieved a Precision/Recall/Dice score of 0.97/0.90/0.93 for IOL, 0.54/0.65/0.55 for retrolental space, and 0.72/0.58/0.59 for BS. For inter-reader variability, Precision/Recall/Dice values were 0.98/0.98/0.98 for IOL, 0.74/0.59/0.62 for retrolental space, and 0.58/0.57/0.57 for BS. No statistical differences were observed between the automated algorithm and the inter-reader variability for BS segmentation.

CONCLUSION

The deep learning model allows for fully automatic segmentation of all investigated structures, achieving human-level performance in BS segmentation. We, therefore, expect promising applications of the algorithm with particular interest in BS in automated big data analysis and real-time intra-operative support in ophthalmology, particularly in conjunction with primary posterior capsulotomy in femtosecond laser-assisted cataract surgery.

Intersession Repeatability of Structural Biomarkers in Early and Intermediate Age-Related Macular Degeneration: A MACUSTAR Study Report

Purpose

To analyze the intersession repeatability of structural biomarkers in eyes with early and intermediate age-related macular degeneration (iAMD) within the cross-sectional part of the observational multicenter MACUSTAR study.

Methods

Certified site personnel obtained multimodal imaging data at two visits (38 ± 20 [mean ± standard deviation] days apart), including spectral-domain optical coherence tomography (SD-OCT). One junior reader performed systematic and blinded grading at the central reading center, followed by senior reader review. Structural biomarkers included maximum drusen size classification (>63 to ≤125 µm vs. >125 µm), presence of large pigment epithelium detachments (PEDs), reticular pseudodrusen (RPD), vitelliform lesions, and refractile deposits. Intrasession variability was assessed using Cohen's κ statistics.

Results

At the first visit, 202 study eyes of 202 participants were graded as manifesting with either early (n = 34) or intermediate (n = 168) AMD. Grading of imaging data between visits revealed perfect agreement for the maximum drusen size classification (κ = 0.817; 95% confidence interval, 0.70–0.94). In iAMD eyes, perfect to substantial agreement was determined for the presence of large PEDs (0.87; 0.69–1.00) and RPD (0.752; 0.63–0.87), while intersession agreement was lower for the presence of vitelliform lesions (0.649; 0.39–0.65) and refractile deposits (0.342; −0.029–0.713), respectively.

Conclusions

Multimodal retinal imaging analysis between sessions showed a higher repeatability for structural biomarkers with predefined cutoff values than purely qualitative defined parameters.

Translational Relevance

A high repeatability of retinal imaging biomarkers will be important to implement automatic grading approaches and to establish robust and meaningful structural clinical endpoints for future interventional clinical trials in patients with iAMD.

Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice

An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.

Deep learning models for screening of high myopia using optical coherence tomography

This study aimed to validate and evaluate deep learning (DL) models for screening of high myopia using spectral-domain optical coherence tomography (OCT). This retrospective cross-sectional study included 690 eyes in 492 patients with OCT images and axial length measurement. Eyes were divided into three groups based on axial length: a “normal group,” a “high myopia group,” and an “other retinal disease” group. The researchers trained and validated three DL models to classify the three groups based on horizontal and vertical OCT images of the 600 eyes. For evaluation, OCT images of 90 eyes were used. Diagnostic agreements of human doctors and DL models were analyzed. The area under the receiver operating characteristic curve of the three DL models was evaluated. Absolute agreement of retina specialists was 99.11% (range: 97.78–100%). Absolute agreement of the DL models with multiple-column model was 100.0% (ResNet 50), 90.0% (Inception V3), and 72.22% (VGG 16). Areas under the receiver operating characteristic curves of the DL models with multiple-column model were 0.99 (ResNet 50), 0.97 (Inception V3), and 0.86 (VGG 16). The DL model based on ResNet 50 showed comparable diagnostic performance with retinal specialists. The DL model using OCT images demonstrated reliable diagnostic performance to identify high myopia.

Artificial intelligence-based predictions in neovascular age-related macular degeneration

PURPOSE OF REVIEW

Predicting treatment response and optimizing treatment regimen in patients with neovascular age-related macular degeneration (nAMD) remains challenging. Artificial intelligence-based tools have the potential to increase confidence in clinical development of new therapeutics, facilitate individual prognostic predictions, and ultimately inform treatment decisions in clinical practice.

RECENT FINDINGS

To date, most advances in applying artificial intelligence to nAMD have focused on facilitating image analysis, particularly for automated segmentation, extraction, and quantification of imaging-based features from optical coherence tomography (OCT) images. No studies in our literature search evaluated whether artificial intelligence could predict the treatment regimen required for an optimal visual response for an individual patient. Challenges identified for developing artificial intelligence-based models for nAMD include the limited number of large datasets with high-quality OCT data, limiting the patient populations included in model development; lack of counterfactual data to inform how individual patients may have fared with an alternative treatment strategy; and absence of OCT data standards, impairing the development of models usable across devices.

SUMMARY

Artificial intelligence has the potential to enable powerful prognostic tools for a complex nAMD treatment landscape; however, additional work remains before these tools are applicable to informing treatment decisions for nAMD in clinical practice.