Clinically relevant deep learning for detection and quantification of geographic atrophy from optical coherence tomography: a model development and external validation study

Strong versus Weak Data Labeling for Artificial Intelligence Algorithms in the Measurement of Geographic Atrophy

Purpose

To gain an understanding of data labeling requirements to train deep learning models for measurement of geographic atrophy (GA) with fundus autofluorescence (FAF) images.

Design

Evaluation of artificial intelligence (AI) algorithms.

Subjects

The Age-Related Eye Disease Study 2 (AREDS2) images were used for training and cross-validation, and GA clinical trial images were used for testing.

Methods

Training data consisted of 2 sets of FAF images; 1 with area measurements only and no indication of GA location (Weakly labeled) and the second with GA segmentation masks (Strongly labeled).

Main Outcome Measures

Bland-Altman plots and scatter plots were used to compare GA area measurement between ground truth and AI measurements. The Dice coefficient was used to compare accuracy of segmentation of the Strong model.

Results

In the cross-validation AREDS2 data set (n = 601), the mean (standard deviation [SD]) area of GA measured by human grader, Weakly labeled AI model, and Strongly labeled AI model was 6.65 (6.3) mm2, 6.83 (6.29) mm2, and 6.58 (6.24) mm2, respectively. The mean difference between ground truth and AI was 0.18 mm2 (95% confidence interval, [CI], -7.57 to 7.92) for the Weakly labeled model and -0.07 mm2 (95% CI, -1.61 to 1.47) for the Strongly labeled model. With GlaxoSmithKline testing data (n = 156), the mean (SD) GA area was 9.79 (5.6) mm2, 8.82 (4.61) mm2, and 9.55 (5.66) mm2 for human grader, Strongly labeled AI model, and Weakly labeled AI model, respectively. The mean difference between ground truth and AI for the 2 models was -0.97 mm2 (95% CI, -4.36 to 2.41) and -0.24 mm2 (95% CI, -4.98 to 4.49), respectively. The Dice coefficient was 0.99 for intergrader agreement, 0.89 for the cross-validation data, and 0.92 for the testing data.

Conclusions

Deep learning models can achieve reasonable accuracy even with Weakly labeled data. Training methods that integrate large volumes of Weakly labeled images with small number of Strongly labeled images offer a promising solution to overcome the burden of cost and time for data labeling.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Disease activity and therapeutic response to pegcetacoplan for geographic atrophy identified by deep learning-based analysis of OCT

PURPOSE

To quantify morphological changes of the photoreceptors (PR) and retinal pigment epithelium (RPE) layers under pegcetacoplan therapy in geographic atrophy (GA) using deep learning-based analysis of optical coherence tomography (OCT) images.

DESIGN

Post-hoc longitudinal image analysis SUBJECTS: Patients with GA due to age-related macular degeneration from two prospective randomized phase III clinical trials (OAKS and DERBY) METHODS: Deep learning-based segmentation of RPE loss and PR degeneration, defined as loss of the ellipsoid zone (EZ) layer on OCT, over 24 months on SD-OCT images MAIN OUTCOME MEASURES: Change in the mean area of RPE loss and EZ loss over time in the pooled sham arms and the monthly (PM)/every other month (PEOM) treatment arms RESULTS: 897 eyes of 897 patients were included. There was a therapeutic reduction of RPE loss growth by 22%/20% in OAKS and 27%/21% in DERBY for PM/PEOM compared to sham, respectively, at 24 months. The reduction on the EZ level was significantly higher with 53%/46% in OAKS and 47%/46% in DERBY for PM/PEOM compared to sham at 24 months. The baseline EZ-RPE difference had an impact on disease activity and therapeutic response. The therapeutic benefit for RPE loss growth increased with larger EZ-RPE difference quartiles from 21.9%, 23.1%, 23.9% to 33.6% for PM vs. sham (all p<0.01) and from 13.6% (p=0.11), 23.8%, 23.8% to 20.0% for PEOM vs. sham (p<0.01) in quartiles 1,2,3 and 4, respectively, at 24 months. Regarding EZ layer maintenance, the therapeutic reduction of loss increased from 14.8% (p=0.09), 33.3%, 46.6% to 77.8% (p<0.0001) between PM and sham and from 15.9% (p=0.08), 33.8%, 52.0% to 64.9% (p<0.0001) between PEOM and sham for quartiles 1-4 at 24 months.

CONCLUSION

OCT-based AI analysis objectively identifies and quantifies PR and RPE degeneration in GA. Reductions in further PR degeneration consistent with EZ loss on OCT are even higher than the effect on RPE loss in phase 3 trials of pegcetacoplan treatment. The EZ-RPE difference has a strong impact on disease progression and therapeutic response. Identification of patients with higher EZ-RPE loss difference may become an important criterion for the management of GA secondary to AMD.

BRSET: A Brazilian Multilabel Ophthalmological Dataset of Retina Fundus Photos

INTRODUCTION

The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups.

METHODS

Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used a ConvNext model trained during 50 epochs using a weighted cross entropy loss to avoid overfitting, with 70% training (20% validation), and 30% testing subsets. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score. Saliency maps were calculated for interpretability.

RESULTS

BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 (± 18.26) years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. A ConvNext V2 model was trained and evaluated BRSET in four prediction tasks: "binary diabetic retinopathy diagnosis (Normal vs Diabetic Retinopathy)" (AUC: 97, F1: 89); "3 class diabetic retinopathy diagnosis (Normal, Proliferative, Non-Proliferative)" (AUC: 97, F1: 82); "diabetes diagnosis" (AUC: 91, F1: 83); "sex classification" (AUC: 87, F1: 70).

DISCUSSION

BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.

Advancements in the treatment of age-related macular degeneration: a comprehensive review

Age-related macular degeneration (AMD) stands as a leading cause of irreversible blindness, particularly affecting central vision and impeding daily tasks. This paper provides a thorough exploration of AMD, distinguishing between its two main subtypes-Wet and Dry AMD-while shedding light on the prevalence and risk factors, including age, genetics, and smoking. The focus shifts to the current and future treatment landscape, examining both Dry and Wet AMD. Regarding Dry AMD, interventions such as antioxidant supplementation and ongoing clinical trials offer hope. Notable among these is Pegcetacoplan which is the only Food and Drug Administration (FDA)-approved medication, displaying promising results in reducing geographic atrophy lesions. For Wet AMD, anti-Vascular Endothelial Growth Factor therapies like Ranibizumab (Lucentis®) have been instrumental, and newer drugs like Faricimab and OPT-302 show comparable efficacy with extended dosing intervals. Additionally, gene therapies such as RGX-314 present a potential paradigm shift, reducing or eliminating the need for frequent injections. Biosimilars offer cost-effective alternatives. The paper also delves into the integration of technology and artificial intelligence in AMD management, highlighting the role of smartphone apps for patient monitoring and artificial intelligence algorithms for diagnosis and surveillance. Furthermore, patient perspectives on artificial intelligence demonstrate a positive correlation between understanding and trust. The narrative concludes with a glimpse into ground-breaking technologies, including retinal implants and bionic chips, offering hope for vision restoration. Overall, this paper underscores the multifaceted approach in addressing AMD, combining traditional and innovative strategies, paving the way for a more promising future in AMD treatment.

Pegcetacoplan Treatment and Consensus Features of Geographic Atrophy Over 24 Months

Importance

Despite widespread availability and consensus on its advantages for detailed imaging of geographic atrophy (GA), spectral-domain optical coherence tomography (SD-OCT) might benefit from automated quantitative OCT analyses in GA diagnosis, monitoring, and reporting of its landmark clinical trials.

Objective

To analyze the association between pegcetacoplan and consensus GA SD-OCT end points.

Design, Setting, and Participants

This was a post hoc analysis of 11 614 SD-OCT volumes from 936 of the 1258 participants in 2 parallel phase 3 studies, the Study to Compare the Efficacy and Safety of Intravitreal APL-2 Therapy With Sham Injections in Patients With Geographic Atrophy (GA) Secondary to Age-Related Macular Degeneration (OAKS) and Study to Compare the Efficacy and Safety of Intravitreal APL-2 Therapy With Sham Injections in Patients With Geographic Atrophy (GA) Secondary to Age-Related Macular Degeneration (DERBY). OAKS and DERBY were 24-month, multicenter, randomized, double-masked, sham-controlled studies conducted from August 2018 to July 2020 among adults with GA with total area 2.5 to 17.5 mm2 on fundus autofluorescence imaging (if multifocal, at least 1 lesion ≥1.25 mm2). This analysis was conducted from September to December 2023.

Interventions

Study participants received pegcetacoplan, 15 mg per 0.1-mL intravitreal injection, monthly or every other month, or sham injection monthly or every other month.

Main Outcomes and Measures

The primary end point was the least squares mean change from baseline in area of retinal pigment epithelium and outer retinal atrophy in each of the 3 treatment arms (pegcetacoplan monthly, pegcetacoplan every other month, and pooled sham [sham monthly and sham every other month]) at 24 months. Feature-specific area analysis was conducted by Early Treatment Diabetic Retinopathy Study (ETDRS) regions of interest (ie, foveal, parafoveal, and perifoveal).

Results

Among 936 participants, the mean (SD) age was 78.5 (7.22) years, and 570 participants (60.9%) were female. Pegcetacoplan, but not sham treatment, was associated with reduced growth rates of SD-OCT biomarkers for GA for up to 24 months. Reductions vs sham in least squares mean (SE) change from baseline of retinal pigment epithelium and outer retinal atrophy area were detectable at every time point from 3 through 24 months (least squares mean difference vs pooled sham at month 24, pegcetacoplan monthly: -0.86 mm2; 95% CI, -1.15 to -0.57; P < .001; pegcetacoplan every other month: -0.69 mm2; 95% CI, -0.98 to -0.39; P < .001). This association was more pronounced with more frequent dosing (pegcetacoplan monthly vs pegcetacoplan every other month at month 24: -0.17 mm2; 95% CI, -0.43 to 0.08; P = .17). Stronger associations were observed in the parafoveal and perifoveal regions for both pegcetacoplan monthly and pegcetacoplan every other month.

Conclusions and Relevance

These findings offer additional insight into the potential effects of pegcetacoplan on the development of GA, including potential effects on the retinal pigment epithelium and photoreceptors.

Trial Registration

ClinicalTrials.gov Identifiers: NCT03525600 and NCT03525613.

Deep Learning Approaches for Detecting of Nascent Geographic Atrophy in Age-Related Macular Degeneration

Purpose

Nascent geographic atrophy (nGA) refers to specific features seen on OCT B-scans, which are strongly associated with the future development of geographic atrophy (GA). This study sought to develop a deep learning model to screen OCT B-scans for nGA that warrant further manual review (an artificial intelligence [AI]-assisted approach), and to determine the extent of reduction in OCT B-scan load requiring manual review while maintaining near-perfect nGA detection performance.

Design

Development and evaluation of a deep learning model.

Participants

One thousand eight hundred and eighty four OCT volume scans (49 B-scans per volume) without neovascular age-related macular degeneration from 280 eyes of 140 participants with bilateral large drusen at baseline, seen at 6-monthly intervals up to a 36-month period (from which 40 eyes developed nGA).

Methods

OCT volume and B-scans were labeled for the presence of nGA. Their presence at the volume scan level provided the ground truth for training a deep learning model to identify OCT B-scans that potentially showed nGA requiring manual review. Using a threshold that provided a sensitivity of 0.99, the B-scans identified were assigned the ground truth label with the AI-assisted approach. The performance of this approach for detecting nGA across all visits, or at the visit of nGA onset, was evaluated using fivefold cross-validation.

Main Outcome Measures

Sensitivity for detecting nGA, and proportion of OCT B-scans requiring manual review.

Results

The AI-assisted approach (utilizing outputs from the deep learning model to guide manual review) had a sensitivity of 0.97 (95% confidence interval [CI] = 0.93-1.00) and 0.95 (95% CI = 0.87-1.00) for detecting nGA across all visits and at the visit of nGA onset, respectively, when requiring manual review of only 2.7% and 1.9% of selected OCT B-scans, respectively.

Conclusions

A deep learning model could be used to enable near-perfect detection of nGA onset while reducing the number of OCT B-scans requiring manual review by over 50-fold. This AI-assisted approach shows promise for substantially reducing the current burden of manual review of OCT B-scans to detect this crucial feature that portends future development of GA.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Atrophic Lesions Associated with Age-Related Macular Degeneration: High-Resolution versus Standard OCT

OBJECTIVE

The objective of this study was to determine whether high-resolution OCT (HR-OCT) could enhance the identification and classification of atrophic features in age-related macular degeneration (AMD) compared with standard resolution OCT.

DESIGN

Prospective, observational, cross-sectional study.

SUBJECTS

The study included 60 eyes from 60 patients > 60 years of age with a diagnosis of AMD.

METHODS

The participants underwent volume OCT scanning using HR-OCT and standard resolution OCT devices. Trained graders reviewed and graded the scans, identifying specific regions of interest for subsequent analysis.

MAIN OUTCOME MEASURES

The study focused on identifying and classifying complete retinal pigment epithelium (RPE) and outer retinal atrophy (cRORA), incomplete RORA (iRORA), and other nonatrophic AMD features. Additionally, qualitative and quantitative features associated with atrophy were assessed.

RESULTS

The agreement among readers for classifying atrophic lesions was substantial to perfect for both HR-OCT (0.88) and standard resolution OCT(0.82). However, HR-OCT showed a higher accuracy in identifying iRORA lesions compared with standard OCT. Qualitative assessment of features demonstrated higher agreement for HR-OCT, particularly in identifying external limiting membrane (ELM) (0.95) and ellipsoid zone (EZ) disruption (0.94). Quantitative measurements of features such as hypertransmission defects, RPE attenuation/disruption, EZ disruption width, and ELM disruption width showed excellent interreader agreement with HR-OCT (> 0.90 for all features) but only moderate agreement with standard OCT (0.51-0.60).

CONCLUSIONS

The study results suggest that HR-OCT improves the accuracy and reliability of classifying and quantifying atrophic lesions associated with AMD compared with standard resolution OCT. The quantitative findings in particular may have implications for future research and clinical practice, especially with the availability of therapeutic agents for treating geographic atrophy and the development of commercially available HR-OCT devices.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Quantification of Fundus Autofluorescence Features in a Molecularly Characterized Cohort of More Than 3000 Inherited Retinal Disease Patients from the United Kingdom

Purpose

To quantify relevant fundus autofluorescence (FAF) image features cross-sectionally and longitudinally in a large cohort of inherited retinal diseases (IRDs) patients.

Design

Retrospective study of imaging data (55-degree blue-FAF on Heidelberg Spectralis) from patients.

Participants

Patients with a clinical and molecularly confirmed diagnosis of IRD who have undergone FAF 55-degree imaging at Moorfields Eye Hospital (MEH) and the Royal Liverpool Hospital (RLH) between 2004 and 2019.

Methods

Five FAF features of interest were defined: vessels, optic disc, perimacular ring of increased signal (ring), relative hypo-autofluorescence (hypo-AF) and hyper-autofluorescence (hyper-AF). Features were manually annotated by six graders in a subset of patients based on a defined grading protocol to produce segmentation masks to train an AI model, AIRDetect, which was then applied to the entire imaging dataset.

Main Outcome Measures

Quantitative FAF imaging features including area in mm 2 and vessel metrics, were analysed cross-sectionally by gene and age, and longitudinally to determine rate of progression. AIRDetect feature segmentation and detection were validated with Dice score and precision/recall, respectively.

Results

A total of 45,749 FAF images from 3,606 IRD patients from MEH covering 170 genes were automatically segmented using AIRDetect. Model-grader Dice scores for disc, hypo-AF, hyper-AF, ring and vessels were respectively 0.86, 0.72, 0.69, 0.68 and 0.65. The five genes with the largest hypo-AF areas were CHM , ABCC6 , ABCA4 , RDH12 , and RPE65 , with mean per-patient areas of 41.5, 30.0, 21.9, 21.4, and 15.1 mm 2 . The five genes with the largest hyper-AF areas were BEST1 , CDH23 , RDH12 , MYO7A , and NR2E3 , with mean areas of 0.49, 0.45, 0.44, 0.39, and 0.34 mm 2 respectively. The five genes with largest ring areas were CDH23 , NR2E3 , CRX , EYS and MYO7A, with mean areas of 3.63, 3.32, 2.84, 2.39, and 2.16 mm 2 . Vessel density was found to be highest in EFEMP1 , BEST1 , TIMP3 , RS1 , and PRPH2 (10.6%, 10.3%, 9.8%, 9.7%, 8.9%) and was lower in Retinitis Pigmentosa (RP) and Leber Congenital Amaurosis genes. Longitudinal analysis of decreasing ring area in four RP genes ( RPGR, USH2A, RHO, EYS ) found EYS to be the fastest progressor at -0.18 mm 2 /year.

Conclusions

We have conducted the first large-scale cross-sectional and longitudinal quantitative analysis of FAF features across a diverse range of IRDs using a novel AI approach.

Deep-learning automated quantification of longitudinal OCT scans demonstrates reduced RPE loss rate, preservation of intact macular area and predictive value of isolated photoreceptor degeneration in geographic atrophy patients receiving C3 inhibition treatment

OBJECTIVE

To evaluate the role of automated optical coherence tomography (OCT) segmentation, using a validated deep-learning model, for assessing the effect of C3 inhibition on the area of geographic atrophy (GA); the constituent features of GA on OCT (photoreceptor degeneration (PRD), retinal pigment epithelium (RPE) loss and hypertransmission); and the area of unaffected healthy macula.To identify OCT predictive biomarkers for GA growth.

METHODS

Post hoc analysis of the FILLY trial using a deep-learning model for spectral domain OCT (SD-OCT) autosegmentation. 246 patients were randomised 1:1:1 into pegcetacoplan monthly (PM), pegcetacoplan every other month (PEOM) and sham treatment (pooled) for 12 months of treatment and 6 months of therapy-free monitoring. Only participants with Heidelberg SD-OCT were included (n=197, single eye per participant).The primary efficacy endpoint was the square root transformed change in area of GA as complete RPE and outer retinal atrophy (cRORA) in each treatment arm at 12 months, with secondary endpoints including RPE loss, hypertransmission, PRD and intact macular area.

RESULTS

Eyes treated PM showed significantly slower mean change of cRORA progression at 12 and 18 months (0.151 and 0.277 mm, p=0.0039; 0.251 and 0.396 mm, p=0.039, respectively) and RPE loss (0.147 and 0.287 mm, p=0.0008; 0.242 and 0.410 mm, p=0.00809). PEOM showed significantly slower mean change of RPE loss compared with sham at 12 months (p=0.0313). Intact macular areas were preserved in PM compared with sham at 12 and 18 months (p=0.0095 and p=0.044). PRD in isolation and intact macula areas was predictive of reduced cRORA growth at 12 months (coefficient 0.0195, p=0.01 and 0.00752, p=0.02, respectively) CONCLUSION: The OCT evidence suggests that pegcetacoplan slows progression of cRORA overall and RPE loss specifically while protecting the remaining photoreceptors and slowing the progression of healthy retina to iRORA.

Deep learning-based detection and quantification of brain metastases on black-blood imaging can provide treatment suggestions: a clinical cohort study

OBJECTIVES

We aimed to evaluate whether deep learning-based detection and quantification of brain metastasis (BM) may suggest treatment options for patients with BMs.

METHODS

The deep learning system (DLS) for detection and quantification of BM was developed in 193 patients and applied to 112 patients that were newly detected on black-blood contrast-enhanced T1-weighted imaging. Patients were assigned to one of 3 treatment suggestion groups according to the European Association of Neuro-Oncology (EANO)-European Society for Medical Oncology (ESMO) recommendations using number and volume of the BMs detected by the DLS: short-term imaging follow-up without treatment (group A), surgery or stereotactic radiosurgery (limited BM, group B), or whole-brain radiotherapy or systemic chemotherapy (extensive BM, group C). The concordance between the DLS-based groups and clinical decisions was analyzed with or without consideration of targeted agents. The performance of distinguishing high-risk (B + C) was calculated.

RESULTS

Among 112 patients (mean age 64.3 years, 63 men), group C had the largest number and volume of BM, followed by group B (4.4 and 851.6 mm3) and A (1.5 and 15.5 mm3). The DLS-based groups were concordant with the actual clinical decisions, with an accuracy of 76.8% (86 of 112). Modified accuracy considering targeted agents was 81.3% (91 of 112). The DLS showed 95% (82/86) sensitivity and 81% (21/26) specificity for distinguishing the high risk.

CONCLUSION

DLS-based detection and quantification of BM have the potential to be helpful in the determination of treatment options for both low- and high-risk groups of limited and extensive BMs.

CLINICAL RELEVANCE STATEMENT

For patients with newly diagnosed brain metastasis, deep learning-based detection and quantification may be used in clinical settings where prompt and accurate treatment decisions are required, which can lead to better patient outcomes.

KEY POINTS

• Deep learning-based brain metastasis detection and quantification showed excellent agreement with ground-truth classifications. • By setting an algorithm to suggest treatment based on the number and volume of brain metastases detected by the deep learning system, the concordance was 81.3%. • When dividing patients into low- and high-risk groups, the sensitivity for detecting the latter was 95%.

RobOCTNet: Robotics and Deep Learning for Referable Posterior Segment Pathology Detection in an Emergency Department Population

Purpose

To evaluate the diagnostic performance of a robotically aligned optical coherence tomography (RAOCT) system coupled with a deep learning model in detecting referable posterior segment pathology in OCT images of emergency department patients.

Methods

A deep learning model, RobOCTNet, was trained and internally tested to classify OCT images as referable versus non-referable for ophthalmology consultation. For external testing, emergency department patients with signs or symptoms warranting evaluation of the posterior segment were imaged with RAOCT. RobOCTNet was used to classify the images. Model performance was evaluated against a reference standard based on clinical diagnosis and retina specialist OCT review.

Results

We included 90,250 OCT images for training and 1489 images for internal testing. RobOCTNet achieved an area under the curve (AUC) of 1.00 (95% confidence interval [CI], 0.99-1.00) for detection of referable posterior segment pathology in the internal test set. For external testing, RAOCT was used to image 72 eyes of 38 emergency department patients. In this set, RobOCTNet had an AUC of 0.91 (95% CI, 0.82-0.97), a sensitivity of 95% (95% CI, 87%-100%), and a specificity of 76% (95% CI, 62%-91%). The model's performance was comparable to two human experts' performance.

Conclusions

A robotically aligned OCT coupled with a deep learning model demonstrated high diagnostic performance in detecting referable posterior segment pathology in a cohort of emergency department patients.

Translational Relevance

Robotically aligned OCT coupled with a deep learning model may have the potential to improve emergency department patient triage for ophthalmology referral.

BRSET: A Brazilian Multilabel Ophthalmological Dataset of Retina Fundus Photos

Introduction

The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups.

Methods

Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used Dino V2 Base for feature extraction, with 70% training and 30% testing subsets. Support Vector Machines (SVM) and Logistic Regression (LR) were employed with weighted training. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score.

Results

BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. Models were trained on BRSET in three prediction tasks: "diabetes diagnosis"; "sex classification"; and "diabetic retinopathy diagnosis".

Discussion

BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.

PREVALENCE AND PERSISTENCE OF HYPERTRANSMISSION DEFECTS OF VARIOUS SIZES IN EYES WITH INTERMEDIATE AGE-RELATED MACULAR DEGENERATION

PURPOSE

To determine the prevalence and rate of persistence over 2 years of various-sized hypertransmission defects (hyperTDs) in eyes with intermediate age-related macular degeneration.

METHODS

Retrospective analysis of optical coherence tomography data from consecutive intermediate age-related macular degeneration patients. Choroidal en face optical coherence tomography images were evaluated for the presence and number of hyperTDs of three different sizes based on greatest linear dimension (small, 63-124 µ m; medium, 125-249 µ m; large, ≥250 µ m) at baseline and at the 2-year follow-up. Interreader agreement was determined by Gwet's agreement coefficient. Disagreements between graders were resolved by the senior investigator to yield a single consensus for all cases.

RESULTS

From 273 intermediate age-related macular degeneration eyes (247 patients), 72 and 76 hyperTD lesions were independently identified by two graders at baseline and overall agreement coefficient was 0.89 (95% CI, 0.86-0.93). After adjudication by the senior grader, the final consensus yielded 78 hyperTD lesions from 46 eyes (16.8%) of 42 patients (17.0%) in this study cohort. Among eyes with follow-up optical coherence tomography, 32 of 45 hyperTD lesions (71.1%) persisted. The rates of persistence were 100.0%, 72.7%, and 53.3% in large, medium, and small hyperTD sizes, respectively.

CONCLUSION

HyperTDs were present in a significant proportion of intermediate age-related macular degeneration eyes. Acceptable interreader agreement was demonstrated in identifying hyperTD. Larger hyperTD lesions were more likely to persist over 2 years.

Evaluating the Effects of C3 Inhibition on Geographic Atrophy Progression from Deep-Learning OCT Quantification: A Split-Person Study

INTRODUCTION

To evaluate the effect pegcetacoplan, a C3 and C3b inhibitor, on the rate of progression of geographic atrophy (GA) as assessed by spectral domain optical coherence tomography (SD-OCT) using a split-person study design and deep-learning quantification.

METHODS

A post hoc analysis of phase 2 FILLY trial data comparing study (treated monthly, treated every other month and sham-treated) and fellow (untreated) eyes in a split-person study design was performed. This analysis included 288 eyes from 144 patients with bilateral GA from the FILLY phase 2 trial (Clinical Trials identifier: NCT02503332). Only patients with bilateral GA and without evidence of choroidal neovascularisation in either eye were included. Patient study eyes were treated with sham injections or with pegcetacoplan monthly (PM) or every other month (PEOM) for 12 months. SD-OCT scans of study and fellow eyes taken at baseline and 12 months were used for the analysis. The main outcomes were the annual change in the area of retinal pigment epithelial and outer retinal atrophy (RORA), its constituent features (photoreceptor degeneration [PRD], retinal pigment epithelium [RPE] loss, hypertransmission) and intact macula as compared to the untreated fellow eye.

RESULTS

Annual GA growth was reduced in eyes treated with PM versus untreated fellow eyes for OCT features, including RORA (study eye 0.792 vs. fellow eye 1.13 mm2; P = 0.003), PRD (0.739 vs. 1.23 mm2; P = 0.015), RPE-loss (0.789 vs. 1.17 mm2; P = 0.007) and intact macula (- 0.735 vs. - 1.29 mm2; P = 0.011). Similar (but not statistically significant) trends were observed with the PEOM treatment or when GA was quantified with fundus autofluorescence (FAF). The sham treatment demonstrated no effect. Pearson correlation coefficients showed concordance in the enlargement rate of GA between the study and fellow eyes in the sham (R = 0.64) and PEOM (R = 0.68) groups, but not in the PM group (R = 0.21).

CONCLUSIONS

Pegcetacoplan-treated eyes demonstrated a reduction in spatial GA progression compared to their untreated counterparts. This effect was more evident on OCT than with FAF.

TRIAL REGISTRATION

Clinical Trials identifier: NCT02503332.

Deep-GA-Net for Accurate and Explainable Detection of Geographic Atrophy on OCT Scans

Objective

To propose Deep-GA-Net, a 3-dimensional (3D) deep learning network with 3D attention layer, for the detection of geographic atrophy (GA) on spectral domain OCT (SD-OCT) scans, explain its decision making, and compare it with existing methods.

Design

Deep learning model development.

Participants

Three hundred eleven participants from the Age-Related Eye Disease Study 2 Ancillary SD-OCT Study.

Methods

A dataset of 1284 SD-OCT scans from 311 participants was used to develop Deep-GA-Net. Cross-validation was used to evaluate Deep-GA-Net, where each testing set contained no participant from the corresponding training set. En face heatmaps and important regions at the B-scan level were used to visualize the outputs of Deep-GA-Net, and 3 ophthalmologists graded the presence or absence of GA in them to assess the explainability (i.e., understandability and interpretability) of its detections.

Main Outcome Measures

Accuracy, area under receiver operating characteristic curve (AUC), area under precision-recall curve (APR).

Results

Compared with other networks, Deep-GA-Net achieved the best metrics, with accuracy of 0.93, AUC of 0.94, and APR of 0.91, and received the best gradings of 0.98 and 0.68 on the en face heatmap and B-scan grading tasks, respectively.

Conclusions

Deep-GA-Net was able to detect GA accurately from SD-OCT scans. The visualizations of Deep-GA-Net were more explainable, as suggested by 3 ophthalmologists. The code and pretrained models are publicly available at https://github.com/ncbi/Deep-GA-Net.

Financial Disclosures

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Interreader Agreement and Longitudinal Progression of Incomplete/Complete Retinal Pigment Epithelium and Outer Retinal Atrophy in Age-Related Macular Degeneration

OBJECTIVE

To analyze the ability to evaluate changes over time of individual lesions of incomplete or complete retinal pigment epithelium (RPE) and outer retinal atrophy (iRORA and cRORA, respectively) in patients with intermediate age-related macular degeneration (iAMD).

DESIGN

OCT images from patients enrolled in Proxima B clinical trial (NCT02399072) were utilized.

PARTICIPANTS

Patients enrolled in the Proxima B clinical trial, from the cohort with geographic atrophy (GA) in 1 eye and iAMD in the other eye at baseline, were included.

METHODS

Junior and senior readers analyzed OCT images for the qualitative presence of 9 distinct early atrophic features (presence of zone of choroidal hypertransmission, attenuation and/or disruption of RPE, disruption of ellipsoid zone [EZ] and external limiting membrane [ELM], outer nuclear layer [ONL] thinning, outer plexiform layer [OPL]/inner nuclear layer [INL] subsidence, and hyporeflective wedge-shaped band). If deemed "present," 7 features were quantified with a predefined tolerance level of 50 μm (diameter for the zone of choroidal hypertransmission, zone of attenuation and/or disruption of the RPE, outer retinal thickness left/right vertical diameter, outer retinal thickness thinnest vertical diameter, annotation of EZ, and ELM disruption).

MAIN OUTCOME MEASURES

Interreader agreements for qualitative assessments (κ-type statistics) and quantitative measurements (Bland-Altman statistics) were assessed. Progression of the lesion features over time was described.

RESULTS

Moderate agreement was found for presence of choroidal hypertransmission (κ = 0.54), followed by ELM disruption (κ = 0.58), OPL/INL subsidence (κ = 0.46), and a hyporeflective wedge-shaped band (κ = 0.47). Quantification measurements showed that choroidal hypertransmission had the highest agreement, whereas RPE attenuation/disruption had the lowest agreement. Longitudinal adjudicated changes for quantitative measurements of lesion progression showed that choroidal hypertransmission and ELM disruption showed significant progression, whereas EZ disruption and RPE attenuation/disruption did not.

CONCLUSIONS

The ability to evaluate changes over time for specific features of iRORA and cRORA was explored. The most robust biomarker was found to be choroidal hypertransmission, followed by ELM disruption and the qualitative markers of OPL/INL subsidence, as well as a wedge-shaped band. Disease progression over time could be assessed by some, but not all, spectral-domain OCT features that were explored.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Deep learning model for automatic differentiation of EMAP from AMD in macular atrophy

To create a deep learning (DL) classifier pre-trained on fundus autofluorescence (FAF) images that can assist the clinician in distinguishing age-related geographic atrophy from extensive macular atrophy and pseudodrusen-like appearance (EMAP). Patients with complete outer retinal and retinal pigment epithelium atrophy secondary to either EMAP (EMAP Group) or to dry age related macular degeneration (AMD group) were retrospectively selected. Fovea-centered posterior pole (30° × 30°) and 55° × 55° degree-field-of-view FAF images of sufficiently high quality were collected and used to train two different deep learning (DL) classifiers based on ResNet-101 design. Testing was performed on a set of images coming from a different center. A total of 300 patients were recruited, 135 belonging to EMAP group and 165 belonging to AMD group. The 30° × 30° FAF based DL classifier showed a sensitivity of 84.6% and a specificity of 85.3% for the diagnosis of EMAP. The 55° × 55° FAF based DL classifier showed a sensitivity of 90% and a specificity of 84.6%, a performance that was significantly higher than that of the 30° × 30° classifer (p = 0.037). Artificial intelligence can accurately distinguish between atrophy caused by AMD or by EMAP on FAF images. Its performance are improved using wide field acquisitions.

A Deep-Learning Algorithm to Predict Short-Term Progression to Geographic Atrophy on Spectral-Domain Optical Coherence Tomography

Importance

The identification of patients at risk of progressing from intermediate age-related macular degeneration (iAMD) to geographic atrophy (GA) is essential for clinical trials aimed at preventing disease progression. DeepGAze is a fully automated and accurate convolutional neural network-based deep learning algorithm for predicting progression from iAMD to GA within 1 year from spectral-domain optical coherence tomography (SD-OCT) scans.

Objective

To develop a deep-learning algorithm based on volumetric SD-OCT scans to predict the progression from iAMD to GA during the year following the scan.

Design, Setting, and Participants

This retrospective cohort study included participants with iAMD at baseline and who either progressed or did not progress to GA within the subsequent 13 months. Participants were included from centers in 4 US states. Data set 1 included patients from the Age-Related Eye Disease Study 2 AREDS2 (Ancillary Spectral-Domain Optical Coherence Tomography) A2A study (July 2008 to August 2015). Data sets 2 and 3 included patients with imaging taken in routine clinical care at a tertiary referral center and associated satellites between January 2013 and January 2023. The stored imaging data were retrieved for the purpose of this study from July 1, 2022, to February 1, 2023. Data were analyzed from May 2021 to July 2023.

Exposure

A position-aware convolutional neural network with proactive pseudointervention was trained and cross-validated on Bioptigen SD-OCT volumes (data set 1) and validated on 2 external data sets comprising Heidelberg Spectralis SD-OCT scans (data sets 2 and 3).

Main Outcomes and Measures

Prediction of progression to GA within 13 months was evaluated with area under the receiver-operator characteristic curves (AUROC) as well as area under the precision-recall curve (AUPRC), sensitivity, specificity, positive predictive value, negative predictive value, and accuracy.

Results

The study included a total of 417 patients: 316 in data set 1 (mean [SD] age, 74 [8]; 185 [59%] female), 53 in data set 2, (mean [SD] age, 83 [8]; 32 [60%] female), and 48 in data set 3 (mean [SD] age, 81 [8]; 32 [67%] female). The AUROC for prediction of progression from iAMD to GA within 1 year was 0.94 (95% CI, 0.92-0.95; AUPRC, 0.90 [95% CI, 0.85-0.95]; sensitivity, 0.88 [95% CI, 0.84-0.92]; specificity, 0.90 [95% CI, 0.87-0.92]) for data set 1. The addition of expert-annotated SD-OCT features to the model resulted in no improvement compared to the fully autonomous model (AUROC, 0.95; 95% CI, 0.92-0.95; P = .19). On an independent validation data set (data set 2), the model predicted progression to GA with an AUROC of 0.94 (95% CI, 0.91-0.96; AUPRC, 0.92 [0.89-0.94]; sensitivity, 0.91 [95% CI, 0.74-0.98]; specificity, 0.80 [95% CI, 0.63-0.91]). At a high-specificity operating point, simulated clinical trial recruitment was enriched for patients progressing to GA within 1 year by 8.3- to 20.7-fold (data sets 2 and 3).

Conclusions and Relevance

The fully automated, position-aware deep-learning algorithm assessed in this study successfully predicted progression from iAMD to GA over a clinically meaningful time frame. The ability to predict imminent GA progression could facilitate clinical trials aimed at preventing the condition and could guide clinical decision-making regarding screening frequency or treatment initiation.

Artificial intelligence in retinal disease: clinical application, challenges, and future directions

Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.

Diagnosis and Management of Patients With Geographic Atrophy Secondary to Age-Related Macular Degeneration: A Delphi Consensus Exercise

Geographic atrophy (GA) is a progressive and irreversible retinal disease with no comprehensive recommendations for diagnosis or monitoring. We used a Delphi approach to determine consensus in key areas around diagnosis and management of GA. A steering committee of eight retina specialists developed two sequential online surveys administered to eye care professionals (ECPs). Consensus was defined as agreement by ≥ 75% of respondents. Up to 177 ECPs from eight countries completed one or both surveys. Consensus was achieved in several topics related to diagnostic imaging, including the use of optical coherence tomography, and the urgent need for treatments and beneficial interventions to reduce the associated burden. Currently, low-vision aids and smoking cessation are considered the most beneficial interventions. We demonstrate consensus for diagnosis and management of patients with GA including best practices in patient identification and monitoring, and unmet needs. [Ophthalmic Surg Lasers Imaging Retina 2023;54:589-598.].

Unleashing the power of optical attenuation coefficients to facilitate segmentation strategies in OCT imaging of age-related macular degeneration: perspective

The use of optical attenuation coefficients (OAC) in optical coherence tomography (OCT) imaging of the retina has improved the segmentation of anatomic layers compared with traditional intensity-based algorithms. Optical attenuation correction has improved our ability to measure the choroidal thickness and choroidal vascularity index using dense volume scans. Algorithms that combine conventional intensity-based segmentation with depth-resolved OAC OCT imaging have been used to detect elevations of the retinal pigment epithelium (RPE) due to drusen and basal laminar deposits, the location of hyperpigmentation within the retina and along the RPE, the identification of macular atrophy, the thickness of the outer retinal (photoreceptor) layer, and the presence of calcified drusen. OAC OCT algorithms can identify the risk-factors that predict disease progression in age-related macular degeneration.

Deep learning to detect macular atrophy in wet age-related macular degeneration using optical coherence tomography

Here, we have developed a deep learning method to fully automatically detect and quantify six main clinically relevant atrophic features associated with macular atrophy (MA) using optical coherence tomography (OCT) analysis of patients with wet age-related macular degeneration (AMD). The development of MA in patients with AMD results in irreversible blindness, and there is currently no effective method of early diagnosis of this condition, despite the recent development of unique treatments. Using OCT dataset of a total of 2211 B-scans from 45 volumetric scans of 8 patients, a convolutional neural network using one-against-all strategy was trained to present all six atrophic features followed by a validation to evaluate the performance of the models. The model predictive performance has achieved a mean dice similarity coefficient score of 0.706 ± 0.039, a mean Precision score of 0.834 ± 0.048, and a mean Sensitivity score of 0.615 ± 0.051. These results show the unique potential of using artificially intelligence-aided methods for early detection and identification of the progression of MA in wet AMD, which can further support and assist clinical decisions.

Detection of macular atrophy in age-related macular degeneration aided by artificial intelligence

INTRODUCTION

Age-related macular degeneration (AMD) is a leading cause of irreversible visual impairment worldwide. The endpoint of AMD, both in its dry or wet form, is macular atrophy (MA) which is characterized by the permanent loss of the RPE and overlying photoreceptors either in dry AMD or in wet AMD. A recognized unmet need in AMD is the early detection of MA development.

AREAS COVERED

Artificial Intelligence (AI) has demonstrated great impact in detection of retinal diseases, especially with its robust ability to analyze big data afforded by ophthalmic imaging modalities, such as color fundus photography (CFP), fundus autofluorescence (FAF), near-infrared reflectance (NIR), and optical coherence tomography (OCT). Among these, OCT has been shown to have great promise in identifying early MA using the new criteria in 2018.

EXPERT OPINION

There are few studies in which AI-OCT methods have been used to identify MA; however, results are very promising when compared to other imaging modalities. In this paper, we review the development and advances of ophthalmic imaging modalities and their combination with AI technology to detect MA in AMD. In addition, we emphasize the application of AI-OCT as an objective, cost-effective tool for the early detection and monitoring of the progression of MA in AMD.

Clinical validation for automated geographic atrophy monitoring on OCT under complement inhibitory treatment

Geographic atrophy (GA) represents a late stage of age-related macular degeneration, which leads to irreversible vision loss. With the first successful therapeutic approach, namely complement inhibition, huge numbers of patients will have to be monitored regularly. Given these perspectives, a strong need for automated GA segmentation has evolved. The main purpose of this study was the clinical validation of an artificial intelligence (AI)-based algorithm to segment a topographic 2D GA area on a 3D optical coherence tomography (OCT) volume, and to evaluate its potential for AI-based monitoring of GA progression under complement-targeted treatment. 100 GA patients from routine clinical care at the Medical University of Vienna for internal validation and 113 patients from the FILLY phase 2 clinical trial for external validation were included. Mean Dice Similarity Coefficient (DSC) was 0.86 ± 0.12 and 0.91 ± 0.05 for total GA area on the internal and external validation, respectively. Mean DSC for the GA growth area at month 12 on the external test set was 0.46 ± 0.16. Importantly, the automated segmentation by the algorithm corresponded to the outcome of the original FILLY trial measured manually on fundus autofluorescence. The proposed AI approach can reliably segment GA area on OCT with high accuracy. The availability of such tools represents an important step towards AI-based monitoring of GA progression under treatment on OCT for clinical management as well as regulatory trials.

Automated Identification of Incomplete and Complete Retinal Epithelial Pigment and Outer Retinal Atrophy Using Machine Learning

OBJECTIVE

To assess and validate a deep learning algorithm to automatically detect incomplete retinal pigment epithelial and outer retinal atrophy (iRORA) and complete retinal pigment epithelial and outer retinal atrophy (cRORA) in eyes with age-related macular degeneration.

DESIGN

In a retrospective machine learning analysis, a deep learning model was trained to jointly classify the presence of iRORA and cRORA within a given B-scan. The algorithm was evaluated using 2 separate and independent datasets.

PARTICIPANTS

OCT B-scan volumes from 71 patients with nonneovascular age-related macular degeneration captured at the Doheny-University of California Los Angeles Eye Centers and the following 2 external OCT B-scans testing datasets: (1) University of Pennsylvania, University of Miami, and Case Western Reserve University and (2) Doheny Image Reading Research Laboratory.

METHODS

The images were annotated by an experienced grader for the presence of iRORA and cRORA. A Resnet18 model was trained to classify these annotations for each B-scan using OCT volumes collected at the Doheny-University of California Los Angeles Eye Centers. The model was applied to 2 testing datasets to assess out-of-sample model performance.

MAIN OUTCOMES MEASURES

Model performance was quantified in terms of area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC). Sensitivity, specificity, and positive predictive value were also compared against additional clinician annotators.

RESULTS

On an independently collected test set, consisting of 1117 volumes from the general population, the model predicted iRORA and cRORA presence within the entire volume with nearly perfect AUROC performance and AUPRC scores (iRORA, 0.61; 95% confidence interval [CI] [0.45, 0.82]: cRORA, 0.83; 95% CI [0.68, 0.95]). On another independently collected set, consisting of 60 OCT B-scans enriched for iRORA and cRORA lesions, the model performed with AUROC (iRORA: 0.68, 95% CI [0.54, 0.81]; cRORA: 0.84, 95% CI [0.75, 0.94]) and AUPRC (iRORA: 0.70, 95% CI [0.55, 0.86]; cRORA: 0.82, 95% CI [0.70, 0.93]).

CONCLUSIONS

A deep learning model can accurately and precisely identify both iRORA and cRORA lesions within the OCT B-scan volume. The model can achieve similar sensitivity compared with human graders, which potentially obviates a laborious and time-consuming annotation process and could be developed into a diagnostic screening tool.

TSSK-Net: Weakly supervised biomarker localization and segmentation with image-level annotation in retinal OCT images

The localization and segmentation of biomarkers in OCT images are critical steps in retina-related disease diagnosis. Although fully supervised deep learning models can segment pathological regions, their performance relies on labor-intensive pixel-level annotations. Compared with dense pixel-level annotation, image-level annotation can reduce the burden of manual annotation. Existing methods for image-level annotation are usually based on class activation maps (CAM). However, current methods still suffer from model collapse, training instability, and anatomical mismatch due to the considerable variation in retinal biomarkers' shape, texture, and size. This paper proposes a novel weakly supervised biomarkers localization and segmentation method, requiring only image-level annotations. The technique is a Teacher-Student network with joint Self-supervised contrastive learning and Knowledge distillation-based anomaly localization, namely TSSK-Net. Specifically, we treat retinal biomarker regions as abnormal regions distinct from normal regions. First, we propose a novel pre-training strategy based on supervised contrastive learning that encourages the model to learn the anatomical structure of normal OCT images. Second, we design a fine-tuning module and propose a novel hybrid network structure. The network includes supervised contrastive loss for feature learning and cross-entropy loss for classification learning. To further improve the performance, we propose an efficient strategy to combine these two losses to preserve the anatomical structure and enhance the encoding representation of features. Finally, we design a knowledge distillation-based anomaly segmentation method that is effectively combined with the previous model to alleviate the challenge of insufficient supervision. Experimental results on a local dataset and a public dataset demonstrated the effectiveness of our proposed method. Our proposed method can effectively reduce the annotation burden of ophthalmologists in OCT images.

iERM: An Interpretable Deep Learning System to Classify Epiretinal Membrane for Different Optical Coherence Tomography Devices: A Multi-Center Analysis

Background: Epiretinal membranes (ERM) have been found to be common among individuals >50 years old. However, the severity grading assessment for ERM based on optical coherence tomography (OCT) images has remained a challenge due to lacking reliable and interpretable analysis methods. Thus, this study aimed to develop a two-stage deep learning (DL) system named iERM to provide accurate automatic grading of ERM for clinical practice. Methods: The iERM was trained based on human segmentation of key features to improve classification performance and simultaneously provide interpretability to the classification results. We developed and tested iERM using a total of 4547 OCT B-Scans of four different commercial OCT devices that were collected from nine international medical centers. Results: As per the results, the integrated network effectively improved the grading performance by 1−5.9% compared with the traditional classification DL model and achieved high accuracy scores of 82.9%, 87.0%, and 79.4% in the internal test dataset and two external test datasets, respectively. This is comparable to retinal specialists whose average accuracy scores are 87.8% and 79.4% in two external test datasets. Conclusion: This study proved to be a benchmark method to improve the performance and enhance the interpretability of the traditional DL model with the implementation of segmentation based on prior human knowledge. It may have the potential to provide precise guidance for ERM diagnosis and treatment.

Machine Learning-Based Automated Detection and Quantification of Geographic Atrophy and Hypertransmission Defects Using Spectral Domain Optical Coherence Tomography

The current study describes the development and assessment of innovative, machine learning (ML)-based approaches for automated detection and pixel-accurate measurements of regions with geographic atrophy (GA) in late-stage age-related macular degeneration (AMD) using optical coherence tomography systems. 900 OCT volumes, 100266 B-scans, and en face OCT images from 341 non-exudative AMD patients with or without GA were included in this study from both Cirrus (Zeiss) and Spectralis (Heidelberg) OCT systems. B-scan and en face level ground truth GA masks were created on OCT B-scan where the segmented ellipsoid zone (EZ) line, retinal pigment epithelium (RPE) line, and bruchs membrane (BM) line overlapped. Two deep learning-based approaches, B-scan level and en face level, were trained. The OCT B-scan model had detection accuracy of 91% and GA area measurement accuracy of 94%. The en face OCT model had detection accuracy of 82% and GA area measurement accuracy of 96% with primary target of hypertransmission on en face OCT. Accuracy was good for both devices tested (92-97%). Automated lesion size stratification for CAM cRORA definition of 250um minimum lesion size was feasible. High-performance models for automatic detection and segmentation of GA area were achieved using OCT systems and deep learning. The automatic measurements showed high correlation with the ground truth. The en face model excelled at identification of hypertransmission defects. The models performance generalized well across device types tested. Future development will include integration of both models to enhance feature detection across GA lesions as well as isolating hypertransmission defects without GA for pre-GA biomarker extraction.

An interpretable artificial intelligence system for detecting risk factors of gastroesophageal variceal bleeding

Bleeding risk factors for gastroesophageal varices (GEV) detected by endoscopy in cirrhotic patients determine the prophylactical treatment patients will undergo in the following 2 years. We propose a methodology for measuring the risk factors. We create an artificial intelligence system (ENDOANGEL-GEV) containing six models to segment GEV and to classify the grades (grades 1-3) and red color signs (RC, RC0-RC3) of varices. It also summarizes changes in the above results with region in real time. ENDOANGEL-GEV is trained using 6034 images from 1156 cirrhotic patients across three hospitals (dataset 1) and validated on multicenter datasets with 11009 images from 141 videos (dataset 2) and in a prospective study recruiting 161 cirrhotic patients from Renmin Hospital of Wuhan University (dataset 3). In dataset 1, ENDOANGEL-GEV achieves intersection over union values of 0.8087 for segmenting esophageal varices and 0.8141 for gastric varices. In dataset 2, the system maintains fairly accuracy across images from three hospitals. In dataset 3, ENDOANGEL-GEV surpasses attended endoscopists in detecting RC of GEV and classifying grades (p < 0.001). When ranking the risk of patients combined with the Child‒Pugh score, ENDOANGEL-GEV outperforms endoscopists for esophageal varices (p < 0.001) and shows comparable performance for gastric varices (p = 0.152). Compared with endoscopists, ENDOANGEL-GEV may help 12.31% (16/130) more patients receive the right intervention. We establish an interpretable system for the endoscopic diagnosis and risk stratification of GEV. It will assist in detecting the first bleeding risk factors accurately and expanding the scope of quantitative measurement of diseases.

Algorithm of segmentation of OCT macular images to analyze the results in patients with age-related macular degeneration

Age-related macular degeneration (AMD) is one of the main causes of loss of sight and hypovision in people over working age. Results of optical coherence tomography (OCT) are essential for diagnostics of the disease. Developing the recommendation system to analyze OCT images will reduce the time to process visual data and decrease the probability of errors while working as a doctor. The purpose of the study was to develop an algorithm of segmentation to analyze the results of macular OCT in patients with AMD. It allows to provide a correct prediction of an AMD stage based on the form of discovered pathologies. A program has been developed in the Python programming language using the Pytorch and TensorFlow libraries. Its quality was estimated using OCT macular images of 51 patients with early, intermediate, late AMD. A segmentation algorithm of OCT images was developed based on convolutional neural network. UNet network was selected as architecture of high-accuracy neural net. The neural net is trained on macular OCT images of 125 patients (197 eyes). The author algorithm displayed 98.1% of properly segmented areas on OCT images, which are the most essential for diagnostics and determination of an AMD stage. Weighted sensitivity and specificity of AMD stage classifier amounted to 83.8% and 84.9% respectively. The developed algorithm is promising as a recommendation system that implements the AMD classification based on data that promote taking decisions regarding the treatment strategy.

Artificial intelligence and deep learning in ophthalmology: Current status and future perspectives

Background

The ophthalmology field was among the first to adopt artificial intelligence (AI) in medicine. The availability of digitized ocular images and substantial data have made deep learning (DL) a popular topic.

Main text

At the moment, AI in ophthalmology is mostly used to improve disease diagnosis and assist decision-making aiming at ophthalmic diseases like diabetic retinopathy (DR), glaucoma, age-related macular degeneration (AMD), cataract and other anterior segment diseases. However, most of the AI systems developed to date are still in the experimental stages, with only a few having achieved clinical applications. There are a number of reasons for this phenomenon, including security, privacy, poor pervasiveness, trust and explainability concerns.

Conclusions

This review summarizes AI applications in ophthalmology, highlighting significant clinical considerations for adopting AI techniques and discussing the potential challenges and future directions.

Research Trends and Hotspots of Retinal Optical Coherence Tomography: A 31-Year Bibliometric Analysis

The emergence of optical coherence tomography (OCT) over the past three decades has sparked great interest in retinal research. However, a comprehensive analysis of the trends and hotspots in retinal OCT research is currently lacking. We searched the publications on retinal OCT in the Web of Science database from 1991 to 2021 and performed the co-occurrence keyword analysis and co-cited reference network using bibliometric tools. A total of 25,175 publications were included. There has been a progressive increase in the number of publications. The keyword co-occurrence network revealed five clusters of hotspots: (1) thickness measurements; (2) therapies for macular degeneration and macular edema; (3) degenerative retinal diseases; (4) OCT angiography (OCTA); and (5) vitrectomy for macular hole and epiretinal membrane. The co-citation analysis displayed 26 highly credible clusters (S = 0.9387) with a well-structured network (Q = 0.879). The major trends of research were: (1) thickness measurements; (2) therapies for macular degeneration and macular edema; and (3) OCTA. Recent emerging frontiers showed a growing interest in OCTA, vessel density, choriocapillaris, central serous chorioretinopathy, Alzheimer’s disease, and deep learning. This review summarized 31 years of retinal OCT research, shedding light on the hotspots, main themes, and emerging frontiers to assist in future research.

Prediction of visual function from automatically quantified optical coherence tomography biomarkers in patients with geographic atrophy using machine learning

Geographic atrophy (GA) is a vision-threatening manifestation of age-related macular degeneration (AMD), one of the leading causes of blindness globally. Objective, rapid, reliable, and scalable quantification of GA from optical coherence tomography (OCT) retinal scans is necessary for disease monitoring, prognostic research, and clinical endpoints for therapy development. Such automatically quantified biomarkers on OCT are likely to further elucidate structure-function correlation in GA and thus the pathophysiological mechanisms of disease development and progression. In this work, we aimed to predict visual function with machine-learning applied to automatically acquired quantitative imaging biomarkers in GA. A post-hoc analysis of data from a clinical trial and routine clinical care was conducted. A deep-learning automated segmentation model was applied on OCT scans from 476 eyes (325 patients) with GA. A separate machine learning prediction model (Random Forest) used the resultant quantitative OCT (qOCT) biomarkers to predict cross-sectional visual acuity under standard (VA) and low luminance (LLVA). The primary outcome was regression coefficient (r2) and mean absolute error (MAE) for cross-sectional VA and LLVA in Early Treatment Diabetic Retinopathy Study (ETDRS) letters. OCT parameters were predictive of VA (r2 0.40 MAE 11.7 ETDRS letters) and LLVA (r2 0.25 MAE 12.1). Normalised random forest feature importance, as a measure of the predictive value of the three constituent features of GA; retinal pigment epithelium (RPE)-loss, photoreceptor degeneration (PDR), hypertransmission and their locations, was reported both on voxel-level heatmaps and ETDRS-grid subfields. The foveal region (46.5%) and RPE-loss (31.1%) had greatest predictive importance for VA. For LLVA, however, non-foveal regions (74.5%) and PDR (38.9%) were most important. In conclusion, automated qOCT biomarkers demonstrate predictive significance for VA and LLVA in GA. LLVA is itself predictive of GA progression, implying that the predictive qOCT biomarkers provided by our model are also prognostic.

Advances in Optical Coherence Tomography Imaging Technology and Techniques for Choroidal and Retinal Disorders

Optical coherence tomography (OCT) imaging has played a pivotal role in the field of retina. This light-based, non-invasive imaging modality provides high-quality, cross-sectional analysis of the retina and has revolutionized the diagnosis and management of retinal and choroidal diseases. Since its introduction in the early 1990s, OCT technology has continued to advance to provide quicker acquisition times and higher resolution. In this manuscript, we discuss some of the most recent advances in OCT technology and techniques for choroidal and retinal diseases. The emerging innovations discussed include wide-field OCT, adaptive optics OCT, polarization sensitive OCT, full-field OCT, hand-held OCT, intraoperative OCT, at-home OCT, and more. The applications of these rising OCT systems and techniques will allow for a closer monitoring of chorioretinal diseases and treatment response, more robust analysis in basic science research, and further insights into surgical management. In addition, these innovations to optimize visualization of the choroid and retina offer a promising future for advancing our understanding of the pathophysiology of chorioretinal diseases.

Machine learning-based 3D modeling and volumetry of human posterior vitreous cavity of optical coherence tomographic images

The structure of the human vitreous varies considerably because of age-related liquefactions of the vitreous gel. These changes are poorly studied in vivo mainly because their high transparency and mobility make it difficult to obtain reliable and repeatable images of the vitreous. Optical coherence tomography can detect the boundaries between the vitreous gel and vitreous fluid, but it is difficult to obtain high resolution images that can be used to convert the images to three-dimensional (3D) images. Thus, the purpose of this study was to determine the shape and characteristics of the vitreous fluid using machine learning-based 3D modeling in which manually labelled fluid areas were used to train deep convolutional neural network (DCNN). The trained DCNN labelled vitreous fluid automatically and allowed us to obtain 3D vitreous model and to quantify the vitreous fluidic cavities. The mean volume and surface area of posterior vitreous fluidic cavities are 19.6 ± 7.8 mm³ and 104.0 ± 18.9 mm² in eyes of 17 school children. The results suggested that vitreous fluidic cavities expanded as the cavities connects with each other, and this modeling system provided novel imaging markers for aging and eye diseases.

Optical coherence tomography and convolutional neural networks can differentiate colorectal liver metastases from liver parenchyma ex vivo

PURPOSE

Optical coherence tomography (OCT) is an imaging technology based on low-coherence interferometry, which provides non-invasive, high-resolution cross-sectional images of biological tissues. A potential clinical application is the intraoperative examination of resection margins, as a real-time adjunct to histological examination. In this ex vivo study, we investigated the ability of OCT to differentiate colorectal liver metastases (CRLM) from healthy liver parenchyma, when combined with convolutional neural networks (CNN).

METHODS

Between June and August 2020, consecutive adult patients undergoing elective liver resections for CRLM were included in this study. Fresh resection specimens were scanned ex vivo, before fixation in formalin, using a table-top OCT device at 1310 nm wavelength. Scanned areas were marked and histologically examined. A pre-trained CNN (Xception) was used to match OCT scans to their corresponding histological diagnoses. To validate the results, a stratified k-fold cross-validation (CV) was carried out.

RESULTS

A total of 26 scans (containing approx. 26,500 images in total) were obtained from 15 patients. Of these, 13 were of normal liver parenchyma and 13 of CRLM. The CNN distinguished CRLM from healthy liver parenchyma with an F1-score of 0.93 (0.03), and a sensitivity and specificity of 0.94 (0.04) and 0.93 (0.04), respectively.

CONCLUSION

Optical coherence tomography combined with CNN can distinguish between healthy liver and CRLM with great accuracy ex vivo. Further studies are needed to improve upon these results and develop in vivo diagnostic technologies, such as intraoperative scanning of resection margins.

Predicting Systemic Health Features from Retinal Fundus Images Using Transfer-Learning-Based Artificial Intelligence Models

While color fundus photos are used in routine clinical practice to diagnose ophthalmic conditions, evidence suggests that ocular imaging contains valuable information regarding the systemic health features of patients. These features can be identified through computer vision techniques including deep learning (DL) artificial intelligence (AI) models. We aim to construct a DL model that can predict systemic features from fundus images and to determine the optimal method of model construction for this task. Data were collected from a cohort of patients undergoing diabetic retinopathy screening between March 2020 and March 2021. Two models were created for each of 12 systemic health features based on the DenseNet201 architecture: one utilizing transfer learning with images from ImageNet and another from 35,126 fundus images. Here, 1277 fundus images were used to train the AI models. Area under the receiver operating characteristics curve (AUROC) scores were used to compare the model performance. Models utilizing the ImageNet transfer learning data were superior to those using retinal images for transfer learning (mean AUROC 0.78 vs. 0.65, p-value < 0.001). Models using ImageNet pretraining were able to predict systemic features including ethnicity (AUROC 0.93), age > 70 (AUROC 0.90), gender (AUROC 0.85), ACE inhibitor (AUROC 0.82), and ARB medication use (AUROC 0.78). We conclude that fundus images contain valuable information about the systemic characteristics of a patient. To optimize DL model performance, we recommend that even domain specific models consider using transfer learning from more generalized image sets to improve accuracy.

Reticular Pseudodrusen on the Risk of Progression in Intermediate Age-Related Macular Degeneration

PURPOSE

To examine the association between reticular pseudodrusen (RPD) and progression to late age-related macular degeneration (AMD) in individuals with intermediate AMD.

DESIGN

Prospective cohort study.

METHODS

Two hundred eighty eyes from 140 participants with bilateral large drusen underwent multimodal imaging (MMI), including optical coherence tomography (OCT), near-infrared reflectance (NIR), fundus autofluorescence, and color fundus photography (CFP), at 6-monthly intervals up over a 36-month follow-up period. The presence of RPD per eye was determined based on either a combined MMI criterion, or each individual imaging modality, and their extent measured on combined OCT and NIR imaging. The association between the presence of RPD on different imaging modalities, and their extent, with the development of late AMD (including OCT-defined atrophy) was evaluated.

RESULTS

The presence of RPD on MMI, or any of its individual modalities, at baseline was not significantly associated with an increased rate of developing late AMD, with or without adjusting for risk factors for AMD progression (age, drusen volume on OCT, and pigmentary abnormalities on CFP; all P ≥ 0.205). The extent of RPD present was also not significantly associated with an increased rate of developing late AMD, with or without adjustment for risk factors for AMD progression (both P ≥ 0.522).

CONCLUSIONS

In this cohort with bilateral large drusen, the presence of RPD was not significantly associated with an increased risk of developing late AMD. Additional longitudinal studies in all stages of AMD are needed to understand the implications of RPD on vision loss in this condition.

The Development and Clinical Application of Innovative Optical Ophthalmic Imaging Techniques

The field of ophthalmic imaging has grown substantially over the last years. Massive improvements in image processing and computer hardware have allowed the emergence of multiple imaging techniques of the eye that can transform patient care. The purpose of this review is to describe the most recent advances in eye imaging and explain how new technologies and imaging methods can be utilized in a clinical setting. The introduction of optical coherence tomography (OCT) was a revolution in eye imaging and has since become the standard of care for a plethora of conditions. Its most recent iterations, OCT angiography, and visible light OCT, as well as imaging modalities, such as fluorescent lifetime imaging ophthalmoscopy, would allow a more thorough evaluation of patients and provide additional information on disease processes. Toward that goal, the application of adaptive optics (AO) and full-field scanning to a variety of eye imaging techniques has further allowed the histologic study of single cells in the retina and anterior segment. Toward the goal of remote eye care and more accessible eye imaging, methods such as handheld OCT devices and imaging through smartphones, have emerged. Finally, incorporating artificial intelligence (AI) in eye images has the potential to become a new milestone for eye imaging while also contributing in social aspects of eye care.

From Data to Deployment: The Collaborative Community on Ophthalmic Imaging Roadmap for Artificial Intelligence in Age-Related Macular Degeneration

OBJECTIVE

Health care systems worldwide are challenged to provide adequate care for the 200 million individuals with age-related macular degeneration (AMD). Artificial intelligence (AI) has the potential to make a significant, positive impact on the diagnosis and management of patients with AMD; however, the development of effective AI devices for clinical care faces numerous considerations and challenges, a fact evidenced by a current absence of Food and Drug Administration (FDA)-approved AI devices for AMD.

PURPOSE

To delineate the state of AI for AMD, including current data, standards, achievements, and challenges.

METHODS

Members of the Collaborative Community on Ophthalmic Imaging Working Group for AI in AMD attended an inaugural meeting on September 7, 2020, to discuss the topic. Subsequently, they undertook a comprehensive review of the medical literature relevant to the topic. Members engaged in meetings and discussion through December 2021 to synthesize the information and arrive at a consensus.

RESULTS

Existing infrastructure for robust AI development for AMD includes several large, labeled data sets of color fundus photography and OCT images; however, image data often do not contain the metadata necessary for the development of reliable, valid, and generalizable models. Data sharing for AMD model development is made difficult by restrictions on data privacy and security, although potential solutions are under investigation. Computing resources may be adequate for current applications, but knowledge of machine learning development may be scarce in many clinical ophthalmology settings. Despite these challenges, researchers have produced promising AI models for AMD for screening, diagnosis, prediction, and monitoring. Future goals include defining benchmarks to facilitate regulatory authorization and subsequent clinical setting generalization.

CONCLUSIONS

Delivering an FDA-authorized, AI-based device for clinical care in AMD involves numerous considerations, including the identification of an appropriate clinical application; acquisition and development of a large, high-quality data set; development of the AI architecture; training and validation of the model; and functional interactions between the model output and clinical end user. The research efforts undertaken to date represent starting points for the medical devices that eventually will benefit providers, health care systems, and patients.