A deep learning system for identifying lattice degeneration and retinal breaks using ultra-widefield fundus images

Deep learning system for screening AIDS-related cytomegalovirus retinitis with ultra-wide-field fundus images

Background

Ophthalmological screening for cytomegalovirus retinitis (CMVR) for HIV/AIDS patients is important to prevent lifelong blindness. Previous studies have shown good properties of automated CMVR screening using digital fundus images. However, the application of a deep learning (DL) system to CMVR with ultra-wide-field (UWF) fundus images has not been studied, and the feasibility and efficiency of this method are uncertain.

Methods

In this study, we developed, internally validated, externally validated, and prospectively validated a DL system to detect AIDS-related from UWF fundus images from different clinical datasets. We independently used the InceptionResnetV2 network to develop and internally validate a DL system for identifying active CMVR, inactive CMVR, and non-CMVR in 6960 UWF fundus images from 862 AIDS patients and validated the system in a prospective and an external validation data set using the area under the curve (AUC), accuracy, sensitivity, and specificity. A heat map identified the most important area (lesions) used by the DL system for differentiating CMVR.

Results

The DL system showed AUCs of 0.945 (95 % confidence interval [CI]: 0.929, 0.962), 0.964 (95 % CI: 0.870, 0.999) and 0.968 (95 % CI: 0.860, 1.000) for detecting active CMVR from non-CMVR and 0.923 (95 % CI: 0.908, 0.938), 0.902 (0.857, 0.948) and 0.884 (0.851, 0.917) for detecting active CMVR from non-CMVR in the internal cross-validation, external validation, and prospective validation, respectively. Deep learning performed promisingly in screening CMVR. It also showed the ability to differentiate active CMVR from non-CMVR and inactive CMVR as well as to identify active CMVR and inactive CMVR from non-CMVR (all AUCs in the three independent data sets >0.900). The heat maps successfully highlighted lesion locations.

Conclusions

Our UWF fundus image-based DL system showed reliable performance for screening AIDS-related CMVR showing its potential for screening CMVR in HIV/AIDS patients, especially in the absence of ophthalmic resources.

Applications of deep learning for detecting ophthalmic diseases with ultrawide-field fundus images

AIM

To summarize the application of deep learning in detecting ophthalmic disease with ultrawide-field fundus images and analyze the advantages, limitations, and possible solutions common to all tasks.

METHODS

We searched three academic databases, including PubMed, Web of Science, and Ovid, with the date of August 2022. We matched and screened according to the target keywords and publication year and retrieved a total of 4358 research papers according to the keywords, of which 23 studies were retrieved on applying deep learning in diagnosing ophthalmic disease with ultrawide-field images.

RESULTS

Deep learning in ultrawide-field images can detect various ophthalmic diseases and achieve great performance, including diabetic retinopathy, glaucoma, age-related macular degeneration, retinal vein occlusions, retinal detachment, and other peripheral retinal diseases. Compared to fundus images, the ultrawide-field fundus scanning laser ophthalmoscopy enables the capture of the ocular fundus up to 200° in a single exposure, which can observe more areas of the retina.

CONCLUSION

The combination of ultrawide-field fundus images and artificial intelligence will achieve great performance in diagnosing multiple ophthalmic diseases in the future.

Retinal Disease Diagnosis Using Deep Learning on Ultra-Wide-Field Fundus Images

Ultra-wide-field fundus imaging (UFI) provides comprehensive visualization of crucial eye components, including the optic disk, fovea, and macula. This in-depth view facilitates doctors in accurately diagnosing diseases and recommending suitable treatments. This study investigated the application of various deep learning models for detecting eye diseases using UFI. We developed an automated system that processes and enhances a dataset of 4697 images. Our approach involves brightness and contrast enhancement, followed by applying feature extraction, data augmentation and image classification, integrated with convolutional neural networks. These networks utilize layer-wise feature extraction and transfer learning from pre-trained models to accurately represent and analyze medical images. Among the five evaluated models, including ResNet152, Vision Transformer, InceptionResNetV2, RegNet and ConVNext, ResNet152 is the most effective, achieving a testing area under the curve (AUC) score of 96.47% (with a 95% confidence interval (CI) of 0.931-0.974). Additionally, the paper presents visualizations of the model's predictions, including confidence scores and heatmaps that highlight the model's focal points-particularly where lesions due to damage are evident. By streamlining the diagnosis process and providing intricate prediction details without human intervention, our system serves as a pivotal tool for ophthalmologists. This research underscores the compatibility and potential of utilizing ultra-wide-field images in conjunction with deep learning.

Insights into artificial intelligence in myopia management: from a data perspective

Given the high incidence and prevalence of myopia, the current healthcare system is struggling to handle the task of myopia management, which is worsened by home quarantine during the ongoing COVID-19 pandemic. The utilization of artificial intelligence (AI) in ophthalmology is thriving, yet not enough in myopia. AI can serve as a solution for the myopia pandemic, with application potential in early identification, risk stratification, progression prediction, and timely intervention. The datasets used for developing AI models are the foundation and determine the upper limit of performance. Data generated from clinical practice in managing myopia can be categorized into clinical data and imaging data, and different AI methods can be used for analysis. In this review, we comprehensively review the current application status of AI in myopia with an emphasis on data modalities used for developing AI models. We propose that establishing large public datasets with high quality, enhancing the model's capability of handling multimodal input, and exploring novel data modalities could be of great significance for the further application of AI for myopia.

DCT-Net: An effective method to diagnose retinal tears from B-scan ultrasound images

Retinal tears (RTs) are usually detected by B-scan ultrasound images, particularly for individuals with complex eye conditions. However, traditional manual techniques for reading ultrasound images have the potential to overlook or inaccurately diagnose conditions. Thus, the development of rapid and accurate approaches for the diagnosis of an RT is highly important and urgent. The present study introduces a novel hybrid deep-learning model called DCT-Net to enable the automatic and precise diagnosis of RTs. The implemented model utilizes a vision transformer as the backbone and feature extractor. Additionally, in order to accommodate the edge characteristics of the lesion areas, a novel module called the residual deformable convolution has been incorporated. Furthermore, normalization is employed to mitigate the issue of overfitting and, a Softmax layer has been included to achieve the final classification following the acquisition of the global and local representations. The study was conducted by using both our proprietary dataset and a publicly available dataset. In addition, interpretability of the trained model was assessed by generating attention maps using the attention rollout approach. On the private dataset, the model demonstrated a high level of performance, with an accuracy of 97.78%, precision of 97.34%, recall rate of 97.13%, and an F1 score of 0.9682. On the other hand, the model developed by using the public funds image dataset demonstrated an accuracy of 83.82%, a sensitivity of 82.69% and a specificity of 82.40%. The findings, therefore present a novel framework for the diagnosis of RTs that is characterized by a high degree of efficiency, accuracy and interpretability. Accordingly, the technology exhibits considerable promise and has the potential to serve as a reliable tool for ophthalmologists.

H-Deep-Net: A deep hybrid network with stationary wavelet packet transforms for Retinal detachment classification through fundus images

Nowadays, automated disease diagnosis has become a vital role in the medical field due to the significant population expansion. An automated disease diagnostic approach assists clinicians in the diagnosis of disease by giving exact, consistent, and prompt results, along with minimizing the mortality rate. Retinal detachment has recently emerged as one of the most severe and acute ocular illnesses, spreading worldwide. Therefore, an automated and quickest diagnostic model should be implemented to diagnose retinal detachment at an early stage. This paper introduces a new hybrid approach of best basis stationary wavelet packet transform and modified VGG19-Bidirectional long short-term memory to detect retinal detachment using retinal fundus images automatically. In this paper, the best basis stationary wavelet packet transform is utilized for image analysis, modified VGG19-Bidirectional long short-term memory is employed as the deep feature extractors, and then obtained features are classified through the Adaptive boosting technique. The experimental outcomes demonstrate that our proposed method obtained 99.67% sensitivity, 95.95% specificity, 98.21% accuracy, 97.43% precision, 98.54% F1-score, and 0.9985 AUC. The model obtained the intended results on the presently accessible database, which may be enhanced further when additional RD images become accessible. The proposed approach aids ophthalmologists in identifying and easily treating RD patients.

Automated analysis of vessel morphometry in retinal images from a Danish high street optician setting

PURPOSE

To evaluate the test performance of the QUARTZ (QUantitative Analysis of Retinal vessel Topology and siZe) software in detecting retinal features from retinal images captured by health care professionals in a Danish high street optician chain, compared with test performance from other large population studies (i.e., UK Biobank) where retinal images were captured by non-experts.

METHOD

The dataset FOREVERP (Finding Ophthalmic Risk and Evaluating the Value of Eye exams and their predictive Reliability, Pilot) contains retinal images obtained from a Danish high street optician chain. The QUARTZ algorithm utilizes both image processing and machine learning methods to determine retinal image quality, vessel segmentation, vessel width, vessel classification (arterioles or venules), and optic disc localization. Outcomes were evaluated by metrics including sensitivity, specificity, and accuracy and compared to human expert ground truths.

RESULTS

QUARTZ's performance was evaluated on a subset of 3,682 images from the FOREVERP database. 80.55% of the FOREVERP images were labelled as being of adequate quality compared to 71.53% of UK Biobank images, with a vessel segmentation sensitivity of 74.64% and specificity of 98.41% (FOREVERP) compared with a sensitivity of 69.12% and specificity of 98.88% (UK Biobank). The mean (± standard deviation) vessel width of the ground truth was 16.21 (4.73) pixels compared to that predicted by QUARTZ of 17.01 (4.49) pixels, resulting in a difference of -0.8 (1.96) pixels. The differences were stable across a range of vessels. The detection rate for optic disc localisation was similar for the two datasets.

CONCLUSION

QUARTZ showed high performance when evaluated on the FOREVERP dataset, and demonstrated robustness across datasets, providing validity to direct comparisons and pooling of retinal feature measures across data sources.

Spotlight on Lattice Degeneration Imaging Techniques

Lattice degeneration (LD), routinely diagnosed with indirect ophthalmoscopy, is one of the most common and clinically significant peripheral retinal findings. In this review, we have summarized the data on currently available imaging techniques which help to improve diagnosis and our understanding of LD pathogenesis. Ultra-wide field imaging provides reliable color fundus capturing for the primary diagnosis of LD and may also be used as a screening tool. Wide-field imaging can be used for targeted documentation of LD lesions using true colors and with minimal optical distortions. Information on the status of the vitreoretinal interface, including detection of retinal holes, detachments, and vitreous tractions, can be obtained with peripheral structural optical coherence tomography (OCT) or scanning laser ophthalmoscopy in retro-mode. These techniques clarify the associated risks of rhegmatogenous retinal detachment. Fundus autofluorescence can provide details on atrophic changes. However, the risk of retinal detachment by means of this technique requires further investigation. OCT angiography may be successfully performed for some lesions. Taken together, OCT and OCT angiography demonstrate thinning of the choroid, alteration of local choroidal microcirculation, and, in severe lesions, involvement of the sclera. OCT angiography confirms loss of retinal microcirculation within LD lesion, which was previously shown with fluorescein angiography. In conclusion, despite relatively simple primary diagnosis, imaging of LD lesions remains challenging due to their peripheral localization. However, several new strategies, including ultra-wide field imaging, peripheral OCT, and scanning laser ophthalmoscopy, make LD imaging possible on a routine basis, improving diagnosis and understanding of LD pathogenesis.

Diagnosis of Acanthamoeba Keratitis: Past, Present and Future

Acanthamoeba keratitis (AK) is a painful and sight-threatening parasitic corneal infection. In recent years, the incidence of AK has increased. Timely and accurate diagnosis is crucial during the management of AK, as delayed diagnosis often results in poor clinical outcomes. Currently, AK diagnosis is primarily achieved through a combination of clinical suspicion, microbiological investigations and corneal imaging. Historically, corneal scraping for microbiological culture has been considered to be the gold standard. Despite its technical ease, accessibility and cost-effectiveness, the long diagnostic turnaround time and variably low sensitivity of microbiological culture limit its use as a sole diagnostic test for AK in clinical practice. In this review, we aim to provide a comprehensive overview of the diagnostic modalities that are currently used to diagnose AK, including microscopy with staining, culture, corneal biopsy, in vivo confocal microscopy, polymerase chain reaction and anterior segment optical coherence tomography. We also highlight emerging techniques, such as next-generation sequencing and artificial intelligence-assisted models, which have the potential to transform the diagnostic landscape of AK.

Semi-automated quantification of vitreal hyperreflective foci in SD-OCT and their relevance in patients with peripheral retinal breaks

BACKGROUND

Retinal breaks (RB) are emergencies that require treatment to prevent progression of rhegmatogenous retinal detachment. Vitreal hyperreflective foci (VHF) representing migration of RPE cell clusters or interphotoreceptor matrix from the RB are potential biomarkers. The aim of this study is to investigate VHF in RB-patients using SD-OCT.

METHODS

The retrospective cross-sectional study included RB patients from our Department of Ophthalmology, HSK Wiesbaden who underwent macular SD-OCT (SPECTRALIS®, Heidelberg Engineering, Germany) on both eyes. VHF, defined and quantified as foci that differ markedly in size and reflectivity from the background speckle pattern, were assessed for presence and frequency. The RB-affected eyes were the study group (G1), the partner eyes the control group (G2).

RESULTS

160 consecutive patients with RB were included. Age was 60 ± 10.2 years (52% female). 89.4% of G1 and 87.5% of G2 were phakic (p = 0.73). 94.4% (n = 151) were symptomatic. Symptom duration was 8.0 ± 10.1 days in G1, 94.4% (n = 151) showed VHF versus 5.6% (p < 0.0001) in G2, of which 75% (n = 6) showed asymptomatic lattice degenerations. Detectable VHF showed a strong association of OR = 320 (95% CI, 110-788, p < 0.0001)) with respect to symptomatic RB. Sensitivity and specificity were 94.7% and 94.7%, respectively.

CONCLUSIONS

Most eyes with symptomatic RB show vitreal VHF in SD-OCT. Detected VHF are strongly associated with RB, and our semi-automated greyscale reflectivity analysis indicates that VHF likely originate from photoreceptor complexes torn out of the RB area that migrate into the vitreous cavity. The presence of VHF may indicate RB and should lead to a thorough fundus examination in both symptomatic and asymptomatic cases.

Artificial intelligence in ophthalmology: The path to the real-world clinic

Artificial intelligence (AI) has great potential to transform healthcare by enhancing the workflow and productivity of clinicians, enabling existing staff to serve more patients, improving patient outcomes, and reducing health disparities. In the field of ophthalmology, AI systems have shown performance comparable with or even better than experienced ophthalmologists in tasks such as diabetic retinopathy detection and grading. However, despite these quite good results, very few AI systems have been deployed in real-world clinical settings, challenging the true value of these systems. This review provides an overview of the current main AI applications in ophthalmology, describes the challenges that need to be overcome prior to clinical implementation of the AI systems, and discusses the strategies that may pave the way to the clinical translation of these systems.

Joint conditional generative adversarial networks for eyelash artifact removal in ultra-wide-field fundus images

Background: Ultra-Wide-Field (UWF) fundus imaging is an essential diagnostic tool for identifying ophthalmologic diseases, as it captures detailed retinal structures within a wider field of view (FOV). However, the presence of eyelashes along the edge of the eyelids can cast shadows and obscure the view of fundus imaging, which hinders reliable interpretation and subsequent screening of fundus diseases. Despite its limitations, there are currently no effective methods or datasets available for removing eyelash artifacts from UWF fundus images. This research aims to develop an effective approach for eyelash artifact removal and thus improve the visual quality of UWF fundus images for accurate analysis and diagnosis. Methods: To address this issue, we first constructed two UWF fundus datasets: the paired synthetic eyelashes (PSE) dataset and the unpaired real eyelashes (uPRE) dataset. Then we proposed a deep learning architecture called Joint Conditional Generative Adversarial Networks (JcGAN) to remove eyelash artifacts from UWF fundus images. JcGAN employs a shared generator with two discriminators for joint learning of both real and synthetic eyelash artifacts. Furthermore, we designed a background refinement module that refines background information and is trained with the generator in an end-to-end manner. Results: Experimental results on both PSE and uPRE datasets demonstrate the superiority of the proposed JcGAN over several state-of-the-art deep learning approaches. Compared with the best existing method, JcGAN improves PSNR and SSIM by 4.82% and 0.23%, respectively. In addition, we also verified that eyelash artifact removal via JcGAN could significantly improve vessel segmentation performance in UWF fundus images. Assessment via vessel segmentation illustrates that the sensitivity, Dice coefficient and area under curve (AUC) of ResU-Net have respectively increased by 3.64%, 1.54%, and 1.43% after eyelash artifact removal using JcGAN. Conclusion: The proposed JcGAN effectively removes eyelash artifacts in UWF images, resulting in improved visibility of retinal vessels. Our method can facilitate better processing and analysis of retinal vessels and has the potential to improve diagnostic outcomes.

Automatic Detection of Peripheral Retinal Lesions From Ultrawide-Field Fundus Images Using Deep Learning

PURPOSE

To establish a multilabel-based deep learning (DL) algorithm for automatic detection and categorization of clinically significant peripheral retinal lesions using ultrawide-field fundus images.

METHODS

A total of 5958 ultrawide-field fundus images from 3740 patients were randomly split into a training set, validation set, and test set. A multilabel classifier was developed to detect rhegmatogenous retinal detachment, cystic retinal tuft, lattice degeneration, and retinal breaks. Referral decision was automatically generated based on the results of each disease class. t -distributed stochastic neighbor embedding heatmaps were used to visualize the features extracted by the neural networks. Gradient-weighted class activation mapping and guided backpropagation heatmaps were generated to investigate the image locations for decision-making by the DL models. The performance of the classifier(s) was evaluated by sensitivity, specificity, accuracy, F 1 score, area under receiver operating characteristic curve (AUROC) with 95% CI, and area under the precision-recall curve.

RESULTS

In the test set, all categories achieved a sensitivity of 0.836-0.918, a specificity of 0.858-0.989, an accuracy of 0.854-0.977, an F 1 score of 0.400-0.931, an AUROC of 0.9205-0.9882, and an area under the precision-recall curve of 0.6723-0.9745. The referral decisions achieved an AUROC of 0.9758 (95% CI= 0.9648-0.9869). The multilabel classifier had significantly better performance in cystic retinal tuft detection than the binary classifier (AUROC= 0.9781 vs 0.6112, P < 0.001). The model showed comparable performance with human experts.

CONCLUSIONS

This new DL model of a multilabel classifier is capable of automatic, accurate, and early detection of clinically significant peripheral retinal lesions with various sample sizes. It can be applied in peripheral retinal screening in clinics.

Intelligent Diagnosis of Multiple Peripheral Retinal Lesions in Ultra-widefield Fundus Images Based on Deep Learning

INTRODUCTION

Compared with traditional fundus examination techniques, ultra-widefield fundus (UWF) images provide 200° panoramic images of the retina, which allows better detection of peripheral retinal lesions. The advent of UWF provides effective solutions only for detection but still lacks efficient diagnostic capabilities. This study proposed a retinal lesion detection model to automatically locate and identify six relatively typical and high-incidence peripheral retinal lesions from UWF images which will enable early screening and rapid diagnosis.

METHODS

A total of 24,602 augmented ultra-widefield fundus images with labels corresponding to 6 peripheral retinal lesions and normal manifestation labelled by 5 ophthalmologists were included in this study. An object detection model named You Only Look Once X (YOLOX) was modified and trained to locate and classify the six peripheral retinal lesions including rhegmatogenous retinal detachment (RRD), retinal breaks (RB), white without pressure (WWOP), cystic retinal tuft (CRT), lattice degeneration (LD), and paving-stone degeneration (PSD). We applied coordinate attention block and generalized intersection over union (GIOU) loss to YOLOX and evaluated it for accuracy, sensitivity, specificity, precision, F1 score, and average precision (AP). This model was able to show the exact location and saliency map of the retinal lesions detected by the model thus contributing to efficient screening and diagnosis.

RESULTS

The model reached an average accuracy of 96.64%, sensitivity of 87.97%, specificity of 98.04%, precision of 87.01%, F1 score of 87.39%, and mAP of 86.03% on test dataset 1 including 248 UWF images and reached an average accuracy of 95.04%, sensitivity of 83.90%, specificity of 96.70%, precision of 78.73%, F1 score of 81.96%, and mAP of 80.59% on external test dataset 2 including 586 UWF images, showing this system performs well in distinguishing the six peripheral retinal lesions.

CONCLUSION

Focusing on peripheral retinal lesions, this work proposed a deep learning model, which automatically recognized multiple peripheral retinal lesions from UWF images and localized exact positions of lesions. Therefore, it has certain potential for early screening and intelligent diagnosis of peripheral retinal lesions.

Deep Learning for the Detection of Multiple Fundus Diseases Using Ultra-widefield Images

INTRODUCTION

To design and evaluate a deep learning model based on ultra-widefield images (UWFIs) that can detect several common fundus diseases.

METHODS

Based on 4574 UWFIs, a deep learning model was trained and validated that can identify normal fundus and eight common fundus diseases, namely referable diabetic retinopathy, retinal vein occlusion, pathologic myopia, retinal detachment, retinitis pigmentosa, age-related macular degeneration, vitreous opacity, and optic neuropathy. The model was tested on three test sets with data volumes of 465, 979, and 525. The performance of the three deep learning networks, EfficientNet-B7, DenseNet, and ResNet-101, was evaluated on the internal test set. Additionally, we compared the performance of the deep learning model with that of doctors in a tertiary referral hospital.

RESULTS

Compared to the other two deep learning models, EfficientNet-B7 achieved the best performance. The area under the receiver operating characteristic curves of the EfficientNet-B7 model on the internal test set, external test set A and external test set B were 0.9708 (0.8772, 0.9849) to 1.0000 (1.0000, 1.0000), 0.9683 (0.8829, 0.9770) to 1.0000 (0.9975, 1.0000), and 0.8919 (0.7150, 0.9055) to 0.9977 (0.9165, 1.0000), respectively. On a data set of 100 images, the total accuracy of the deep learning model was 93.00%, the average accuracy of three ophthalmologists who had been working for 2 years and three ophthalmologists who had been working in fundus imaging for more than 5 years was 88.00% and 94.00%, respectively.

CONCLUSION

High performance was achieved on all three test sets using our UWFI multidisease classification model with a small sample size and fast model inference. The performance of the artificial intelligence model was comparable to that of a physician with 2-5 years of experience in fundus diseases at a tertiary referral hospital. The model is expected to be used as an effective aid for fundus disease screening.

Which data subset should be augmented for deep learning? a simulation study using urothelial cell carcinoma histopathology images

BACKGROUND

Applying deep learning to digital histopathology is hindered by the scarcity of manually annotated datasets. While data augmentation can ameliorate this obstacle, its methods are far from standardized. Our aim was to systematically explore the effects of skipping data augmentation; applying data augmentation to different subsets of the whole dataset (training set, validation set, test set, two of them, or all of them); and applying data augmentation at different time points (before, during, or after dividing the dataset into three subsets). Different combinations of the above possibilities resulted in 11 ways to apply augmentation. The literature contains no such comprehensive systematic comparison of these augmentation ways.

RESULTS

Non-overlapping photographs of all tissues on 90 hematoxylin-and-eosin-stained urinary bladder slides were obtained. Then, they were manually classified as either inflammation (5948 images), urothelial cell carcinoma (5811 images), or invalid (3132 images; excluded). If done, augmentation was eight-fold by flipping and rotation. Four convolutional neural networks (Inception-v3, ResNet-101, GoogLeNet, and SqueezeNet), pre-trained on the ImageNet dataset, were fine-tuned to binary classify images of our dataset. This task was the benchmark for our experiments. Model testing performance was evaluated using accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve. Model validation accuracy was also estimated. The best testing performance was achieved when augmentation was done to the remaining data after test-set separation, but before division into training and validation sets. This leaked information between the training and the validation sets, as evidenced by the optimistic validation accuracy. However, this leakage did not cause the validation set to malfunction. Augmentation before test-set separation led to optimistic results. Test-set augmentation yielded more accurate evaluation metrics with less uncertainty. Inception-v3 had the best overall testing performance.

CONCLUSIONS

In digital histopathology, augmentation should include both the test set (after its allocation), and the remaining combined training/validation set (before being split into separate training and validation sets). Future research should try to generalize our results.

Ultra-wide-field fundus photography compared to ophthalmoscopy in diagnosing and classifying major retinal diseases

To analyze the performance of ultra-wide-field (UWF) fundus photography compared with ophthalmoscopy in identifying and classifying retinal diseases. Patients examined for presumed major retinal disorders were consecutively enrolled. Each patient underwent indirect ophthalmoscopic evaluation, with scleral depression and/or fundus biomicroscopy, when clinically indicated, and mydriatic UWF fundus imaging by means of CLARUS 500™ fundus camera. Each eye was classified by a clinical grader and two image graders in the following groups: normal retina, diabetic retinopathy, vascular abnormalities, macular degenerations and dystrophies, retinal and choroidal tumors, peripheral degenerative lesions and retinal detachment and myopic alterations. 7024 eyes of new patients were included. The inter-grader agreement for images classification was perfect (kappa = 0.998, 95% Confidence Interval (95%CI) = 0.997-0.999), as the two methods concordance for retinal diseases diagnosis (kappa = 0.997, 95%CI = 0.996-0.999) without statistically significant difference. UWF fundus imaging might be an alternative to ophthalmoscopy, since it allows to accurately classify major retinal diseases, widening the range of disorders possibly diagnosed with teleophthalmology. Although the clinician should be aware of the possibility that a minority of the most peripheral lesions may be not entirely visualized, it might be considered a first line diagnostic modality, in the context of a full ophthalmological examination.

Automated detection of retinal exudates and drusen in ultra-widefield fundus images based on deep learning

BACKGROUND

Retinal exudates and/or drusen (RED) can be signs of many fundus diseases that can lead to irreversible vision loss. Early detection and treatment of these diseases are critical for improving vision prognosis. However, manual RED screening on a large scale is time-consuming and labour-intensive. Here, we aim to develop and assess a deep learning system for automated detection of RED using ultra-widefield fundus (UWF) images.

METHODS

A total of 26,409 UWF images from 14,994 subjects were used to develop and evaluate the deep learning system. The Zhongshan Ophthalmic Center (ZOC) dataset was selected to compare the performance of the system to that of retina specialists in RED detection. The saliency map visualization technique was used to understand which areas in the UWF image had the most influence on our deep learning system when detecting RED.

RESULTS

The system for RED detection achieved areas under the receiver operating characteristic curve of 0.994 (95% confidence interval [CI]: 0.991-0.996), 0.972 (95% CI: 0.957-0.984), and 0.988 (95% CI: 0.983-0.992) in three independent datasets. The performance of the system in the ZOC dataset was comparable to that of an experienced retina specialist. Regions of RED were highlighted by saliency maps in UWF images.

CONCLUSIONS

Our deep learning system is reliable in the automated detection of RED in UWF images. As a screening tool, our system may promote the early diagnosis and management of RED-related fundus diseases.

Deep learning for ultra-widefield imaging: a scoping review

PURPOSE

This article is a scoping review of published and peer-reviewed articles using deep-learning (DL) applied to ultra-widefield (UWF) imaging. This study provides an overview of the published uses of DL and UWF imaging for the detection of ophthalmic and systemic diseases, generative image synthesis, quality assessment of images, and segmentation and localization of ophthalmic image features.

METHODS

A literature search was performed up to August 31st, 2021 using PubMed, Embase, Cochrane Library, and Google Scholar. The inclusion criteria were as follows: (1) deep learning, (2) ultra-widefield imaging. The exclusion criteria were as follows: (1) articles published in any language other than English, (2) articles not peer-reviewed (usually preprints), (3) no full-text availability, (4) articles using machine learning algorithms other than deep learning. No study design was excluded from consideration.

RESULTS

A total of 36 studies were included. Twenty-three studies discussed ophthalmic disease detection and classification, 5 discussed segmentation and localization of ultra-widefield images (UWFIs), 3 discussed generative image synthesis, 3 discussed ophthalmic image quality assessment, and 2 discussed detecting systemic diseases via UWF imaging.

CONCLUSION

The application of DL to UWF imaging has demonstrated significant effectiveness in the diagnosis and detection of ophthalmic diseases including diabetic retinopathy, retinal detachment, and glaucoma. DL has also been applied in the generation of synthetic ophthalmic images. This scoping review highlights and discusses the current uses of DL with UWF imaging, and the future of DL applications in this field.

Novel technical and privacy-preserving technology for artificial intelligence in ophthalmology

PURPOSE OF REVIEW

The application of artificial intelligence (AI) in medicine and ophthalmology has experienced exponential breakthroughs in recent years in diagnosis, prognosis, and aiding clinical decision-making. The use of digital data has also heralded the need for privacy-preserving technology to protect patient confidentiality and to guard against threats such as adversarial attacks. Hence, this review aims to outline novel AI-based systems for ophthalmology use, privacy-preserving measures, potential challenges, and future directions of each.

RECENT FINDINGS

Several key AI algorithms used to improve disease detection and outcomes include: Data-driven, imagedriven, natural language processing (NLP)-driven, genomics-driven, and multimodality algorithms. However, deep learning systems are susceptible to adversarial attacks, and use of data for training models is associated with privacy concerns. Several data protection methods address these concerns in the form of blockchain technology, federated learning, and generative adversarial networks.

SUMMARY

AI-applications have vast potential to meet many eyecare needs, consequently reducing burden on scarce healthcare resources. A pertinent challenge would be to maintain data privacy and confidentiality while supporting AI endeavors, where data protection methods would need to rapidly evolve with AI technology needs. Ultimately, for AI to succeed in medicine and ophthalmology, a balance would need to be found between innovation and privacy.

Artificial intelligence to detect malignant eyelid tumors from photographic images

Malignant eyelid tumors can invade adjacent structures and pose a threat to vision and even life. Early identification of malignant eyelid tumors is crucial to avoiding substantial morbidity and mortality. However, differentiating malignant eyelid tumors from benign ones can be challenging for primary care physicians and even some ophthalmologists. Here, based on 1,417 photographic images from 851 patients across three hospitals, we developed an artificial intelligence system using a faster region-based convolutional neural network and deep learning classification networks to automatically locate eyelid tumors and then distinguish between malignant and benign eyelid tumors. The system performed well in both internal and external test sets (AUCs ranged from 0.899 to 0.955). The performance of the system is comparable to that of a senior ophthalmologist, indicating that this system has the potential to be used at the screening stage for promoting the early detection and treatment of malignant eyelid tumors.

Comparison of deep learning systems and cornea specialists in detecting corneal diseases from low-quality images

The performance of deep learning in disease detection from high-quality clinical images is identical to and even greater than that of human doctors. However, in low-quality images, deep learning performs poorly. Whether human doctors also have poor performance in low-quality images is unknown. Here, we compared the performance of deep learning systems with that of cornea specialists in detecting corneal diseases from low-quality slit lamp images. The results showed that the cornea specialists performed better than our previously established deep learning system (PEDLS) trained on only high-quality images. The performance of the system trained on both high- and low-quality images was superior to that of the PEDLS while inferior to that of a senior corneal specialist. This study highlights that cornea specialists perform better in low-quality images than the system trained on high-quality images. Adding low-quality images with sufficient diagnostic certainty to the training set can reduce this performance gap.

•

Deep learning performs poorly in low-quality images for detecting corneal diseases

•

Corneal specialists perform better than the PEDLS in low-quality images

•

The performance of the NDLS is better than that of the PEDLS in low-quality images

•

Adding low-quality images to the training set can improve the system's performance

Artificial intelligence in myopia: current and future trends

PURPOSE OF REVIEW

Myopia is one of the leading causes of visual impairment, with a projected increase in prevalence globally. One potential approach to address myopia and its complications is early detection and treatment. However, current healthcare systems may not be able to cope with the growing burden. Digital technological solutions such as artificial intelligence (AI) have emerged as a potential adjunct for myopia management.

RECENT FINDINGS

There are currently four significant domains of AI in myopia, including machine learning (ML), deep learning (DL), genetics and natural language processing (NLP). ML has been demonstrated to be a useful adjunctive for myopia prediction and biometry for cataract surgery in highly myopic individuals. DL techniques, particularly convoluted neural networks, have been applied to various image-related diagnostic and predictive solutions. Applications of AI in genomics and NLP appear to be at a nascent stage.

SUMMARY

Current AI research is mainly focused on disease classification and prediction in myopia. Through greater collaborative research, we envision AI will play an increasingly critical role in big data analysis by aggregating a greater variety of parameters including genomics and environmental factors. This may enable the development of generalizable adjunctive DL systems that could help realize predictive and individualized precision medicine for myopic patients.

Application of Comprehensive Artificial intelligence Retinal Expert (CARE) system: a national real-world evidence study

BACKGROUND

Medical artificial intelligence (AI) has entered the clinical implementation phase, although real-world performance of deep-learning systems (DLSs) for screening fundus disease remains unsatisfactory. Our study aimed to train a clinically applicable DLS for fundus diseases using data derived from the real world, and externally test the model using fundus photographs collected prospectively from the settings in which the model would most likely be adopted.

METHODS

In this national real-world evidence study, we trained a DLS, the Comprehensive AI Retinal Expert (CARE) system, to identify the 14 most common retinal abnormalities using 207 228 colour fundus photographs derived from 16 clinical settings with different disease distributions. CARE was internally validated using 21 867 photographs and externally tested using 18 136 photographs prospectively collected from 35 real-world settings across China where CARE might be adopted, including eight tertiary hospitals, six community hospitals, and 21 physical examination centres. The performance of CARE was further compared with that of 16 ophthalmologists and tested using datasets with non-Chinese ethnicities and previously unused camera types. This study was registered with ClinicalTrials.gov, NCT04213430, and is currently closed.

FINDINGS

The area under the receiver operating characteristic curve (AUC) in the internal validation set was 0·955 (SD 0·046). AUC values in the external test set were 0·965 (0·035) in tertiary hospitals, 0·983 (0·031) in community hospitals, and 0·953 (0·042) in physical examination centres. The performance of CARE was similar to that of ophthalmologists. Large variations in sensitivity were observed among the ophthalmologists in different regions and with varying experience. The system retained strong identification performance when tested using the non-Chinese dataset (AUC 0·960, 95% CI 0·957-0·964 in referable diabetic retinopathy).

INTERPRETATION

Our DLS (CARE) showed satisfactory performance for screening multiple retinal abnormalities in real-world settings using prospectively collected fundus photographs, and so could allow the system to be implemented and adopted for clinical care.

FUNDING

This study was funded by the National Key R&D Programme of China, the Science and Technology Planning Projects of Guangdong Province, the National Natural Science Foundation of China, the Natural Science Foundation of Guangdong Province, and the Fundamental Research Funds for the Central Universities.

TRANSLATION

For the Chinese translation of the abstract see Supplementary Materials section.

Preventing corneal blindness caused by keratitis using artificial intelligence

Keratitis is the main cause of corneal blindness worldwide. Most vision loss caused by keratitis can be avoidable via early detection and treatment. The diagnosis of keratitis often requires skilled ophthalmologists. However, the world is short of ophthalmologists, especially in resource-limited settings, making the early diagnosis of keratitis challenging. Here, we develop a deep learning system for the automated classification of keratitis, other cornea abnormalities, and normal cornea based on 6,567 slit-lamp images. Our system exhibits remarkable performance in cornea images captured by the different types of digital slit lamp cameras and a smartphone with the super macro mode (all AUCs>0.96). The comparable sensitivity and specificity in keratitis detection are observed between the system and experienced cornea specialists. Our system has the potential to be applied to both digital slit lamp cameras and smartphones to promote the early diagnosis and treatment of keratitis, preventing the corneal blindness caused by keratitis.

Digital technology, tele-medicine and artificial intelligence in ophthalmology: A global perspective

The simultaneous maturation of multiple digital and telecommunications technologies in 2020 has created an unprecedented opportunity for ophthalmology to adapt to new models of care using tele-health supported by digital innovations. These digital innovations include artificial intelligence (AI), 5th generation (5G) telecommunication networks and the Internet of Things (IoT), creating an inter-dependent ecosystem offering opportunities to develop new models of eye care addressing the challenges of COVID-19 and beyond. Ophthalmology has thrived in some of these areas partly due to its many image-based investigations. Tele-health and AI provide synchronous solutions to challenges facing ophthalmologists and healthcare providers worldwide. This article reviews how countries across the world have utilised these digital innovations to tackle diabetic retinopathy, retinopathy of prematurity, age-related macular degeneration, glaucoma, refractive error correction, cataract and other anterior segment disorders. The review summarises the digital strategies that countries are developing and discusses technologies that may increasingly enter the clinical workflow and processes of ophthalmologists. Furthermore as countries around the world have initiated a series of escalating containment and mitigation measures during the COVID-19 pandemic, the delivery of eye care services globally has been significantly impacted. As ophthalmic services adapt and form a "new normal", the rapid adoption of some of telehealth and digital innovation during the pandemic is also discussed. Finally, challenges for validation and clinical implementation are considered, as well as recommendations on future directions.

Development of a deep learning-based image quality control system to detect and filter out ineligible slit-lamp images: A multicenter study

BACKGROUND AND OBJECTIVE

Previous studies developed artificial intelligence (AI) diagnostic systems only using eligible slit-lamp images for detecting corneal diseases. However, images of ineligible quality (including poor-field, defocused, and poor-location images), which are inevitable in the real world, can cause diagnostic information loss and thus affect downstream AI-based image analysis. Manual evaluation for the eligibility of slit-lamp images often requires an ophthalmologist, and this procedure can be time-consuming and labor-intensive when applied on a large scale. Here, we aimed to develop a deep learning-based image quality control system (DLIQCS) to automatically detect and filter out ineligible slit-lamp images (poor-field, defocused, and poor-location images).

METHODS

We developed and externally evaluated the DLIQCS based on 48,530 slit-lamp images (19,890 individuals) that were derived from 4 independent institutions using different types of digital slit lamp cameras. To find the best deep learning model for the DLIQCS, we used 3 algorithms (AlexNet, DenseNet121, and InceptionV3) to train models. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy were leveraged to assess the performance of each algorithm for the classification of poor-field, defocused, poor-location, and eligible images.

RESULTS

In an internal test dataset, the best algorithm DenseNet121 had AUCs of 0.999, 1.000, 1.000, and 1.000 in the detection of poor-field, defocused, poor-location, and eligible images, respectively. In external test datasets, the AUCs of the best algorithm DenseNet121 for identifying poor-field, defocused, poor-location, and eligible images were ranged from 0.997 to 0.997, 0.983 to 0.995, 0.995 to 0.998, and 0.999 to 0.999, respectively.

CONCLUSIONS

Our DLIQCS can accurately detect poor-field, defocused, poor-location, and eligible slit-lamp images in an automated fashion. This system may serve as a prescreening tool to filter out ineligible images and enable that only eligible images would be transferred to the subsequent AI diagnostic systems.

Development of a deep-learning system for detection of lattice degeneration, retinal breaks, and retinal detachment in tessellated eyes using ultra-wide-field fundus images: a pilot study

PURPOSE

To investigate the detection of lattice degeneration, retinal breaks, and retinal detachment in tessellated eyes using ultra-wide-field fundus imaging system (Optos) with convolutional neural network technology.

METHODS

This study included 1500 Optos color images for tessellated fundus confirmation and peripheral retinal lesion (lattice degeneration, retinal breaks, and retinal detachment) assessment. Three retinal specialists evaluated all images and proposed the reference standard when an agreement was achieved. Then, 722 images were used to train and verify a combined deep-learning system of 3 optimal binary classification models trained using seResNext50 algorithm with 2 preprocessing methods (original resizing and cropping), and a test set of 189 images were applied to verify the performance compared to the reference standard.

RESULTS

With optimal preprocessing approach (original resizing method for lattice degeneration and retinal detachment, cropping method for retinal breaks), the combined deep-learning system exhibited an area under curve of 0.888, 0.953, and 1.000 for detection of lattice degeneration, retinal breaks, and retinal detachment respectively in tessellated eyes. The referral accuracy of this system was 79.8% compared to the reference standard.

CONCLUSION

A deep-learning system is feasible to detect lattice degeneration, retinal breaks, and retinal detachment in tessellated eyes using ultra-wide-field images. And this system may be considered for screening and telemedicine.

Deep learning from "passive feeding" to "selective eating" of real-world data

Artificial intelligence (AI) based on deep learning has shown excellent diagnostic performance in detecting various diseases with good-quality clinical images. Recently, AI diagnostic systems developed from ultra-widefield fundus (UWF) images have become popular standard-of-care tools in screening for ocular fundus diseases. However, in real-world settings, these systems must base their diagnoses on images with uncontrolled quality ("passive feeding"), leading to uncertainty about their performance. Here, using 40,562 UWF images, we develop a deep learning-based image filtering system (DLIFS) for detecting and filtering out poor-quality images in an automated fashion such that only good-quality images are transferred to the subsequent AI diagnostic system ("selective eating"). In three independent datasets from different clinical institutions, the DLIFS performed well with sensitivities of 96.9%, 95.6% and 96.6%, and specificities of 96.6%, 97.9% and 98.8%, respectively. Furthermore, we show that the application of our DLIFS significantly improves the performance of established AI diagnostic systems in real-world settings. Our work demonstrates that "selective eating" of real-world data is necessary and needs to be considered in the development of image-based AI systems.

Deep learning for automated glaucomatous optic neuropathy detection from ultra-widefield fundus images

Background/Aims

To develop a deep learning system for automated glaucomatous optic neuropathy (GON) detection using ultra-widefield fundus (UWF) images.

Methods

We trained, validated and externally evaluated a deep learning system for GON detection based on 22 972 UWF images from 10 590 subjects that were collected at 4 different institutions in China and Japan. The InceptionResNetV2 neural network architecture was used to develop the system. The area under the receiver operating characteristic curve (AUC), sensitivity and specificity were used to assess the performance of detecting GON by the system. The data set from the Zhongshan Ophthalmic Center (ZOC) was selected to compare the performance of the system to that of ophthalmologists who mainly conducted UWF image analysis in clinics.

Results

The system for GON detection achieved AUCs of 0.983–0.999 with sensitivities of 97.5–98.2% and specificities of 94.3–98.4% in four independent data sets. The most common reasons for false-negative results were confounding optic disc characteristics caused by high myopia or pathological myopia (n=39 (53%)). The leading cause for false-positive results was having other fundus lesions (n=401 (96%)). The performance of the system in the ZOC data set was comparable to that of an experienced ophthalmologist (p>0.05).

Conclusion

Our deep learning system can accurately detect GON from UWF images in an automated fashion. It may be used as a screening tool to improve the accessibility of screening and promote the early diagnosis and management of glaucoma.

Interpretation of artificial intelligence studies for the ophthalmologist

PURPOSE OF REVIEW

The use of artificial intelligence (AI) in ophthalmology has increased dramatically. However, interpretation of these studies can be a daunting prospect for the ophthalmologist without a background in computer or data science. This review aims to share some practical considerations for interpretation of AI studies in ophthalmology.

RECENT FINDINGS

It can be easy to get lost in the technical details of studies involving AI. Nevertheless, it is important for clinicians to remember that the fundamental questions in interpreting these studies remain unchanged - What does this study show, and how does this affect my patients? Being guided by familiar principles like study purpose, impact, validity, and generalizability, these studies become more accessible to the ophthalmologist. Although it may not be necessary for nondomain experts to understand the exact AI technical details, we explain some broad concepts in relation to AI technical architecture and dataset management.

SUMMARY

The expansion of AI into healthcare and ophthalmology is here to stay. AI systems have made the transition from bench to bedside, and are already being applied to patient care. In this context, 'AI education' is crucial for ophthalmologists to be confident in interpretation and translation of new developments in this field to their own clinical practice.

Fundamentals of artificial intelligence for ophthalmologists

PURPOSE OF REVIEW

As artificial intelligence continues to develop new applications in ophthalmic image recognition, we provide here an introduction for ophthalmologists and a primer on the mechanisms of deep learning systems.

RECENT FINDINGS

Deep learning has lent itself to the automated interpretation of various retinal imaging modalities, including fundus photography and optical coherence tomography. Convolutional neural networks (CNN) represent the primary class of deep neural networks applied to these image analyses. These have been configured to aid in the detection of diabetes retinopathy, AMD, retinal detachment, glaucoma, and ROP, among other ocular disorders. Predictive models for retinal disease prognosis and treatment are also being validated.

SUMMARY

Deep learning systems have begun to demonstrate a reliable level of diagnostic accuracy equal or better to human graders for narrow image recognition tasks. However, challenges regarding the use of deep learning systems in ophthalmology remain. These include trust of unsupervised learning systems and the limited ability to recognize broad ranges of disorders.