Feasibility study to improve deep learning in OCT diagnosis of rare retinal diseases with few-shot classification

Developing a 10-Layer Retinal Segmentation for MacTel Using Semi-Supervised Learning

Purpose

Automated segmentation software in optical coherence tomography (OCT) devices is usually developed for and primarily tested on common diseases. Therefore segmentation accuracy of automated software can be limited in eyes with rare pathologies.

Methods

We sought to develop a semisupervised deep learning segmentation model that segments 10 retinal layers and four retinal features in eyes with Macular Telangiectasia Type II (MacTel) using a small labeled dataset by leveraging unlabeled images. We compared our model against popular supervised and semisupervised models, as well as conducted ablation studies on the model itself.

Results

Our model significantly outperformed all other models in terms of intersection over union on the 10 retinal layers and two retinal features in the test dataset. For the remaining two features, the pre-retinal space above the internal limiting membrane and the background below the retinal pigment epithelium, all of the models performed similarly. Furthermore, we showed that using more unlabeled images improved the performance of our semisupervised model.

Conclusions

Our model improves segmentation performance over supervised models by leveraging unlabeled data. This approach has the potential to improve segmentation performance for other diseases, where labeled data is limited but unlabeled data abundant.

Translational Relevance

Improving automated segmentation of MacTel pathology on OCT imaging by leveraging unlabeled data may enable more accurate assessment of disease progression, and this approach may be useful for improving feature identification and location on OCT in other rare diseases as well.

Wavelet-based selection-and-recalibration network for Parkinson's disease screening in OCT images

BACKGROUND AND OBJECTIVE

Parkinson's disease (PD) is one of the most prevalent neurodegenerative brain diseases worldwide. Therefore, accurate PD screening is crucial for early clinical intervention and treatment. Recent clinical research indicates that changes in pathology, such as the texture and thickness of the retinal layers, can serve as biomarkers for clinical PD diagnosis based on optical coherence tomography (OCT) images. However, the pathological manifestations of PD in the retinal layers are subtle compared to the more salient lesions associated with retinal diseases.

METHODS

Inspired by textural edge feature extraction in frequency domain learning, we aim to explore a potential approach to enhance the distinction between the feature distributions in retinal layers of PD cases and healthy controls. In this paper, we introduce a simple yet novel wavelet-based selection and recalibration module to effectively enhance the feature representations of the deep neural network by aggregating the unique clinical properties, such as the retinal layers in each frequency band. We combine this module with the residual block to form a deep network named Wavelet-based Selection and Recalibration Network (WaveSRNet) for automatic PD screening.

RESULTS

The extensive experiments on a clinical PD-OCT dataset and two publicly available datasets demonstrate that our approach outperforms state-of-the-art methods. Visualization analysis and ablation studies are conducted to enhance the explainability of WaveSRNet in the decision-making process.

CONCLUSIONS

Our results suggest the potential role of the retina as an assessment tool for PD. Visual analysis shows that PD-related elements include not only certain retinal layers but also the location of the fovea in OCT images.

Exploring Publicly Accessible Optical Coherence Tomography Datasets: A Comprehensive Overview

Artificial intelligence has transformed medical diagnostic capabilities, particularly through medical image analysis. AI algorithms perform well in detecting abnormalities with a strong performance, enabling computer-aided diagnosis by analyzing the extensive amounts of patient data. The data serve as a foundation upon which algorithms learn and make predictions. Thus, the importance of data cannot be underestimated, and clinically corresponding datasets are required. Many researchers face a lack of medical data due to limited access, privacy concerns, or the absence of available annotations. One of the most widely used diagnostic tools in ophthalmology is Optical Coherence Tomography (OCT). Addressing the data availability issue is crucial for enhancing AI applications in the field of OCT diagnostics. This review aims to provide a comprehensive analysis of all publicly accessible retinal OCT datasets. Our main objective is to compile a list of OCT datasets and their properties, which can serve as an accessible reference, facilitating data curation for medical image analysis tasks. For this review, we searched through the Zenodo repository, Mendeley Data repository, MEDLINE database, and Google Dataset search engine. We systematically evaluated all the identified datasets and found 23 open-access datasets containing OCT images, which significantly vary in terms of size, scope, and ground-truth labels. Our findings indicate the need for improvement in data-sharing practices and standardized documentation. Enhancing the availability and quality of OCT datasets will support the development of AI algorithms and ultimately improve diagnostic capabilities in ophthalmology. By providing a comprehensive list of accessible OCT datasets, this review aims to facilitate better utilization and development of AI in medical image analysis.

A hybrid model for the detection of retinal disorders using artificial intelligence techniques

The prevalence of vision impairment is increasing at an alarming rate. The goal of the study was to create an automated method that uses optical coherence tomography (OCT) to classify retinal disorders into four categories: choroidal neovascularization, diabetic macular edema, drusen, and normal cases. This study proposed a new framework that combines machine learning and deep learning-based techniques. The utilized classifiers were support vector machine (SVM), K-nearest neighbor (K-NN), decision tree (DT), and ensemble model (EM). A feature extractor, the InceptionV3 convolutional neural network, was also employed. The performance of the models was evaluated against nine criteria using a dataset of 18000 OCT images. For the SVM, K-NN, DT, and EM classifiers, the analysis exhibited state-of-the-art performance, with classification accuracies of 99.43%, 99.54%, 97.98%, and 99.31%, respectively. A promising methodology has been introduced for the automatic identification and classification of retinal disorders, leading to reduced human error and saved time.

Latest developments of generative artificial intelligence and applications in ophthalmology

The emergence of generative artificial intelligence (AI) has revolutionized various fields. In ophthalmology, generative AI has the potential to enhance efficiency, accuracy, personalization and innovation in clinical practice and medical research, through processing data, streamlining medical documentation, facilitating patient-doctor communication, aiding in clinical decision-making, and simulating clinical trials. This review focuses on the development and integration of generative AI models into clinical workflows and scientific research of ophthalmology. It outlines the need for development of a standard framework for comprehensive assessments, robust evidence, and exploration of the potential of multimodal capabilities and intelligent agents. Additionally, the review addresses the risks in AI model development and application in clinical service and research of ophthalmology, including data privacy, data bias, adaptation friction, over interdependence, and job replacement, based on which we summarized a risk management framework to mitigate these concerns. This review highlights the transformative potential of generative AI in enhancing patient care, improving operational efficiency in the clinical service and research in ophthalmology. It also advocates for a balanced approach to its adoption.

Multi-contrast learning-guided lightweight few-shot learning scheme for predicting breast cancer molecular subtypes

Invasive gene expression profiling studies have exposed prognostically significant breast cancer subtypes: normal-like, luminal, HER-2 enriched, and basal-like, which is defined in large part by human epidermal growth factor receptor 2 (HER-2), progesterone receptor (PR), and estrogen receptor (ER). However, while dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has been generally employed in the screening and therapy of breast cancer, there is a challenging problem to noninvasively predict breast cancer molecular subtypes, which have extremely low-data regimes. In this paper, a novel few-shot learning scheme, which combines lightweight contrastive convolutional neural network (LC-CNN) and multi-contrast learning strategy (MCLS), is worthwhile to be developed for predicting molecular subtype of breast cancer in DCE-MRI. Moreover, MCLS is designed to construct One-vs-Rest and One-vs-One classification tasks, which addresses inter-class similarity among normal-like, luminal, HER-2 enriched, and basal-like. Extensive experiments demonstrate the superiority of our proposed scheme over state-of-the-art methods. Furthermore, our scheme is able to achieve competitive results on few samples due to joint LC-CNN and MCLS for excavating contrastive correlations of a pair of DCE-MRI.

Diagnostic accuracy of artificial intelligence in detecting retinitis pigmentosa: A systematic review and meta-analysis

Retinitis pigmentosa (RP) is often undetected in its early stages. Artificial intelligence (AI) has emerged as a promising tool in medical diagnostics. Therefore, we conducted a systematic review and meta-analysis to evaluate the diagnostic accuracy of AI in detecting RP using various ophthalmic images. We conducted a systematic search on PubMed, Scopus, and Web of Science databases on December 31, 2022. We included studies in the English language that used any ophthalmic imaging modality, such as OCT or fundus photography, used any AI technologies, had at least an expert in ophthalmology as a reference standard, and proposed an AI algorithm able to distinguish between images with and without retinitis pigmentosa features. We considered the sensitivity, specificity, and area under the curve (AUC) as the main measures of accuracy. We had a total of 14 studies in the qualitative analysis and 10 studies in the quantitative analysis. In total, the studies included in the meta-analysis dealt with 920,162 images. Overall, AI showed an excellent performance in detecting RP with pooled sensitivity and specificity of 0.985 [95%CI: 0.948-0.996], 0.993 [95%CI: 0.982-0.997] respectively. The area under the receiver operating characteristic (AUROC), using a random-effect model, was calculated to be 0.999 [95%CI: 0.998-1.000; P < 0.001]. The Zhou and Dendukuri I² test revealed a low level of heterogeneity between the studies, with [I2 = 19.94%] for sensitivity and [I2 = 21.07%] for specificity. The bivariate I² [20.33%] also suggested a low degree of heterogeneity. We found evidence supporting the accuracy of AI in the detection of RP; however, the level of heterogeneity between the studies was low.

Hybrid unsupervised representation learning and pseudo-label supervised self-distillation for rare disease imaging phenotype classification with dispersion-aware imbalance correction

Rare diseases are characterized by low prevalence and are often chronically debilitating or life-threatening. Imaging phenotype classification of rare diseases is challenging due to the severe shortage of training examples. Few-shot learning (FSL) methods tackle this challenge by extracting generalizable prior knowledge from a large base dataset of common diseases and normal controls and transferring the knowledge to rare diseases. Yet, most existing methods require the base dataset to be labeled and do not make full use of the precious examples of rare diseases. In addition, the extremely small size of the training samples may result in inter-class performance imbalance due to insufficient sampling of the true distributions. To this end, we propose in this work a novel hybrid approach to rare disease imaging phenotype classification, featuring three key novelties targeted at the above drawbacks. First, we adopt the unsupervised representation learning (URL) based on self-supervising contrastive loss, whereby to eliminate the overhead in labeling the base dataset. Second, we integrate the URL with pseudo-label supervised classification for effective self-distillation of the knowledge about the rare diseases, composing a hybrid approach taking advantage of both unsupervised and (pseudo-) supervised learning on the base dataset. Third, we use the feature dispersion to assess the intra-class diversity of training samples, to alleviate the inter-class performance imbalance via dispersion-aware correction. Experimental results of imaging phenotype classification of both simulated (skin lesions and cervical smears) and real clinical rare diseases (retinal diseases) show that our hybrid approach substantially outperforms existing FSL methods (including those using a fully supervised base dataset) via effective integration of the URL, pseudo-label driven self-distillation, and dispersion-aware imbalance correction, thus establishing a new state of the art.

Deep learning prediction of steep and flat corneal curvature using fundus photography in post-COVID telemedicine era

Recently, fundus photography (FP) is being increasingly used. Corneal curvature is an essential factor in refractive errors and is associated with several pathological corneal conditions. As FP-based examination systems have already been widely distributed, it would be helpful for telemedicine to extract information such as corneal curvature using FP. This study aims to develop a deep learning model based on FP for corneal curvature prediction by categorizing corneas into steep, regular, and flat groups. The EfficientNetB0 architecture with transfer learning was used to learn FP patterns to predict flat, regular, and steep corneas. In validation, the model achieved a multiclass accuracy of 0.727, a Matthews correlation coefficient of 0.519, and an unweighted Cohen's κ of 0.590. The areas under the receiver operating characteristic curves for binary prediction of flat and steep corneas were 0.863 and 0.848, respectively. The optic nerve and its peripheral areas were the main focus of the model. The developed algorithm shows that FP can potentially be used as an imaging modality to estimate corneal curvature in the post-COVID-19 era, whereby patients may benefit from the detection of abnormal corneal curvatures using FP in the telemedicine setting.

Few-shot learning based oral cancer diagnosis using a dual feature extractor prototypical network

A large global health issue is cancer, wherein early diagnosis and treatment have proven to be life-saving. This holds true for oral cancer, thus emphasizing the significance of timely intervention. Deep learning techniques have gained traction in early cancer detection, exhibiting promising outcomes in accurate diagnosis. However, collecting a substantial amount of training data poses a challenge for deep learning models in cancer diagnosis. To address this limitation, this study proposes an oral cancer diagnosis approach based on a few-shot learning framework that circumvents the need for extensive training data. Specifically, a prototypical network is employed to construct a diagnostic model, wherein two feature extractors are utilized to extract prototypical features and query features respectively, departing from the conventional use of a single feature extraction function in prototypical networks. Moreover, a customized loss function is designed for the proposed method. Rigorous experimentation using a histopathological image dataset demonstrates the superior performance of our proposed approach over comparison methods.

A difficulty-aware and task-augmentation method based on meta-learning model for few-shot diabetic retinopathy classification

Background

Accurate classification techniques are essential for the early diagnosis and treatment of patients with diabetic retinopathy (DR). However, the limited amount of annotated DR data poses a challenge for existing deep-learning models. This article proposes a difficulty-aware and task-augmentation method based on meta-learning (DaTa-ML) model for few-shot DR classification with fundus images.

Methods

The difficulty-aware (Da) method operates by dynamically modifying the cross-entropy loss function applied to learning tasks. This methodology has the ability to intelligently down-weight simpler tasks, while simultaneously prioritizing more challenging tasks. These adjustments occur automatically and aim to optimize the learning process. Additionally, the task-augmentation (Ta) method is used to enhance the meta-training process by augmenting the number of tasks through image rotation and improving the feature-extraction capability. To implement the expansion of the meta-training tasks, various task instances can be sampled during the meta-training stage. Ultimately, the proposed Ta method was introduced to optimize the initialization parameters and enhance the meta-generalization performance of the model. The DaTa-ML model showed promising results by effectively addressing the challenges associated with few-shot DR classification.

Results

The Asia Pacific Tele-Ophthalmology Society (APTOS) 2019 blindness detection data set was used to evaluate the DaTa-ML model. The results showed that with only 1% of the training data (5-way, 20-shot) and a single update step (training time reduced by 90%), the DaTa-ML model had an accuracy rate of 89.6% on the test data, which is a 1.7% improvement over the transfer-learning method [i.e., residual neural network (ResNet)50 pre-trained on ImageNet], and a 16.8% improvement over scratch-built models (i.e., ResNet50 without pre-trained weights), despite having fewer trainable parameters (the parameters used by the DaTa-ML model are only 0.47% of the ResNet50 parameters).

Conclusions

The DaTa-ML model provides a more efficient DR classification solution with little annotated data and has significant advantages over state-of-the-art methods. Thus, it could be used to guide and assist ophthalmologists to determine the severity of DR.

A few-shot disease diagnosis decision making model based on meta-learning for general practice

BACKGROUND

Diagnostic errors have become the biggest threat to the safety of patients in primary health care. General practitioners, as the "gatekeepers" of primary health care, have a responsibility to accurately diagnose patients. However, many general practitioners have insufficient knowledge and clinical experience in some diseases. Clinical decision making tools need to be developed to effectively improve the diagnostic process in primary health care. The long-tailed class distributions of medical datasets are challenging for many popular decision making models based on deep learning, which have difficulty predicting few-shot diseases. Meta-learning is a new strategy for solving few-shot problems.

METHODS AND MATERIALS

In this study, a few-shot disease diagnosis decision making model based on a model-agnostic meta-learning algorithm (FSDD-MAML) is proposed. The MAML algorithm is applied in a knowledge graph-based disease diagnosis model to find the optimal model parameters. Moreover, FSDD-MAML can learn learning rates for all modules of the knowledge graph-based disease diagnosis model. For n-way, k-shot learning tasks, the inner loop of FSDD-MAML performs multiple gradient update steps to learn internal features in disease classification tasks using n×k examples, and the outer loop of FSDD-MAML optimizes the meta-objective to find the associated optimal parameters and learning rates. FSDD-MAML is compared with the original knowledge graph-based disease diagnosis model and other meta-learning algorithms based on an abdominal disease dataset.

RESULT

Meta-learning algorithms can greatly improve the performance of models in top-1 evaluation compared with top-3, top-5, and top-10 evaluations. The proposed decision making model FSDD-MAML outperforms all the other models, with a precision@1 of 90.02 %. We achieve state-of-the-art performance in the diagnosis of all diseases, and the prediction performance for few-shot diseases is greatly improved. For the two groups with the fewest examples of diseases, FSDD-MAML achieves relative increases in precision@1 of 29.13 % and 21.63 % compared with the original knowledge graph-based disease diagnosis model. In addition, we analyze the reasoning process of several few-shot disease predictions and provide an explanation for the results.

CONCLUSION

The decision making model based on meta-learning proposed in this paper can support the rapid diagnosis of diseases in general practice and is especially capable of helping general practitioners diagnose few-shot diseases. This study is of profound significance for the exploration and application of meta-learning to few-shot disease assessment in general practice.

What can GPT-4 do for Diagnosing Rare Eye Diseases? A Pilot Study

INTRODUCTION

Generative pretrained transformer-4 (GPT-4) has gained widespread attention from society, and its potential has been extensively evaluated in many areas. However, investigation of GPT-4's use in medicine, especially in the ophthalmology field, is still limited. This study aims to evaluate GPT-4's capability to identify rare ophthalmic diseases in three simulated scenarios for different end-users, including patients, family physicians, and junior ophthalmologists.

METHODS

We selected ten treatable rare ophthalmic disease cases from the publicly available EyeRounds service. We gradually increased the amount of information fed into GPT-4 to simulate the scenarios of patient, family physician, and junior ophthalmologist using GPT-4. GPT-4's responses were evaluated from two aspects: suitability (appropriate or inappropriate) and accuracy (right or wrong) by senior ophthalmologists (> 10 years' experiences).

RESULTS

Among the 30 responses, 83.3% were considered "appropriate" by senior ophthalmologists. In the scenarios of simulated patient, family physician, and junior ophthalmologist, seven (70%), ten (100%), and eight (80%) responses were graded as "appropriate" by senior ophthalmologists. However, compared to the ground truth, GPT-4 could only output several possible diseases generally without "right" responses in the simulated patient scenarios. In contrast, in the simulated family physician scenario, 50% of GPT-4's responses were "right," and in the simulated junior ophthalmologist scenario, the model achieved a higher "right" rate of 90%.

CONCLUSION

To our knowledge, this is the first proof-of-concept study that evaluates GPT-4's capacity to identify rare eye diseases in simulated scenarios involving patients, family physicians, and junior ophthalmologists. The results indicate that GPT-4 has the potential to serve as a consultation assisting tool for patients and family physicians to receive referral suggestions and an assisting tool for junior ophthalmologists to diagnose rare eye diseases. However, it is important to approach GPT-4 with caution and acknowledge the need for verification and careful referrals in clinical settings.

Deep learning in optical coherence tomography: Where are the gaps?

Optical coherence tomography (OCT) is a non-invasive optical imaging modality, which provides rapid, high-resolution and cross-sectional morphology of macular area and optic nerve head for diagnosis and managing of different eye diseases. However, interpreting OCT images requires experts in both OCT images and eye diseases since many factors such as artefacts and concomitant diseases can affect the accuracy of quantitative measurements made by post-processing algorithms. Currently, there is a growing interest in applying deep learning (DL) methods to analyse OCT images automatically. This review summarises the trends in DL-based OCT image analysis in ophthalmology, discusses the current gaps, and provides potential research directions. DL in OCT analysis shows promising performance in several tasks: (1) layers and features segmentation and quantification; (2) disease classification; (3) disease progression and prognosis; and (4) referral triage level prediction. Different studies and trends in the development of DL-based OCT image analysis are described and the following challenges are identified and described: (1) public OCT data are scarce and scattered; (2) models show performance discrepancies in real-world settings; (3) models lack of transparency; (4) there is a lack of societal acceptance and regulatory standards; and (5) OCT is still not widely available in underprivileged areas. More work is needed to tackle the challenges and gaps, before DL is further applied in OCT image analysis for clinical use.

Multi-Fundus Diseases Classification Using Retinal Optical Coherence Tomography Images with Swin Transformer V2

Fundus diseases cause damage to any part of the retina. Untreated fundus diseases can lead to severe vision loss and even blindness. Analyzing optical coherence tomography (OCT) images using deep learning methods can provide early screening and diagnosis of fundus diseases. In this paper, a deep learning model based on Swin Transformer V2 was proposed to diagnose fundus diseases rapidly and accurately. In this method, calculating self-attention within local windows was used to reduce computational complexity and improve its classification efficiency. Meanwhile, the PolyLoss function was introduced to further improve the model's accuracy, and heat maps were generated to visualize the predictions of the model. Two independent public datasets, OCT 2017 and OCT-C8, were applied to train the model and evaluate its performance, respectively. The results showed that the proposed model achieved an average accuracy of 99.9% on OCT 2017 and 99.5% on OCT-C8, performing well in the automatic classification of multi-fundus diseases using retinal OCT images.

Addressing inter-device variations in optical coherence tomography angiography: will image-to-image translation systems help?

BACKGROUND

Optical coherence tomography angiography (OCTA) is an innovative technology providing visual and quantitative data on retinal microvasculature in a non-invasive manner.

MAIN BODY

Due to variations in the technical specifications of different OCTA devices, there are significant inter-device differences in OCTA data, which can limit their comparability and generalizability. These variations can also result in a domain shift problem that may interfere with applicability of machine learning models on data obtained from different OCTA machines. One possible approach to address this issue may be unsupervised deep image-to-image translation leveraging systems such as Cycle-Consistent Generative Adversarial Networks (Cycle-GANs) and Denoising Diffusion Probabilistic Models (DDPMs). Through training on unpaired images from different device domains, Cycle-GANs and DDPMs may enable cross-domain translation of images. They have been successfully applied in various medical imaging tasks, including segmentation, denoising, and cross-modality image-to-image translation. In this commentary, we briefly describe how Cycle-GANs and DDPMs operate, and review the recent experiments with these models on medical and ocular imaging data. We then discuss the benefits of applying such techniques for inter-device translation of OCTA data and the potential challenges ahead.

CONCLUSION

Retinal imaging technologies and deep learning-based domain adaptation techniques are rapidly evolving. We suggest exploring the potential of image-to-image translation methods in improving the comparability of OCTA data from different centers or devices. This may facilitate more efficient analysis of heterogeneous data and broader applicability of machine learning models trained on limited datasets in this field.

Learning from small data: Classifying sex from retinal images via deep learning

Deep learning (DL) techniques have seen tremendous interest in medical imaging, particularly in the use of convolutional neural networks (CNNs) for the development of automated diagnostic tools. The facility of its non-invasive acquisition makes retinal fundus imaging particularly amenable to such automated approaches. Recent work in the analysis of fundus images using CNNs relies on access to massive datasets for training and validation, composed of hundreds of thousands of images. However, data residency and data privacy restrictions stymie the applicability of this approach in medical settings where patient confidentiality is a mandate. Here, we showcase results for the performance of DL on small datasets to classify patient sex from fundus images-a trait thought not to be present or quantifiable in fundus images until recently. Specifically, we fine-tune a Resnet-152 model whose last layer has been modified to a fully-connected layer for binary classification. We carried out several experiments to assess performance in the small dataset context using one private (DOVS) and one public (ODIR) data source. Our models, developed using approximately 2500 fundus images, achieved test AUC scores of up to 0.72 (95% CI: [0.67, 0.77]). This corresponds to a mere 25% decrease in performance despite a nearly 1000-fold decrease in the dataset size compared to prior results in the literature. Our results show that binary classification, even with a hard task such as sex categorization from retinal fundus images, is possible with very small datasets. Our domain adaptation results show that models trained with one distribution of images may generalize well to an independent external source, as in the case of models trained on DOVS and tested on ODIR. Our results also show that eliminating poor quality images may hamper training of the CNN due to reducing the already small dataset size even further. Nevertheless, using high quality images may be an important factor as evidenced by superior generalizability of results in the domain adaptation experiments. Finally, our work shows that ensembling is an important tool in maximizing performance of deep CNNs in the context of small development datasets.

A deep learning model using hyperspectral image for EUS-FNA cytology diagnosis in pancreatic ductal adenocarcinoma

BACKGROUND AND AIMS

Endoscopic ultrasonography-guided fine-needle aspiration/biopsy (EUS-FNA/B) is considered to be a first-line procedure for the pathological diagnosis of pancreatic cancer owing to its high accuracy and low complication rate. The number of new cases of pancreatic ductal adenocarcinoma (PDAC) is increasing, and its accurate pathological diagnosis poses a challenge for cytopathologists. Our aim was to develop a hyperspectral imaging (HSI)-based convolution neural network (CNN) algorithm to aid in the diagnosis of pancreatic EUS-FNA cytology specimens.

METHODS

HSI images were captured of pancreatic EUS-FNA cytological specimens from benign pancreatic tissues (n = 33) and PDAC (n = 39) prepared using a liquid-based cytology method. A CNN was established to test the diagnostic performance, and Attribution Guided Factorization Visualization (AGF-Visualization) was used to visualize the regions of important classification features identified by the model.

RESULTS

A total of 1913 HSI images were obtained. Our ResNet18-SimSiam model achieved an accuracy of 0.9204, sensitivity of 0.9310 and specificity of 0.9123 (area under the curve of 0.9625) when trained on HSI images for the differentiation of PDAC cytological specimens from benign pancreatic cells. AGF-Visualization confirmed that the diagnoses were based on the features of tumor cell nuclei.

CONCLUSIONS

An HSI-based model was developed to diagnose cytological PDAC specimens obtained using EUS-guided sampling. Under the supervision of experienced cytopathologists, we performed multi-staged consecutive in-depth learning of the model. Its superior diagnostic performance could be of value for cytologists when diagnosing PDAC.

Artificial Intelligence-Assisted Processing of Anterior Segment OCT Images in the Diagnosis of Vitreoretinal Lymphoma

Anterior segment optical coherence tomography (AS-OCT) allows the explore not only the anterior chamber but also the front part of the vitreous cavity. Our cross-sectional single-centre study investigated whether AS-OCT can distinguish between vitreous involvement due to vitreoretinal lymphoma (VRL) and vitritis in uveitis. We studied AS-OCT images from 28 patients (11 with biopsy-proven VRL and 17 with differential diagnosis uveitis) using publicly available radiomics software written in MATLAB. Patients were divided into two balanced groups: training and testing. Overall, 3260/3705 (88%) AS-OCT images met our defined quality criteria, making them eligible for analysis. We studied five different sets of grey-level samplings (16, 32, 64, 128, and 256 levels), finding that 128 grey levels performed the best. We selected the five most effective radiomic features ranked by the ability to predict the class (VRL or uveitis). We built a classification model using the xgboost python function; through our model, 87% of eyes were correctly diagnosed as VRL or uveitis, regardless of exam technique or lens status. Areas under the receiver operating characteristic curves (AUC) in the 128 grey-level model were 0.95 [CI 0.94, 0.96] and 0.84 for training and testing datasets, respectively. This preliminary retrospective study highlights how AS-OCT can support ophthalmologists when there is clinical suspicion of VRL.

Using Artificial Intelligence to Analyse the Retinal Vascular Network: The Future of Cardiovascular Risk Assessment Based on Oculomics? A Narrative Review

The healthcare burden of cardiovascular diseases remains a major issue worldwide. Understanding the underlying mechanisms and improving identification of people with a higher risk profile of systemic vascular disease through noninvasive examinations is crucial. In ophthalmology, retinal vascular network imaging is simple and noninvasive and can provide in vivo information of the microstructure and vascular health. For more than 10 years, different research teams have been working on developing software to enable automatic analysis of the retinal vascular network from different imaging techniques (retinal fundus photographs, OCT angiography, adaptive optics, etc.) and to provide a description of the geometric characteristics of its arterial and venous components. Thus, the structure of retinal vessels could be considered a witness of the systemic vascular status. A new approach called "oculomics" using retinal image datasets and artificial intelligence algorithms recently increased the interest in retinal microvascular biomarkers. Despite the large volume of associated research, the role of retinal biomarkers in the screening, monitoring, or prediction of systemic vascular disease remains uncertain. A PubMed search was conducted until August 2022 and yielded relevant peer-reviewed articles based on a set of inclusion criteria. This literature review is intended to summarize the state of the art in oculomics and cardiovascular disease research.

Classification of endoscopic image and video frames using distance metric-based learning with interpolated latent features

Conventional Endoscopy (CE) and Wireless Capsule Endoscopy (WCE) are well known tools for diagnosing gastrointestinal (GI) tract related disorders. Defining the anatomical location within the GI tract helps clinicians determine appropriate treatment options, which can reduce the need for repetitive endoscopy. Limited research addresses the localization of the anatomical location of WCE and CE images using classification, mainly due to the difficulty in collecting annotated data. In this study, we present a few-shot learning method based on distance metric learning which combines transfer-learning and manifold mixup schemes to localize and classify endoscopic images and video frames. The proposed method allows us to develop a pipeline for endoscopy video sequence localization that can be trained with only a few samples. The use of manifold mixup improves learning by increasing the number of training epochs while reducing overfitting and providing more accurate decision boundaries. A dataset is collected from 10 different anatomical positions of the human GI tract. Two models were trained using only 78 CE and 27 WCE annotated frames to predict the location of 25,700 and 1825 video frames from CE and WCE respectively. We performed subjective evaluation using nine gastroenterologists to validate the need of having such an automated system to localize endoscopic images and video frames. Our method achieved higher accuracy and a higher F1-score when compared with the scores from subjective evaluation. In addition, the results show improved performance with less cross-entropy loss when compared with several existing methods trained on the same datasets. This indicates that the proposed method has the potential to be used in endoscopy image classification.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11042-023-14982-1.

A Deep Learning-Based Framework for Retinal Disease Classification

This study addresses the problem of the automatic detection of disease states of the retina. In order to solve the abovementioned problem, this study develops an artificially intelligent model. The model is based on a customized 19-layer deep convolutional neural network called VGG-19 architecture. The model (VGG-19 architecture) is empowered by transfer learning. The model is designed so that it can learn from a large set of images taken with optical coherence tomography (OCT) and classify them into four conditions of the retina: (1) choroidal neovascularization, (2) drusen, (3) diabetic macular edema, and (4) normal form. The training datasets (taken from publicly available sources) consist of 84,568 instances of OCT retinal images. The datasets exhibit all four classes of retinal disease mentioned above. The proposed model achieved a 99.17% classification accuracy with 0.995 specificities and 0.99 sensitivity, making it better than the existing models. In addition, the proper statistical evaluation is done on the predictions using such performance measures as (1) area under the receiver operating characteristic curve, (2) Cohen's kappa parameter, and (3) confusion matrix. Experimental results show that the proposed VGG-19 architecture coupled with transfer learning is an effective technique for automatically detecting the disease state of a retina.

DeepPDT-Net: predicting the outcome of photodynamic therapy for chronic central serous chorioretinopathy using two-stage multimodal transfer learning

Central serous chorioretinopathy (CSC), characterized by serous detachment of the macular retina, can cause permanent vision loss in the chronic course. Chronic CSC is generally treated with photodynamic therapy (PDT), which is costly and quite invasive, and the results are unpredictable. In a retrospective case-control study design, we developed a two-stage deep learning model to predict 1-year outcome of PDT using initial multimodal clinical data. The training dataset included 166 eyes with chronic CSC and an additional learning dataset containing 745 healthy control eyes. A pre-trained ResNet50-based convolutional neural network was first trained with normal fundus photographs (FPs) to detect CSC and then adapted to predict CSC treatability through transfer learning. The domain-specific ResNet50 successfully predicted treatable and refractory CSC (accuracy, 83.9%). Then other multimodal clinical data were integrated with the FP deep features using XGBoost.The final combined model (DeepPDT-Net) outperformed the domain-specific ResNet50 (accuracy, 88.0%). The FP deep features had the greatest impact on DeepPDT-Net performance, followed by central foveal thickness and age. In conclusion, DeepPDT-Net could solve the PDT outcome prediction task challenging even to retinal specialists. This two-stage strategy, adopting transfer learning and concatenating multimodal data, can overcome the clinical prediction obstacles arising from insufficient datasets.

Deep Ensemble Learning for Retinal Image Classification

Purpose

Vision impairment affects 2.2 billion people worldwide, half of which is preventable with early detection and treatment. Currently, automatic screening of ocular pathologies using convolutional neural networks (CNNs) on retinal fundus photographs is limited to a few pathologies. Simultaneous detection of multiple ophthalmic pathologies would increase clinical usability and uptake.

Methods

Two thousand five hundred sixty images were used from the Retinal Fundus Multi-Disease Image Dataset (RFMiD). Models were trained (n = 1920) and validated (n = 640). Five selected CNN architectures were trained to predict the presence of any pathology and categorize the 28 pathologies. All models were trained to minimize asymmetric loss, a modified form of binary cross-entropy. Individual model predictions were averaged to obtain a final ensembled model and assessed for mean area under the receiver-operator characteristic curve (AUROC) for disease screening (healthy versus pathologic image) and classification (AUROC for each class).

Results

The ensemble network achieved a disease screening (healthy versus pathologic) AUROC score of 0.9613. The highest single network score was 0.9586 using the SE-ResNeXt architecture. For individual disease classification, the average AUROC score for each class was 0.9295.

Conclusions

Retinal fundus images analyzed by an ensemble of CNNs trained to minimize asymmetric loss were effective in detection and classification of ocular pathologies than individual models. External validation is needed to translate machine learning models to diverse clinical contexts.

Translational Relevance

This study demonstrates the potential benefit of ensemble-based deep learning methods on improving automatic screening and diagnosis of multiple ocular pathologies from fundoscopy imaging.

A review of generative adversarial network applications in optical coherence tomography image analysis

Optical coherence tomography (OCT) has revolutionized ophthalmic clinical practice and research, as a result of the high-resolution images that the method is able to capture in a fast, non-invasive manner. Although clinicians can interpret OCT images qualitatively, the ability to quantitatively and automatically analyse these images represents a key goal for eye care by providing clinicians with immediate and relevant metrics to inform best clinical practice. The range of applications and methods to analyse OCT images is rich and rapidly expanding. With the advent of deep learning methods, the field has experienced significant progress with state-of-the-art-performance for several OCT image analysis tasks. Generative adversarial networks (GANs) represent a subfield of deep learning that allows for a range of novel applications not possible in most other deep learning methods, with the potential to provide more accurate and robust analyses. In this review, the progress in this field and clinical impact are reviewed and the potential future development of applications of GANs to OCT image processing are discussed.

Deep learning for predicting uncorrected refractive error using posterior segment optical coherence tomography images

BACKGROUND/OBJECTIVES

This study aimed to evaluate a deep learning model for estimating uncorrected refractive error using posterior segment optical coherence tomography (OCT) images.

METHODS

In this retrospective study, we assigned healthy subjects to development (N = 688 eyes of 344 subjects) and test (N = 248 eyes of 124 subjects) datasets (prospective validation design). We developed and validated OCT-based deep learning models to estimate refractive error. A regression model based on a pretrained ResNet50 architecture was trained using horizontal OCT images to predict the spherical equivalent (SE). The performance of the deep learning model for detecting high myopia was also evaluated. A saliency map was generated using the Grad-CAM technique to visualize the characteristic features.

RESULTS

The developed model showed a low mean absolute error for SE prediction (2.66 D) and a significant Pearson correlation coefficient of 0.588 (P < 0.001) in the test dataset validation. To detect high myopia, the model yielded an area under the receiver operating characteristic curve of 0.813 (95% confidence interval [CI], 0.744-0.881) and an accuracy of 71.4% (95% CI, 65.3-76.9%). The inner retinal layers and relatively steepened curvatures were highlighted using a saliency map to detect high myopia.

CONCLUSION

A deep learning algorithm showed that OCT could potentially be used as an imaging modality to estimate refractive error. This method will facilitate the evaluation of refractive error to prevent clinicians from overlooking the risks associated with refractive error during OCT assessment.

Event recognition method based on dual-augmentation for a Φ-OTDR system with a few training samples

Thanks to the development of machine learning and deep learning, data-driven pattern recognition based on neural network is a trend for Φ-OTDR system intrusion event recognition. The data-driven pattern recognition needs a large number of samples for training. However, in some scenarios, intrusion signals are difficult to collect, resulting in the lack of training samples. At the same time, labeling a large number of samples is also a very time-consuming work. This paper presents a few-shot learning classification method based on time series transfer and cycle generative adversarial network (CycleGAN) data augmentation for Φ-OTDR system. By expanding the rare samples based on time series transfer and CycleGAN, the number of samples in the dataset can finally meet the requirement of network training. The experimental result shows that even when the training set has two minor classes with only two samples, the average accuracy of the validation set with 5 classification tasks can still reach 90.84%, and the classification accuracy of minor classes can reach 79.28% with the proposed method.

A deep learning approach for detection of shallow anterior chamber depth based on the hidden features of fundus photographs

BACKGROUND AND OBJECTIVES

Patients with angle-closure glaucoma (ACG) are asymptomatic until they experience a painful attack. Shallow anterior chamber depth (ACD) is considered a significant risk factor for ACG. We propose a deep learning approach to detect shallow ACD using fundus photographs and to identify the hidden features of shallow ACD.

METHODS

This retrospective study assigned healthy subjects to the training (n = 1188 eyes) and test (n = 594) datasets (prospective validation design). We used a deep learning approach to estimate ACD and build a classification model to identify eyes with a shallow ACD. The proposed method, including subtraction of the input and output images of CycleGAN and a thresholding algorithm, was adopted to visualize the characteristic features of fundus photographs with a shallow ACD.

RESULTS

The deep learning model integrating fundus photographs and clinical variables achieved areas under the receiver operating characteristic curve of 0.978 (95% confidence interval [CI], 0.963-0.988) for an ACD ≤ 2.60 mm and 0.895 (95% CI, 0.868-0.919) for an ACD ≤ 2.80 mm, and outperformed the regression model using only clinical variables. However, the difference between shallow and deep ACD classes on fundus photographs was difficult to be detected with the naked eye. We were unable to identify the features of shallow ACD using the Grad-CAM. The CycleGAN-based feature images showed that area around the macula and optic disk significantly contributed to the classification of fundus photographs with a shallow ACD.

CONCLUSIONS

We demonstrated the feasibility of a novel deep learning model to detect a shallow ACD as a screening tool for ACG using fundus photographs. The CycleGAN-based feature map showed the hidden characteristic features of shallow ACD that were previously undetectable by conventional techniques and ophthalmologists. This framework will facilitate the early detection of shallow ACD to prevent overlooking the risks associated with ACG.

Automatic Segmentation of Retinal Fluid and Photoreceptor Layer from Optical Coherence Tomography Images of Diabetic Macular Edema Patients Using Deep Learning and Associations with Visual Acuity

Diabetic macular edema (DME) is a highly common cause of vision loss in patients with diabetes. Optical coherence tomography (OCT) is crucial in classifying DME and tracking the results of DME treatment. The presence of intraretinal cystoid fluid (IRC) and subretinal fluid (SRF) and the disruption of the ellipsoid zone (EZ), which is part of the photoreceptor layer, are three crucial factors affecting the best corrected visual acuity (BCVA). However, the manual segmentation of retinal fluid and the EZ from retinal OCT images is laborious and time-consuming. Current methods focus only on the segmentation of retinal features, lacking a correlation with visual acuity. Therefore, we proposed a modified U-net, a deep learning algorithm, to segment these features from OCT images of patients with DME. We also correlated these features with visual acuity. The IRC, SRF, and EZ of the OCT retinal images were manually labeled and checked by doctors. We trained the modified U-net model on these labeled images. Our model achieved Sørensen-Dice coefficients of 0.80 and 0.89 for IRC and SRF, respectively. The area under the receiver operating characteristic curve (ROC) for EZ disruption was 0.88. Linear regression indicated that EZ disruption was the factor most strongly correlated with BCVA. This finding agrees with that of previous studies on OCT images. Thus, we demonstrate that our segmentation network can be feasibly applied to OCT image segmentation and assist physicians in assessing the severity of the disease.

Development and Clinical Validation of Semi-Supervised Generative Adversarial Networks for Detection of Retinal Disorders in Optical Coherence Tomography Images Using Small Dataset

PURPOSE

To develop and test semi-supervised generative adversarial networks (GANs) that detect retinal disorders on optical coherence tomography (OCT) images using a small-labeled dataset.

METHODS

From a public database, we randomly chose a small supervised dataset with 400 OCT images (100 choroidal neovascularization, 100 diabetic macular edema, 100 drusen, and 100 normal) and assigned all other OCT images to unsupervised dataset (107,912 images without labeling). We adopted a semi-supervised GAN and a supervised deep learning (DL) model for automatically detecting retinal disorders from OCT images. The performance of the 2 models was compared in 3 testing datasets with different OCT devices. The evaluation metrics included accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curves.

RESULTS

The local validation dataset included 1000 images with 250 from each category. The independent clinical dataset included 366 OCT images using Cirrus OCT Shanghai Shibei Hospital and 511 OCT images using RTVue OCT from Xinhua Hospital respectively. The semi-supervised GANs classifier achieved better accuracy than supervised DL model (0.91 vs 0.86 for local cell validation dataset, 0.91 vs 0.86 in the Shanghai Shibei Hospital testing dataset, and 0.93 vs 0.92 in Xinhua Hospital testing dataset). For detecting urgent referrals (choroidal neo-vascularization and diabetic macular edema) from nonurgent referrals (drusen and normal) on OCT images, the semi-supervised GANs classifier also achieved better area under the receiver operating characteristic curves than supervised DL model (0.99 vs 0.97, 0.97 vs 0.96, and 0.99 vs 0.99, respectively).

CONCLUSIONS

A semi-supervised GAN can achieve better performance than that of a supervised DL model when the labeled dataset is limited. The current study offers utility to various research and clinical studies using DL with relatively small datasets. Semi-supervised GANs can detect retinal disorders from OCT images using relatively small dataset.

Novel technical and privacy-preserving technology for artificial intelligence in ophthalmology

PURPOSE OF REVIEW

The application of artificial intelligence (AI) in medicine and ophthalmology has experienced exponential breakthroughs in recent years in diagnosis, prognosis, and aiding clinical decision-making. The use of digital data has also heralded the need for privacy-preserving technology to protect patient confidentiality and to guard against threats such as adversarial attacks. Hence, this review aims to outline novel AI-based systems for ophthalmology use, privacy-preserving measures, potential challenges, and future directions of each.

RECENT FINDINGS

Several key AI algorithms used to improve disease detection and outcomes include: Data-driven, imagedriven, natural language processing (NLP)-driven, genomics-driven, and multimodality algorithms. However, deep learning systems are susceptible to adversarial attacks, and use of data for training models is associated with privacy concerns. Several data protection methods address these concerns in the form of blockchain technology, federated learning, and generative adversarial networks.

SUMMARY

AI-applications have vast potential to meet many eyecare needs, consequently reducing burden on scarce healthcare resources. A pertinent challenge would be to maintain data privacy and confidentiality while supporting AI endeavors, where data protection methods would need to rapidly evolve with AI technology needs. Ultimately, for AI to succeed in medicine and ophthalmology, a balance would need to be found between innovation and privacy.

Diagnosis of Retinal Diseases Based on Bayesian Optimization Deep Learning Network Using Optical Coherence Tomography Images

Retinal abnormalities have emerged as a serious public health concern in recent years and can manifest gradually and without warning. These diseases can affect any part of the retina, causing vision impairment and indeed blindness in extreme cases. This necessitates the development of automated approaches to detect retinal diseases more precisely and, preferably, earlier. In this paper, we examine transfer learning of pretrained convolutional neural network (CNN) and then transfer it to detect retinal problems from Optical Coherence Tomography (OCT) images. In this study, pretrained CNN models, namely, VGG16, DenseNet201, InceptionV3, and Xception, are used to classify seven different retinal diseases from a dataset of images with and without retinal diseases. In addition, to choose optimum values for hyperparameters, Bayesian optimization is applied, and image augmentation is used to increase the generalization capabilities of the developed models. This research also provides a comparison of the proposed models as well as an analysis of them. The accuracy achieved using DenseNet201 on the Retinal OCT Image dataset is more than 99% and offers a good level of accuracy in classifying retinal diseases compared to other approaches, which only detect a small number of retinal diseases.

Evaluating the Diagnostic Accuracy of a Novel Bayesian Decision-Making Algorithm for Vision Loss

The current diagnostic aids for acute vision loss are static flowcharts that do not provide dynamic, stepwise workups. We tested the diagnostic accuracy of a novel dynamic Bayesian algorithm for acute vision loss. Seventy-nine “participants” with acute vision loss in Windsor, Canada were assessed by an emergency medicine or primary care provider who completed a questionnaire about ocular symptoms/findings (without requiring fundoscopy). An ophthalmologist then attributed an independent “gold-standard diagnosis”. The algorithm employed questionnaire data to produce a differential diagnosis. The referrer diagnostic accuracy was 30.4%, while the algorithm’s accuracy was 70.9%, increasing to 86.1% with the algorithm’s top two diagnoses included and 88.6% with the top three included. In urgent cases of vision loss (n = 54), the referrer diagnostic accuracy was 38.9%, while the algorithm’s top diagnosis was correct in 72.2% of cases, increasing to 85.2% (top two included) and 87.0% (top three included). The algorithm’s sensitivity for urgent cases using the top diagnosis was 94.4% (95% CI: 85–99%), with a specificity of 76.0% (95% CI: 55–91%). This novel algorithm adjusts its workup at each step using clinical symptoms. In doing so, it successfully improves diagnostic accuracy for vision loss using clinical data collected by non-ophthalmologists.

Clinical explainable differential diagnosis of polypoidal choroidal vasculopathy and age-related macular degeneration using deep learning

BACKGROUND

This study aims to achieve an automatic differential diagnosis between two types of retinal pathologies with similar pathological features - Polypoidal choroidal vasculopathy (PCV) and wet age-related macular degeneration (AMD) from volumetric optical coherence tomography (OCT) images, and identify clinically-relevant pathological features, using an explainable deep-learning-based framework.

METHODS

This is a retrospective study with data from a cross-sectional cohort. The OCT volume of 73 eyes from 59 patients was included in this study. Disease differentiation was achieved through single-B-scan-based classification followed by a volumetric probability prediction aggregation step. We compared different labeling strategies with and without identifying pathological B-scans within each OCT volume. Clinical interpretability was achieved through normalized aggregation of B-scan-based saliency maps followed by maximum-intensity-projection onto the en face plane. We derived the PCV score from the proposed differential diagnosis framework with different labeling strategies. The en face projection of saliency map was validated with the pathologies identified in Indocyanine green angiography (ICGA).

RESULTS

Model trained with both labeling strategies achieved similar level differentiation power (>90%), with good correspondence between pathological features detected from the projected en face saliency map and ICGA.

CONCLUSIONS

This study demonstrated the potential clinical application of non-invasive differential diagnosis using AI-driven OCT-based analysis, with minimal requirement of labeling efforts, along with clinical explainability achieved through automatically detected disease-related pathologies.

Deep Learning Model Based on 3D Optical Coherence Tomography Images for the Automated Detection of Pathologic Myopia

Pathologic myopia causes vision impairment and blindness, and therefore, necessitates a prompt diagnosis. However, there is no standardized definition of pathologic myopia, and its interpretation by 3D optical coherence tomography images is subjective, requiring considerable time and money. Therefore, there is a need for a diagnostic tool that can automatically and quickly diagnose pathologic myopia in patients. This study aimed to develop an algorithm that uses 3D optical coherence tomography volumetric images (C-scan) to automatically diagnose patients with pathologic myopia. The study was conducted using 367 eyes of patients who underwent optical coherence tomography tests at the Ophthalmology Department of Incheon St. Mary’s Hospital and Seoul St. Mary’s Hospital from January 2012 to May 2020. To automatically diagnose pathologic myopia, a deep learning model was developed using 3D optical coherence tomography images. The model was developed using transfer learning based on four pre-trained convolutional neural networks (ResNet18, ResNext50, EfficientNetB0, EfficientNetB4). Grad-CAM was used to visualize features affecting the detection of pathologic myopia. The performance of each model was evaluated and compared based on accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC). The model based on EfficientNetB4 showed the best performance (95% accuracy, 93% sensitivity, 96% specificity, and 98% AUROC) in identifying pathologic myopia.

Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey

BACKGROUND

Recent advances in deep learning techniques have led to improved diagnostic abilities in ophthalmology. A generative adversarial network (GAN), which consists of two competing types of deep neural networks, including a generator and a discriminator, has demonstrated remarkable performance in image synthesis and image-to-image translation. The adoption of GAN for medical imaging is increasing for image generation and translation, but it is not familiar to researchers in the field of ophthalmology. In this work, we present a literature review on the application of GAN in ophthalmology image domains to discuss important contributions and to identify potential future research directions.

METHODS

We performed a survey on studies using GAN published before June 2021 only, and we introduced various applications of GAN in ophthalmology image domains. The search identified 48 peer-reviewed papers in the final review. The type of GAN used in the analysis, task, imaging domain, and the outcome were collected to verify the usefulness of the GAN.

RESULTS

In ophthalmology image domains, GAN can perform segmentation, data augmentation, denoising, domain transfer, super-resolution, post-intervention prediction, and feature extraction. GAN techniques have established an extension of datasets and modalities in ophthalmology. GAN has several limitations, such as mode collapse, spatial deformities, unintended changes, and the generation of high-frequency noises and artifacts of checkerboard patterns.

CONCLUSIONS

The use of GAN has benefited the various tasks in ophthalmology image domains. Based on our observations, the adoption of GAN in ophthalmology is still in a very early stage of clinical validation compared with deep learning classification techniques because several problems need to be overcome for practical use. However, the proper selection of the GAN technique and statistical modeling of ocular imaging will greatly improve the performance of each image analysis. Finally, this survey would enable researchers to access the appropriate GAN technique to maximize the potential of ophthalmology datasets for deep learning research.

Simple Code Implementation for Deep Learning-Based Segmentation to Evaluate Central Serous Chorioretinopathy in Fundus Photography

Purpose

Central serous chorioretinopathy (CSC) is a retinal disease that frequently shows resolution and recurrence with serous detachment of the neurosensory retina. Here, we present a deep learning analysis of subretinal fluid (SRF) lesion segmentation in fundus photographs to evaluate CSC.

Methods

We collected 194 fundus photographs of SRF lesions from the patients with CSC. Three graders manually annotated of the entire SRF area in the retinal images. The dataset was randomly separated into training (90%) and validation (10%) datasets. We used the U-Net segmentation model based on conditional generative adversarial networks (pix2pix) to detect the SRF lesions. The algorithms were trained and validated using Google Colaboratory. Researchers did not need prior knowledge of coding skills or computing resources to implement this code.

Results

The validation results showed that the Jaccard index and Dice coefficient scores were 0.619 and 0.763, respectively. In most cases, the segmentation results overlapped with most of the reference areas in the annotated images. However, cases with exceptional SRFs were not accurate in terms of prediction. Using Colaboratory, the proposed segmentation task ran easily in a web-based environment without setup or personal computing resources.

Conclusions

The results suggest that the deep learning model based on U-Net from the pix2pix algorithm is suitable for the automatic segmentation of SRF lesions to evaluate CSC.

Translational Relevance

Our code implementation has the potential to facilitate ophthalmology research; in particular, deep learning–based segmentation can assist in the development of pathological lesion detection solutions.

Artificial intelligence promotes the diagnosis and screening of diabetic retinopathy

Deep learning evolves into a new form of machine learning technology that is classified under artificial intelligence (AI), which has substantial potential for large-scale healthcare screening and may allow the determination of the most appropriate specific treatment for individual patients. Recent developments in diagnostic technologies facilitated studies on retinal conditions and ocular disease in metabolism and endocrinology. Globally, diabetic retinopathy (DR) is regarded as a major cause of vision loss. Deep learning systems are effective and accurate in the detection of DR from digital fundus photographs or optical coherence tomography. Thus, using AI techniques, systems with high accuracy and efficiency can be developed for diagnosing and screening DR at an early stage and without the resources that are only accessible in special clinics. Deep learning enables early diagnosis with high specificity and sensitivity, which makes decisions based on minimally handcrafted features paving the way for personalized DR progression real-time monitoring and in-time ophthalmic or endocrine therapies. This review will discuss cutting-edge AI algorithms, the automated detecting systems of DR stage grading and feature segmentation, the prediction of DR outcomes and therapeutics, and the ophthalmic indications of other systemic diseases revealed by AI.

Synthetic OCT data in challenging conditions: three-dimensional OCT and presence of abnormalities

Nowadays, retinal optical coherence tomography (OCT) plays an important role in ophthalmology and automatic analysis of the OCT is of real importance: image denoising facilitates a better diagnosis and image segmentation and classification are undeniably critical in treatment evaluation. Synthetic OCT was recently considered to provide a benchmark for quantitative comparison of automatic algorithms and to be utilized in the training stage of novel solutions based on deep learning. Due to complicated data structure in retinal OCTs, a limited number of delineated OCT datasets are already available in presence of abnormalities; furthermore, the intrinsic three-dimensional (3D) structure of OCT is ignored in many public 2D datasets. We propose a new synthetic method, applicable to 3D data and feasible in presence of abnormalities like diabetic macular edema (DME). In this method, a limited number of OCT data is used during the training step and the Active Shape Model is used to produce synthetic OCTs plus delineation of retinal boundaries and location of abnormalities. Statistical comparison of thickness maps showed that synthetic dataset can be used as a statistically acceptable representative of the original dataset (p > 0.05). Visual inspection of the synthesized vessels was also promising. Regarding the texture features of the synthesized datasets, Q-Q plots were used, and even in cases that the points have slightly digressed from the straight line, the p-values of the Kolmogorov–Smirnov test rejected the null hypothesis and showed the same distribution in texture features of the real and the synthetic data. The proposed algorithm provides a unique benchmark for comparison of OCT enhancement methods and a tailored augmentation method to overcome the limited number of OCTs in deep learning algorithms.

Adopting low-shot deep learning for the detection of conjunctival melanoma using ocular surface images

BACKGROUND AND OBJECTIVE

The purpose of the present study was to investigate low-shot deep learning models applied to conjunctival melanoma detection using a small dataset with ocular surface images.

METHODS

A dataset was composed of anonymized images of four classes; conjunctival melanoma (136), nevus or melanosis (93), pterygium (75), and normal conjunctiva (94). Before training involving conventional deep learning models, two generative adversarial networks (GANs) were constructed to augment the training dataset for low-shot learning. The collected data were randomly divided into training (70%), validation (10%), and test (20%) datasets. Moreover, 3D melanoma phantoms were designed to build an external validation set using a smartphone. The GoogleNet, InceptionV3, NASNet, ResNet50, and MobileNetV2 architectures were trained through transfer learning and validated using the test and external validation datasets.

RESULTS

The deep learning model demonstrated a significant improvement in the classification accuracy of conjunctival lesions using synthetic images generated by the GAN models. MobileNetV2 with GAN-based augmentation displayed the highest accuracy of 87.5% in the four-class classification and 97.2% in the binary classification for the detection of conjunctival melanoma. It showed an accuracy of 94.0% using 3D melanoma phantom images captured using a smartphone camera.

CONCLUSIONS

The present study described a low-shot deep learning model that can detect conjunctival melanomas using ocular surface images. To the best of our knowledge, this study is the first to develop a deep learning model to detect conjunctival melanoma using a digital imaging device such as smartphone camera.

Deep Learning-Based Optical Coherence Tomography and Optical Coherence Tomography Angiography Image Analysis: An Updated Summary

ABSTRACT

Deep learning (DL) is a subset of artificial intelligence based on deep neural networks. It has made remarkable breakthroughs in medical imaging, particularly for image classification and pattern recognition. In ophthalmology, there are rising interests in applying DL methods to analyze optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images. Studies showed that OCT and OCTA image evaluation by DL algorithms achieved good performance for disease detection, prognosis prediction, and image quality control, suggesting that the incorporation of DL technology could potentially enhance the accuracy of disease evaluation and the efficiency of clinical workflow. However, substantial issues, such as small training sample size, data preprocessing standardization, model robustness, results explanation, and performance cross-validation, are yet to be tackled before deploying these DL models in real-time clinics. This review summarized recent studies on DL-based image analysis models for OCT and OCTA images and discussed the potential challenges of clinical deployment and future research directions.