Lesion Detection and Grading of Diabetic Retinopathy via Two-Stages Deep Convolutional Neural Networks

Diabetic retinopathy screening through artificial intelligence algorithms: A systematic review

Diabetic retinopathy (DR) poses a significant challenge in diabetes management, with its progression often asymptomatic until advanced stages. This underscores the urgent need for cost-effective and reliable screening methods. Consequently, the integration of artificial intelligence (AI) tools presents a promising avenue to address this need effectively. We provide an overview of the current state of the art results and techniques in DR screening using AI, while also identifying gaps in research for future exploration. By synthesizing existing database and pinpointing areas requiring further investigation, this paper seeks to guide the direction of future research in the field of automatic diabetic retinopathy screening. There has been a continuous rise in the number of articles detailing deep learning (DL) methods designed for the automatic screening of diabetic retinopathy especially by the year 2021. Researchers utilized various databases, with a primary focus on the IDRiD dataset. This dataset consists of color fundus images captured at an ophthalmological clinic situated in India. It comprises 516 images that depict various stages of DR and diabetic macular edema. Each of the chosen papers concentrates on various DR signs. Nevertheless, a significant portion primarily focused on detecting exudates, which remains insufficient to assess the overall presence of this disease. Various AI methods have been employed to identify DR signs. Among the chosen papers, 4.7 % utilized detection methods, 46.5 % employed classification techniques, 41.9 % relied on segmentation, and 7 % opted for a combination of classification and segmentation. Metrics calculated from 80 % of the articles employing preprocessing techniques demonstrated the significant benefits of this approach in enhancing results quality. In addition, multiple DL techniques, starting by classification, detection then segmentation. Researchers used mostly YOLO for detection, ViT for classification, and U-Net for segmentation. Another perspective on the evolving landscape of AI models for diabetic retinopathy screening lies in the increasing adoption of Convolutional Neural Networks for classification tasks and U-Net architectures for segmentation purposes; however, there is a growing realization within the research community that these techniques, while powerful individually, can be even more effective when integrated. This integration holds promise for not only diagnosing DR, but also accurately classifying its different stages, thereby enabling more tailored treatment strategies. Despite this potential, the development of AI models for DR screening is fraught with challenges. Chief among these is the difficulty in obtaining the high-quality, labeled data necessary for training models to perform effectively. This scarcity of data poses significant barriers to achieving robust performance and can hinder progress in developing accurate screening systems. Moreover, managing the complexity of these models, particularly deep neural networks, presents its own set of challenges. Additionally, interpreting the outputs of these models and ensuring their reliability in real-world clinical settings remain ongoing concerns. Furthermore, the iterative process of training and adapting these models to specific datasets can be time-consuming and resource-intensive. These challenges underscore the multifaceted nature of developing effective AI models for DR screening. Addressing these obstacles requires concerted efforts from researchers, clinicians, and technologists to develop new approaches and overcome existing limitations. By doing so, a full potential of AI may transform DR screening and improve patient outcomes.

Role of Machine Learning and Artificial Intelligence in the Diagnosis and Treatment of Refractive Errors for Enhanced Eye Care: A Systematic Review

A significant contributor to blindness and visual impairment globally is uncorrected refractive error. To plan effective interventions, eye care professionals must promptly identify people at a high risk of acquiring myopia, and monitor disease progress. Artificial intelligence (AI) and machine learning (ML) have enormous potential to improve diagnosis and treatment. This systematic review explores the current state of ML and AI applications in the diagnoses and treatment of refractory errors in optometry. A systematic review and meta-analysis of studies evaluating the diagnostic performance of AI-based tools in PubMed was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. To find relevant studies on the use of ML or AI in the diagnosis or treatment of refractive errors in optometry, a thorough search was conducted in various electronic databases such as PubMed, Google Scholar, and Web of Science. The search was limited to studies published between January 2015 and December 2022. The search terms used were "refractive errors," "myopia," "optometry," "machine learning," "ophthalmology," and "artificial intelligence." A total of nine studies met the inclusion criteria and were included in the final analysis. ML is increasingly being utilized for automating clinical data processing as AI technology progresses, making the formerly labor-intensive work possible. AI models that primarily use a neural network demonstrated exceptional efficiency and performance in the analysis of vast medical data, rivaling board-certified, healthcare professionals. Several studies showed that ML models could support diagnosis and clinical decision-making. Moreover, an ML algorithm predicted future refraction values in patients with myopia. AI and ML models have great potential to improve the diagnosis and treatment of refractive errors in optometry.

An advanced deep learning method to detect and classify diabetic retinopathy based on color fundus images

BACKGROUND

In this article, we present a computerized system for the analysis and assessment of diabetic retinopathy (DR) based on retinal fundus photographs. DR is a chronic ophthalmic disease and a major reason for blindness in people with diabetes. Consistent examination and prompt diagnosis are the vital approaches to control DR.

METHODS

With the aim of enhancing the reliability of DR diagnosis, we utilized the deep learning model called You Only Look Once V3 (YOLO V3) to recognize and classify DR from retinal images. The DR was classified into five major stages: normal, mild, moderate, severe, and proliferative. We evaluated the performance of the YOLO V3 algorithm based on color fundus images.

RESULTS

We have achieved high precision and sensitivity on the train and test data for the DR classification and mean average precision (mAP) is calculated on DR lesion detection.

CONCLUSIONS

The results indicate that the suggested model distinguishes all phases of DR and performs better than existing models in terms of accuracy and implementation time.

Multi-scale multi-attention network for diabetic retinopathy grading

Objective.Diabetic retinopathy (DR) grading plays an important role in clinical diagnosis. However, automatic grading of DR is challenging due to the presence of intra-class variation and small lesions. On the one hand, deep features learned by convolutional neural networks often lose valid information about these small lesions. On the other hand, the great variability of lesion features, including differences in type and quantity, can exhibit considerable divergence even among fundus images of the same grade. To address these issues, we propose a novel multi-scale multi-attention network (MMNet).Approach.Firstly, to focus on different lesion features of fundus images, we propose a lesion attention module, which aims to encode multiple different lesion attention feature maps by combining channel attention and spatial attention, thus extracting global feature information and preserving diverse lesion features. Secondly, we propose a multi-scale feature fusion module to learn more feature information for small lesion regions, which combines complementary relationships between different convolutional layers to capture more detailed feature information. Furthermore, we introduce a Cross-layer Consistency Constraint Loss to overcome semantic differences between multi-scale features.Main results.The proposed MMNet obtains a high accuracy of 86.4% and a high kappa score of 88.4% for multi-class DR grading tasks on the EyePACS dataset, while 98.6% AUC, 95.3% accuracy, 92.7% recall, 95.0% precision, and 93.3% F1-score for referral and non-referral classification on the Messidor-1 dataset. Extensive experiments on two challenging benchmarks demonstrate that our MMNet achieves significant improvements and outperforms other state-of-the-art DR grading methods.Significance.MMNet has improved the diagnostic efficiency and accuracy of diabetes retinopathy and promoted the application of computer-aided medical diagnosis in DR screening.

A Cross-Domain Weakly Supervised Diabetic Retinopathy Lesion Identification Method Based on Multiple Instance Learning and Domain Adaptation

Accurate identification of lesions and their use across different medical institutions are the foundation and key to the clinical application of automatic diabetic retinopathy (DR) detection. Existing detection or segmentation methods can achieve acceptable results in DR lesion identification, but they strongly rely on a large number of fine-grained annotations that are not easily accessible and suffer severe performance degradation in the cross-domain application. In this paper, we propose a cross-domain weakly supervised DR lesion identification method using only easily accessible coarse-grained lesion attribute labels. We first propose the novel lesion-patch multiple instance learning method (LpMIL), which leverages the lesion attribute label for patch-level supervision to complete weakly supervised lesion identification. Then, we design a semantic constraint adaptation method (LpSCA) that improves the lesion identification performance of our model in different domains with semantic constraint loss. Finally, we perform secondary annotation on the open-source dataset EyePACS, to obtain the largest fine-grained annotated dataset EyePACS-pixel, and validate the performance of our model on it. Extensive experimental results on the public dataset FGADR and our EyePACS-pixel demonstrate that compared with the existing detection and segmentation methods, the proposed method can identify lesions accurately and comprehensively, and obtain competitive results using only coarse-grained annotations.

Hybrid Methods for Fundus Image Analysis for Diagnosis of Diabetic Retinopathy Development Stages Based on Fusion Features

Diabetic retinopathy (DR) is a complication of diabetes that damages the delicate blood vessels of the retina and leads to blindness. Ophthalmologists rely on diagnosing the retina by imaging the fundus. The process takes a long time and needs skilled doctors to diagnose and determine the stage of DR. Therefore, automatic techniques using artificial intelligence play an important role in analyzing fundus images for the detection of the stages of DR development. However, diagnosis using artificial intelligence techniques is a difficult task and passes through many stages, and the extraction of representative features is important in reaching satisfactory results. Convolutional Neural Network (CNN) models play an important and distinct role in extracting features with high accuracy. In this study, fundus images were used for the detection of the developmental stages of DR by two proposed methods, each with two systems. The first proposed method uses GoogLeNet with SVM and ResNet-18 with SVM. The second method uses Feed-Forward Neural Networks (FFNN) based on the hybrid features extracted by first using GoogLeNet, Fuzzy color histogram (FCH), Gray Level Co-occurrence Matrix (GLCM), and Local Binary Pattern (LBP); followed by ResNet-18, FCH, GLCM and LBP. All the proposed methods obtained superior results. The FFNN network with hybrid features of ResNet-18, FCH, GLCM, and LBP obtained 99.7% accuracy, 99.6% precision, 99.6% sensitivity, 100% specificity, and 99.86% AUC.

Identifying the Key Components in ResNet-50 for Diabetic Retinopathy Grading from Fundus Images: A Systematic Investigation

Although deep learning-based diabetic retinopathy (DR) classification methods typically benefit from well-designed architectures of convolutional neural networks, the training setting also has a non-negligible impact on prediction performance. The training setting includes various interdependent components, such as an objective function, a data sampling strategy, and a data augmentation approach. To identify the key components in a standard deep learning framework (ResNet-50) for DR grading, we systematically analyze the impact of several major components. Extensive experiments are conducted on a publicly available dataset EyePACS. We demonstrate that (1) the DR grading framework is sensitive to input resolution, objective function, and composition of data augmentation; (2) using mean square error as the loss function can effectively improve the performance with respect to a task-specific evaluation metric, namely the quadratically weighted Kappa; (3) utilizing eye pairs boosts the performance of DR grading and; (4) using data resampling to address the problem of imbalanced data distribution in EyePACS hurts the performance. Based on these observations and an optimal combination of the investigated components, our framework, without any specialized network design, achieves a state-of-the-art result (0.8631 for Kappa) on the EyePACS test set (a total of 42,670 fundus images) with only image-level labels. We also examine the proposed training practices on other fundus datasets and other network architectures to evaluate their generalizability. Our codes and pre-trained model are available online.

Retinal Disease Detection Using Deep Learning Techniques: A Comprehensive Review

Millions of people are affected by retinal abnormalities worldwide. Early detection and treatment of these abnormalities could arrest further progression, saving multitudes from avoidable blindness. Manual disease detection is time-consuming, tedious and lacks repeatability. There have been efforts to automate ocular disease detection, riding on the successes of the application of Deep Convolutional Neural Networks (DCNNs) and vision transformers (ViTs) for Computer-Aided Diagnosis (CAD). These models have performed well, however, there remain challenges owing to the complex nature of retinal lesions. This work reviews the most common retinal pathologies, provides an overview of prevalent imaging modalities and presents a critical evaluation of current deep-learning research for the detection and grading of glaucoma, diabetic retinopathy, Age-Related Macular Degeneration and multiple retinal diseases. The work concluded that CAD, through deep learning, will increasingly be vital as an assistive technology. As future work, there is a need to explore the potential impact of using ensemble CNN architectures in multiclass, multilabel tasks. Efforts should also be expended on the improvement of model explainability to win the trust of clinicians and patients.

CoT-XNet: contextual transformer with Xception network for diabetic retinopathy grading

Objective.Diabetic retinopathy (DR) grading is primarily performed by assessing fundus images. Many types of lesions, such as microaneurysms, hemorrhages, and soft exudates, are available simultaneously in a single image. However, their sizes may be small, making it difficult to differentiate adjacent DR grades even using deep convolutional neural networks (CNNs). Recently, a vision transformer has shown comparable or even superior performance to CNNs, and it also learns different visual representations from CNNs. Inspired by this finding, we propose a two-path contextual transformer with Xception network (CoT-XNet) to improve the accuracy of DR grading.Approach.The representations learned by CoT through one path and those by the Xception network through another path are concatenated before the fully connected layer. Meanwhile, the dedicated pre-processing, data resampling, and test time augmentation strategies are implemented. The performance of CoT-XNet is evaluated in the publicly available datasets of DDR, APTOS2019, and EyePACS, which include over 50 000 images. Ablation experiments and comprehensive comparisons with various state-of-the-art (SOTA) models have also been performed.Main results.Our proposed CoT-XNet shows better performance than available SOTA models, and the accuracy and Kappa are 83.10% and 0.8496, 84.18% and 0.9000 and 84.10% and 0.7684 respectively, in the three datasets (listed above). Class activation maps of CoT and Xception networks are different and complementary in most images.Significance.By concatenating the different visual representations learned by CoT and Xception networks, CoT-XNet can accurately grade DR from fundus images and present good generalizability. CoT-XNet will promote the application of artificial intelligence-based systems in the DR screening of large-scale populations.

A New Deep Learning Algorithm with Activation Mapping for Diabetic Retinopathy: Backtesting after 10 Years of Tele-Ophthalmology

We report the development of a deep learning algorithm (AI) to detect signs of diabetic retinopathy (DR) from fundus images. For this, we use a ResNet-50 neural network with a double resolution, the addition of Squeeze–Excitation blocks, pre-trained in ImageNet, and trained for 50 epochs using the Adam optimizer. The AI-based algorithm not only classifies an image as pathological or not but also detects and highlights those signs that allow DR to be identified. For development, we have used a database of about half a million images classified in a real clinical environment by family doctors (FDs), ophthalmologists, or both. The AI was able to detect more than 95% of cases worse than mild DR and had 70% fewer misclassifications of healthy cases than FDs. In addition, the AI was able to detect DR signs in 1258 patients before they were detected by FDs, representing 7.9% of the total number of DR patients detected by the FDs. These results suggest that AI is at least comparable to the evaluation of FDs. We suggest that it may be useful to use signaling tools such as an aid to diagnosis rather than an AI as a stand-alone tool.

Weakly-Supervised Learning With Complementary Heatmap for Retinal Disease Detection

There are many types of retinal disease, and accurately detecting these diseases is crucial for proper diagnosis. Convolutional neural networks (CNNs) typically perform well on detection tasks, and the attention module of CNNs can generate heatmaps as visual explanations of the model. However, the generated heatmap can only detect the most discriminative part, which is problematic because many object regions may exist in the region beside the heatmap in an area known as a complementary heatmap. In this study, we developed a method specifically designed multi-retinal diseases detection from fundus images with the complementary heatmap. The proposed CAM-based method is designed for 2D color images of the retina, rather than MRI images or other forms of data. Moreover, unlike other visual images for disease detection, fundus images of multiple retinal diseases have features such as distinguishable lesion region boundaries, overlapped lesion regions between diseases, and specific pathological structures (e.g. scattered blood spots) that lead to mis-classifications. Based on these considerations, we designed two new loss functions, attention-explore loss and attention-refine loss, to generate accurate heatmaps. We select both "bad" and "good" heatmaps based on the prediction score of ground truth and train them with the two loss functions. When the detection accuracy increases, the classification performance of the model is also improved. Experiments on a dataset consisting of five diseases showed that our approach improved both the detection accuracy and the classification accuracy, and the improved heatmaps were closer to the lesion regions than those of current state-of-the-art methods.

CCNET: Cross Coordinate Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading

With the rapid development of the world economy and increasing improvement of people's living standards, the number of diabetic patients has been growing quickly. Meanwhile, the complications of diabetes especially retinopathy have been affecting their daily life seriously. The only way to prevent it from getting worse and even leading to blindness is to make corresponding diagnosis as early as possible. However, it's extremely impossible for professionals to diagnose all the patients through their fundus images. It couldn't be better to solve the problem by automatic systems, so we present a novel network to learn the features of diabetic retinopathy (DR) and its complication diabetic macular edema (DME) and the relationship between them, focus on some vital areas in the pictures and eventually obtain the grades of the two diseases at the same time. Experimental results further prove the effectiveness of our proposed module comparing to the only joint grading network before.

Image based early detection of diabetic retinopathy: A systematic review on Artificial Intelligence (AI) based recent trends and approaches

Diabetic Retinopathy (DR) is a disease that damages the retina of the human eye due to diabetic complications, resulting in a loss of vision. Blindness may be avoided If the DR disease is detected at an early stage. Unfortunately, DR is irreversible process, however, early detection and treatment of DR can significantly reduce the risk of vision loss. The manual diagnosis done by ophthalmologists on DR retina fundus images is time consuming, and error prone process. Nowadays, machine learning and deep learning have become one of the most effective approaches, which have even surpassed the human performance as well as performance of traditional image processing-based algorithms and other computer aided diagnosis systems in the analysis and classification of medical images. This paper addressed and evaluated the various recent state-of-the-art methodologies that have been used for detection and classification of Diabetic Retinopathy disease using machine learning and deep learning approaches in the past decade. Furthermore, this study also provides the authors observation and performance evaluation of available research using several parameters, such as accuracy, disease status, and sensitivity. Finally, we conclude with limitations, remedies, and future directions in DR detection. In addition, various challenging issues that need further study are also discussed.

Comparative analysis of detection and classification of diabetic retinopathy by using transfer learning of CNN based models

 Machine learning and deep learning methods have become exponentially more accurate. These methods are now as precise as experts of respective fields, so it is being used in almost all areas of life. Nowadays, people have more faith in machines than men, so, in this vein, deep learning models with the concept of transfer learning of CNN are used to detect and classify diabetic retinopathy and its different stages. The backbone of various CNN-based models such as InceptionResNetV2, InceptionV3, Xception, MobileNetV2, VGG19, and DenceNet201 are used to classify this vision loss disease. In these base models, transfer learning has been applied by adding some layers like batch normalization, dropout, and dense layers to make the model more effective and accurate for the given problem. The training of the resulting models has been done for the Kaggle retinopathy 2019 dataset with about 3662 fundus fluorescein angiography colored images. Performance of all six trained models have been measured on the test dataset in terms of precision, recall, F1 score, macro average, weighted average, confusion matrix, and accuracy. A confusion matrix is based on maximum class probability prediction that is the incapability of the confusion matrix. The ROC-AUC of different classes and the models are analyzed. ROC-AUC is based on the actual probability of different categories. The results obtained from this study show that InceptionResNetV2 is proven the best model for diabetic retinopathy detection and classification, among other models considered here. It can work accurately in case of less training data. Thus, this model may detect and classify diabetic retinopathy automatically and accurately at an early stage. So it would be beneficial for humans to reduce the effects of diabetes. As a result of this, the impact of diabetes on vision loss can be minimized, and that would be a blessing in the medical field.

CARNet: Cascade attentive RefineNet for multi-lesion segmentation of diabetic retinopathy images

Diabetic retinopathy is the leading cause of blindness in working population. Lesion segmentation from fundus images helps ophthalmologists accurately diagnose and grade of diabetic retinopathy. However, the task of lesion segmentation is full of challenges due to the complex structure, the various sizes and the interclass similarity with other fundus tissues. To address the issue, this paper proposes a cascade attentive RefineNet (CARNet) for automatic and accurate multi-lesion segmentation of diabetic retinopathy. It can make full use of the fine local details and coarse global information from the fundus image. CARNet is composed of global image encoder, local image encoder and attention refinement decoder. We take the whole image and the patch image as the dual input, and feed them to ResNet50 and ResNet101, respectively, for downsampling to extract lesion features. The high-level refinement decoder uses dual attention mechanism to integrate the same-level features in the two encoders with the output of the low-level attention refinement module for multiscale information fusion, which focus the model on the lesion area to generate accurate predictions. We evaluated the segmentation performance of the proposed CARNet on the IDRiD, E-ophtha and DDR data sets. Extensive comparison experiments and ablation studies on various data sets demonstrate the proposed framework outperforms the state-of-the-art approaches and has better accuracy and robustness. It not only overcomes the interference of similar tissues and noises to achieve accurate multi-lesion segmentation, but also preserves the contour details and shape features of small lesions without overloading GPU memory usage.

Segmentation in Diabetic Retinopathy using Deeply-Supervised Multiscalar Attention

Diabetic Retinopathy (DR) is a progressive chronic eye disease that leads to irreversible blindness. Detection of DR at an early stage of the disease is crucial and requires proper detection of minute DR pathologies. A novel Deeply-Supervised Multiscale Attention U-Net (Mult-Attn-U-Net) is proposed for segmentation of different DR pathologies viz. Microaneurysms (MA), Hemorrhages (HE), Soft and Hard Exudates (SE and EX). A publicly available dataset (IDRiD) is used to evaluate the performance. Comparative study with four state-of-the-art models establishes its superiority. The best segmentation accuracy obtained by the model for MA, HE, SE are 0.65, 0.70, 0.72, respectively.

Vision Transformer-based recognition of diabetic retinopathy grade

BACKGROUND

In the domain of natural language processing, Transformers are recognized as state-of-the-art models, which opposing to typical convolutional neural networks (CNNs) do not rely on convolution layers. Instead, Transformers employ multi-head attention mechanisms as the main building block to capture long-range contextual relations between image pixels. Recently, CNNs dominated the deep learning solutions for diabetic retinopathy grade recognition. However, spurred by the advantages of Transformers, we propose a Transformer-based method that is appropriate for recognizing the grade of diabetic retinopathy.

PURPOSE

The purposes of this work are to demonstrate that (i) the pure attention mechanism is suitable for diabetic retinopathy grade recognition and (ii) Transformers can replace traditional CNNs for diabetic retinopathy grade recognition.

METHODS

This paper proposes a Vision Transformer-based method to recognize the grade of diabetic retinopathy. Fundus images are subdivided into non-overlapping patches, which are then converted into sequences by flattening, and undergo a linear and positional embedding process to preserve positional information. Then, the generated sequence is input into several multi-head attention layers to generate the final representation. The first token sequence is input to a softmax classification layer to produce the recognition output in the classification stage.

RESULTS

The dataset for training and testing employs fundus images of different resolutions, subdivided into patches. We challenge our method against current CNNs and extreme learning machines and achieve an appealing performance. Specifically, the suggested deep learning architecture attains an accuracy of 91.4%, specificity = 0.977 (95% confidence interval (CI) (0.951-1)), precision = 0.928 (95% CI (0.852-1)), sensitivity = 0.926 (95% CI (0.863-0.989)), quadratic weighted kappa score = 0.935, and area under curve (AUC) = 0.986.

CONCLUSION

Our comparative experiments against current methods conclude that our model is competitive and highlight that an attention mechanism based on a Vision Transformer model is promising for the diabetic retinopathy grade recognition task.

Explainable Diabetic Retinopathy Detection and Retinal Image Generation

Though deep learning has shown successful performance in classifying the label and severity stage of certain diseases, most of them give few explanations on how to make predictions. Inspired by Koch's Postulates, the foundation in evidence-based medicine (EBM) to identify the pathogen, we propose to exploit the interpretability of deep learning application in medical diagnosis. By isolating neuron activation patterns from a diabetic retinopathy (DR) detector and visualizing them, we can determine the symptoms that the DR detector identifies as evidence to make prediction. To be specific, we first define novel pathological descriptors using activated neurons of the DR detector to encode both spatial and appearance information of lesions. Then, to visualize the symptom encoded in the descriptor, we propose Patho-GAN, a new network to synthesize medically plausible retinal images. By manipulating these descriptors, we could even arbitrarily control the position, quantity, and categories of generated lesions. We also show that our synthesized images carry the symptoms directly related to diabetic retinopathy diagnosis. Our generated images are both qualitatively and quantitatively superior to the ones by previous methods. Besides, compared to existing methods that take hours to generate an image, our second level speed endows the potential to be an effective solution for data augmentation.

Diabetic retinopathy classification based on multipath CNN and machine learning classifiers

Eye care professionals generally use fundoscopy to confirm the occurrence of Diabetic Retinopathy (DR) in patients. Early DR detection and accurate DR grading are critical for the care and management of this disease. This work proposes an automated DR grading method in which features can be extracted from the fundus images and categorized based on severity using deep learning and Machine Learning (ML) algorithms. A Multipath Convolutional Neural Network (M-CNN) is used for global and local feature extraction from images. Then, a machine learning classifier is used to categorize the input according to the severity. The proposed model is evaluated across different publicly available databases (IDRiD, Kaggle (for DR detection), and MESSIDOR) and different ML classifiers (Support Vector Machine (SVM), Random Forest, and J48). The metrics selected for model evaluation are the False Positive Rate (FPR), Specificity, Precision, Recall, F1-score, K-score, and Accuracy. The experiments show that the best response is produced by the M-CNN network with the J48 classifier. The classifiers are evaluated across the pre-trained network features and existing DR grading methods. The average accuracy obtained for the proposed work is 99.62% for DR grading. The experiments and evaluation results show that the proposed method works well for accurate DR grading and early disease detection.

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

People with diabetes are at risk of developing an eye disease called diabetic retinopathy (DR). This disease occurs when high blood glucose levels cause damage to blood vessels in the retina. Computer-aided DR diagnosis has become a promising tool for the early detection and severity grading of DR, due to the great success of deep learning. However, most current DR diagnosis systems do not achieve satisfactory performance or interpretability for ophthalmologists, due to the lack of training data with consistent and fine-grained annotations. To address this problem, we construct a large fine-grained annotated DR dataset containing 2,842 images (FGADR). Specifically, this dataset has 1,842 images with pixel-level DR-related lesion annotations, and 1,000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency. The proposed dataset will enable extensive studies on DR diagnosis. Further, we establish three benchmark tasks for evaluation: 1. DR lesion segmentation; 2. DR grading by joint classification and segmentation; 3. Transfer learning for ocular multi-disease identification. Moreover, a novel inductive transfer learning method is introduced for the third task. Extensive experiments using different state-of-the-art methods are conducted on our FGADR dataset, which can serve as baselines for future research. Our dataset will be released in https://csyizhou.github.io/FGADR/.

CABNet: Category Attention Block for Imbalanced Diabetic Retinopathy Grading

Diabetic Retinopathy (DR) grading is challenging due to the presence of intra-class variations, small lesions and imbalanced data distributions. The key for solving fine-grained DR grading is to find more discriminative features corresponding to subtle visual differences, such as microaneurysms, hemorrhages and soft exudates. However, small lesions are quite difficult to identify using traditional convolutional neural networks (CNNs), and an imbalanced DR data distribution will cause the model to pay too much attention to DR grades with more samples, greatly affecting the final grading performance. In this article, we focus on developing an attention module to address these issues. Specifically, for imbalanced DR data distributions, we propose a novel Category Attention Block (CAB), which explores more discriminative region-wise features for each DR grade and treats each category equally. In order to capture more detailed small lesion information, we also propose the Global Attention Block (GAB), which can exploit detailed and class-agnostic global attention feature maps for fundus images. By aggregating the attention blocks with a backbone network, the CABNet is constructed for DR grading. The attention blocks can be applied to a wide range of backbone networks and trained efficiently in an end-to-end manner. Comprehensive experiments are conducted on three publicly available datasets, showing that CABNet produces significant performance improvements for existing state-of-the-art deep architectures with few additional parameters and achieves the state-of-the-art results for DR grading. Code and models will be available at https://github.com/he2016012996/CABnet.

Machine Learning Techniques for Ophthalmic Data Processing: A Review

Machine learning and especially deep learning techniques are dominating medical image and data analysis. This article reviews machine learning approaches proposed for diagnosing ophthalmic diseases during the last four years. Three diseases are addressed in this survey, namely diabetic retinopathy, age-related macular degeneration, and glaucoma. The review covers over 60 publications and 25 public datasets and challenges related to the detection, grading, and lesion segmentation of the three considered diseases. Each section provides a summary of the public datasets and challenges related to each pathology and the current methods that have been applied to the problem. Furthermore, the recent machine learning approaches used for retinal vessels segmentation, and methods of retinal layers and fluid segmentation are reviewed. Two main imaging modalities are considered in this survey, namely color fundus imaging, and optical coherence tomography. Machine learning approaches that use eye measurements and visual field data for glaucoma detection are also included in the survey. Finally, the authors provide their views, expectations and the limitations of the future of these techniques in the clinical practice.

Leveraging Multimodal Deep Learning Architecture with Retina Lesion Information to Detect Diabetic Retinopathy

Purpose

To improve disease severity classification from fundus images using a hybrid architecture with symptom awareness for diabetic retinopathy (DR).

Methods

We used 26,699 fundus images of 17,834 diabetic patients from three Taiwanese hospitals collected in 2007 to 2018 for DR severity classification. Thirty-seven ophthalmologists verified the images using lesion annotation and severity classification as the ground truth. Two deep learning fusion architectures were proposed: late fusion, which combines lesion and severity classification models in parallel using a postprocessing procedure, and two-stage early fusion, which combines lesion detection and classification models sequentially and mimics the decision-making process of ophthalmologists. Messidor-2 was used with 1748 images to evaluate and benchmark the performance of the architecture. The primary evaluation metrics were classification accuracy, weighted κ statistic, and area under the receiver operating characteristic curve (AUC).

Results

For hospital data, a hybrid architecture achieved a good detection rate, with accuracy and weighted κ of 84.29% and 84.01%, respectively, for five-class DR grading. It also classified the images of early stage DR more accurately than conventional algorithms. The Messidor-2 model achieved an AUC of 97.09% in referral DR detection compared to AUC of 85% to 99% for state-of-the-art algorithms that learned from a larger database.

Conclusions

Our hybrid architectures strengthened and extracted characteristics from DR images, while improving the performance of DR grading, thereby increasing the robustness and confidence of the architectures for general use.

Translational Relevance

The proposed fusion architectures can enable faster and more accurate diagnosis of various DR pathologies than that obtained in current manual clinical practice.

CANet: Cross-Disease Attention Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading

Diabetic retinopathy (DR) and diabetic macular edema (DME) are the leading causes of permanent blindness in the working-age population. Automatic grading of DR and DME helps ophthalmologists design tailored treatments to patients, thus is of vital importance in the clinical practice. However, prior works either grade DR or DME, and ignore the correlation between DR and its complication, i.e., DME. Moreover, the location information, e.g., macula and soft hard exhaust annotations, are widely used as a prior for grading. Such annotations are costly to obtain, hence it is desirable to develop automatic grading methods with only image-level supervision. In this article, we present a novel cross-disease attention network (CANet) to jointly grade DR and DME by exploring the internal relationship between the diseases with only image-level supervision. Our key contributions include the disease-specific attention module to selectively learn useful features for individual diseases, and the disease-dependent attention module to further capture the internal relationship between the two diseases. We integrate these two attention modules in a deep network to produce disease-specific and disease-dependent features, and to maximize the overall performance jointly for grading DR and DME. We evaluate our network on two public benchmark datasets, i.e., ISBI 2018 IDRiD challenge dataset and Messidor dataset. Our method achieves the best result on the ISBI 2018 IDRiD challenge dataset and outperforms other methods on the Messidor dataset. Our code is publicly available at https://github.com/xmengli999/CANet.

Different fundus imaging modalities and technical factors in AI screening for diabetic retinopathy: a review

BACKGROUND

Effective screening is a desirable method for the early detection and successful treatment for diabetic retinopathy, and fundus photography is currently the dominant medium for retinal imaging due to its convenience and accessibility. Manual screening using fundus photographs has however involved considerable costs for patients, clinicians and national health systems, which has limited its application particularly in less-developed countries. The advent of artificial intelligence, and in particular deep learning techniques, has however raised the possibility of widespread automated screening.

MAIN TEXT

In this review, we first briefly survey major published advances in retinal analysis using artificial intelligence. We take care to separately describe standard multiple-field fundus photography, and the newer modalities of ultra-wide field photography and smartphone-based photography. Finally, we consider several machine learning concepts that have been particularly relevant to the domain and illustrate their usage with extant works.

CONCLUSIONS

In the ophthalmology field, it was demonstrated that deep learning tools for diabetic retinopathy show clinically acceptable diagnostic performance when using colour retinal fundus images. Artificial intelligence models are among the most promising solutions to tackle the burden of diabetic retinopathy management in a comprehensive manner. However, future research is crucial to assess the potential clinical deployment, evaluate the cost-effectiveness of different DL systems in clinical practice and improve clinical acceptance.

Recent Development on Detection Methods for the Diagnosis of Diabetic Retinopathy

Diabetic retinopathy (DR) is a complication of diabetes that exists throughout the world. DR occurs due to a high ratio of glucose in the blood, which causes alterations in the retinal microvasculature. Without preemptive symptoms of DR, it leads to complete vision loss. However, early screening through computer-assisted diagnosis (CAD) tools and proper treatment have the ability to control the prevalence of DR. Manual inspection of morphological changes in retinal anatomic parts are tedious and challenging tasks. Therefore, many CAD systems were developed in the past to assist ophthalmologists for observing inter- and intra-variations. In this paper, a recent review of state-of-the-art CAD systems for diagnosis of DR is presented. We describe all those CAD systems that have been developed by various computational intelligence and image processing techniques. The limitations and future trends of current CAD systems are also described in detail to help researchers. Moreover, potential CAD systems are also compared in terms of statistical parameters to quantitatively evaluate them. The comparison results indicate that there is still a need for accurate development of CAD systems to assist in the clinical diagnosis of diabetic retinopathy.

Weakly Supervised Lesion Detection From Fundus Images

Early diagnosis and continuous monitoring of patients suffering from eye diseases have been major concerns in the computer-aided detection techniques. Detecting one or several specific types of retinal lesions has made a significant breakthrough in computer-aided screen in the past few decades. However, due to the variety of retinal lesions and complex normal anatomical structures, automatic detection of lesions with unknown and diverse types from a retina remains a challenging task. In this paper, a weakly supervised method, requiring only a series of normal and abnormal retinal images without need to specifically annotate their locations and types, is proposed for this task. Specifically, a fundus image is understood as a superposition of background, blood vessels, and background noise (lesions included for abnormal images). Background is formulated as a low-rank structure after a series of simple preprocessing steps, including spatial alignment, color normalization, and blood vessels removal. Background noise is regarded as stochastic variable and modeled through Gaussian for normal images and mixture of Gaussian for abnormal images, respectively. The proposed method encodes both the background knowledge of fundus images and the background noise into one unique model, and corporately optimizes the model using normal and abnormal images, which fully depict the low-rank subspace of the background and distinguish the lesions from the background noise in abnormal fundus images. Experimental results demonstrate that the proposed method is of fine arts accuracy and outperforms the previous related methods.