An Interpretable Ensemble Deep Learning Model for Diabetic Retinopathy Disease Classification

A Survey on Explainable Artificial Intelligence (XAI) Techniques for Visualizing Deep Learning Models in Medical Imaging

The combination of medical imaging and deep learning has significantly improved diagnostic and prognostic capabilities in the healthcare domain. Nevertheless, the inherent complexity of deep learning models poses challenges in understanding their decision-making processes. Interpretability and visualization techniques have emerged as crucial tools to unravel the black-box nature of these models, providing insights into their inner workings and enhancing trust in their predictions. This survey paper comprehensively examines various interpretation and visualization techniques applied to deep learning models in medical imaging. The paper reviews methodologies, discusses their applications, and evaluates their effectiveness in enhancing the interpretability, reliability, and clinical relevance of deep learning models in medical image analysis.

Artificial Intelligence Applications in Diabetic Retinopathy: What We Have Now and What to Expect in the Future

Diabetic retinopathy (DR) is a major complication of diabetes mellitus and is a leading cause of vision loss globally. A prompt and accurate diagnosis is crucial for ensuring favorable visual outcomes, highlighting the need for increased access to medical care. The recent remarkable advancements in artificial intelligence (AI) have raised high expectations for its role in disease diagnosis and prognosis prediction across various medical fields. In addition to achieving high precision comparable to that of ophthalmologists, AI-based diagnosis of DR has the potential to improve medical accessibility, especially through telemedicine. In this review paper, we aim to examine the current role of AI in the diagnosis of DR and explore future directions.

Artificial Intelligence (AI) for Early Diagnosis of Retinal Diseases

Artificial intelligence (AI) has emerged as a transformative tool in the field of ophthalmology, revolutionizing disease diagnosis and management. This paper provides a comprehensive overview of AI applications in various retinal diseases, highlighting its potential to enhance screening efficiency, facilitate early diagnosis, and improve patient outcomes. Herein, we elucidate the fundamental concepts of AI, including machine learning (ML) and deep learning (DL), and their application in ophthalmology, underscoring the significance of AI-driven solutions in addressing the complexity and variability of retinal diseases. Furthermore, we delve into the specific applications of AI in retinal diseases such as diabetic retinopathy (DR), age-related macular degeneration (AMD), Macular Neovascularization, retinopathy of prematurity (ROP), retinal vein occlusion (RVO), hypertensive retinopathy (HR), Retinitis Pigmentosa, Stargardt disease, best vitelliform macular dystrophy, and sickle cell retinopathy. We focus on the current landscape of AI technologies, including various AI models, their performance metrics, and clinical implications. Furthermore, we aim to address challenges and pitfalls associated with the integration of AI in clinical practice, including the "black box phenomenon", biases in data representation, and limitations in comprehensive patient assessment. In conclusion, this review emphasizes the collaborative role of AI alongside healthcare professionals, advocating for a synergistic approach to healthcare delivery. It highlights the importance of leveraging AI to augment, rather than replace, human expertise, thereby maximizing its potential to revolutionize healthcare delivery, mitigate healthcare disparities, and improve patient outcomes in the evolving landscape of medicine.

UC-stack: a deep learning computer automatic detection system for diabetic retinopathy classification

Object. The existing diagnostic paradigm for diabetic retinopathy (DR) greatly relies on subjective assessments by medical practitioners utilizing optical imaging, introducing susceptibility to individual interpretation. This work presents a novel system for the early detection and grading of DR, providing an automated alternative to the manual examination.Approach. First, we use advanced image preprocessing techniques, specifically contrast-limited adaptive histogram equalization and Gaussian filtering, with the goal of enhancing image quality and module learning capabilities. Second, a deep learning-based automatic detection system is developed. The system consists of a feature segmentation module, a deep learning feature extraction module, and an ensemble classification module. The feature segmentation module accomplishes vascular segmentation, the deep learning feature extraction module realizes the global feature and local feature extraction of retinopathy images, and the ensemble module performs the diagnosis and classification of DR for the extracted features. Lastly, nine performance evaluation metrics are applied to assess the quality of the model's performance.Main results. Extensive experiments are conducted on four retinal image databases (APTOS 2019, Messidor, DDR, and EyePACS). The proposed method demonstrates promising performance in the binary and multi-classification tasks for DR, evaluated through nine indicators, including AUC and quadratic weighted Kappa score. The system shows the best performance in the comparison of three segmentation methods, two convolutional neural network architecture models, four Swin Transformer structures, and the latest literature methods.Significance. In contrast to existing methods, our system demonstrates superior performance across multiple indicators, enabling accurate screening of DR and providing valuable support to clinicians in the diagnostic process. Our automated approach minimizes the reliance on subjective assessments, contributing to more consistent and reliable DR evaluations.

A Deep Learning Approach for Meibomian Gland Appearance Evaluation

Purpose

To develop and evaluate a deep learning algorithm for Meibomian gland characteristics calculation.

Design

Evaluation of diagnostic technology.

Subjects

A total of 1616 meibography images of both the upper (697) and lower (919) eyelids from a total of 282 individuals.

Methods

Images were collected using the LipiView II device. All the provided data were split into 3 sets: the training, validation, and test sets. Data partitions used proportions of 70/10/20% and included data from 2 optometry settings. Each set was separately partitioned with these proportions, resulting in a balanced distribution of data from both settings. The images were divided based on patient identifiers, such that all images collected for one participant could end up only in one set. The labeled images were used to train a deep learning model, which was subsequently used for Meibomian gland segmentation. The model was then applied to calculate individual Meibomian gland metrics. Interreader agreement and agreement between manual and automated methods for Meibomian gland segmentation were also carried out to assess the accuracy of the automated approach.

Main Outcome Measures

Meibomian gland metrics, including length ratio, area, tortuosity, intensity, and width, were measured. Additionally, the performance of the automated algorithms was evaluated using the aggregated Jaccard index.

Results

The proposed semantic segmentation-based approach achieved average aggregated Jaccard index of mean 0.4718 (95% confidence interval [CI], 0.4680-0.4771) for the 'gland' class and a mean of 0.8470 (95% CI, 0.8432-0.8508) for the 'eyelid' class. The result for object detection-based approach was a mean of 0.4476 (95% CI, 0.4426-0.4533). Both artificial intelligence-based algorithms underestimated area, length ratio, tortuosity, widthmean, widthmedian, width10th, and width90th. Meibomian gland intensity was overestimated by both algorithms compared with the manual approach. The object detection-based algorithm seems to be as reliable as the manual approach only for Meibomian gland width10th calculation.

Conclusions

The proposed approach can successfully segment Meibomian glands; however, to overcome problems with gland overlap and lack of image sharpness, the proposed method requires further development. The study presents another approach to utilizing automated, artificial intelligence-based methods in Meibomian gland health assessment that may assist clinicians in the diagnosis, treatment, and management of Meibomian gland dysfunction.

Financial Disclosures

The authors have no proprietary or commercial interest in any materials discussed in this article.

Attention-based deep learning framework to recognize diabetes disease from cellular retinal images

A medical disorder known as diabetic retinopathy (DR) affects people who suffer from diabetes. Many people are visually impaired due to DR. Primary cause of DR in patients is high blood sugar, and it affects blood vessels available in the retinal cell. The recent advancement in deep learning and computer vision methods, and their automation applications can recognize the presence of DR in retinal cells and vessel images. Authors have proposed an attention-based hybrid model to recognize diabetes in early stage to prevent harmful clauses. Proposed methodology uses DenseNet121 architecture for convolution learning and then, the feature vector will be enhanced with channel and spatial attention model. The proposed architecture also simulates binary and multiclass classification to recognize the infection and the spreading of disease. Binary classification recognizes DR images either positive or negative, while multiclass classification represents an infection on a scale of 0-4. Simulation of the proposed methodology has achieved 98.57% and 99.01% accuracy for multiclass and binary classification, respectively. Simulation of the study also explored the impact of data augmentation to make the proposed model robust and generalized. Attention-based deep learning model has achieved remarkable accuracy to detect diabetic infection from retinal cellular images.

RDS-DR: An Improved Deep Learning Model for Classifying Severity Levels of Diabetic Retinopathy

A well-known eye disorder called diabetic retinopathy (DR) is linked to elevated blood glucose levels. Cotton wool spots, confined veins in the cranial nerve, AV nicking, and hemorrhages in the optic disc are some of its symptoms, which often appear later. Serious side effects of DR might include vision loss, damage to the visual nerves, and obstruction of the retinal arteries. Researchers have devised an automated method utilizing AI and deep learning models to enable the early diagnosis of this illness. This research gathered digital fundus images from renowned Pakistani eye hospitals to generate a new "DR-Insight" dataset and known online sources. A novel methodology named the residual-dense system (RDS-DR) was then devised to assess diabetic retinopathy. To develop this model, we have integrated residual and dense blocks, along with a transition layer, into a deep neural network. The RDS-DR system is trained on the collected dataset of 9860 fundus images. The RDS-DR categorization method demonstrated an impressive accuracy of 97.5% on this dataset. These findings show that the model produces beneficial outcomes and may be used by healthcare practitioners as a diagnostic tool. It is important to emphasize that the system's goal is to augment optometrists' expertise rather than replace it. In terms of accuracy, the RDS-DR technique fared better than the cutting-edge models VGG19, VGG16, Inception V-3, and Xception. This emphasizes how successful the suggested method is for classifying diabetic retinopathy (DR).

Contrastive self-supervised learning for diabetic retinopathy early detection

Diabetic Retinopathy (DR) is the major cause of blindness, which seriously threatens the world's vision health. Limited medical resources make early diagnosis and a large-scale screening of DR difficult. Most of the current automatic diagnostic methods are mostly based on deep learning and large-scale labeled data. However, the insufficiency of manual annotations for medical images still is a great challenge of training deep neural networks. Self-supervised learning methods are proposed to learn general features from dataset without manual annotations. Inspired by this, we proposed a deep learning based DR classification model (SimCLR-DR). In this paper, we first use contrastive self-learning algorithm to pre-train the encoder based on convolution network with unlabeled retinal images, then retrain the encoder with classifier on a small annotated training data to detect referable DR. The experimental results on Kaggle dataset show that this proposed method can overcome the training data insufficiency problem and performs better than transfer learning. SimCLR-DR is a good beginning for other deep learning based medical image detection approaches facing the challenge of insufficient annotated data. Figure presents an overview of the proposed framework, which contains three main steps: (i) Data preprocessing; (ii) Pretext task of SimCLR-DR based on contrastive learning; (iii) Downstream Task of SimCLRDR based on CNN.

MIL-CT: Multiple Instance Learning via a Cross-Scale Transformer for Enhanced Arterial Light Reflex Detection

One of the early manifestations of systemic atherosclerosis, which leads to blood circulation issues, is the enhanced arterial light reflex (EALR). Fundus images are commonly used for regular screening purposes to intervene and assess the severity of systemic atherosclerosis in a timely manner. However, there is a lack of automated methods that can meet the demands of large-scale population screening. Therefore, this study introduces a novel cross-scale transformer-based multi-instance learning method, named MIL-CT, for the detection of early arterial lesions (e.g., EALR) in fundus images. MIL-CT utilizes the cross-scale vision transformer to extract retinal features in a multi-granularity perceptual domain. It incorporates a multi-head cross-scale attention fusion module to enhance global perceptual capability and feature representation. By integrating information from different scales and minimizing information loss, the method significantly improves the performance of the EALR detection task. Furthermore, a multi-instance learning module is implemented to enable the model to better comprehend local details and features in fundus images, facilitating the classification of patch tokens related to retinal lesions. To effectively learn the features associated with retinal lesions, we utilize weights pre-trained on a large fundus image Kaggle dataset. Our validation and comparison experiments conducted on our collected EALR dataset demonstrate the effectiveness of the MIL-CT method in reducing generalization errors while maintaining efficient attention to retinal vascular details. Moreover, the method surpasses existing models in EALR detection, achieving an accuracy, precision, sensitivity, specificity, and F1 score of 97.62%, 97.63%, 97.05%, 96.48%, and 97.62%, respectively. These results exhibit the significant enhancement in diagnostic accuracy of fundus images brought about by the MIL-CT method. Thus, it holds potential for various applications, particularly in the early screening of cardiovascular diseases such as hypertension and atherosclerosis.

Ensemble Deep Learning Derived from Transfer Learning for Classification of COVID-19 Patients on Hybrid Deep-Learning-Based Lung Segmentation: A Data Augmentation and Balancing Framework

BACKGROUND AND MOTIVATION

Lung computed tomography (CT) techniques are high-resolution and are well adopted in the intensive care unit (ICU) for COVID-19 disease control classification. Most artificial intelligence (AI) systems do not undergo generalization and are typically overfitted. Such trained AI systems are not practical for clinical settings and therefore do not give accurate results when executed on unseen data sets. We hypothesize that ensemble deep learning (EDL) is superior to deep transfer learning (TL) in both non-augmented and augmented frameworks.

METHODOLOGY

The system consists of a cascade of quality control, ResNet-UNet-based hybrid deep learning for lung segmentation, and seven models using TL-based classification followed by five types of EDL's. To prove our hypothesis, five different kinds of data combinations (DC) were designed using a combination of two multicenter cohorts-Croatia (80 COVID) and Italy (72 COVID and 30 controls)-leading to 12,000 CT slices. As part of generalization, the system was tested on unseen data and statistically tested for reliability/stability.

RESULTS

Using the K5 (80:20) cross-validation protocol on the balanced and augmented dataset, the five DC datasets improved TL mean accuracy by 3.32%, 6.56%, 12.96%, 47.1%, and 2.78%, respectively. The five EDL systems showed improvements in accuracy of 2.12%, 5.78%, 6.72%, 32.05%, and 2.40%, thus validating our hypothesis. All statistical tests proved positive for reliability and stability.

CONCLUSION

EDL showed superior performance to TL systems for both (a) unbalanced and unaugmented and (b) balanced and augmented datasets for both (i) seen and (ii) unseen paradigms, validating both our hypotheses.

Data Diversity in Convolutional Neural Network Based Ensemble Model for Diabetic Retinopathy

The medical and healthcare domains require automatic diagnosis systems (ADS) for the identification of health problems with technological advancements. Biomedical imaging is one of the techniques used in computer-aided diagnosis systems. Ophthalmologists examine fundus images (FI) to detect and classify stages of diabetic retinopathy (DR). DR is a chronic disease that appears in patients with long-term diabetes. Unattained patients can lead to severe conditions of DR, such as retinal eye detachments. Therefore, early detection and classification of DR are crucial to ward off advanced stages of DR and preserve the vision. Data diversity in an ensemble model refers to the use of multiple models trained on different subsets of data to improve the ensemble's overall performance. In the context of an ensemble model based on a convolutional neural network (CNN) for diabetic retinopathy, this could involve training multiple CNNs on various subsets of retinal images, including images from different patients or those captured using distinct imaging techniques. By combining the predictions of these multiple models, the ensemble model can potentially make more accurate predictions than a single prediction. In this paper, an ensemble model (EM) of three CNN models is proposed for limited and imbalanced DR data using data diversity. Detecting the Class 1 stage of DR is important to control this fatal disease in time. CNN-based EM is incorporated to classify the five classes of DR while giving attention to the early stage, i.e., Class 1. Furthermore, data diversity is created by applying various augmentation and generation techniques with affine transformation. Compared to the single model and other existing work, the proposed EM has achieved better multi-class classification accuracy, precision, sensitivity, and specificity of 91.06%, 91.00%, 95.01%, and 98.38%, respectively.

Retinal Disease Detection Using Deep Learning Techniques: A Comprehensive Review

Millions of people are affected by retinal abnormalities worldwide. Early detection and treatment of these abnormalities could arrest further progression, saving multitudes from avoidable blindness. Manual disease detection is time-consuming, tedious and lacks repeatability. There have been efforts to automate ocular disease detection, riding on the successes of the application of Deep Convolutional Neural Networks (DCNNs) and vision transformers (ViTs) for Computer-Aided Diagnosis (CAD). These models have performed well, however, there remain challenges owing to the complex nature of retinal lesions. This work reviews the most common retinal pathologies, provides an overview of prevalent imaging modalities and presents a critical evaluation of current deep-learning research for the detection and grading of glaucoma, diabetic retinopathy, Age-Related Macular Degeneration and multiple retinal diseases. The work concluded that CAD, through deep learning, will increasingly be vital as an assistive technology. As future work, there is a need to explore the potential impact of using ensemble CNN architectures in multiclass, multilabel tasks. Efforts should also be expended on the improvement of model explainability to win the trust of clinicians and patients.

Automated Diagnosis of Diabetic Retinopathy Using Deep Learning: On the Search of Segmented Retinal Blood Vessel Images for Better Performance

Diabetic retinopathy is one of the most significant retinal diseases that can lead to blindness. As a result, it is critical to receive a prompt diagnosis of the disease. Manual screening can result in misdiagnosis due to human error and limited human capability. In such cases, using a deep learning-based automated diagnosis of the disease could aid in early detection and treatment. In deep learning-based analysis, the original and segmented blood vessels are typically used for diagnosis. However, it is still unclear which approach is superior. In this study, a comparison of two deep learning approaches (Inception v3 and DenseNet-121) was performed on two different datasets of colored images and segmented images. The study’s findings revealed that the accuracy for original images on both Inception v3 and DenseNet-121 equaled 0.8 or higher, whereas the segmented retinal blood vessels under both approaches provided an accuracy of just greater than 0.6, demonstrating that the segmented vessels do not add much utility to the deep learning-based analysis. The study’s findings show that the original-colored images are more significant in diagnosing retinopathy than the extracted retinal blood vessels.

A Survey on Deep-Learning-Based Diabetic Retinopathy Classification

The number of people who suffer from diabetes in the world has been considerably increasing recently. It affects people of all ages. People who have had diabetes for a long time are affected by a condition called Diabetic Retinopathy (DR), which damages the eyes. Automatic detection using new technologies for early detection can help avoid complications such as the loss of vision. Currently, with the development of Artificial Intelligence (AI) techniques, especially Deep Learning (DL), DL-based methods are widely preferred for developing DR detection systems. For this purpose, this study surveyed the existing literature on diabetic retinopathy diagnoses from fundus images using deep learning and provides a brief description of the current DL techniques that are used by researchers in this field. After that, this study lists some of the commonly used datasets. This is followed by a performance comparison of these reviewed methods with respect to some commonly used metrics in computer vision tasks.

CoT-XNet: contextual transformer with Xception network for diabetic retinopathy grading

Objective.Diabetic retinopathy (DR) grading is primarily performed by assessing fundus images. Many types of lesions, such as microaneurysms, hemorrhages, and soft exudates, are available simultaneously in a single image. However, their sizes may be small, making it difficult to differentiate adjacent DR grades even using deep convolutional neural networks (CNNs). Recently, a vision transformer has shown comparable or even superior performance to CNNs, and it also learns different visual representations from CNNs. Inspired by this finding, we propose a two-path contextual transformer with Xception network (CoT-XNet) to improve the accuracy of DR grading.Approach.The representations learned by CoT through one path and those by the Xception network through another path are concatenated before the fully connected layer. Meanwhile, the dedicated pre-processing, data resampling, and test time augmentation strategies are implemented. The performance of CoT-XNet is evaluated in the publicly available datasets of DDR, APTOS2019, and EyePACS, which include over 50 000 images. Ablation experiments and comprehensive comparisons with various state-of-the-art (SOTA) models have also been performed.Main results.Our proposed CoT-XNet shows better performance than available SOTA models, and the accuracy and Kappa are 83.10% and 0.8496, 84.18% and 0.9000 and 84.10% and 0.7684 respectively, in the three datasets (listed above). Class activation maps of CoT and Xception networks are different and complementary in most images.Significance.By concatenating the different visual representations learned by CoT and Xception networks, CoT-XNet can accurately grade DR from fundus images and present good generalizability. CoT-XNet will promote the application of artificial intelligence-based systems in the DR screening of large-scale populations.

Detection of Diabetic Retinopathy using Convolutional Neural Networks for Feature Extraction and Classification (DRFEC)

Diabetic Retinopathy (DR) is caused as a result of Diabetes Mellitus which causes development of various retinal abrasions in the human retina. These lesions cause hindrance in vision and in severe cases, DR can lead to blindness. DR is observed amongst 80% of patients who have been diagnosed from prolonged diabetes for a period of 10-15 years. The manual process of periodic DR diagnosis and detection for necessary treatment, is time consuming and unreliable due to unavailability of resources and expert opinion. Therefore, computerized diagnostic systems which use Deep Learning (DL) Convolutional Neural Network (CNN) architectures, are proposed to learn DR patterns from fundus images and identify the severity of the disease. This paper proposes a comprehensive model using 26 state-of-the-art DL networks to assess and evaluate their performance, and which contribute for deep feature extraction and image classification of DR fundus images. In the proposed model, ResNet50 has shown highest overfitting in comparison to Inception V3, which has shown lowest overfitting when trained using the Kaggle's EyePACS fundus image dataset. EfficientNetB4 is the most optimal, efficient and reliable DL algorithm in detection of DR, followed by InceptionResNetV2, NasNetLarge and DenseNet169. EfficientNetB4 has achieved a training accuracy of 99.37% and the highest validation accuracy of 79.11%. DenseNet201 has achieved the highest training accuracy of 99.58% and a validation accuracy of 76.80% which is less than the top-4 best performing models.

Deep Learning for Diabetic Retinopathy Analysis: A Review, Research Challenges, and Future Directions

Deep learning (DL) enables the creation of computational models comprising multiple processing layers that learn data representations at multiple levels of abstraction. In the recent past, the use of deep learning has been proliferating, yielding promising results in applications across a growing number of fields, most notably in image processing, medical image analysis, data analysis, and bioinformatics. DL algorithms have also had a significant positive impact through yielding improvements in screening, recognition, segmentation, prediction, and classification applications across different domains of healthcare, such as those concerning the abdomen, cardiac, pathology, and retina. Given the extensive body of recent scientific contributions in this discipline, a comprehensive review of deep learning developments in the domain of diabetic retinopathy (DR) analysis, viz., screening, segmentation, prediction, classification, and validation, is presented here. A critical analysis of the relevant reported techniques is carried out, and the associated advantages and limitations highlighted, culminating in the identification of research gaps and future challenges that help to inform the research community to develop more efficient, robust, and accurate DL models for the various challenges in the monitoring and diagnosis of DR.

A survey on the interpretability of deep learning in medical diagnosis

Deep learning has demonstrated remarkable performance in the medical domain, with accuracy that rivals or even exceeds that of human experts. However, it has a significant problem that these models are "black-box" structures, which means they are opaque, non-intuitive, and difficult for people to understand. This creates a barrier to the application of deep learning models in clinical practice due to lack of interpretability, trust, and transparency. To overcome this problem, several studies on interpretability have been proposed. Therefore, in this paper, we comprehensively review the interpretability of deep learning in medical diagnosis based on the current literature, including some common interpretability methods used in the medical domain, various applications with interpretability for disease diagnosis, prevalent evaluation metrics, and several disease datasets. In addition, the challenges of interpretability and future research directions are also discussed here. To the best of our knowledge, this is the first time that various applications of interpretability methods for disease diagnosis have been summarized.

Image based early detection of diabetic retinopathy: A systematic review on Artificial Intelligence (AI) based recent trends and approaches

Diabetic Retinopathy (DR) is a disease that damages the retina of the human eye due to diabetic complications, resulting in a loss of vision. Blindness may be avoided If the DR disease is detected at an early stage. Unfortunately, DR is irreversible process, however, early detection and treatment of DR can significantly reduce the risk of vision loss. The manual diagnosis done by ophthalmologists on DR retina fundus images is time consuming, and error prone process. Nowadays, machine learning and deep learning have become one of the most effective approaches, which have even surpassed the human performance as well as performance of traditional image processing-based algorithms and other computer aided diagnosis systems in the analysis and classification of medical images. This paper addressed and evaluated the various recent state-of-the-art methodologies that have been used for detection and classification of Diabetic Retinopathy disease using machine learning and deep learning approaches in the past decade. Furthermore, this study also provides the authors observation and performance evaluation of available research using several parameters, such as accuracy, disease status, and sensitivity. Finally, we conclude with limitations, remedies, and future directions in DR detection. In addition, various challenging issues that need further study are also discussed.

EDR-Net: Lightweight Deep Neural Network Architecture for Detecting Referable Diabetic Retinopathy

Present architecture of convolution neural network for diabetic retinopathy (DR-Net) is based on normal convolution (NC). It incurs high computational cost as NC uses a multiplicative weight that measures a combined correlation in both cross-channel and spatial dimension of layer's inputs. This might cause the overall DR-Net architecture to be over-parameterised and computationally inefficient. This paper proposes EDR-Net - a new end-to-end, DR-Net architecture with depth-wise separable convolution module. The EDR-Net architecture was trained with DRKaggle-train dataset (35,126 images), and tested on two datasets, i.e. DRKaggle-test (53,576 images) and Messidor-2 (1,748 images). Results showed that the proposed EDR-Net achieved predictive performance comparable with current state-of-the-arts in detecting referable diabetic retinopathy (rDR) from fundus images and outperformed other light weight architectures, with at least two times less computation cost. This makes it more amenable for mobile device based computer-assisted rDR screening applications.

Artificial Intelligence Algorithms in Diabetic Retinopathy Screening

PURPOSE OF REVIEW

In this review, we focus on artificial intelligence (AI) algorithms for diabetic retinopathy (DR) screening and risk stratification and factors to consider when implementing AI algorithms in the clinic.

RECENT FINDINGS

AI algorithms have been adopted, and have received regulatory approval, for automated detection of referable DR with clinically acceptable diagnostic performance. While these metrics are an important first step, performance metrics that go beyond measures of technical accuracy are needed to fully evaluate the impact of AI algorithm on patient outcomes. Recent advances in AI present an exciting opportunity to improve patient care. Using DR as an example, we have reviewed factors to consider in the implementation of AI algorithms in real-world clinical practice. These include real-world evaluation of safety, efficacy, and equity (bias); impact on patient outcomes; ethical, logistical, and regulatory factors.

Attentional Mechanisms and Improved Residual Networks for Diabetic Retinopathy Severity Classification

Diabetic retinopathy is a main cause of blindness in diabetic patients; therefore, detection and treatment of diabetic retinopathy at an early stage has an important effect on delaying and avoiding vision loss. In this paper, we propose a feasible solution for diabetic retinopathy classification using ResNet as the backbone network. By modifying the structure of the residual blocks and improving the downsampling level, we can increase the feature information of the hidden layer feature maps. In addition, attention mechanism is utilized to enhance the feature extraction effect. The experimental results show that the proposed model can effectively detect and classify diabetic retinopathy and achieve better results than the original model.

Untangling Computer-Aided Diagnostic System for Screening Diabetic Retinopathy Based on Deep Learning Techniques

Diabetic Retinopathy (DR) is a predominant cause of visual impairment and loss. Approximately 285 million worldwide population is affected with diabetes, and one-third of these patients have symptoms of DR. Specifically, it tends to affect the patients with 20 years or more with diabetes, but it can be reduced by early detection and proper treatment. Diagnosis of DR by using manual methods is a time-consuming and expensive task which involves trained ophthalmologists to observe and evaluate DR using digital fundus images of the retina. This study aims to systematically find and analyze high-quality research work for the diagnosis of DR using deep learning approaches. This research comprehends the DR grading, staging protocols and also presents the DR taxonomy. Furthermore, identifies, compares, and investigates the deep learning-based algorithms, techniques, and, methods for classifying DR stages. Various publicly available dataset used for deep learning have also been analyzed and dispensed for descriptive and empirical understanding for real-time DR applications. Our in-depth study shows that in the last few years there has been an increasing inclination towards deep learning approaches. 35% of the studies have used Convolutional Neural Networks (CNNs), 26% implemented the Ensemble CNN (ECNN) and, 13% Deep Neural Networks (DNN) are amongst the most used algorithms for the DR classification. Thus using the deep learning algorithms for DR diagnostics have future research potential for DR early detection and prevention based solution.

Review of Machine Learning Applications Using Retinal Fundus Images

Automating screening and diagnosis in the medical field saves time and reduces the chances of misdiagnosis while saving on labor and cost for physicians. With the feasibility and development of deep learning methods, machines are now able to interpret complex features in medical data, which leads to rapid advancements in automation. Such efforts have been made in ophthalmology to analyze retinal images and build frameworks based on analysis for the identification of retinopathy and the assessment of its severity. This paper reviews recent state-of-the-art works utilizing the color fundus image taken from one of the imaging modalities used in ophthalmology. Specifically, the deep learning methods of automated screening and diagnosis for diabetic retinopathy (DR), age-related macular degeneration (AMD), and glaucoma are investigated. In addition, the machine learning techniques applied to the retinal vasculature extraction from the fundus image are covered. The challenges in developing these systems are also discussed.

RF-GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Adversarial Network

Diabetic retinopathy (DR) is a diabetic complication affecting the eyes, which is the main cause of blindness in young and middle-aged people. In order to speed up the diagnosis of DR, a mass of deep learning methods have been used for the detection of this disease, but they failed to attain excellent results due to unbalanced training data, i.e., the lack of DR fundus images. To address the problem of data imbalance, this paper proposes a method dubbed retinal fundus images generative adversarial networks (RF-GANs), which is based on generative adversarial network, to synthesize retinal fundus images. RF-GANs is composed of two generation models, RF-GAN1 and RF-GAN2. Firstly, RF-GAN1 is employed to translate retinal fundus images from source domain (the domain of semantic segmentation datasets) to target domain (the domain of EyePACS dataset connected to Kaggle (EyePACS)). Then, we train the semantic segmentation models with the translated images, and employ the trained models to extract the structural and lesion masks (hereafter, we refer to it as Masks) of EyePACS. Finally, we employ RF-GAN2 to synthesize retinal fundus images using the Masks and DR grading labels. This paper verifies the effectiveness of the method: RF-GAN1 can narrow down the domain gap between different datasets to improve the performance of the segmentation models. RF-GAN2 can synthesize realistic retinal fundus images. Adopting the synthesized images for data augmentation, the accuracy and quadratic weighted kappa of the state-of-the-art DR grading model on the testing set of EyePACS increase by 1.53% and 1.70%, respectively.

A novel four-step feature selection technique for diabetic retinopathy grading

Diabetic retinopathy is a microvascular complication of diabetes mellitus that develops over time. Diabetic retinopathy is one of the retinal disorders. Early detection of diabetic retinopathy reduces the chances of permanent vision loss. However, the identification and regular diagnosis of diabetic retinopathy is a time-consuming task and requires expert ophthalmologists and radiologists. In addition, an automatic diabetic retinopathy detection technique is necessary for real-time applications to facilitate and minimize potential human errors. Therefore, we propose an ensemble deep neural network and a novel four-step feature selection technique in this paper. In the first step, the preprocessed entropy images improve the quality of the retinal features. Second, the features are extracted using a deep ensemble model include InceptionV3, ResNet101, and Vgg19 from the retinal fundus images. Then, these features are combined to create an ample feature space. To reduce the feature space, we propose four-step feature selection techniques: minimum redundancy, maximum relevance, Chi-Square, ReliefF, and F test for selecting efficient features. Further, appropriate features are chosen from the majority voting techniques to reduce the computational complexity. Finally, the standard machine learning classifier, support vector machines, is used in diabetic retinopathy classification. The proposed method is tested on Kaggle, MESSIDOR-2, and IDRiD databases, available publicly. The proposed algorithm provided an accuracy of 97.78%, a sensitivity of 97.6%, and a specificity of 99.3%, using top 300 features, which are better than other state-of-the-art methods.

Automated Detection and Diagnosis of Diabetic Retinopathy: A Comprehensive Survey

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world. In the past few years, artificial intelligence (AI) based approaches have been used to detect and grade DR. Early detection enables appropriate treatment and thus prevents vision loss. For this purpose, both fundus and optical coherence tomography (OCT) images are used to image the retina. Next, Deep-learning (DL)-/machine-learning (ML)-based approaches make it possible to extract features from the images and to detect the presence of DR, grade its severity and segment associated lesions. This review covers the literature dealing with AI approaches to DR such as ML and DL in classification and segmentation that have been published in the open literature within six years (2016-2021). In addition, a comprehensive list of available DR datasets is reported. This list was constructed using both the PICO (P-Patient, I-Intervention, C-Control, O-Outcome) and Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) 2009 search strategies. We summarize a total of 114 published articles which conformed to the scope of the review. In addition, a list of 43 major datasets is presented.

Deep learning for diabetic retinopathy detection and classification based on fundus images: A review

Diabetic Retinopathy is a retina disease caused by diabetes mellitus and it is the leading cause of blindness globally. Early detection and treatment are necessary in order to delay or avoid vision deterioration and vision loss. To that end, many artificial-intelligence-powered methods have been proposed by the research community for the detection and classification of diabetic retinopathy on fundus retina images. This review article provides a thorough analysis of the use of deep learning methods at the various steps of the diabetic retinopathy detection pipeline based on fundus images. We discuss several aspects of that pipeline, ranging from the datasets that are widely used by the research community, the preprocessing techniques employed and how these accelerate and improve the models' performance, to the development of such deep learning models for the diagnosis and grading of the disease as well as the localization of the disease's lesions. We also discuss certain models that have been applied in real clinical settings. Finally, we conclude with some important insights and provide future research directions.

Diabetic Retinopathy Fundus Image Classification and Lesions Localization System Using Deep Learning

Diabetic retinopathy (DR) is a disease resulting from diabetes complications, causing non-reversible damage to retina blood vessels. DR is a leading cause of blindness if not detected early. The currently available DR treatments are limited to stopping or delaying the deterioration of sight, highlighting the importance of regular scanning using high-efficiency computer-based systems to diagnose cases early. The current work presented fully automatic diagnosis systems that exceed manual techniques to avoid misdiagnosis, reducing time, effort and cost. The proposed system classifies DR images into five stages—no-DR, mild, moderate, severe and proliferative DR—as well as localizing the affected lesions on retain surface. The system comprises two deep learning-based models. The first model (CNN512) used the whole image as an input to the CNN model to classify it into one of the five DR stages. It achieved an accuracy of 88.6% and 84.1% on the DDR and the APTOS Kaggle 2019 public datasets, respectively, compared to the state-of-the-art results. Simultaneously, the second model used an adopted YOLOv3 model to detect and localize the DR lesions, achieving a 0.216 mAP in lesion localization on the DDR dataset, which improves the current state-of-the-art results. Finally, both of the proposed structures, CNN512 and YOLOv3, were fused to classify DR images and localize DR lesions, obtaining an accuracy of 89% with 89% sensitivity, 97.3 specificity and that exceeds the current state-of-the-art results.

Understanding inherent image features in CNN-based assessment of diabetic retinopathy

Diabetic retinopathy (DR) is a leading cause of blindness and affects millions of people throughout the world. Early detection and timely checkups are key to reduce the risk of blindness. Automated grading of DR is a cost-effective way to ensure early detection and timely checkups. Deep learning or more specifically convolutional neural network (CNN)-based methods produce state-of-the-art performance in DR detection. Whilst CNN based methods have been proposed, no comparisons have been done between the extracted image features and their clinical relevance. Here we first adopt a CNN visualization strategy to discover the inherent image features involved in the CNN's decision-making process. Then, we critically analyze those features with respect to commonly known pathologies namely microaneurysms, hemorrhages and exudates, and other ocular components. We also critically analyze different CNNs by considering what image features they pick up during learning to predict and justify their clinical relevance. The experiments are executed on publicly available fundus datasets (EyePACS and DIARETDB1) achieving an accuracy of 89 ~ 95% with AUC, sensitivity and specificity of respectively 95 ~ 98%, 74 ~ 86%, and 93 ~ 97%, for disease level grading of DR. Whilst different CNNs produce consistent classification results, the rate of picked-up image features disagreement between models could be as high as 70%.

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

People with diabetes are at risk of developing an eye disease called diabetic retinopathy (DR). This disease occurs when high blood glucose levels cause damage to blood vessels in the retina. Computer-aided DR diagnosis has become a promising tool for the early detection and severity grading of DR, due to the great success of deep learning. However, most current DR diagnosis systems do not achieve satisfactory performance or interpretability for ophthalmologists, due to the lack of training data with consistent and fine-grained annotations. To address this problem, we construct a large fine-grained annotated DR dataset containing 2,842 images (FGADR). Specifically, this dataset has 1,842 images with pixel-level DR-related lesion annotations, and 1,000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency. The proposed dataset will enable extensive studies on DR diagnosis. Further, we establish three benchmark tasks for evaluation: 1. DR lesion segmentation; 2. DR grading by joint classification and segmentation; 3. Transfer learning for ocular multi-disease identification. Moreover, a novel inductive transfer learning method is introduced for the third task. Extensive experiments using different state-of-the-art methods are conducted on our FGADR dataset, which can serve as baselines for future research. Our dataset will be released in https://csyizhou.github.io/FGADR/.

A Multi-Label Deep Learning Model with Interpretable Grad-CAM for Diabetic Retinopathy Classification

The characteristics of diabetic retinopathy (DR) fundus images generally consist of multiple types of lesions which provided strong evidence for the ophthalmologists to make diagnosis. It is particularly significant to figure out an efficient method to not only accurately classify DR fundus images but also recognize all kinds of lesions on them. In this paper, a deep learning-based multi-label classification model with Gradient-weighted Class Activation Mapping (Grad-CAM) was proposed, which can both make DR classification and automatically locate the regions of different lesions. To reducing laborious annotation work and improve the efficiency of labeling, this paper innovatively considered different types of lesions as different labels for a fundus image so that this paper changed the task of lesion detection into that of image classification. A total of five labels were pre-defined and 3228 fundus images were collected for developing our model. The architecture of deep learning model was designed by ourselves based on ResNet. Through experiments on the test images, this method acquired a sensitive of 93.9% and a specificity of 94.4% on DR classification. Moreover, the corresponding regions of lesions were reasonably outlined on the DR fundus images.