An ensemble deep learning based approach for red lesion detection in fundus images

Trends and hotspots in the field of diabetic retinopathy imaging research from 2000-2023

Background

Diabetic retinopathy (DR) poses a major threat to diabetic patients' vision and is a critical public health issue. Imaging applications for DR have grown since the 21st century, aiding diagnosis, grading, and screening. This study uses bibliometric analysis to assess the field's advancements and key areas of interest.

Methods

This study performed a bibliometric analysis of DR imaging articles collected from the Web of Science Core Collection database between January 1st, 2000, and December 31st, 2023. The literature information was then analyzed through CiteSpace.

Results

The United States and China led in the number of publications, with 719 and 609, respectively. The University of London topped the institution list with 139 papers. Tien Yin Wong was the most prolific researcher. Invest. Ophthalmol. Vis. Sci. published the most articles (105). Notable burst keywords were "deep learning," "artificial intelligence," et al.

Conclusion

The United States is at the forefront of DR research, with the University of London as the top institution and Invest. Ophthalmol. Vis. Sci. as the most published journal. Tien Yin Wong is the most influential researcher. Hotspots like "deep learning," and "artificial intelligence," have seen a significant rise, indicating artificial intelligence's growing role in DR imaging.

Enhancing deep learning pre-trained networks on diabetic retinopathy fundus photographs with SLIC-G

Diabetic retinopathy disease contains lesions (e.g., exudates, hemorrhages, and microaneurysms) that are minute to the naked eye. Determining the lesions at pixel level poses a challenge as each pixel does not reflect any semantic entities. Furthermore, the computational cost of inspecting each pixel is expensive because the number of pixels is high even at low resolution. In this work, we propose a hybrid image processing method. Simple Linear Iterative Clustering with Gaussian Filter (SLIC-G) for the purpose of overcoming pixel constraints. The SLIC-G image processing method is divided into two stages: (1) simple linear iterative clustering superpixel segmentation and (2) Gaussian smoothing operation. In such a way, a large number of new transformed datasets are generated and then used for model training. Finally, two performance evaluation metrics that are suitable for imbalanced diabetic retinopathy datasets were used to validate the effectiveness of the proposed SLIC-G. The results indicate that, in comparison to prior published works' results, the proposed SLIC-G shows better performance on image classification of class imbalanced diabetic retinopathy datasets. This research reveals the importance of image processing and how it influences the performance of deep learning networks. The proposed SLIC-G enhances pre-trained network performance by eliminating the local redundancy of an image, which preserves local structures, but avoids over-segmented, noisy clips. It closes the research gap by introducing the use of superpixel segmentation and Gaussian smoothing operation as image processing methods in diabetic retinopathy-related tasks.

Medical-informed machine learning: integrating prior knowledge into medical decision systems

BACKGROUND

Clinical medicine offers a promising arena for applying Machine Learning (ML) models. However, despite numerous studies employing ML in medical data analysis, only a fraction have impacted clinical care. This article underscores the importance of utilising ML in medical data analysis, recognising that ML alone may not adequately capture the full complexity of clinical data, thereby advocating for the integration of medical domain knowledge in ML.

METHODS

The study conducts a comprehensive review of prior efforts in integrating medical knowledge into ML and maps these integration strategies onto the phases of the ML pipeline, encompassing data pre-processing, feature engineering, model training, and output evaluation. The study further explores the significance and impact of such integration through a case study on diabetes prediction. Here, clinical knowledge, encompassing rules, causal networks, intervals, and formulas, is integrated at each stage of the ML pipeline, resulting in a spectrum of integrated models.

RESULTS

The findings highlight the benefits of integration in terms of accuracy, interpretability, data efficiency, and adherence to clinical guidelines. In several cases, integrated models outperformed purely data-driven approaches, underscoring the potential for domain knowledge to enhance ML models through improved generalisation. In other cases, the integration was instrumental in enhancing model interpretability and ensuring conformity with established clinical guidelines. Notably, knowledge integration also proved effective in maintaining performance under limited data scenarios.

CONCLUSIONS

By illustrating various integration strategies through a clinical case study, this work provides guidance to inspire and facilitate future integration efforts. Furthermore, the study identifies the need to refine domain knowledge representation and fine-tune its contribution to the ML model as the two main challenges to integration and aims to stimulate further research in this direction.

Diabetic retinopathy prediction based on vision transformer and modified capsule network

Diabetic retinopathy is considered one of the most common diseases that can lead to blindness in the working age, and the chance of developing it increases as long as a person suffers from diabetes. Protecting the sight of the patient or decelerating the evolution of this disease depends on its early detection as well as identifying the exact levels of this pathology, which is done manually by ophthalmologists. This manual process is very consuming in terms of the time and experience of an expert ophthalmologist, which makes developing an automated method to aid in the diagnosis of diabetic retinopathy an essential and urgent need. In this paper, we aim to propose a new hybrid deep learning method based on a fine-tuning vision transformer and a modified capsule network for automatic diabetic retinopathy severity level prediction. The proposed approach consists of a new range of computer vision operations, including the power law transformation technique and the contrast-limiting adaptive histogram equalization technique in the preprocessing step. While the classification step builds up on a fine-tuning vision transformer, a modified capsule network, and a classification model combined with a classification model, The effectiveness of our approach was evaluated using four datasets, including the APTOS, Messidor-2, DDR, and EyePACS datasets, for the task of severity levels of diabetic retinopathy. We have attained excellent test accuracy scores on the four datasets, respectively: 88.18%, 87.78%, 80.36%, and 78.64%. Comparing our results with the state-of-the-art, we reached a better performance.

Artificial Intelligence (AI) for Early Diagnosis of Retinal Diseases

Artificial intelligence (AI) has emerged as a transformative tool in the field of ophthalmology, revolutionizing disease diagnosis and management. This paper provides a comprehensive overview of AI applications in various retinal diseases, highlighting its potential to enhance screening efficiency, facilitate early diagnosis, and improve patient outcomes. Herein, we elucidate the fundamental concepts of AI, including machine learning (ML) and deep learning (DL), and their application in ophthalmology, underscoring the significance of AI-driven solutions in addressing the complexity and variability of retinal diseases. Furthermore, we delve into the specific applications of AI in retinal diseases such as diabetic retinopathy (DR), age-related macular degeneration (AMD), Macular Neovascularization, retinopathy of prematurity (ROP), retinal vein occlusion (RVO), hypertensive retinopathy (HR), Retinitis Pigmentosa, Stargardt disease, best vitelliform macular dystrophy, and sickle cell retinopathy. We focus on the current landscape of AI technologies, including various AI models, their performance metrics, and clinical implications. Furthermore, we aim to address challenges and pitfalls associated with the integration of AI in clinical practice, including the "black box phenomenon", biases in data representation, and limitations in comprehensive patient assessment. In conclusion, this review emphasizes the collaborative role of AI alongside healthcare professionals, advocating for a synergistic approach to healthcare delivery. It highlights the importance of leveraging AI to augment, rather than replace, human expertise, thereby maximizing its potential to revolutionize healthcare delivery, mitigate healthcare disparities, and improve patient outcomes in the evolving landscape of medicine.

Automatic Detection and Classification of Hypertensive Retinopathy with Improved Convolution Neural Network and Improved SVM

Hypertensive retinopathy (HR) results from the microvascular retinal changes triggered by hypertension, which is the most common leading cause of preventable blindness worldwide. Therefore, it is necessary to develop an automated system for HR detection and evaluation using retinal images. We aimed to propose an automated approach to identify and categorize the various degrees of HR severity. A new network called the spatial convolution module (SCM) combines cross-channel and spatial information, and the convolution operations extract helpful features. The present model is evaluated using publicly accessible datasets ODIR, INSPIREVR, and VICAVR. We applied the augmentation to artificially increase the dataset of 1200 fundus images. The different HR severity levels of normal, mild, moderate, severe, and malignant are finally classified with the reduced time when compared to the existing models because in the proposed model, convolutional layers run only once on the input fundus images, which leads to a speedup and reduces the processing time in detecting the abnormalities in the vascular structure. According to the findings, the improved SVM had the highest detection and classification accuracy rate in the vessel classification with an accuracy of 98.99% and completed the task in 160.4 s. The ten-fold classification achieved the highest accuracy of 98.99%, i.e., 0.27 higher than the five-fold classification accuracy and the improved KNN classifier achieved an accuracy of 98.72%. When computation efficiency is a priority, the proposed model's ability to quickly recognize different HR severity levels is significant.

Micro-segmentation of retinal image lesions in diabetic retinopathy using energy-based fuzzy C-Means clustering (EFM-FCM)

Diabetic retinopathy (DR) is a prevalent cause of global visual impairment, contributing to approximately 4.8% of blindness cases worldwide as reported by the World Health Organization (WHO). The condition is characterized by pathological abnormalities in the retinal layer, including microaneurysms, vitreous hemorrhages, and exudates. Microscopic analysis of retinal images is crucial in diagnosing and treating DR. This article proposes a novel method for early DR screening using segmentation and unsupervised learning techniques. The approach integrates a neural network energy-based model into the Fuzzy C-Means (FCM) algorithm to enhance convergence criteria, aiming to improve the accuracy and efficiency of automated DR screening tools. The evaluation of results includes the primary dataset from the Shiva Netralaya Centre, IDRiD, and DIARETDB1. The performance of the proposed method is compared against FCM, EFCM, FLICM, and M-FLICM techniques, utilizing metrics such as accuracy in noiseless and noisy conditions and average execution time. The results showcase auspicious performance on both primary and secondary datasets, achieving accuracy rates of 99.03% in noiseless conditions and 93.13% in noisy images, with an average execution time of 16.1 s. The proposed method holds significant potential in medical image analysis and could pave the way for future advancements in automated DR diagnosis and management. RESEARCH HIGHLIGHTS: A novel approach is proposed in the article, integrating a neural network energy-based model into the FCM algorithm to enhance the convergence criteria and the accuracy of automated DR screening tools. By leveraging the microscopic characteristics of retinal images, the proposed method significantly improves the accuracy of lesion segmentation, facilitating early detection and monitoring of DR. The evaluation of the method's performance includes primary datasets from reputable sources such as the Shiva Netralaya Centre, IDRiD, and DIARETDB1, demonstrating its effectiveness in comparison to other techniques (FCM, EFCM, FLICM, and M-FLICM) in terms of accuracy in both noiseless and noisy conditions. It achieves impressive accuracy rates of 99.03% in noiseless conditions and 93.13% in noisy images, with an average execution time of 16.1 s.

Artificial intelligence in retinal disease: clinical application, challenges, and future directions

Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.

A new retinal OCT-angiography diabetic retinopathy dataset for segmentation and DR grading

PURPOSE

Diabetic retinopathy (DR) is one of the most common diseases caused by diabetes and can lead to vision loss or even blindness. The wide-field optical coherence tomography (OCT) angiography is non-invasive imaging technology and convenient to diagnose DR.

METHODS

A newly constructed Retinal OCT-Angiography Diabetic retinopathy (ROAD) dataset is utilized for segmentation and grading tasks. It contains 1200 normal images, 1440 DR images, and 1440 ground truths for DR image segmentation. To handle the problem of grading DR, we propose a novel and effective framework, named projective map attention-based convolutional neural network (PACNet).

RESULTS

The experimental results demonstrate the effectiveness of our PACNet. The accuracy of the proposed framework for grading DR is 87.5% on the ROAD dataset.

CONCLUSIONS

The information on ROAD can be viewed at URL https://mip2019.github.io/ROAD. The ROAD dataset will be helpful for the development of the early detection of DR field and future research.

TRANSLATIONAL RELEVANCE

The novel framework for grading DR is a valuable research and clinical diagnosis method.

A Hybrid Technique for Diabetic Retinopathy Detection Based on Ensemble-Optimized CNN and Texture Features

One of the most prevalent chronic conditions that can result in permanent vision loss is diabetic retinopathy (DR). Diabetic retinopathy occurs in five stages: no DR, and mild, moderate, severe, and proliferative DR. The early detection of DR is essential for preventing vision loss in diabetic patients. In this paper, we propose a method for the detection and classification of DR stages to determine whether patients are in any of the non-proliferative stages or in the proliferative stage. The hybrid approach based on image preprocessing and ensemble features is the foundation of the proposed classification method. We created a convolutional neural network (CNN) model from scratch for this study. Combining Local Binary Patterns (LBP) and deep learning features resulted in the creation of the ensemble features vector, which was then optimized using the Binary Dragonfly Algorithm (BDA) and the Sine Cosine Algorithm (SCA). Moreover, this optimized feature vector was fed to the machine learning classifiers. The SVM classifier achieved the highest classification accuracy of 98.85% on a publicly available dataset, i.e., Kaggle EyePACS. Rigorous testing and comparisons with state-of-the-art approaches in the literature indicate the effectiveness of the proposed methodology.

An Ensembled Framework for Human Breast Cancer Survivability Prediction Using Deep Learning

Breast cancer is categorized as an aggressive disease, and it is one of the leading causes of death. Accurate survival predictions for both long-term and short-term survivors, when delivered on time, can help physicians make effective treatment decisions for their patients. Therefore, there is a dire need to design an efficient and rapid computational model for breast cancer prognosis. In this study, we propose an ensemble model for breast cancer survivability prediction (EBCSP) that utilizes multi-modal data and stacks the output of multiple neural networks. Specifically, we design a convolutional neural network (CNN) for clinical modalities, a deep neural network (DNN) for copy number variations (CNV), and a long short-term memory (LSTM) architecture for gene expression modalities to effectively handle multi-dimensional data. The independent models' results are then used for binary classification (long term > 5 years and short term < 5 years) based on survivability using the random forest method. The EBCSP model's successful application outperforms models that utilize a single data modality for prediction and existing benchmarks.

Detecting red-lesions from retinal fundus images using unique morphological features

One of the most important retinal diseases is Diabetic Retinopathy (DR) which can lead to serious damage to vision if remains untreated. Red-lesions are from important demonstrations of DR helping its identification in early stages. The detection and verification of them is helpful in the evaluation of disease severity and progression. In this paper, a novel image processing method is proposed for extracting red-lesions from fundus images. The method works based on finding and extracting the unique morphological features of red-lesions. After quality improvement of images, a pixel-based verification is performed in the proposed method to find the ones which provide a significant intensity change in a curve-like neighborhood. In order to do so, a curve is considered around each pixel and the intensity changes around the curve boundary are considered. The pixels for which it is possible to find such curves in at least two directions are considered as parts of red-lesions. The simplicity of computations, the high accuracy of results, and no need to post-processing operations are the important characteristics of the proposed method endorsing its good performance.

Deep learning-based hemorrhage detection for diabetic retinopathy screening

Diabetic retinopathy is a retinal compilation that causes visual impairment. Hemorrhage is one of the pathological symptoms of diabetic retinopathy that emerges during disease development. Therefore, hemorrhage detection reveals the presence of diabetic retinopathy in the early phase. Diagnosing the disease in its initial stage is crucial to adopt proper treatment so the repercussions can be prevented. The automatic deep learning-based hemorrhage detection method is proposed that can be used as the second interpreter for ophthalmologists to reduce the time and complexity of conventional screening methods. The quality of the images was enhanced, and the prospective hemorrhage locations were estimated in the preprocessing stage. Modified gamma correction adaptively illuminates fundus images by using gradient information to address the nonuniform brightness levels of images. The algorithm estimated the locations of potential candidates by using a Gaussian match filter, entropy thresholding, and mathematical morphology. The required objects were segmented using the regional diversity at estimated locations. The novel hemorrhage network is propounded for hemorrhage classification and compared with the renowned deep models. Two datasets benchmarked the model's performance using sensitivity, specificity, precision, and accuracy metrics. Despite being the shallowest network, the proposed network marked competitive results than LeNet-5, AlexNet, ResNet50, and VGG-16. The hemorrhage network was assessed using training time and classification accuracy through synthetic experimentation. Results showed promising accuracy in the classification stage while significantly reducing training time. The research concluded that increasing deep network layers does not guarantee good results but rather increases training time. The suitable architecture of a deep model and its appropriate parameters are critical for obtaining excellent outcomes.

Recent trends and advances in fundus image analysis: A review

Automated retinal image analysis holds prime significance in the accurate diagnosis of various critical eye diseases that include diabetic retinopathy (DR), age-related macular degeneration (AMD), atherosclerosis, and glaucoma. Manual diagnosis of retinal diseases by ophthalmologists takes time, effort, and financial resources, and is prone to error, in comparison to computer-aided diagnosis systems. In this context, robust classification and segmentation of retinal images are primary operations that aid clinicians in the early screening of patients to ensure the prevention and/or treatment of these diseases. This paper conducts an extensive review of the state-of-the-art methods for the detection and segmentation of retinal image features. Existing notable techniques for the detection of retinal features are categorized into essential groups and compared in depth. Additionally, a summary of quantifiable performance measures for various important stages of retinal image analysis, such as image acquisition and preprocessing, is provided. Finally, the widely used in the literature datasets for analyzing retinal images are described and their significance is emphasized.

Detection of Diabetic Retinopathy using Convolutional Neural Networks for Feature Extraction and Classification (DRFEC)

Diabetic Retinopathy (DR) is caused as a result of Diabetes Mellitus which causes development of various retinal abrasions in the human retina. These lesions cause hindrance in vision and in severe cases, DR can lead to blindness. DR is observed amongst 80% of patients who have been diagnosed from prolonged diabetes for a period of 10-15 years. The manual process of periodic DR diagnosis and detection for necessary treatment, is time consuming and unreliable due to unavailability of resources and expert opinion. Therefore, computerized diagnostic systems which use Deep Learning (DL) Convolutional Neural Network (CNN) architectures, are proposed to learn DR patterns from fundus images and identify the severity of the disease. This paper proposes a comprehensive model using 26 state-of-the-art DL networks to assess and evaluate their performance, and which contribute for deep feature extraction and image classification of DR fundus images. In the proposed model, ResNet50 has shown highest overfitting in comparison to Inception V3, which has shown lowest overfitting when trained using the Kaggle's EyePACS fundus image dataset. EfficientNetB4 is the most optimal, efficient and reliable DL algorithm in detection of DR, followed by InceptionResNetV2, NasNetLarge and DenseNet169. EfficientNetB4 has achieved a training accuracy of 99.37% and the highest validation accuracy of 79.11%. DenseNet201 has achieved the highest training accuracy of 99.58% and a validation accuracy of 76.80% which is less than the top-4 best performing models.

ADAM Challenge: Detecting Age-Related Macular Degeneration From Fundus Images

Age-related macular degeneration (AMD) is the leading cause of visual impairment among elderly in the world. Early detection of AMD is of great importance, as the vision loss caused by this disease is irreversible and permanent. Color fundus photography is the most cost-effective imaging modality to screen for retinal disorders. Cutting edge deep learning based algorithms have been recently developed for automatically detecting AMD from fundus images. However, there are still lack of a comprehensive annotated dataset and standard evaluation benchmarks. To deal with this issue, we set up the Automatic Detection challenge on Age-related Macular degeneration (ADAM), which was held as a satellite event of the ISBI 2020 conference. The ADAM challenge consisted of four tasks which cover the main aspects of detecting and characterizing AMD from fundus images, including detection of AMD, detection and segmentation of optic disc, localization of fovea, and detection and segmentation of lesions. As part of the ADAM challenge, we have released a comprehensive dataset of 1200 fundus images with AMD diagnostic labels, pixel-wise segmentation masks for both optic disc and AMD-related lesions (drusen, exudates, hemorrhages and scars, among others), as well as the coordinates corresponding to the location of the macular fovea. A uniform evaluation framework has been built to make a fair comparison of different models using this dataset. During the ADAM challenge, 610 results were submitted for online evaluation, with 11 teams finally participating in the onsite challenge. This paper introduces the challenge, the dataset and the evaluation methods, as well as summarizes the participating methods and analyzes their results for each task. In particular, we observed that the ensembling strategy and the incorporation of clinical domain knowledge were the key to improve the performance of the deep learning models.

Recent developments on computer aided systems for diagnosis of diabetic retinopathy: a review

Diabetes is a long-term condition in which the pancreas quits producing insulin or the body's insulin isn't utilised properly. One of the signs of diabetes is Diabetic Retinopathy. Diabetic retinopathy is the most prevalent type of diabetes, if remains unaddressed, diabetic retinopathy can affect all diabetics and become very serious, raising the chances of blindness. It is a chronic systemic condition that affects up to 80% of patients for more than ten years. Many researchers believe that if diabetes individuals are diagnosed early enough, they can be rescued from the condition in 90% of cases. Diabetes damages the capillaries, which are microscopic blood vessels in the retina. On images, blood vessel damage is usually noticeable. Therefore, in this study, several traditional, as well as deep learning-based approaches, are reviewed for the classification and detection of this particular diabetic-based eye disease known as diabetic retinopathy, and also the advantage of one approach over the other is also described. Along with the approaches, the dataset and the evaluation metrics useful for DR detection and classification are also discussed. The main finding of this study is to aware researchers about the different challenges occurs while detecting diabetic retinopathy using computer vision, deep learning techniques. Therefore, a purpose of this review paper is to sum up all the major aspects while detecting DR like lesion identification, classification and segmentation, security attacks on the deep learning models, proper categorization of datasets and evaluation metrics. As deep learning models are quite expensive and more prone to security attacks thus, in future it is advisable to develop a refined, reliable and robust model which overcomes all these aspects which are commonly found while designing deep learning models.

Automated Diabetic Retinopathy Detection Using Horizontal and Vertical Patch Division-Based Pre-Trained DenseNET with Digital Fundus Images

Diabetic retinopathy (DR) is a common complication of diabetes that can lead to progressive vision loss. Regular surveillance with fundal photography, early diagnosis, and prompt intervention are paramount to reducing the incidence of DR-induced vision loss. However, manual interpretation of fundal photographs is subject to human error. In this study, a new method based on horizontal and vertical patch division was proposed for the automated classification of DR images on fundal photographs. The novel sides of this study are given as follows. We proposed a new non-fixed-size patch division model to obtain high classification results and collected a new fundus image dataset. Moreover, two datasets are used to test the model: a newly collected three-class (normal, non-proliferative DR, and proliferative DR) dataset comprising 2355 DR images and the established open-access five-class Asia Pacific Tele-Ophthalmology Society (APTOS) 2019 dataset comprising 3662 images. Two analysis scenarios, Case 1 and Case 2, with three (normal, non-proliferative DR, and proliferative DR) and five classes (normal, mild DR, moderate DR, severe DR, and proliferative DR), respectively, were derived from the APTOS 2019 dataset. These datasets and these cases have been used to demonstrate the general classification performance of our proposal. By applying transfer learning, the last fully connected and global average pooling layers of the DenseNet201 architecture were used to extract deep features from input DR images and each of the eight subdivided horizontal and vertical patches. The most discriminative features are then selected using neighborhood component analysis. These were fed as input to a standard shallow cubic support vector machine for classification. Our new DR dataset obtained 94.06% and 91.55% accuracy values for three-class classification with 80:20 hold-out validation and 10-fold cross-validation, respectively. As can be seen from steps of the proposed model, a new patch-based deep-feature engineering model has been proposed. The proposed deep-feature engineering model is a cognitive model, since it uses efficient methods in each phase. Similar excellent results were seen for three-class classification with the Case 1 dataset. In addition, the model attained 87.43% and 84.90% five-class classification accuracy rates using 80:20 hold-out validation and 10-fold cross-validation, respectively, on the Case 2 dataset, which outperformed prior DR classification studies based on the five-class APTOS 2019 dataset. Our model attained about >2% classification results compared to others. These findings demonstrate the accuracy and robustness of the proposed model for classification of DR images.

Image based early detection of diabetic retinopathy: A systematic review on Artificial Intelligence (AI) based recent trends and approaches

Diabetic Retinopathy (DR) is a disease that damages the retina of the human eye due to diabetic complications, resulting in a loss of vision. Blindness may be avoided If the DR disease is detected at an early stage. Unfortunately, DR is irreversible process, however, early detection and treatment of DR can significantly reduce the risk of vision loss. The manual diagnosis done by ophthalmologists on DR retina fundus images is time consuming, and error prone process. Nowadays, machine learning and deep learning have become one of the most effective approaches, which have even surpassed the human performance as well as performance of traditional image processing-based algorithms and other computer aided diagnosis systems in the analysis and classification of medical images. This paper addressed and evaluated the various recent state-of-the-art methodologies that have been used for detection and classification of Diabetic Retinopathy disease using machine learning and deep learning approaches in the past decade. Furthermore, this study also provides the authors observation and performance evaluation of available research using several parameters, such as accuracy, disease status, and sensitivity. Finally, we conclude with limitations, remedies, and future directions in DR detection. In addition, various challenging issues that need further study are also discussed.

Automatic Detection of Abnormalities and Grading of Diabetic Retinopathy in 6-Field Retinal Images: Integration of Segmentation Into Classification

Purpose

Classification of diabetic retinopathy (DR) is traditionally based on severity grading, given by the most advanced lesion, but potentially leaving out relevant information for risk stratification. In this study, we aimed to develop a deep learning model able to individually segment seven different DR-lesions, in order to test if this would improve a subsequently developed classification model.

Methods

First, manual segmentation of 34,075 different DR-lesions was used to construct a segmentation model, with performance subsequently compared to another retinal specialist. Second, we constructed a 5-step classification model using a data set of 31,325 expert-annotated retinal 6-field images and evaluated if performance was improved with the integration of presegmentation given by the segmentation model.

Results

The segmentation model had higher average sensitivity across all abnormalities compared to the retinal expert (0.68 and 0.62) at a comparable average F1-score (0.60 and 0.62). Model sensitivity for microaneurysms, retinal hemorrhages and intraretinal microvascular abnormalities was higher by 42.5%, 8.8%, and 67.5% and F1-scores by 15.8%, 6.5%, and 12.5%, respectively. When presegmentation was included, grading performance increased by 29.7%, 6.0%, and 4.5% for average per class accuracy, quadratic weighted kappa, and multiclass macro area under the curve, with values of 70.4%, 0.90, and 0.92, respectively.

Conclusions

The segmentation model matched an expert in detecting retinal abnormalities, and presegmentation substantially improved accuracy of the automated classification model.

Translational Relevance

Presegmentation may yield more accurate automated DR grading models and increase interpretability and trust in model decisions.

Hybrid multi-kernel SVM algorithm for detection of microaneurysm in color fundus images

Diabetic retinopathy (DR) is a chronic disease that may cause vision loss in diabetic patients. Microaneurysms which are characterized by small red spots on the retina due to fluid or blood leakage from the weak capillary wall often occur during the early stage of DR, making screening at this stage is essential. In this paper, an automatic screening system for early detection of DR in retinal images is developed using a combined shape and texture features. Due to minimum number of hand-crafted features, the computational burden is much reduced. The proposed hybrid multi-kernel support vector machine classifier is constructed by learning a kernel model formed as a combination of the base kernels. This approach outperforms the recent deep learning techniques in terms of the evaluation metrics. The efficiency of the proposed scheme is experimentally validated on three public datasets - Retinopathy Online Challenge, DIARETdB1, MESSIDOR, and AGAR300 (developed for this study). Studies reveal that the proposed model produced the best results of 0.503 in ROC dataset, 0.481 in DIARETdB1, and 0.464 in the MESSIDOR dataset in terms of FROC score. The AGAR300 database outperforms the existing MA detection algorithm in terms of FROC, AUC, F1 score, precision, sensitivity, and specificity which guarantees the robustness of this system.

A critical review on diagnosis of diabetic retinopathy using machine learning and deep learning

Diabetic Retinopathy (DR) is a health condition caused due to Diabetes Mellitus (DM). It causes vision problems and blindness due to disfigurement of human retina. According to statistics, 80% of diabetes patients battling from long diabetic period of 15 to 20 years, suffer from DR. Hence, it has become a dangerous threat to the health and life of people. To overcome DR, manual diagnosis of the disease is feasible but overwhelming and cumbersome at the same time and hence requires a revolutionary method. Thus, such a health condition necessitates primary recognition and diagnosis to prevent DR from developing into severe stages and prevent blindness. Innumerable Machine Learning (ML) models are proposed by researchers across the globe, to achieve this purpose. Various feature extraction techniques are proposed for extraction of DR features for early detection. However, traditional ML models have shown either meagre generalization throughout feature extraction and classification for deploying smaller datasets or consumes more of training time causing inefficiency in prediction while using larger datasets. Hence Deep Learning (DL), a new domain of ML, is introduced. DL models can handle a smaller dataset with help of efficient data processing techniques. However, they generally incorporate larger datasets for their deep architectures to enhance performance in feature extraction and image classification. This paper gives a detailed review on DR, its features, causes, ML models, state-of-the-art DL models, challenges, comparisons and future directions, for early detection of DR.

Deep Red Lesion Classification for Early Screening of Diabetic Retinopathy

Diabetic retinopathy (DR) is an asymptotic and vision-threatening complication among working-age adults. To prevent blindness, a deep convolutional neural network (CNN) based diagnosis can help to classify less-discriminative and small-sized red lesions in early screening of DR patients. However, training deep models with minimal data is a challenging task. Fine-tuning through transfer learning is a useful alternative, but performance degradation, overfitting, and domain adaptation issues further demand architectural amendments to effectively train deep models. Various pre-trained CNNs are fine-tuned on an augmented set of image patches. The best-performing ResNet50 model is modified by introducing reinforced skip connections, a global max-pooling layer, and the sum-of-squared-error loss function. The performance of the modified model (DR-ResNet50) on five public datasets is found to be better than state-of-the-art methods in terms of well-known metrics. The highest scores (0.9851, 0.991, 0.991, 0.991, 0.991, 0.9939, 0.0029, 0.9879, and 0.9879) for sensitivity, specificity, AUC, accuracy, precision, F1-score, false-positive rate, Matthews’s correlation coefficient, and kappa coefficient are obtained within a 95% confidence interval for unseen test instances from e-Ophtha_MA. This high sensitivity and low false-positive rate demonstrate the worth of a proposed framework. It is suitable for early screening due to its performance, simplicity, and robustness.

A holistic overview of deep learning approach in medical imaging

Medical images are a rich source of invaluable necessary information used by clinicians. Recent technologies have introduced many advancements for exploiting the most of this information and use it to generate better analysis. Deep learning (DL) techniques have been empowered in medical images analysis using computer-assisted imaging contexts and presenting a lot of solutions and improvements while analyzing these images by radiologists and other specialists. In this paper, we present a survey of DL techniques used for variety of tasks along with the different medical image's modalities to provide critical review of the recent developments in this direction. We have organized our paper to provide significant contribution of deep leaning traits and learn its concepts, which is in turn helpful for non-expert in medical society. Then, we present several applications of deep learning (e.g., segmentation, classification, detection, etc.) which are commonly used for clinical purposes for different anatomical site, and we also present the main key terms for DL attributes like basic architecture, data augmentation, transfer learning, and feature selection methods. Medical images as inputs to deep learning architectures will be the mainstream in the coming years, and novel DL techniques are predicted to be the core of medical images analysis. We conclude our paper by addressing some research challenges and the suggested solutions for them found in literature, and also future promises and directions for further developments.

Local Structure Awareness-Based Retinal Microaneurysm Detection with Multi-Feature Combination

Retinal microaneurysm (MA) is the initial symptom of diabetic retinopathy (DR). The automatic detection of MA is helpful to assist doctors in diagnosis and treatment. Previous algorithms focused on the features of the target itself; however, the local structural features of the target and background are also worth exploring. To achieve MA detection, an efficient local structure awareness-based retinal MA detection with the multi-feature combination (LSAMFC) is proposed in this paper. We propose a novel local structure feature called a ring gradient descriptor (RGD) to describe the structural differences between an object and its surrounding area. Then, a combination of RGD with the salience and texture features is used by a Gradient Boosting Decision Tree (GBDT) for candidate classification. We evaluate our algorithm on two public datasets, i.e., the e-ophtha MA dataset and retinopathy online challenge (ROC) dataset. The experimental results show that the performance of the trained model significantly improved after combining traditional features with RGD, and the area under the receiver operating characteristic curve (AUC) values in the test results of the datasets e-ophtha MA and ROC increased from 0.9615 to 0.9751 and from 0.9066 to 0.9409, respectively.

Artificial intelligence promotes the diagnosis and screening of diabetic retinopathy

Deep learning evolves into a new form of machine learning technology that is classified under artificial intelligence (AI), which has substantial potential for large-scale healthcare screening and may allow the determination of the most appropriate specific treatment for individual patients. Recent developments in diagnostic technologies facilitated studies on retinal conditions and ocular disease in metabolism and endocrinology. Globally, diabetic retinopathy (DR) is regarded as a major cause of vision loss. Deep learning systems are effective and accurate in the detection of DR from digital fundus photographs or optical coherence tomography. Thus, using AI techniques, systems with high accuracy and efficiency can be developed for diagnosing and screening DR at an early stage and without the resources that are only accessible in special clinics. Deep learning enables early diagnosis with high specificity and sensitivity, which makes decisions based on minimally handcrafted features paving the way for personalized DR progression real-time monitoring and in-time ophthalmic or endocrine therapies. This review will discuss cutting-edge AI algorithms, the automated detecting systems of DR stage grading and feature segmentation, the prediction of DR outcomes and therapeutics, and the ophthalmic indications of other systemic diseases revealed by AI.

A Magnified Adaptive Feature Pyramid Network for automatic microaneurysms detection

Diabetic retinopathy (DR), as an important complication of diabetes, is the primary cause of blindness in adults. Automatic DR detection poses a challenge which is crucial for early DR screening. Currently, the vast majority of DR is diagnosed through fundus images, where the microaneurysm (MA) has been widely used as the most distinguishable marker. Research works on automatic DR detection have traditionally utilized manually designed operators, while a few recent researchers have explored deep learning techniques for this topic. But due to issues such as the extremely small size of microaneurysms, low resolution of fundus pictures, and insufficient imaging depth, the DR detection problem is quite challenging and remains unsolved. To address these issues, this research proposes a new deep learning model (Magnified Adaptive Feature Pyramid Network, MAFP-Net) for DR detection, which conducts super-resolution on low quality fundus images and integrates an improved feature pyramid structure while utilizing a standard two-stage detection network as the backbone. Our proposed detection model needs no pre-segmented patches to train the CNN network. When tested on the E-ophtha-MA dataset, the sensitivity value of our method reached as high as 83.5% at false positives per image (FPI) of 8 and the F1 value achieved 0.676, exceeding all those of the state-of-the-art algorithms as well as the human performance of experienced physicians. Similar results were achieved on another public dataset of IDRiD.

Severity Grading and Early Retinopathy Lesion Detection through Hybrid Inception-ResNet Architecture

Diabetic retinopathy (DR) is a diabetes disorder that disturbs human vision. It starts due to the damage in the light-sensitive tissues of blood vessels at the retina. In the beginning, DR may show no symptoms or only slight vision issues, but in the long run, it could be a permanent source of impaired vision, simply known as blindness in the advanced as well as in developing nations. This could be prevented if DR is identified early enough, but it can be challenging as we know the disease frequently shows rare signs until it is too late to deliver an effective cure. In our work, we recommend a framework for severity grading and early DR detection through hybrid deep learning Inception-ResNet architecture with smart data preprocessing. Our proposed method is composed of three steps. Firstly, the retinal images are preprocessed with the help of augmentation and intensity normalization. Secondly, the preprocessed images are given to the hybrid Inception-ResNet architecture to extract the vector image features for the categorization of different stages. Lastly, to identify DR and decide its stage (e.g., mild DR, moderate DR, severe DR, or proliferative DR), a classification step is used. The studies and trials have to reveal suitable outcomes when equated with some other previously deployed approaches. However, there are specific constraints in our study that are also discussed and we suggest methods to enhance further research in this field.

Red-lesion extraction in retinal fundus images by directional intensity changes' analysis

Diabetic retinopathy (DR) is an important retinal disease threatening people with the long diabetic history. Blood leakage in retina leads to the formation of red lesions in retina the analysis of which is helpful in the determination of severity of disease. In this paper, a novel red-lesion extraction method is proposed. The new method firstly determines the boundary pixels of blood vessel and red lesions. Then, it determines the distinguishing features of boundary pixels of red-lesions to discriminate them from other boundary pixels. The main point utilized here is that a red lesion can be observed as significant intensity changes in almost all directions in the fundus image. This can be feasible through considering special neighborhood windows around the extracted boundary pixels. The performance of the proposed method has been evaluated for three different datasets including Diaretdb0, Diaretdb1 and Kaggle datasets. It is shown that the method is capable of providing the values of 0.87 and 0.88 for sensitivity and specificity of Diaretdb1, 0.89 and 0.9 for sensitivity and specificity of Diaretdb0, 0.82 and 0.9 for sensitivity and specificity of Kaggle. Also, the proposed method has a time-efficient performance in the red-lesion extraction process.

Automated Detection and Diagnosis of Diabetic Retinopathy: A Comprehensive Survey

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world. In the past few years, artificial intelligence (AI) based approaches have been used to detect and grade DR. Early detection enables appropriate treatment and thus prevents vision loss. For this purpose, both fundus and optical coherence tomography (OCT) images are used to image the retina. Next, Deep-learning (DL)-/machine-learning (ML)-based approaches make it possible to extract features from the images and to detect the presence of DR, grade its severity and segment associated lesions. This review covers the literature dealing with AI approaches to DR such as ML and DL in classification and segmentation that have been published in the open literature within six years (2016-2021). In addition, a comprehensive list of available DR datasets is reported. This list was constructed using both the PICO (P-Patient, I-Intervention, C-Control, O-Outcome) and Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) 2009 search strategies. We summarize a total of 114 published articles which conformed to the scope of the review. In addition, a list of 43 major datasets is presented.

Performance and Limitation of Machine Learning Algorithms for Diabetic Retinopathy Screening: Meta-analysis

Background

Diabetic retinopathy (DR), whose standard diagnosis is performed by human experts, has high prevalence and requires a more efficient screening method. Although machine learning (ML)–based automated DR diagnosis has gained attention due to recent approval of IDx-DR, performance of this tool has not been examined systematically, and the best ML technique for use in a real-world setting has not been discussed.

Objective

The aim of this study was to systematically examine the overall diagnostic accuracy of ML in diagnosing DR of different categories based on color fundus photographs and to determine the state-of-the-art ML approach.

Methods

Published studies in PubMed and EMBASE were searched from inception to June 2020. Studies were screened for relevant outcomes, publication types, and data sufficiency, and a total of 60 out of 2128 (2.82%) studies were retrieved after study selection. Extraction of data was performed by 2 authors according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and the quality assessment was performed according to the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Meta-analysis of diagnostic accuracy was pooled using a bivariate random effects model. The main outcomes included diagnostic accuracy, sensitivity, and specificity of ML in diagnosing DR based on color fundus photographs, as well as the performances of different major types of ML algorithms.

Results

The primary meta-analysis included 60 color fundus photograph studies (445,175 interpretations). Overall, ML demonstrated high accuracy in diagnosing DR of various categories, with a pooled area under the receiver operating characteristic (AUROC) ranging from 0.97 (95% CI 0.96-0.99) to 0.99 (95% CI 0.98-1.00). The performance of ML in detecting more-than-mild DR was robust (sensitivity 0.95; AUROC 0.97), and by subgroup analyses, we observed that robust performance of ML was not limited to benchmark data sets (sensitivity 0.92; AUROC 0.96) but could be generalized to images collected in clinical practice (sensitivity 0.97; AUROC 0.97). Neural network was the most widely used method, and the subgroup analysis revealed a pooled AUROC of 0.98 (95% CI 0.96-0.99) for studies that used neural networks to diagnose more-than-mild DR.

Conclusions

This meta-analysis demonstrated high diagnostic accuracy of ML algorithms in detecting DR on color fundus photographs, suggesting that state-of-the-art, ML-based DR screening algorithms are likely ready for clinical applications. However, a significant portion of the earlier published studies had methodology flaws, such as the lack of external validation and presence of spectrum bias. The results of these studies should be interpreted with caution.

Deep learning for diabetic retinopathy detection and classification based on fundus images: A review

Diabetic Retinopathy is a retina disease caused by diabetes mellitus and it is the leading cause of blindness globally. Early detection and treatment are necessary in order to delay or avoid vision deterioration and vision loss. To that end, many artificial-intelligence-powered methods have been proposed by the research community for the detection and classification of diabetic retinopathy on fundus retina images. This review article provides a thorough analysis of the use of deep learning methods at the various steps of the diabetic retinopathy detection pipeline based on fundus images. We discuss several aspects of that pipeline, ranging from the datasets that are widely used by the research community, the preprocessing techniques employed and how these accelerate and improve the models' performance, to the development of such deep learning models for the diagnosis and grading of the disease as well as the localization of the disease's lesions. We also discuss certain models that have been applied in real clinical settings. Finally, we conclude with some important insights and provide future research directions.

ResNet Based Deep Features and Random Forest Classifier for Diabetic Retinopathy Detection

Diabetic retinopathy, an eye disease commonly afflicting diabetic patients, can result in loss of vision if prompt detection and treatment are not done in the early stages. Once the symptoms are identified, the severity level of the disease needs to be classified for prescribing the right medicine. This study proposes a deep learning-based approach, for the classification and grading of diabetic retinopathy images. The proposed approach uses the feature map of ResNet-50 and passes it to Random Forest for classification. The proposed approach is compared with five state-of-the-art approaches using two category Messidor-2 and five category EyePACS datasets. These two categories on the Messidor-2 dataset include 'No Referable Diabetic Macular Edema Grade (DME)' and 'Referable DME' while five categories consist of 'Proliferative diabetic retinopathy', 'Severe', 'Moderate', 'Mild', and 'No diabetic retinopathy'. The results show that the proposed approach outperforms compared approaches and achieves an accuracy of 96% and 75.09% for these datasets, respectively. The proposed approach outperforms six existing state-of-the-art architectures, namely ResNet-50, VGG-19, Inception-v3, MobileNet, Xception, and VGG16.

A method for the automatic detection of myopia in Optos fundus images based on deep learning

Myopia detection is significant for preventing irreversible visual impairment and diagnosing myopic retinopathy. To improve the detection efficiency and accuracy, a Myopia Detection Network (MDNet) that combines the advantages of dense connection and Residual Squeeze-and-Excitation attention is proposed in this paper to automatically detect myopia in Optos fundus images. First, an automatic optic disc recognition method is applied to extract the Regions of Interest and remove the noise disturbances; then, data augmentation techniques are implemented to enlarge the data set and prevent overfitting; moreover, an MDNet composed of Attention Dense blocks is constructed to detect myopia in Optos fundus images. The results show that the Mean Absolute Error of the Spherical Equivalent detected by this network can reach 1.1150 D (diopter), which verifies the feasibility and applicability of this method for the automatic detection of myopia in Optos fundus images.

Automated Microaneurysm Counts on Ultrawide Field Color and Fluorescein Angiography Images

BACKGROUND

The severity and extent of microaneurysms (MAs) have been used to determine diabetic retinopathy (DR) severity and estimate the risk of DR progression over time. The recent introduction of ultrawide field (UWF) imaging has allowed ophthalmologists to readily image nearly the entire retina. Manual counting of MAs, especially on UWF images, is laborious and time-consuming, limiting its potential use in clinical settings. Automated MA counting techniques are potentially more accurate and reproducible compared to manual methods.

METHOD

Review of available literature on current techniques of automated MA counting techniques on both ultrawide field (UWF) color images (CI) and fluorescein angiography (FA) images.

RESULTS

Automated MA counting techniques on UWF images are still in the early phases of development with UWF-FA counts being further along. Early studies have demonstrated that these techniques are accurate and reproducible.

CONCLUSION

Automated techniques may be an appropriate option for detecting and quantifying MAs on UWF images, especially in eyes with earlier DR severity. Larger studies are needed to appropriately validate these techniques and determine if they add substantially to clinical practice compared to standard DR grading.

Automatic Lumen Border Detection in IVUS Images Using Deep Learning Model and Handcrafted Features

In the clinical analysis of Intravascular ultrasound (IVUS) images, the lumen size is an important indicator of coronary atherosclerosis, and is also the premise of coronary artery disease diagnosis and interventional treatment. In this study, a fully automatic method based on deep learning model and handcrafted features is presented for the detection of the lumen borders in IVUS images. First, 193 handcrafted features are extracted from the IVUS images. Then hybrid feature vectors are constructed by combining handcrafted features with 64 high-level features extracted from U-Net. In order to obtain the feature subsets with larger contribution, we employ the extended binary cuckoo search for feature selection. Finally, the selected 36-dimensional hybrid feature subset is used to classify the test images using dictionary learning based on kernel sparse coding. The proposed algorithm is tested on the publicly available dataset and evaluated using three indicators. Through ablation experiments, mean value of the experimental results (Jaccard: 0.88, Hausdorff distance: 0.36, Percentage of the area difference: 0.06) prove to be effective improving lumen border detection. Furthermore, compared with the recent methods used on the same dataset, the proposed method shows good performance and high accuracy.

Pathological myopia classification with simultaneous lesion segmentation using deep learning

BACKGROUND AND OBJECTIVES

Pathological myopia (PM) is the seventh leading cause of blindness, with a reported global prevalence up to 3%. Early and automated PM detection from fundus images could aid to prevent blindness in a world population that is characterized by a rising myopia prevalence. We aim to assess the use of convolutional neural networks (CNNs) for the detection of PM and semantic segmentation of myopia-induced lesions from fundus images on a recently introduced reference data set.

METHODS

This investigation reports on the results of CNNs developed for the recently introduced Pathological Myopia (PALM) dataset, which consists of 1200 images. Our CNN bundles lesion segmentation and PM classification, as the two tasks are heavily intertwined. Domain knowledge is also inserted through the introduction of a new Optic Nerve Head (ONH)-based prediction enhancement for the segmentation of atrophy and fovea localization. Finally, we are the first to approach fovea localization using segmentation instead of detection or regression models. Evaluation metrics include area under the receiver operating characteristic curve (AUC) for PM detection, Euclidean distance for fovea localization, and Dice and F1 metrics for the semantic segmentation tasks (optic disc, retinal atrophy and retinal detachment).

RESULTS

Models trained with 400 available training images achieved an AUC of 0.9867 for PM detection, and a Euclidean distance of 58.27 pixels on the fovea localization task, evaluated on a test set of 400 images. Dice and F1 metrics for semantic segmentation of lesions scored 0.9303 and 0.9869 on optic disc, 0.8001 and 0.9135 on retinal atrophy, and 0.8073 and 0.7059 on retinal detachment, respectively.

CONCLUSIONS

We report a successful approach for a simultaneous classification of pathological myopia and segmentation of associated lesions. Our work was acknowledged with an award in the context of the "Pathological Myopia detection from retinal images" challenge held during the IEEE International Symposium on Biomedical Imaging (April 2019). Considering that (pathological) myopia cases are often identified as false positives and negatives in glaucoma deep learning models, we envisage that the current work could aid in future research to discriminate between glaucomatous and highly-myopic eyes, complemented by the localization and segmentation of landmarks such as fovea, optic disc and atrophy.

Survey on recent developments in automatic detection of diabetic retinopathy

Diabetic retinopathy (DR) is a disease facilitated by the rapid spread of diabetes worldwide. DR can blind diabetic individuals. Early detection of DR is essential to restoring vision and providing timely treatment. DR can be detected manually by an ophthalmologist, examining the retinal and fundus images to analyze the macula, morphological changes in blood vessels, hemorrhage, exudates, and/or microaneurysms. This is a time consuming, costly, and challenging task. An automated system can easily perform this function by using artificial intelligence, especially in screening for early DR. Recently, much state-of-the-art research relevant to the identification of DR has been reported. This article describes the current methods of detecting non-proliferative diabetic retinopathy, exudates, hemorrhage, and microaneurysms. In addition, the authors point out future directions in overcoming current challenges in the field of DR research.

Testing Segmentation Popular Loss and Variations in Three Multiclass Medical Imaging Problems

Image structures are segmented automatically using deep learning (DL) for analysis and processing. The three most popular base loss functions are cross entropy (crossE), intersect-over-the-union (IoU), and dice. Which should be used, is it useful to consider simple variations, such as modifying formula coefficients? How do characteristics of different image structures influence scores? Taking three different medical image segmentation problems (segmentation of organs in magnetic resonance images (MRI), liver in computer tomography images (CT) and diabetic retinopathy lesions in eye fundus images (EFI)), we quantify loss functions and variations, as well as segmentation scores of different targets. We first describe the limitations of metrics, since loss is a metric, then we describe and test alternatives. Experimentally, we observed that DeeplabV3 outperforms UNet and fully convolutional network (FCN) in all datasets. Dice scored 1 to 6 percentage points (pp) higher than cross entropy over all datasets, IoU improved 0 to 3 pp. Varying formula coefficients improved scores, but the best choices depend on the dataset: compared to crossE, different false positive vs. false negative weights improved MRI by 12 pp, and assigning zero weight to background improved EFI by 6 pp. Multiclass segmentation scored higher than n-uniclass segmentation in MRI by 8 pp. EFI lesions score low compared to more constant structures (e.g., optic disk or even organs), but loss modifications improve those scores significantly 6 to 9 pp. Our conclusions are that dice is best, it is worth assigning 0 weight to class background and to test different weights on false positives and false negatives.

Automatic prediction of treatment outcomes in patients with diabetic macular edema using ensemble machine learning

Background

This study aimed to predict the treatment outcomes in patients with diabetic macular edema (DME) after 3 monthly anti-vascular endothelial growth factor (VEGF) injections using machine learning (ML) based on pretreatment optical coherence tomography (OCT) images and clinical variables.

Methods

An ensemble ML system consisting of four deep learning (DL) models and five classical machine learning (CML) models was developed to predict the posttreatment central foveal thickness (CFT) and the best-corrected visual acuity (BCVA). A total of 363 OCT images and 7,587 clinical data records from 363 eyes were included in the training set (304 eyes) and external validation set (59 eyes). The DL models were trained using the OCT images, and the CML models were trained using the OCT images features and clinical variables. The predictive posttreatment CFT and BCVA values were compared with true outcomes obtained from the medical records.

Results

For CFT prediction, the mean absolute error (MAE), root mean square error (RMSE), and R² of the best-performing model in the training set was 66.59, 93.73, and 0.71, respectively, with an area under receiver operating characteristic curve (AUC) of 0.90 for distinguishing the eyes with good anatomical response. The MAE, RMSE, and R² was 68.08, 97.63, and 0.74, respectively, with an AUC of 0.94 in the external validation set. For BCVA prediction, the MAE, RMSE, and R² of the best-performing model in the training set was 0.19, 0.29, and 0.60, respectively, with an AUC of 0.80 for distinguishing eyes with a good functional response. The external validation achieved a MAE, RMSE, and R² of 0.13, 0.20, and 0.68, respectively, with an AUC of 0.81.

Conclusions

Our ensemble ML system accurately predicted posttreatment CFT and BCVA after anti-VEGF injections in DME patients, and can be used to prospectively assess the efficacy of anti-VEGF therapy in DME patients.

Machine Learning Techniques for Ophthalmic Data Processing: A Review

Machine learning and especially deep learning techniques are dominating medical image and data analysis. This article reviews machine learning approaches proposed for diagnosing ophthalmic diseases during the last four years. Three diseases are addressed in this survey, namely diabetic retinopathy, age-related macular degeneration, and glaucoma. The review covers over 60 publications and 25 public datasets and challenges related to the detection, grading, and lesion segmentation of the three considered diseases. Each section provides a summary of the public datasets and challenges related to each pathology and the current methods that have been applied to the problem. Furthermore, the recent machine learning approaches used for retinal vessels segmentation, and methods of retinal layers and fluid segmentation are reviewed. Two main imaging modalities are considered in this survey, namely color fundus imaging, and optical coherence tomography. Machine learning approaches that use eye measurements and visual field data for glaucoma detection are also included in the survey. Finally, the authors provide their views, expectations and the limitations of the future of these techniques in the clinical practice.