A survey on medical image analysis in diabetic retinopathy

Attention-based deep learning framework for automatic fundus image processing to aid in diabetic retinopathy grading

BACKGROUND AND OBJECTIVE

Early detection and grading of Diabetic Retinopathy (DR) is essential to determine an adequate treatment and prevent severe vision loss. However, the manual analysis of fundus images is time consuming and DR screening programs are challenged by the availability of human graders. Current automatic approaches for DR grading attempt the joint detection of all signs at the same time. However, the classification can be optimized if red lesions and bright lesions are independently processed since the task gets divided and simplified. Furthermore, clinicians would greatly benefit from explainable artificial intelligence (XAI) to support the automatic model predictions, especially when the type of lesion is specified. As a novelty, we propose an end-to-end deep learning framework for automatic DR grading (5 severity degrees) based on separating the attention of the dark structures from the bright structures of the retina. As the main contribution, this approach allowed us to generate independent interpretable attention maps for red lesions, such as microaneurysms and hemorrhages, and bright lesions, such as hard exudates, while using image-level labels only.

METHODS

Our approach is based on a novel attention mechanism which focuses separately on the dark and the bright structures of the retina by performing a previous image decomposition. This mechanism can be seen as a XAI approach which generates independent attention maps for red lesions and bright lesions. The framework includes an image quality assessment stage and deep learning-related techniques, such as data augmentation, transfer learning and fine-tuning. We used the architecture Xception as a feature extractor and the focal loss function to deal with data imbalance.

RESULTS

The Kaggle DR detection dataset was used for method development and validation. The proposed approach achieved 83.7 % accuracy and a Quadratic Weighted Kappa of 0.78 to classify DR among 5 severity degrees, which outperforms several state-of-the-art approaches. Nevertheless, the main result of this work is the generated attention maps, which reveal the pathological regions on the image distinguishing the red lesions and the bright lesions. These maps provide explainability to the model predictions.

CONCLUSIONS

Our results suggest that our framework is effective to automatically grade DR. The separate attention approach has proven useful for optimizing the classification. On top of that, the obtained attention maps facilitate visual interpretation for clinicians. Therefore, the proposed method could be a diagnostic aid for the early detection and grading of DR.

Source-free active domain adaptation for diabetic retinopathy grading based on ultra-wide-field fundus images

Domain adaptation (DA) is commonly employed in diabetic retinopathy (DR) grading using unannotated fundus images, allowing knowledge transfer from labeled color fundus images. Existing DAs often struggle with domain disparities, hindering DR grading performance compared to clinical diagnosis. A source-free active domain adaptation method (SFADA), which generates features of color fundus images by noise, selects valuable ultra-wide-field (UWF) fundus images through local representation matching, and adapts models using DR lesion prototypes, is proposed to upgrade DR diagnostic accuracy. Importantly, SFADA enhances data security and patient privacy by excluding source domain data. It reduces image resolution and boosts model training speed by modeling DR grade relationships directly. Experiments show SFADA significantly improves DR grading performance, increasing accuracy by 20.90% and quadratic weighted kappa by 18.63% over baseline, reaching 85.36% and 92.38%, respectively. This suggests SFADA's promise for real clinical applications.

A generative adversarial neural network with multi-attention feature extraction for fundus lesion segmentation

PURPOSE

Fundus lesion segmentation determines the location and size of diabetes retinopathy in fundus image, which assists doctors in developing the best eye treatment plan. However, owing to the scattered distribution and the similarity of lesions, it is extremely difficult to extract representative lesions feature and accurately segment lesions area.

METHODS

To solve the thorny problem, a generative adversarial network with multi-attention feature extraction is developed to segment diabetic retinopathy region. The main contributions are as follows: (1) An improved residual U-Net network combining with self-attention mechanism is designed as generative network to fully extract local and global feature of lesions while reducing the loss of key feature information. Considering the correlation between the same lesions feature of different samples, external attention mechanism is introduced in the residual U-Net network to focus on the relevant features of the same lesions in different samples throughout the entire dataset. (2) A discriminative network based on the PatchGAN structure is designed to further enhance the segmentation ability of generation network by discriminating between true and false samples.

RESULTS

The proposed network is evaluated on the public dataset IDRiD, which achieved the Dice correlation coefficients of 75.7%, 76.53%, 50.06%, and 45.89% for EX, SE, MA, and HE, respectively.

CONCLUSION

The experimental results show the generative adversarial neural network qualified for accurate segmentation of diabetic retinopathy from fundus image well.

Novel in-house knowledge-based automated planning system for lung cancer treated with intensity-modulated radiotherapy

PURPOSE

The goal of this study was to propose a knowledge-based planning system which could automatically design plans for lung cancer patients treated with intensity-modulated radiotherapy (IMRT).

METHODS AND MATERIALS

From May 2018 to June 2020, 612 IMRT treatment plans of lung cancer patients were retrospectively selected to construct a planning database. Knowledge-based planning (KBP) architecture named αDiar was proposed in this study. It consisted of two parts separated by a firewall. One was the in-hospital workstation, and the other was the search engine in the cloud. Based on our previous study, A‑Net in the in-hospital workstation was used to generate predicted virtual dose images. A search engine including a three-dimensional convolutional neural network (3D CNN) was constructed to derive the feature vectors of dose images. By comparing the similarity of the features between virtual dose images and the clinical dose images in the database, the most similar feature was found. The optimization parameters (OPs) of the treatment plan corresponding to the most similar feature were assigned to the new plan, and the design of a new treatment plan was automatically completed. After αDiar was developed, we performed two studies. The first retrospective study was conducted to validate whether this architecture was qualified for clinical practice and involved 96 patients. The second comparative study was performed to investigate whether αDiar could assist dosimetrists in improving the quality of planning for the patients. Two dosimetrists were involved and designed plans for only one trial with and without αDiar; 26 patients were involved in this study.

RESULTS

The first study showed that about 54% (52/96) of the automatically generated plans would achieve the dosimetric constraints of the Radiation Therapy Oncology Group (RTOG) and about 93% (89/96) of the automatically generated plans would achieve the dosimetric constraints of the National Comprehensive Cancer Network (NCCN). The second study showed that the quality of treatment planning designed by junior dosimetrists was improved with the help of αDiar.

CONCLUSIONS

Our results showed that αDiar was an effective tool to improve planning quality. Over half of the patients' plans could be designed automatically. For the remaining patients, although the automatically designed plans did not fully meet the clinical requirements, their quality was also better than that of manual plans.

Secure medical image transmission using deep neural network in e-health applications

Recently, medical technologies have developed, and the diagnosis of diseases through medical images has become very important. Medical images often pass through the branches of the network from one end to the other. Hence, high-level security is required. Problems arise due to unauthorized use of data in the image. One of the methods used to secure data in the image is encryption, which is one of the most effective techniques in this field. Confusion and diffusion are the two main steps addressed here. The contribution here is the adaptation of the deep neural network by the weight that has the highest impact on the output, whether it is an intermediate output or a semi-final output in additional to a chaotic system that is not detectable using deep neural network algorithm. The colour and grayscale images were used in the proposed method by dividing the images according to the Region of Interest by the deep neural network algorithm. The algorithm was then used to generate random numbers to randomly create a chaotic system based on the replacement of columns and rows, and randomly distribute the pixels on the designated area. The proposed algorithm evaluated in several ways, and compared with the existing methods to prove the worth of the proposed method.

Elastic Deformation of Optical Coherence Tomography Images of Diabetic Macular Edema for Deep-Learning Models Training: How Far to Go?

- Objective: To explore the clinical validity of elastic deformation of optical coherence tomography (OCT) images for data augmentation in the development of deep-learning model for detection of diabetic macular edema (DME).

METHODS

Prospective evaluation of OCT images of DME (n = 320) subject to elastic transformation, with the deformation intensity represented by ([Formula: see text]). Three sets of images, each comprising 100 pairs of scans (100 original & 100 modified), were grouped according to the range of ([Formula: see text]), including low-, medium- and high-degree of augmentation; ([Formula: see text] = 1-6), ([Formula: see text] = 7-12), and ([Formula: see text] = 13-18), respectively. Three retina specialists evaluated all datasets in a blinded manner and designated each image as 'original' versus 'modified'. The rate of assignment of 'original' value to modified images (false-negative) was determined for each grader in each dataset.

RESULTS

The false-negative rates ranged between 71-77% for the low-, 63-76% for the medium-, and 50-75% for the high-augmentation categories. The corresponding rates of correct identification of original images ranged between 75-85% ([Formula: see text]0.05) in the low-, 73-85% ([Formula: see text]0.05 for graders 1 & 2, p = 0.01 for grader 3) in the medium-, and 81-91% ([Formula: see text]) in the high-augmentation categories. In the subcategory ([Formula: see text] = 7-9) the false-negative rates were 93-83%, whereas the rates of correctly identifying original images ranged between 89-99% ([Formula: see text]0.05 for all graders).

CONCLUSIONS

Deformation of low-medium intensity ([Formula: see text] = 1-9) may be applied without compromising OCT image representativeness in DME. Clinical and Translational Impact Statement-Elastic deformation may efficiently augment the size, robustness, and diversity of training datasets without altering their clinical value, enhancing the development of high-accuracy algorithms for automated interpretation of OCT images.

Pathophysiology and diagnosis of diabetic retinopathy: a narrative review

Diabetes is an endocrine disorder which is known by abnormal high blood glucose levels. There are two main categories of diabetes: type I (10%-15%) and type II (85%-90%). Although type II is more common, type I is the most common form in children. Diabetic retinopathy (DR), which remains the foremost cause of losing vision in working-age populations, can be considered as the main complication of diabetes mellitus. So choosing the best method for diagnosing, tracking, and treating the DR is vital to enhance the quality of life and decrease the medical expenses. Each method for diagnosing DR has some advantages and the best way must be selected according to the points that we need to find. For writing this manuscript, we made a list of relevant keywords including diabetes, DR, pathophysiology, ultrawide field imaging, fluorescein angiography, optical coherence tomography, and optical coherence tomography-angiography, and then we started searching for studies in PubMed, Scopus, and Web of Science databases. This review article covers the pathophysiology of DR and medical imaging techniques to monitor DR. First, we introduce DR and its pathophysiology and then we present the medical imaging techniques to monitor it.

Blood flow perfusion in visual pathway detected by arterial spin labeling magnetic resonance imaging for differential diagnosis of ocular ischemic syndrome

Background

Ocular ischemic syndrome (OIS), attributable to chronic hypoperfusion caused by marked carotid stenosis, is one of the important factors that cause ocular neurodegenerative diseases such as optic atrophy. The current study aimed to detect blood flow perfusion in a visual pathway by arterial spin labeling (ASL) and magnetic resonance imaging (MRI) for the differential diagnosis of OIS.

Methods

This diagnostic, cross-sectional study at a single institution was performed to detect blood flow perfusion in a visual pathway based on 3D pseudocontinuous ASL (3D-pCASL) using 3.0T MRI. A total of 91 participants (91 eyes) consisting of 30 eyes with OIS and 61 eyes with noncarotid artery stenosis-related retinal vascular diseases (39 eyes with diabetic retinopathy and 22 eyes with high myopic retinopathy) were consecutively included. Blood flow perfusion values in visual pathways derived from regions of interest in ASL images, including the retinal-choroidal complex, the intraorbital segments of the optic nerve, the tractus optics, and the visual center, were obtained and compared with arm-retinal circulation time and retinal circulation time derived from fundus fluorescein angiography (FFA). Receiver operating characteristic (ROC) curve analyses and the intraclass correlation coefficient (ICC) were performed to evaluate the accuracy and consistency.

Results

Patients with OIS had the lowest blood flow perfusion values in the visual pathway (all p < 0.05). The relative intraorbital segments of optic nerve blood flow values at post-labeling delays (PLDs) of 1.5 s (area under the curve, AUC = 0.832) and the relative retinal-choroidal complex blood flow values at PLDs of 2.5 s (AUC = 0.805) were effective for the differential diagnosis of OIS. The ICC of the blood flow values derived from the retinal-choroidal complex and the intraorbital segments of the optic nerve between the two observers showed satisfactory concordance (all ICC > 0.932, p < 0.001). The adverse reaction rates of ASL and FFA were 2.20 and 3.30%, respectively.

Conclusion

3D-pCASL showed that the participants with OIS had lower blood flow perfusion values in the visual pathway, which presented satisfactory accuracy, reproducibility, and safety. It is a noninvasive and comprehensive differential diagnostic tool to assess blood flow perfusion in a visual pathway for the differential diagnosis of OIS.

Artificial intelligence in screening, diagnosis, and classification of diabetic macular edema: A systematic review

We review the application of artificial intelligence (AI) techniques in the screening, diagnosis, and classification of diabetic macular edema (DME) by searching six databases- PubMed, Scopus, Web of Science, Science Direct, IEEE, and ACM- from January 1, 2005 to July 4, 2021. A total of 879 articles were extracted, and by applying inclusion and exclusion criteria, 38 articles were selected for more evaluation. The methodological quality of included studies was evaluated using the Quality Assessment for Diagnostic Accuracy Studies (QUADAS-2). We provide an overview of the current state of various AI techniques for DME screening, diagnosis, and classification using retinal imaging modalities such as optical coherence tomography (OCT) and color fundus photography (CFP). Based on our findings, deep learning models have an extraordinary capacity to provide an accurate and efficient system for DME screening and diagnosis. Using these in the processing of modalities leads to a significant increase in sensitivity and specificity values. The use of decision support systems and applications based on AI in processing retinal images provided by OCT and CFP increases the sensitivity and specificity in DME screening and detection.

Developments in the detection of diabetic retinopathy: a state-of-the-art review of computer-aided diagnosis and machine learning methods

The exponential increase in the number of diabetics around the world has led to an equally large increase in the number of diabetic retinopathy (DR) cases which is one of the major complications caused by diabetes. Left unattended, DR worsens the vision and would lead to partial or complete blindness. As the number of diabetics continue to increase exponentially in the coming years, the number of qualified ophthalmologists need to increase in tandem in order to meet the demand for screening of the growing number of diabetic patients. This makes it pertinent to develop ways to automate the detection process of DR. A computer aided diagnosis system has the potential to significantly reduce the burden currently placed on the ophthalmologists. Hence, this review paper is presented with the aim of summarizing, classifying, and analyzing all the recent development on automated DR detection using fundus images from 2015 up to this date. Such work offers an unprecedentedly thorough review of all the recent works on DR, which will potentially increase the understanding of all the recent studies on automated DR detection, particularly on those that deploys machine learning algorithms. Firstly, in this paper, a comprehensive state-of-the-art review of the methods that have been introduced in the detection of DR is presented, with a focus on machine learning models such as convolutional neural networks (CNN) and artificial neural networks (ANN) and various hybrid models. Each AI will then be classified according to its type (e.g. CNN, ANN, SVM), its specific task(s) in performing DR detection. In particular, the models that deploy CNN will be further analyzed and classified according to some important properties of the respective CNN architectures of each model. A total of 150 research articles related to the aforementioned areas that were published in the recent 5 years have been utilized in this review to provide a comprehensive overview of the latest developments in the detection of DR.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10462-022-10185-6.

Enhancing diabetic retinopathy classification using deep learning

Prolonged hyperglycemia can cause diabetic retinopathy (DR), which is a major contributor to blindness. Numerous incidences of DR may be avoided if it were identified and addressed promptly. Throughout recent years, many deep learning (DL)-based algorithms have been proposed to facilitate psychometric testing. Utilizing DL model that encompassed four scenarios, DR and its stages were identified in this study using retinal scans from the "Asia Pacific Tele-Ophthalmology Society (APTOS) 2019 Blindness Detection" dataset. Adopting a DL model then led to the use of augmentation strategies that produced a comprehensive dataset with consistent hyper parameters across all test cases. As a further step in the classification process, we used a Convolutional Neural Network model. Different enhancement methods have been used to raise visual quality. The proposed approach detected the DR with a highest experimental result of 97.83%, a top-2 accuracy of 99.31%, and a top-3 accuracy of 99.88% across all the 5 severity stages of the APTOS 2019 evaluation employing CLAHE and ESRGAN techniques for image enhancement. In addition, we employed APTOS 2019 to develop a set of evaluation metrics (precision, recall, and F1-score) to use in analyzing the efficacy of the suggested model. The proposed approach was also proven to be more efficient at DR location than both state-of-the-art technology and conventional DL.

Recent trends and advances in fundus image analysis: A review

Automated retinal image analysis holds prime significance in the accurate diagnosis of various critical eye diseases that include diabetic retinopathy (DR), age-related macular degeneration (AMD), atherosclerosis, and glaucoma. Manual diagnosis of retinal diseases by ophthalmologists takes time, effort, and financial resources, and is prone to error, in comparison to computer-aided diagnosis systems. In this context, robust classification and segmentation of retinal images are primary operations that aid clinicians in the early screening of patients to ensure the prevention and/or treatment of these diseases. This paper conducts an extensive review of the state-of-the-art methods for the detection and segmentation of retinal image features. Existing notable techniques for the detection of retinal features are categorized into essential groups and compared in depth. Additionally, a summary of quantifiable performance measures for various important stages of retinal image analysis, such as image acquisition and preprocessing, is provided. Finally, the widely used in the literature datasets for analyzing retinal images are described and their significance is emphasized.

Retinal optical coherence tomography image analysis by a restricted Boltzmann machine

Optical coherence tomography (OCT) is an emerging imaging technique for ophthalmic disease diagnosis. Two major problems in OCT image analysis are image enhancement and image segmentation. Deep learning methods have achieved excellent performance in image analysis. However, most of the deep learning-based image analysis models are supervised learning-based approaches and need a high volume of training data (e.g., reference clean images for image enhancement and accurate annotated images for segmentation). Moreover, acquiring reference clean images for OCT image enhancement and accurate annotation of the high volume of OCT images for segmentation is hard. So, it is difficult to extend these deep learning methods to the OCT image analysis. We propose an unsupervised learning-based approach for OCT image enhancement and abnormality segmentation, where the model can be trained without reference images. The image is reconstructed by Restricted Boltzmann Machine (RBM) by defining a target function and minimizing it. For OCT image enhancement, each image is independently learned by the RBM network and is eventually reconstructed. In the reconstruction phase, we use the ReLu function instead of the Sigmoid function. Reconstruction of images given by the RBM network leads to improved image contrast in comparison to other competitive methods in terms of contrast to noise ratio (CNR). For anomaly detection, hyper-reflective foci (HF) as one of the first signs in retinal OCTs of patients with diabetic macular edema (DME) are identified based on image reconstruction by RBM and post-processing by removing the HFs candidates outside the area between the first and the last retinal layers. Our anomaly detection method achieves a high ability to detect abnormalities.

A systematic review of transfer learning-based approaches for diabetic retinopathy detection

Cases of diabetes and related diabetic retinopathy (DR) have been increasing at an alarming rate in modern times. Early detection of DR is an important problem since it may cause permanent blindness in the late stages. In the last two decades, many different approaches have been applied in DR detection. Reviewing academic literature shows that deep neural networks (DNNs) have become the most preferred approach for DR detection. Among these DNN approaches, Convolutional Neural Network (CNN) models are the most used ones in the field of medical image classification. Designing a new CNN architecture is a tedious and time-consuming approach. Additionally, training an enormous number of parameters is also a difficult task. Due to this reason, instead of training CNNs from scratch, using pre-trained models has been suggested in recent years as transfer learning approach. Accordingly, the present study as a review focuses on DNN and Transfer Learning based applications of DR detection considering 43 publications between 2015 and 2021. The published papers are summarized using 3 figures and 10 tables, giving information about 29 pre-trained CNN models, 13 DR data sets and standard performance metrics.

Multimodal image encoding pre-training for diabetic retinopathy grading

Diabetic retinopathy is an increasingly prevalent eye disorder that can lead to severe vision impairment. The severity grading of the disease using retinal images is key to provide an adequate treatment. However, in order to learn the diverse patterns and complex relations that are required for the grading, deep neural networks require very large annotated datasets that are not always available. This has been typically addressed by reusing networks that were pre-trained for natural image classification, hence relying on additional annotated data from a different domain. In contrast, we propose a novel pre-training approach that takes advantage of unlabeled multimodal visual data commonly available in ophthalmology. The use of multimodal visual data for pre-training purposes has been previously explored by training a network in the prediction of one image modality from another. However, that approach does not ensure a broad understanding of the retinal images, given that the network may exclusively focus on the similarities between modalities while ignoring the differences. Thus, we propose a novel self-supervised pre-training that explicitly teaches the networks to learn the common characteristics between modalities as well as the characteristics that are exclusive to the input modality. This provides a complete comprehension of the input domain and facilitates the training of downstream tasks that require a broad understanding of the retinal images, such as the grading of diabetic retinopathy. To validate and analyze the proposed approach, we performed an exhaustive experimentation on different public datasets. The transfer learning performance for the grading of diabetic retinopathy is evaluated under different settings while also comparing against previous state-of-the-art pre-training approaches. Additionally, a comparison against relevant state-of-the-art works for the detection and grading of diabetic retinopathy is also provided. The results show a satisfactory performance of the proposed approach, which outperforms previous pre-training alternatives in the grading of diabetic retinopathy.

Evaluating a Deep Learning Diabetic Retinopathy Grading System Developed on Mydriatic Retinal Images When Applied to Non-Mydriatic Community Screening

Artificial Intelligence has showcased clear capabilities to automatically grade diabetic retinopathy (DR) on mydriatic retinal images captured by clinical experts on fixed table-top retinal cameras within hospital settings. However, in many low- and middle-income countries, screening for DR revolves around minimally trained field workers using handheld non-mydriatic cameras in community settings. This prospective study evaluated the diagnostic accuracy of a deep learning algorithm developed using mydriatic retinal images by the Singapore Eye Research Institute, commercially available as Zeiss VISUHEALTH-AI DR, on images captured by field workers on a Zeiss Visuscout^® 100 non-mydriatic handheld camera from people with diabetes in a house-to-house cross-sectional study across 20 regions in India. A total of 20,489 patient eyes from 11,199 patients were used to evaluate algorithm performance in identifying referable DR, non-referable DR, and gradability. For each category, the algorithm achieved precision values of 29.60 (95% CI 27.40, 31.88), 92.56 (92.13, 92.97), and 58.58 (56.97, 60.19), recall values of 62.69 (59.17, 66.12), 85.65 (85.11, 86.18), and 65.06 (63.40, 66.69), and F-score values of 40.22 (38.25, 42.21), 88.97 (88.62, 89.31), and 61.65 (60.50, 62.80), respectively. Model performance reached 91.22 (90.79, 91.64) sensitivity and 65.06 (63.40, 66.69) specificity at detecting gradability and 72.08 (70.68, 73.46) sensitivity and 85.65 (85.11, 86.18) specificity for the detection of all referable eyes. Algorithm accuracy is dependent on the quality of acquired retinal images, and this is a major limiting step for its global implementation in community non-mydriatic DR screening using handheld cameras. This study highlights the need to develop and train deep learning-based screening tools in such conditions before implementation.

CARNet: Cascade attentive RefineNet for multi-lesion segmentation of diabetic retinopathy images

Diabetic retinopathy is the leading cause of blindness in working population. Lesion segmentation from fundus images helps ophthalmologists accurately diagnose and grade of diabetic retinopathy. However, the task of lesion segmentation is full of challenges due to the complex structure, the various sizes and the interclass similarity with other fundus tissues. To address the issue, this paper proposes a cascade attentive RefineNet (CARNet) for automatic and accurate multi-lesion segmentation of diabetic retinopathy. It can make full use of the fine local details and coarse global information from the fundus image. CARNet is composed of global image encoder, local image encoder and attention refinement decoder. We take the whole image and the patch image as the dual input, and feed them to ResNet50 and ResNet101, respectively, for downsampling to extract lesion features. The high-level refinement decoder uses dual attention mechanism to integrate the same-level features in the two encoders with the output of the low-level attention refinement module for multiscale information fusion, which focus the model on the lesion area to generate accurate predictions. We evaluated the segmentation performance of the proposed CARNet on the IDRiD, E-ophtha and DDR data sets. Extensive comparison experiments and ablation studies on various data sets demonstrate the proposed framework outperforms the state-of-the-art approaches and has better accuracy and robustness. It not only overcomes the interference of similar tissues and noises to achieve accurate multi-lesion segmentation, but also preserves the contour details and shape features of small lesions without overloading GPU memory usage.

Comparisons of deep learning algorithms for diagnosing bacterial keratitis via external eye photographs

Bacterial keratitis (BK), a painful and fulminant bacterial infection of the cornea, is the most common type of vision-threatening infectious keratitis (IK). A rapid clinical diagnosis by an ophthalmologist may often help prevent BK patients from progression to corneal melting or even perforation, but many rural areas cannot afford an ophthalmologist. Thanks to the rapid development of deep learning (DL) algorithms, artificial intelligence via image could provide an immediate screening and recommendation for patients with red and painful eyes. Therefore, this study aims to elucidate the potentials of different DL algorithms for diagnosing BK via external eye photos. External eye photos of clinically suspected IK were consecutively collected from five referral centers. The candidate DL frameworks, including ResNet50, ResNeXt50, DenseNet121, SE-ResNet50, EfficientNets B0, B1, B2, and B3, were trained to recognize BK from the photo toward the target with the greatest area under the receiver operating characteristic curve (AUROC). Via five-cross validation, EfficientNet B3 showed the most excellent average AUROC, in which the average percentage of sensitivity, specificity, positive predictive value, and negative predictive value was 74, 64, 77, and 61. There was no statistical difference in diagnostic accuracy and AUROC between any two of these DL frameworks. The diagnostic accuracy of these models (ranged from 69 to 72%) is comparable to that of the ophthalmologist (66% to 74%). Therefore, all these models are promising tools for diagnosing BK in first-line medical care units without ophthalmologists.

Classification of diabetic retinopathy using unlabeled data and knowledge distillation

Over the last decade, advances in Machine Learning and Artificial Intelligence have highlighted their potential as a diagnostic tool in the healthcare domain. Despite the widespread availability of medical images, their usefulness is severely hampered by a lack of access to labeled data. For example, while Convolutional Neural Networks (CNNs) have emerged as an essential analytical tool in image processing, their impact is curtailed by training limitations due to insufficient labeled data availability. Transfer Learning enables models developed for one task to be reused for a second task. Knowledge distillation enables transferring knowledge from a pre-trained model to another. However, it suffers from limitations, and the two models' constraints need to be architecturally similar. Knowledge distillation addresses some of the shortcomings of transfer learning by generalizing a complex model to a lighter model. However, some parts of the knowledge may not be distilled by knowledge distillation sufficiently. In this paper, a novel knowledge distillation approach using transfer learning is proposed. The proposed approach transfers the complete knowledge of a model to a new smaller one. Unlabeled data are used in an unsupervised manner to transfer the new smaller model's maximum amount of knowledge. The proposed method can be beneficial in medical image analysis, where labeled data are typically scarce. The proposed approach is evaluated in classifying images for diagnosing Diabetic Retinopathy on two publicly available datasets, including Messidor and EyePACS. Simulation results demonstrate that the approach effectively transfers knowledge from a complex model to a lighter one. Furthermore, experimental results illustrate that different small models' performance is improved significantly using unlabeled data and knowledge distillation.

An interpretable multiple-instance approach for the detection of referable diabetic retinopathy in fundus images

Diabetic retinopathy (DR) is one of the leading causes of vision loss across the world. Yet despite its wide prevalence, the majority of affected people lack access to the specialized ophthalmologists and equipment required for monitoring their condition. This can lead to delays in the start of treatment, thereby lowering their chances for a successful outcome. Machine learning systems that automatically detect the disease in eye fundus images have been proposed as a means of facilitating access to retinopathy severity estimates for patients in remote regions or even for complementing the human expert’s diagnosis. Here we propose a machine learning system for the detection of referable diabetic retinopathy in fundus images, which is based on the paradigm of multiple-instance learning. Our method extracts local information independently from multiple rectangular image patches and combines it efficiently through an attention mechanism that focuses on the abnormal regions of the eye (i.e. those that contain DR-induced lesions), thus resulting in a final image representation that is suitable for classification. Furthermore, by leveraging the attention mechanism our algorithm can seamlessly produce informative heatmaps that highlight the regions where the lesions are located. We evaluate our approach on the publicly available Kaggle, Messidor-2 and IDRiD retinal image datasets, in which it exhibits near state-of-the-art classification performance (AUC of 0.961 in Kaggle and 0.976 in Messidor-2), while also producing valid lesion heatmaps (AUPRC of 0.869 in the 81 images of IDRiD that contain pixel-level lesion annotations). Our results suggest that the proposed approach provides an efficient and interpretable solution against the problem of automated diabetic retinopathy grading.

Bone marrow mesenchymal stem cells-induced exosomal microRNA-486-3p protects against diabetic retinopathy through TLR4/NF-κB axis repression

AIM

Diabetic retinopathy (DR) is a chronic disease causing health and economic burdens on individuals and society. Thus, this study is conducted to figure out the mechanisms of bone marrow mesenchymal stem cells (BMSCs)-induced exosomal microRNA-486-3p (miR-486-3p) in DR.

METHODS

The putative miR-486-3p binding sites to 3'untranslated region of Toll-like receptor 4 (TLR4) was verified by luciferase reporter assay. High glucose (HG)-treated Muller cells were transfected with miR-486-3p or TLR4-related oligonucleotides and plasmids to explore theirs functions in DR. Additionally, HG-treated Muller cells were co-cultured with BMSC-derived exosomes, exosomes collected from BMSCs that had been transfected with miR-486-3p or TLR4-related oligonucleotides and plasmids to explore their functions in DR. MiR-486-3p, TLR4 and nuclear factor-kappaB (NF-κB) expression, angiogenesis-related factors, oxidative stress factors, viability and apoptosis in HG-treated Muller cells were detected by RT-qPCR, western blot analysis, ELISA, MTT assay and flow cytometry, respectively.

RESULTS

MiR-486-3p was poorly expressed while TLR4 and NF-κB were highly expressed in HG-treated Muller cells. TLR4 was a target of miR-486-3p. Upregulating miR-486-3p or down-regulating TLR4 inhibited oxidative stress, inflammation and apoptosis, and promoted proliferation of HG-treated Muller cells. Meanwhile, BMSC-derived exosomes inhibited oxidative stress, inflammation and apoptosis, and promoted proliferation of HG-treated Muller cells. Restoring miR-486-3p further enhanced, while up-regulating TLR4 reversed, the improvement of exosomes treatment.

CONCLUSION

Our study highlights that up-regulation of miR-486-3p induced by BMSC-derived exosomes played a protective role in DR mice via TLR4/NF-κB axis repression.

An Enhanced Histopathology Analysis: An AI-Based System for Multiclass Grading of Oral Squamous Cell Carcinoma and Segmenting of Epithelial and Stromal Tissue

Oral squamous cell carcinoma is most frequent histological neoplasm of head and neck cancers, and although it is localized in a region that is accessible to see and can be detected very early, this usually does not occur. The standard procedure for the diagnosis of oral cancer is based on histopathological examination, however, the main problem in this kind of procedure is tumor heterogeneity where a subjective component of the examination could directly impact patient-specific treatment intervention. For this reason, artificial intelligence (AI) algorithms are widely used as computational aid in the diagnosis for classification and segmentation of tumors, in order to reduce inter- and intra-observer variability. In this research, a two-stage AI-based system for automatic multiclass grading (the first stage) and segmentation of the epithelial and stromal tissue (the second stage) from oral histopathological images is proposed in order to assist the clinician in oral squamous cell carcinoma diagnosis. The integration of Xception and SWT resulted in the highest classification value of 0.963 (σ = 0.042) AUCmacro and 0.966 (σ = 0.027) AUCmicro while using DeepLabv3+ along with Xception_65 as backbone and data preprocessing, semantic segmentation prediction resulted in 0.878 (σ = 0.027) mIOU and 0.955 (σ = 0.014) F1 score. Obtained results reveal that the proposed AI-based system has great potential in the diagnosis of OSCC.

Brain Tumor Classification & Segmentation by Using Advanced DNN, CNN & ResNet-50 Neural Networks

In the medical domain, brain image classification is an extremely challenging field. Medical images play a vital role in making the doctor's precise diagnosis and in the surgery process. Adopting intelligent algorithms makes it feasible to detect the lesions of medical images quickly, and it is especially necessary to extract features from medical images. Several studies have integrated multiple algorithms toward medical images domain. Concerning feature extraction from the medical image, a vast amount of data is analyzed to achieve processing results, helping physicians deliver more precise case diagnoses. Image processing mechanism becomes extensive usage in medical science to advance the early detection and treatment aspects. In this aspect, this paper takes tumor, and healthy images as the research object and primarily performs image processing and data augmentation process to feed the dataset to the neural networks. Deep neural networks (DNN), to date, have shown outstanding achievement in classification and segmentation tasks. Carrying this concept into consideration, in this study, we adopted a pre-trained model Resnet_50 for image analysis. The paper proposed three diverse neural networks, particularly DNN, CNN, and ResNet-50. Finally, the splitting dataset is individually assigned to each simplified neural network. Once the image is classified as a tumor accurately, the OTSU segmentation is employed to extract the tumor alone. It can be examined from the experimental outcomes that the ResNet-50 algorithm shows high accuracy 0.996, precision 1.00 with best F1 score 1.0, and minimum test losses of 0.0269 in terms of Brain tumor classification. Extensive experiments prove our offered tumor detection segmentation efficiency and accuracy. To this end, our approach is comprehensive sufficient and only requires minimum pre-and post-processing, which allows its adoption in various medical image classification & segmentation tasks.

Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography

Automatic semantic segmentation in 2D echocardiography is vital in clinical practice for assessing various cardiac functions and improving the diagnosis of cardiac diseases. However, two distinct problems have persisted in automatic segmentation in 2D echocardiography, namely the lack of an effective feature enhancement approach for contextual feature capture and lack of label coherence in category prediction for individual pixels. Therefore, in this study, we propose a deep learning model, called deep pyramid local attention neural network (PLANet), to improve the segmentation performance of automatic methods in 2D echocardiography. Specifically, we propose a pyramid local attention module to enhance features by capturing supporting information within compact and sparse neighboring contexts. We also propose a label coherence learning mechanism to promote prediction consistency for pixels and their neighbors by guiding the learning with explicit supervision signals. The proposed PLANet was extensively evaluated on the dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS) and sub-EchoNet-Dynamic, which are two large-scale and public 2D echocardiography datasets. The experimental results show that PLANet performs better than traditional and deep learning-based segmentation methods on geometrical and clinical metrics. Moreover, PLANet can complete the segmentation of heart structures in 2D echocardiography in real time, indicating a potential to assist cardiologists accurately and efficiently.