Weakly Supervised Lesion Detection From Fundus Images

A Critical Analysis of Deep Semi-Supervised Learning Approaches for Enhanced Medical Image Classification

Deep semi-supervised learning (DSSL) is a machine learning paradigm that blends supervised and unsupervised learning techniques to improve the performance of various models in computer vision tasks. Medical image classification plays a crucial role in disease diagnosis, treatment planning, and patient care. However, obtaining labeled medical image data is often expensive and time-consuming for medical practitioners, leading to limited labeled datasets. DSSL techniques aim to address this challenge, particularly in various medical image tasks, to improve model generalization and performance. DSSL models leverage both the labeled information, which provides explicit supervision, and the unlabeled data, which can provide additional information about the underlying data distribution. That offers a practical solution to resource-intensive demands of data annotation, and enhances the model’s ability to generalize across diverse and previously unseen data landscapes. The present study provides a critical review of various DSSL approaches and their effectiveness and challenges in enhancing medical image classification tasks. The study categorized DSSL techniques into six classes: consistency regularization method, deep adversarial method, pseudo-learning method, graph-based method, multi-label method, and hybrid method. Further, a comparative analysis of performance for six considered methods is conducted using existing studies. The referenced studies have employed metrics such as accuracy, sensitivity, specificity, AUC-ROC, and F1 score to evaluate the performance of DSSL methods on different medical image datasets. Additionally, challenges of the datasets, such as heterogeneity, limited labeled data, and model interpretability, were discussed and highlighted in the context of DSSL for medical image classification. The current review provides future directions and considerations to researchers to further address the challenges and take full advantage of these methods in clinical practices.

URNet: System for recommending referrals for community screening of diabetic retinopathy based on deep learning

Diabetic retinopathy (DR) will cause blindness if the detection and treatment are not carried out in the early stages. To create an effective treatment strategy, the severity of the disease must first be divided into referral-warranted diabetic retinopathy (RWDR) and non-referral diabetic retinopathy (NRDR). However, there are usually no sufficient fundus examinations due to lack of professional service in the communities, particularly in the developing countries. In this study, we introduce UGAN_Resnet_CBAM (URNet; UGAN is a generative adversarial network that uses Unet for feature extraction), a two-stage end-to-end deep learning technique for the automatic detection of diabetic retinopathy. The characteristics of DDR fundus data set were used to design an adaptive image preprocessing module in the first stage. Gradient-weighted Class Activation Mapping (Grad-CAM) and t-distribution and stochastic neighbor embedding (t-SNE) were used as the evaluation indices to analyze the preprocessing results. In the second stage, we enhanced the performance of the Resnet50 network by integrating the convolutional block attention module (CBAM). The outcomes demonstrate that our proposed solution outperformed other current structures, achieving 94.5% and 94.4% precisions, and 96.2% and 91.9% recall for NRDR and RWDR, respectively.

Individualized Statistical Modeling of Lesions in Fundus Images for Anomaly Detection

Anomaly detection in fundus images remains challenging due to the fact that fundus images often contain diverse types of lesions with various properties in locations, sizes, shapes, and colors. Current methods achieve anomaly detection mainly through reconstructing or separating the fundus image background from a fundus image under the guidance of a set of normal fundus images. The reconstruction methods, however, ignore the constraint from lesions. The separation methods primarily model the diverse lesions with pixel-based independent and identical distributed (i.i.d.) properties, neglecting the individualized variations of different types of lesions and their structural properties. And hence, these methods may have difficulty to well distinguish lesions from fundus image backgrounds especially with the normal personalized variations (NPV). To address these challenges, we propose a patch-based non-i.i.d. mixture of Gaussian (MoG) to model diverse lesions for adapting to their statistical distribution variations in different fundus images and their patch-like structural properties. Further, we particularly introduce the weighted Schatten p-norm as the metric of low-rank decomposition for enhancing the accuracy of the learned fundus image backgrounds and reducing false-positives caused by NPV. With the individualized modeling of the diverse lesions and the background learning, fundus image backgrounds and NPV are finely learned and subsequently distinguished from diverse lesions, to ultimately improve the anomaly detection. The proposed method is evaluated on two real-world databases and one artificial database, outperforming the state-of-the-art methods.

Exudate identification in retinal fundus images using precise textural verifications

One of the most salient diseases of retina is Diabetic Retinopathy (DR) which may lead to irreparable damages to eye vision in the advanced phases. A large number of the people infected with diabetes experience DR. The early identification of DR signs facilitates the treatment process and prevents from blindness. Hard Exudates (HE) are bright lesions appeared in retinal fundus images of DR patients. Thus, the detection of HEs is an important task preventing the progress of DR. However, the detection of HEs is a challenging process due to their different appearance features. In this paper, an automatic method for the identification of HEs with various sizes and shapes is proposed. The method works based on a pixel-wise approach. It considers several semi-circular regions around each pixel. For each semi-circular region, the intensity changes around several directions and non-necessarily equal radiuses are computed. All pixels for which several semi-circular regions include considerable intensity changes are considered as the pixels located in HEs. In order to reduce false positives, an optic disc localization method is proposed in the post-processing phase. The performance of the proposed method has been evaluated on DIARETDB0 and DIARETDB1 datasets. The experimental results confirm the improved performance of the suggested method in term of accuracy.

Weakly Supervised Object Localization and Detection: A Survey

As an emerging and challenging problem in the computer vision community, weakly supervised object localization and detection plays an important role for developing new generation computer vision systems and has received significant attention in the past decade. As methods have been proposed, a comprehensive survey of these topics is of great importance. In this work, we review (1) classic models, (2) approaches with feature representations from off-the-shelf deep networks, (3) approaches solely based on deep learning, and (4) publicly available datasets and standard evaluation metrics that are widely used in this field. We also discuss the key challenges in this field, development history of this field, advantages/disadvantages of the methods in each category, the relationships between methods in different categories, applications of the weakly supervised object localization and detection methods, and potential future directions to further promote the development of this research field.

Anomaly detection in fundus images by self-adaptive decomposition via local and color based sparse coding

Anomaly detection in color fundus images is challenging due to the diversity of anomalies. The current studies detect anomalies from fundus images by learning their background images, however, ignoring the affluent characteristics of anomalies. In this paper, we propose a simultaneous modeling strategy in both sequential sparsity and local and color saliency property of anomalies are utilized for the multi-perspective anomaly modeling. In the meanwhile, the Schatten p-norm based metric is employed to better learn the heterogeneous background images, from where the anomalies are better discerned. Experiments and comparisons demonstrate the outperforming and effectiveness of the proposed method.

Weakly supervised training for eye fundus lesion segmentation in patients with diabetic retinopathy

OBJECTIVE

Diabetic retinopathy is the leading cause of vision loss in working-age adults. Early screening and diagnosis can help to facilitate subsequent treatment and prevent vision loss. Deep learning has been applied in various fields of medical identification. However, current deep learning-based lesion segmentation techniques rely on a large amount of pixel-level labeled ground truth data, which limits their performance and application. In this work, we present a weakly supervised deep learning framework for eye fundus lesion segmentation in patients with diabetic retinopathy.

METHODS

First, an efficient segmentation algorithm based on grayscale and morphological features is proposed for rapid coarse segmentation of lesions. Then, a deep learning model named Residual-Attention Unet (RAUNet) is proposed for eye fundus lesion segmentation. Finally, a data sample of fundus images with labeled lesions and unlabeled images with coarse segmentation results is jointly used to train RAUNet to broaden the diversity of lesion samples and increase the robustness of the segmentation model.

RESULTS

A dataset containing 582 fundus images with labels verified by doctors, including hemorrhage (HE), microaneurysm (MA), hard exudate (EX) and soft exudate (SE), and 903 images without labels was used to evaluate the model. In ablation test, the proposed RAUNet achieved the highest intersection over union (IOU) on the labeled dataset, and the proposed attention and residual modules both improved the IOU of the UNet benchmark. Using both the images labeled by doctors and the proposed coarse segmentation method, the weakly supervised framework based on RAUNet architecture significantly improved the mean segmentation accuracy by over 7% on the lesions.

SIGNIFICANCE

This study demonstrates that combining unlabeled medical images with coarse segmentation results can effectively improve the robustness of the lesion segmentation model and proposes a practical framework for improving the performance of medical image segmentation given limited labeled data samples.

Research on the Segmentation of Biomarker for Chronic Central Serous Chorioretinopathy Based on Multimodal Fundus Image

At present, laser surgery is one of the effective ways to treat the chronic central serous chorioretinopathy (CSCR), in which the location of the leakage area is of great importance. In order to alleviate the pressure on ophthalmologists to manually label the biomarkers as well as elevate the biomarker segmentation quality, a semiautomatic biomarker segmentation method is proposed in this paper, aiming to facilitate the accurate and rapid acquisition of biomarker location information. Firstly, the multimodal fundus images are introduced into the biomarker segmentation task, which can effectively weaken the interference of highlighted vessels in the angiography images to the location of biomarkers. Secondly, a semiautomatic localization technique is adopted to reduce the search range of biomarkers, thus enabling the improvement of segmentation efficiency. On the basis of the above, the low-rank and sparse decomposition (LRSD) theory is introduced to construct the baseline segmentation scheme for segmentation of the CSCR biomarkers. Moreover, a joint segmentation framework consisting of the above method and region growing (RG) method is further designed to improve the performance of the baseline scheme. On the one hand, the LRSD is applied to offer the initial location information of biomarkers for the RG method, so as to ensure that the RG method can capture effective biomarkers. On the other hand, the biomarkers obtained by RG are fused with those gained by LRSD to make up for the defect of undersegmentation of the baseline scheme. Finally, the quantitative and qualitative ablation experiments have been carried out to demonstrate that the joint segmentation framework performs well than the baseline scheme in most cases, especially in the sensitivity and F1-score indicators, which not only confirms the effectiveness of the framework in the CSCR biomarker segmentation scene but also implies its potential application value in CSCR laser surgery.

Red-lesion extraction in retinal fundus images by directional intensity changes' analysis

Diabetic retinopathy (DR) is an important retinal disease threatening people with the long diabetic history. Blood leakage in retina leads to the formation of red lesions in retina the analysis of which is helpful in the determination of severity of disease. In this paper, a novel red-lesion extraction method is proposed. The new method firstly determines the boundary pixels of blood vessel and red lesions. Then, it determines the distinguishing features of boundary pixels of red-lesions to discriminate them from other boundary pixels. The main point utilized here is that a red lesion can be observed as significant intensity changes in almost all directions in the fundus image. This can be feasible through considering special neighborhood windows around the extracted boundary pixels. The performance of the proposed method has been evaluated for three different datasets including Diaretdb0, Diaretdb1 and Kaggle datasets. It is shown that the method is capable of providing the values of 0.87 and 0.88 for sensitivity and specificity of Diaretdb1, 0.89 and 0.9 for sensitivity and specificity of Diaretdb0, 0.82 and 0.9 for sensitivity and specificity of Kaggle. Also, the proposed method has a time-efficient performance in the red-lesion extraction process.

Retinal fundus image enhancement with image decomposition and visual adaptation

Retinal fundus photography has been widely used to diagnose various prevalent diseases because many important diseases manifest themselves on the retina. However, the quality of fundus images obtained from practical clinical environments is not always good enough for diagnosis due to uneven illumination, blurring, low contrast, etc. In this paper, we propose a simple yet efficient method for fundus image enhancement. We first conduct image decomposition to divide the input image into three layers: base, detail, and noise layers; and then illumination correction, detail enhancement and denoising are conducted respectively at these three layers. Specifically, a simple visual adaptation model is used to correct the uneven illumination at the base layer and a weighted fusion is employed to enhance details and suppress noise and artifacts. The proposed method was evaluated on public datasets (DIARETDB0 and DIARETDB1), and also on some challenging images collected by us from the hospital. In addition, quality assessments by ophthalmologists were implemented to further verify the contribution of the proposed method in helping make diagnosis. Experimental results show that the proposed method outperforms other related methods and can simultaneously handle the tasks of illumination correction, detail enhancement and noise (and artifact) suppression.

A Multi-Label Deep Learning Model with Interpretable Grad-CAM for Diabetic Retinopathy Classification

The characteristics of diabetic retinopathy (DR) fundus images generally consist of multiple types of lesions which provided strong evidence for the ophthalmologists to make diagnosis. It is particularly significant to figure out an efficient method to not only accurately classify DR fundus images but also recognize all kinds of lesions on them. In this paper, a deep learning-based multi-label classification model with Gradient-weighted Class Activation Mapping (Grad-CAM) was proposed, which can both make DR classification and automatically locate the regions of different lesions. To reducing laborious annotation work and improve the efficiency of labeling, this paper innovatively considered different types of lesions as different labels for a fundus image so that this paper changed the task of lesion detection into that of image classification. A total of five labels were pre-defined and 3228 fundus images were collected for developing our model. The architecture of deep learning model was designed by ourselves based on ResNet. Through experiments on the test images, this method acquired a sensitive of 93.9% and a specificity of 94.4% on DR classification. Moreover, the corresponding regions of lesions were reasonably outlined on the DR fundus images.

Robust Multiview Subspace Learning With Nonindependently and Nonidentically Distributed Complex Noise

Multiview Subspace Learning (MSL), which aims at obtaining a low-dimensional latent subspace from multiview data, has been widely used in practical applications. Most recent MSL approaches, however, only assume a simple independent identically distributed (i.i.d.) Gaussian or Laplacian noise for all views of data, which largely underestimates the noise complexity in practical multiview data. Actually, in real cases, noises among different views generally have three specific characteristics. First, in each view, the data noise always has a complex configuration beyond a simple Gaussian or Laplacian distribution. Second, the noise distributions of different views of data are generally nonidentical and with evident distinctiveness. Third, noises among all views are nonindependent but obviously correlated. Based on such understandings, we elaborately construct a new MSL model by more faithfully and comprehensively considering all these noise characteristics. First, the noise in each view is modeled as a Dirichlet process (DP) Gaussian mixture model (DPGMM), which can fit a wider range of complex noise types than conventional Gaussian or Laplacian. Second, the DPGMM parameters in each view are different from one another, which encodes the "nonidentical" noise property. Third, the DPGMMs on all views share the same high-level priors by using the technique of hierarchical DP, which encodes the "nonindependent" noise property. All the aforementioned ideas are incorporated into an integrated graphics model which can be appropriately solved by the variational Bayes algorithm. The superiority of the proposed method is verified by experiments on 3-D reconstruction simulations, multiview face modeling, and background subtraction, as compared with the current state-of-the-art MSL methods.