Fast and Automated Hyperreflective Foci Segmentation Based on Image Enhancement and Improved 3D U-Net in SD-OCT Volumes with Diabetic Retinopathy

Exploration on OCT biomarker candidate related to macular edema caused by diabetic retinopathy and retinal vein occlusion in SD-OCT images

To improve the understanding of potential pathological mechanisms of macular edema (ME), we try to discover biomarker candidates related to ME caused by diabetic retinopathy (DR) and retinal vein occlusion (RVO) in spectral-domain optical coherence tomography images by means of deep learning (DL). 32 eyes of 26 subjects with non-proliferative DR (NPDR), 77 eyes of 61 subjects with proliferative DR (PDR), 120 eyes of 116 subjects with branch RVO (BRVO), and 17 eyes of 15 subjects with central RVO (CRVO) were collected. A DL model was implemented to guide biomarker candidate discovery. The disorganization of the retinal outer layers (DROL), i.e., the gray value of the retinal tissues between the external limiting membrane (ELM) and retinal pigment epithelium (RPE), the disrupted and obscured rate of the ELM, ellipsoid zone (EZ), and RPE, was measured. In addition, the occurrence, number, volume, and projected area of hyperreflective foci (HRF) were recorded. ELM, EZ, and RPE are more likely to be obscured in RVO group and HRFs are observed more frequently in DR group (all P ≤ 0.001). In conclusion, the features of DROL and HRF can be possible biomarkers related to ME caused by DR and RVO in OCT modality.

Application and progress of artificial intelligence technology in the segmentation of hyperreflective foci in OCT images for ophthalmic disease research

With the advancement of retinal imaging, hyperreflective foci (HRF) on optical coherence tomography (OCT) images have gained significant attention as potential biological biomarkers for retinal neuroinflammation. However, these biomarkers, represented by HRF, present pose challenges in terms of localization, quantification, and require substantial time and resources. In recent years, the progress and utilization of artificial intelligence (AI) have provided powerful tools for the analysis of biological markers. AI technology enables use machine learning (ML), deep learning (DL) and other technologies to precise characterization of changes in biological biomarkers during disease progression and facilitates quantitative assessments. Based on ophthalmic images, AI has significant implications for early screening, diagnostic grading, treatment efficacy evaluation, treatment recommendations, and prognosis development in common ophthalmic diseases. Moreover, it will help reduce the reliance of the healthcare system on human labor, which has the potential to simplify and expedite clinical trials, enhance the reliability and professionalism of disease management, and improve the prediction of adverse events. This article offers a comprehensive review of the application of AI in combination with HRF on OCT images in ophthalmic diseases including age-related macular degeneration (AMD), diabetic macular edema (DME), retinal vein occlusion (RVO) and other retinal diseases and presents prospects for their utilization.

Diabetic Macular Edema Optical Coherence Tomography Biomarkers Detected with EfficientNetV2B1 and ConvNeXt

(1) Background: Diabetes mellitus (DM) is a growing challenge, both for patients and physicians, in order to control the impact on health and prevent complications. Millions of patients with diabetes require medical attention, which generates problems regarding the limited time for screening but also addressability difficulties for consultation and management. As a result, screening programs for vision-threatening complications due to DM have to be more efficient in the future in order to cope with such a great healthcare burden. Diabetic macular edema (DME) is a severe complication of DM that can be prevented if it is timely screened with the help of optical coherence tomography (OCT) devices. Newly developing state-of-the-art artificial intelligence (AI) algorithms can assist physicians in analyzing large datasets and flag potential risks. By using AI algorithms in order to process OCT images of large populations, the screening capacity and speed can be increased so that patients can be timely treated. This quick response gives the physicians a chance to intervene and prevent disability. (2) Methods: This study evaluated ConvNeXt and EfficientNet architectures in correctly identifying DME patterns on real-life OCT images for screening purposes. (3) Results: Firstly, we obtained models that differentiate between diabetic retinopathy (DR) and healthy scans with an accuracy of 0.98. Secondly, we obtained a model that can indicate the presence of edema, detachment of the subfoveolar neurosensory retina, and hyperreflective foci (HF) without using pixel level annotation. Lastly, we analyzed the extent to which the pretrained weights on natural images "understand" OCT scans. (4) Conclusions: Pretrained networks such as ConvNeXt or EfficientNet correctly identify features relevant to the differentiation between healthy retinas and DR, even though they were pretrained on natural images. Another important aspect of our research is that the differentiation between biomarkers and their localization can be obtained even without pixel-level annotation. The "three biomarkers model" is able to identify obvious subfoveal neurosensory detachments, retinal edema, and hyperreflective foci, as well as very small subfoveal detachments. In conclusion, our study points out the possible usefulness of AI-assisted diagnosis of DME for lowering healthcare costs, increasing the quality of life of patients with diabetes, and reducing the waiting time until an appropriate ophthalmological consultation and treatment can be performed.

Automated evaluation of retinal hyperreflective foci changes in diabetic macular edema patients before and after intravitreal injection

Purpose

Fast and automated reconstruction of retinal hyperreflective foci (HRF) is of great importance for many eye-related disease understanding. In this paper, we introduced a new automated framework, driven by recent advances in deep learning to automatically extract 12 three-dimensional parameters from the segmented hyperreflective foci in optical coherence tomography (OCT).

Methods

Unlike traditional convolutional neural networks, which struggle with long-range feature correlations, we introduce a spatial and channel attention module within the bottleneck layer, integrated into the nnU-Net architecture. Spatial Attention Block aggregates features across spatial locations to capture related features, while Channel Attention Block heightens channel feature contrasts. The proposed model was trained and tested on 162 retinal OCT volumes of patients with diabetic macular edema (DME), yielding robust segmentation outcomes. We further investigate HRF's potential as a biomarker of DME.

Results

Results unveil notable discrepancies in the amount and volume of HRF subtypes. In the whole retinal layer (WR), the mean distance from HRF to the retinal pigmented epithelium was significantly reduced after treatment. In WR, the improvement in central macular thickness resulting from intravitreal injection treatment was positively correlated with the mean distance from HRF subtypes to the fovea.

Conclusion

Our study demonstrates the applicability of OCT for automated quantification of retinal HRF in DME patients, offering an objective, quantitative approach for clinical and research applications.

Segmentation of paracentral acute middle maculopathy lesions in spectral-domain optical coherence tomography images through weakly supervised deep convolutional networks

BACKGROUND AND OBJECTIVES

Spectral-domain optical coherence tomography (SD-OCT) is a valuable tool for non-invasive imaging of the retina, allowing the discovery and visualization of localized lesions, the presence of which is associated with eye diseases. The present study introduces X-Net, a weakly supervised deep-learning framework for automated segmentation of paracentral acute middle maculopathy (PAMM) lesions in retinal SD-OCT images. Despite recent advances in the development of automatic methods for clinical analysis of OCT scans, there remains a scarcity of studies focusing on the automated detection of small retinal focal lesions. Additionally, most existing solutions depend on supervised learning, which can be time-consuming and require extensive image labeling, whereas X-Net offers a solution to these challenges. As far as we can determine, no prior study has addressed the segmentation of PAMM lesions in SD-OCT images.

METHODS

This study leverages 133 SD-OCT retinal images, each containing instances of paracentral acute middle maculopathy lesions. A team of eye experts annotated the PAMM lesions in these images using bounding boxes. Then, labeled data were used to train a U-Net that performs pre-segmentation, producing region labels of pixel-level accuracy. To attain a highly-accurate final segmentation, we introduced X-Net, a novel neural network made up of a master and a slave U-Net. During training, it takes the expert annotated, and pixel-level pre-segment annotated images and employs sophisticated strategies to ensure the highest segmentation accuracy.

RESULTS

The proposed method was rigorously evaluated on clinical retinal images excluded from training and achieved an accuracy of 99% with a high level of similarity between the automatic segmentation and expert annotation, as demonstrated by a mean Intersection-over-Union of 0.8. Alternative methods were tested on the same data. Single-stage neural networks proved insufficient for achieving satisfactory results, confirming that more advanced solutions, such as the proposed method, are necessary. We also found that X-Net using Attention U-net for both the pre-segmentation and X-Net arms for the final segmentation shows comparable performance to the proposed method, suggesting that the proposed approach remains a viable solution even when implemented with variants of the classic U-Net.

CONCLUSIONS

The proposed method exhibits reasonably high performance, validated through quantitative and qualitative evaluations. Medical eye specialists have also verified its validity and accuracy. Thus, it could be a viable tool in the clinical assessment of the retina. Additionally, the demonstrated approach for annotating the training set has proven to be effective in reducing the expert workload.

Application of Artificial Intelligence in Precision Medicine for Diabetic Macular Edema

Diabetic macular edema (DME) is the primary cause of central vision impairment in patients with diabetes and the leading cause of preventable blindness in working-age people. With the advent of optical coherence tomography and antivascular endothelial growth factor (anti-VEGF) therapy, the diagnosis, evaluation, and treatment of DME were greatly revolutionized in the last decade. However, there is tremendous heterogeneity among DME patients, and 30%-50% of DME patients do not respond well to anti-VEGF agents. In addition, there is no evidence-based and universally accepted administration regimen. The identification of DME patients not responding to anti-VEGF agents and the determination of the optimal administration interval are the 2 major challenges of DME, which are difficult to achieve with the coarse granularity of conventional health care modality. Therefore, more and more retina specialists have pointed out the necessity of introducing precision medicine into the management of DME and have conducted related studies in recent years. One of the most frontier methods is the targeted extraction of individualized disease features from optical coherence tomography images based on artificial intelligence technology, which provides precise evaluation and risk classification of DME. This review aims to provide an overview of the progress of artificial intelligence-enabled precision medicine in automated screening, precise evaluation, prognosis prediction, and follow-up monitoring of DME. Further, the challenges ahead of real-world applications and the future development of precision medicine in DME will be discussed.

Fuzzy Logic-Based System for Identifying the Severity of Diabetic Macular Edema from OCT B-Scan Images Using DRIL, HRF, and Cystoids

Diabetic Macular Edema (DME) is a severe ocular complication commonly found in patients with diabetes. The condition can precipitate a significant drop in VA and, in extreme cases, may result in irreversible vision loss. Optical Coherence Tomography (OCT), a technique that yields high-resolution retinal images, is often employed by clinicians to assess the extent of DME in patients. However, the manual interpretation of OCT B-scan images for DME identification and severity grading can be error-prone, with false negatives potentially resulting in serious repercussions. In this paper, we investigate an Artificial Intelligence (AI) driven system that offers an end-to-end automated model, designed to accurately determine DME severity using OCT B-Scan images. This model operates by extracting specific biomarkers such as Disorganization of Retinal Inner Layers (DRIL), Hyper Reflective Foci (HRF), and cystoids from the OCT image, which are then utilized to ascertain DME severity. The rules guiding the fuzzy logic engine are derived from contemporary research in the field of DME and its association with various biomarkers evident in the OCT image. The proposed model demonstrates high efficacy, identifying images with DRIL with 93.3% accuracy and successfully segmenting HRF and cystoids from OCT images with dice similarity coefficients of 91.30% and 95.07% respectively. This study presents a comprehensive system capable of accurately grading DME severity using OCT B-scan images, serving as a potentially invaluable tool in the clinical assessment and treatment of DME.

Boundary-enhanced semi-supervised retinal layer segmentation in optical coherence tomography images using fewer labels

Automatic segmentation of multiple layers in retinal optical coherence tomography (OCT) images is crucial for eye disease diagnosis and treatment. Despite the success of deep learning algorithms, it still remains a challenge due to the blurry layer boundaries and lack of adequate pixel-wise annotations. To tackle these issues, we propose a Boundary-Enhanced Semi-supervised Network (BE-SemiNet) that exploits an auxiliary distance map regression task to improve retinal layer segmentation with scarce labeled data and abundant unlabeled data. Specifically, a novel Unilaterally Truncated Distance Map (UTDM) is firstly introduced to alleviate the class imbalance problem and enhance the layer boundary learning in the regression task. Then for the pixel-wise segmentation and UTDM regression branches, we impose task-level and data-level consistency regularization on unlabeled data to enrich the diversity of unsupervised information and improve the regularization effects. Pseudo supervision is incorporated in consistency regularization to bridge the task prediction spaces for consistency and expand training labeled data. Experiments on two public retinal OCT datasets show that our method can greatly improve the supervised baseline performance with only 5 annotations and outperform the state-of-the-art methods. Since it is difficult and labor-expensive to obtain adequate pixel-wise annotations in practice, our method has a promising application future in clinical retinal OCT image analysis.

Automatic segmentation of hyperreflective dots via focal priors and visual saliency

PURPOSE

Hyperreflective dots (HRDs) can be observed in spectral domain optical coherence tomography (SD-OCT), which can provide a sensitive marker in the treatment decision process. Quantitative analyses of HRDs are the key to make appropriate decisions on observation, treatment, and retreatment. The purpose of this study is to automatically and accurately segment HRDs in SD-OCT B-scans with diabetic retinopathy (DR).

METHODS

The authors propose an automatic segmentation algorithm of HRDs via focal priors and visual saliency. The algorithm is divided into three stages: segmentation of retinal layers, calculation of the multiscale local contrast saliency map, and adaptive threshold segmentation. First, a method based on improved graph search is used to segment retinal layers to obtain the region of interest (ROI) and the reflectivity estimation of the retinal pigment epithelium (RPE) layer; then, the multiscale local contrast saliency map is obtained by using a local contrast measure, which measures the dissimilarity between the current pixels and corresponding neighborhoods; finally, an adaptive threshold is applied to segment HRDs.

RESULTS

Experimental results on 20 SD-OCT B-scans demonstrate that our method is effective for HRDs segmentation. The average dice similarity coefficient (DSC) and detection accuracy are 71.12% and 85.07%, respectively.

CONCLUSIONS

The proposed method can accurately segment HRDs in SD-OCT B-scans with DR and outperforms current state-of-the-art methods. Our method can provide reliable HRDs segmentation to assist ophthalmologists in clinical diagnosis, treatment, disease monitoring, and progression.

Retinal optical coherence tomography image analysis by a restricted Boltzmann machine

Optical coherence tomography (OCT) is an emerging imaging technique for ophthalmic disease diagnosis. Two major problems in OCT image analysis are image enhancement and image segmentation. Deep learning methods have achieved excellent performance in image analysis. However, most of the deep learning-based image analysis models are supervised learning-based approaches and need a high volume of training data (e.g., reference clean images for image enhancement and accurate annotated images for segmentation). Moreover, acquiring reference clean images for OCT image enhancement and accurate annotation of the high volume of OCT images for segmentation is hard. So, it is difficult to extend these deep learning methods to the OCT image analysis. We propose an unsupervised learning-based approach for OCT image enhancement and abnormality segmentation, where the model can be trained without reference images. The image is reconstructed by Restricted Boltzmann Machine (RBM) by defining a target function and minimizing it. For OCT image enhancement, each image is independently learned by the RBM network and is eventually reconstructed. In the reconstruction phase, we use the ReLu function instead of the Sigmoid function. Reconstruction of images given by the RBM network leads to improved image contrast in comparison to other competitive methods in terms of contrast to noise ratio (CNR). For anomaly detection, hyper-reflective foci (HF) as one of the first signs in retinal OCTs of patients with diabetic macular edema (DME) are identified based on image reconstruction by RBM and post-processing by removing the HFs candidates outside the area between the first and the last retinal layers. Our anomaly detection method achieves a high ability to detect abnormalities.

Depth-resolved visualization and automated quantification of hyperreflective foci on OCT scans using optical attenuation coefficients

An automated depth-resolved algorithm using optical attenuation coefficients (OACs) was developed to visualize, localize, and quantify hyperreflective foci (HRF) seen on OCT imaging that are associated with macular hyperpigmentation and represent an increased risk of disease progression in age related macular degeneration. To achieve this, we first transformed the OCT scans to linear representation, which were then contrasted by OACs. HRF were visualized and localized within the entire scan by differentiating HRF within the retina from HRF along the retinal pigment epithelium (RPE). The total pigment burden was quantified using the en face sum projection of an OAC slab between the inner limiting membrane (ILM) to Bruch's membrane (BM). The manual total pigment burden measurements were also obtained by combining manual outlines of HRF in the B-scans with the total area of hypotransmission defects outlined on sub-RPE slabs, which was used as the reference to compare with those obtained from the automated algorithm. 6×6 mm swept-source OCT scans were collected from a total of 49 eyes from 42 patients with macular HRF. We demonstrate that the algorithm was able to automatically distinguish between HRF within the retina and HRF along the RPE. In 24 test eyes, the total pigment burden measurements by the automated algorithm were compared with measurements obtained from manual segmentations. A significant correlation was found between the total pigment area measurements from the automated and manual segmentations (P < 0.001). The proposed automated algorithm based on OACs should be useful in studying eye diseases involving HRF.

Research on Semantic Segmentation Method of Macular Edema in Retinal OCT Images Based on Improved Swin-Unet

Optical coherence tomography (OCT), as a new type of tomography technology, has the characteristics of non-invasive, real-time imaging and high sensitivity, and is currently an important medical imaging tool to assist ophthalmologists in the screening, diagnosis, and follow-up treatment of patients with macular disease. In order to solve the problem of irregular occurrence area of diabetic retinopathy macular edema (DME), multi-scale and multi-region cluster of macular edema, which leads to inaccurate segmentation of the edema area, an improved Swin-Unet networks model was proposed for automatic semantic segmentation of macular edema lesion areas in OCT images. Firstly, in the deep bottleneck of the Swin-Unet network, the Resnet network layer was used to increase the extraction of pairs of sub-feature images. Secondly, the Swin Transformer block and skip connection structure were used for global and local learning, and the regions after semantic segmentation were morphologically smoothed and post-processed. Finally, the proposed method was performed on the macular edema patient dataset publicly available at Duke University, and was compared with previous segmentation methods. The experimental results show that the proposed method can not only improve the overall semantic segmentation accuracy of retinal macular edema, but also further to improve the semantic segmentation effect of multi-scale and multi-region edema regions.

Automatic Segmentation of Macular Edema in Retinal OCT Images Using Improved U-Net++

The number and volume of retinal macular edemas are important indicators for screening and diagnosing retinopathy. Aiming at the problem that the segmentation method of macular edemas in a retinal optical coherence tomography (OCT) image is not ideal in segmentation of diverse edemas, this paper proposes a new method of automatic segmentation of macular edema regions in retinal OCT images using the improved U-Net++. The proposed method makes full use of the U-Net++ re-designed skip pathways and dense convolution block; reduces the semantic gap of the feature maps in the encoder/decoder sub-network; and adds the improved Resnet network as the backbone, which make the extraction of features in the edema regions more accurate and improves the segmentation effect. The proposed method was trained and validated on the public dataset of Duke University, and the experiments demonstrated the proposed method can not only improve the overall segmentation effect, but also can significantly improve the segmented precision for diverse edema in multi-regions, as well as reducing the error of the number of edema regions.