Automated Layer Segmentation of Retinal Optical Coherence Tomography Images Using a Deep Feature Enhanced Structured Random Forests Classifier

Boundary-Repairing Dual-Path Network for Retinal Layer Segmentation in OCT Image with Pigment Epithelial Detachment

Automatic retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis of ocular diseases. Currently, automatic retinal layer segmentation works well with normal OCT images. However, pigment epithelial detachment (PED) dramatically alters the retinal structure, causing blurred boundaries and partial disappearance of the Bruch's Membrane (BM), thus posing challenges to the segmentation. To tackle these problems, we propose a novel dual-path U-shaped network for simultaneous layer segmentation and boundary regression. This network first designs a feature interaction fusion (FIF) module to strengthen the boundary shape constraints in the layer path. To address the challenge posed by partial BM disappearance and boundary-blurring, we propose a layer boundary repair (LBR) module. This module aims to use contrastive loss to enhance the confidence of blurred boundary regions and refine the segmentation of layer boundaries through the re-prediction head. In addition, we introduce a novel bilateral threshold distance map (BTDM) designed for the boundary path. The BTDM serves to emphasize information within boundary regions. This map, combined with the updated probability map, culminates in topology-guaranteed segmentation results achieved through a topology correction (TC) module. We investigated the proposed network on two severely deformed datasets (i.e., OCTA-500 and Aier-PED) and one slightly deformed dataset (i.e., DUKE). The proposed method achieves an average Dice score of 94.26% on the OCTA-500 dataset, which was 1.5% higher than BAU-Net and outperformed other methods. In the DUKE and Aier-PED datasets, the proposed method achieved average Dice scores of 91.65% and 95.75%, respectively.

Simultaneous alignment and surface regression using hybrid 2D-3D networks for 3D coherent layer segmentation of retinal OCT images with full and sparse annotations

Layer segmentation is important to quantitative analysis of retinal optical coherence tomography (OCT). Recently, deep learning based methods have been developed to automate this task and yield remarkable performance. However, due to the large spatial gap and potential mismatch between the B-scans of an OCT volume, all of them were based on 2D segmentation of individual B-scans, which may lose the continuity and diagnostic information of the retinal layers in 3D space. Besides, most of these methods required dense annotation of the OCT volumes, which is labor-intensive and expertise-demanding. This work presents a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) to obtain continuous 3D retinal layer surfaces from OCT volumes, which works well with both full and sparse annotations. The 2D features of individual B-scans are extracted by an encoder consisting of 2D convolutions. These 2D features are then used to produce the alignment displacement vectors and layer segmentation by two 3D decoders coupled via a spatial transformer module. Two losses are proposed to utilize the retinal layers' natural property of being smooth for B-scan alignment and layer segmentation, respectively, and are the key to the semi-supervised learning with sparse annotation. The entire framework is trained end-to-end. To the best of our knowledge, this is the first work that attempts 3D retinal layer segmentation in volumetric OCT images based on CNNs. Experiments on a synthetic dataset and three public clinical datasets show that our framework can effectively align the B-scans for potential motion correction, and achieves superior performance to state-of-the-art 2D deep learning methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity in both fully and semi-supervised settings, thus offering more clinical values than previous works.

MCFSA-Net: A multi-scale channel fusion and spatial activation network for retinal vessel segmentation

As the only vascular tissue that can be directly viewed in vivo, retinal vessels are medically important in assisting the diagnosis of ocular and cardiovascular diseases. They generally appear as different morphologies and uneven thickness in fundus images. Therefore, the single-scale segmentation method may fail to capture abundant morphological features, suffering from the deterioration in vessel segmentation, especially for tiny vessels. To alleviate this issue, we propose a multi-scale channel fusion and spatial activation network (MCFSA-Net) for retinal vessel segmentation with emphasis on tiny ones. Specifically, the Hybrid Convolution-DropBlock (HC-Drop) is first used to extract deep features of vessels and construct multi-scale feature maps by progressive down-sampling. Then, the Channel Cooperative Attention Fusion (CCAF) module is designed to handle different morphological vessels in a multi-scale manner. Finally, the Global Spatial Activation (GSA) module is introduced to aggregate global feature information for improving the attention on tiny vessels in the spatial domain and realizing effective segmentation for them. Experiments are carried out on three datasets including DRIVE, CHASE_DB1, and STARE. Our retinal vessel segmentation method achieves Accuracy of 96.95%, 97.57%, and 97.83%, and F1 score of 82.67%, 81.82%, and 82.95% in the above datasets, respectively. Qualitative and quantitative analysis show that the proposed method outperforms current advanced vessel segmentation methods, especially for tiny vessels.

CTS-Net: A Segmentation Network for Glaucoma Optical Coherence Tomography Retinal Layer Images

Optical Coherence Tomography (OCT) technology is essential to obtain glaucoma diagnostic data non-invasively and rapidly. Early diagnosis of glaucoma can be achieved by analyzing the thickness and shape of retinal layers. Accurate retinal layer segmentation assists ophthalmologists in improving the efficiency of disease diagnosis. Deep learning technology is one of the most effective methods for processing OCT retinal layer images, which can segment different retinal layers and effectively obtain the topological structure of the boundary. This paper proposes a neural network method for retinal layer segmentation based on the CSWin Transformer (CTS-Net), which can achieve pixel-level segmentation and obtain smooth boundaries. A Dice loss function based on boundary areas (BADice Loss) is proposed to make CTS-Net learn more features of edge regions and improve the accuracy of boundary segmentation. We applied the model to the publicly available dataset of glaucoma retina, and the test results showed that mean absolute distance (MAD), root mean square error (RMSE), and dice-similarity coefficient (DSC) metrics were 1.79 pixels, 2.15 pixels, and 92.79%, respectively, which are better than those of the compared model. In the cross-validation experiment, the ranges of MAD, RMSE, and DSC are 0.05 pixels, 0.03 pixels, and 0.33%, respectively, with a slight difference, which further verifies the generalization ability of CTS-Net.

TSSK-Net: Weakly supervised biomarker localization and segmentation with image-level annotation in retinal OCT images

The localization and segmentation of biomarkers in OCT images are critical steps in retina-related disease diagnosis. Although fully supervised deep learning models can segment pathological regions, their performance relies on labor-intensive pixel-level annotations. Compared with dense pixel-level annotation, image-level annotation can reduce the burden of manual annotation. Existing methods for image-level annotation are usually based on class activation maps (CAM). However, current methods still suffer from model collapse, training instability, and anatomical mismatch due to the considerable variation in retinal biomarkers' shape, texture, and size. This paper proposes a novel weakly supervised biomarkers localization and segmentation method, requiring only image-level annotations. The technique is a Teacher-Student network with joint Self-supervised contrastive learning and Knowledge distillation-based anomaly localization, namely TSSK-Net. Specifically, we treat retinal biomarker regions as abnormal regions distinct from normal regions. First, we propose a novel pre-training strategy based on supervised contrastive learning that encourages the model to learn the anatomical structure of normal OCT images. Second, we design a fine-tuning module and propose a novel hybrid network structure. The network includes supervised contrastive loss for feature learning and cross-entropy loss for classification learning. To further improve the performance, we propose an efficient strategy to combine these two losses to preserve the anatomical structure and enhance the encoding representation of features. Finally, we design a knowledge distillation-based anomaly segmentation method that is effectively combined with the previous model to alleviate the challenge of insufficient supervision. Experimental results on a local dataset and a public dataset demonstrated the effectiveness of our proposed method. Our proposed method can effectively reduce the annotation burden of ophthalmologists in OCT images.

MSAL-Net: improve accurate segmentation of nuclei in histopathology images by multiscale attention learning network

Background

The digital pathology images obtain the essential information about the patient’s disease, and the automated nuclei segmentation results can help doctors make better decisions about diagnosing the disease. With the speedy advancement of convolutional neural networks in image processing, deep learning has been shown to play a significant role in the various analysis of medical images, such as nuclei segmentation, mitosis detection and segmentation etc. Recently, several U-net based methods have been developed to solve the automated nuclei segmentation problems. However, these methods fail to deal with the weak features representation from the initial layers and introduce the noise into the decoder path. In this paper, we propose a multiscale attention learning network (MSAL-Net), where the dense dilated convolutions block captures more comprehensive nuclei context information, and a newly modified decoder part is introduced, which integrates with efficient channel attention and boundary refinement modules to effectively learn spatial information for better prediction and further refine the nuclei cell of boundaries.

Results

Both qualitative and quantitative results are obtained on the publicly available MoNuseg dataset. Extensive experiment results verify that our proposed method significantly outperforms state-of-the-art methods as well as the vanilla Unet method in the segmentation task. Furthermore, we visually demonstrate the effect of our modified decoder part.

Conclusion

The MSAL-Net shows superiority with a novel decoder to segment the touching and blurred background nuclei cells obtained from histopathology images with better performance for accurate decoding.

Automated retinal boundary segmentation of optical coherence tomography images using an improved Canny operator

Retinal segmentation is a prerequisite for quantifying retinal structural features and diagnosing related ophthalmic diseases. Canny operator is recognized as the best boundary detection operator so far, and is often used to obtain the initial boundary of the retina in retinal segmentation. However, the traditional Canny operator is susceptible to vascular shadows, vitreous artifacts, or noise interference in retinal segmentation, causing serious misdetection or missed detection. This paper proposed an improved Canny operator for automatic segmentation of retinal boundaries. The improved algorithm solves the problems of the traditional Canny operator by adding a multi-point boundary search step on the basis of the original method, and adjusts the convolution kernel. The algorithm was used to segment the retinal images of healthy subjects and age-related macular degeneration (AMD) patients; eleven retinal boundaries were identified and compared with the results of manual segmentation by the ophthalmologists. The average difference between the automatic and manual methods is: 2–6 microns (1–2 pixels) for healthy subjects and 3–10 microns (1–3 pixels) for AMD patients. Qualitative method is also used to verify the accuracy and stability of the algorithm. The percentage of “perfect segmentation” and “good segmentation” is 98% in healthy subjects and 94% in AMD patients. This algorithm can be used alone or in combination with other methods as an initial boundary detection algorithm. It is easy to understand and improve, and may become a useful tool for analyzing and diagnosing eye diseases.

Robust region encoding and layer attribute protection for the segmentation of retina with multifarious abnormalities

PURPOSE

To robustly segment retinal layers that are affected by complex variety of retinal diseases for optical coherence tomography angiography (OCTA) en face projection generation.

METHODS

In this paper, we propose a robust retinal layer segmentation model to reduce the impact of multifarious abnormalities on model performance. OCTA vascular distribution that is regarded as the supplements of spectral domain optical coherence tomography (SD-OCT) structural information is introduced to improve the robustness of layer region encoding. To further reduce the sensitivity of region encoding to retinal abnormalities, we propose a multitask layer-wise refinement (MLR) module that can refine the initial layer region segmentation results layer-by-layer. Finally, we design a region-to-surface transformation (RtST) module without additional training parameters to convert the encoding layer regions to their corresponding layer surfaces. This transformation from layer regions to layer surfaces can remove the inaccurate segmentation regions, and the layer surfaces are easier to be used to protect the retinal layer natures than layer regions.

RESULTS

Experimental data includes 273 eyes, where 95 eyes are normal and 178 eyes contain complex retinal diseases, including age-related macular degeneration (AMD), diabetic retinopathy (DR), central serous chorioretinopathy (CSC), choroidal neovascularization (CNV), and so forth. The dice similarity coefficient (DSC: %) of superficial, deep and outer retina achieves 98.92, 97.48, and 98.87 on normal eyes and 98.35, 95.33, and 98.17 on abnormal eyes. Compared with other commonly used layer segmentation models, our model achieves the state-of-the-art layer segmentation performance.

CONCLUSIONS

The final results prove that our proposed model obtains outstanding performance and has enough ability to resist retinal abnormalities. Besides, OCTA modality is helpful for retinal layer segmentation.

Deep learning architectures analysis for age-related macular degeneration segmentation on optical coherence tomography scans

BACKGROUND AND OBJECTIVES

Aged people usually are more to be diagnosed with retinal diseases in developed countries. Retinal capillaries leakage into the retina swells and causes an acute vision loss, which is called age-related macular degeneration (AMD). The disease can not be adequately diagnosed solely using fundus images as depth information is not available. The variations in retina volume assist in monitoring ophthalmological abnormalities. Therefore, high-fidelity AMD segmentation in optical coherence tomography (OCT) imaging modality has raised the attention of researchers as well as those of the medical doctors. Many methods across the years encompassing machine learning approaches and convolutional neural networks (CNN) strategies have been proposed for object detection and image segmentation.

METHODS

In this paper, we analyze four wide-spread deep learning models designed for the segmentation of three retinal fluids outputting dense predictions in the RETOUCH challenge data. We aim to demonstrate how a patch-based approach could push the performance for each method. Besides, we also evaluate the methods using the OPTIMA challenge dataset for generalizing network performance. The analysis is driven into two sections: the comparison between the four approaches and the significance of patching the images.

RESULTS

The performance of networks trained on the RETOUCH dataset is higher than human performance. The analysis further generalized the performance of the best network obtained by fine-tuning it and achieved a mean Dice similarity coefficient (DSC) of 0.85. Out of the three types of fluids, intraretinal fluid (IRF) is more recognized, and the highest DSC value of 0.922 is achieved using Spectralis dataset. Additionally, the highest average DSC score is 0.84, which is achieved by PaDeeplabv3+ model using Cirrus dataset.

CONCLUSIONS

The proposed method segments the three fluids in the retina with high DSC value. Fine-tuning the networks trained on the RETOUCH dataset makes the network perform better and faster than training from scratch. Enriching the networks with inputting a variety of shapes by extracting patches helped to segment the fluids better than using a full image.