Boundary Aware U-Net for Retinal Layers Segmentation in Optical Coherence Tomography Images

SD-LayerNet: Robust and label-efficient retinal layer segmentation via anatomical priors

BACKGROUND AND OBJECTIVES

Automated, anatomically coherent retinal layer segmentation in optical coherence tomography (OCT) is one of the most important components of retinal disease management. However, current methods rely on large amounts of labeled data, which can be difficult and expensive to obtain. In addition, these systems tend often propose anatomically impossible results, which undermines their clinical reliability.

METHODS

This study introduces a semi-supervised approach to retinal layer segmentation that leverages large amounts of unlabeled data and anatomical prior knowledge related to the structure of the retina. During training, we use a novel topological engine that converts inferred retinal layer boundaries into pixel-wise structured segmentations. These compose a set of anatomically valid disentangled representations which, together with predicted style factors, are used to reconstruct the input image. At training time, the retinal layer boundaries and pixel-wise predictions are both guided by reference annotations, where available, but more importantly by innovatively exploiting anatomical priors that improve the performance, robustness and coherence of the method even if only a small amount of labeled data is available.

RESULTS

Exhaustive experiments with respect to label efficiency, contribution of unsupervised data and robustness to different acquisition settings were conducted. The proposed method showed state of-the-art performance on all the studied public and internal datasets, specially in low annotated data regimes. Additionally, the model was able to make use of unlabeled data from a different domain with only a small performance drop in comparison to a fully-supervised setting.

CONCLUSION

A novel, robust, label-efficient retinal layer segmentation method was proposed. The approach has shown state-of-the-art layer segmentation performance with a fraction of the training data available, while at the same time, its robustness against domain shift was also shown.

General retinal layer segmentation in OCT images via reinforcement constraint

The change of layer thickness of retina is closely associated with the development of ocular diseases such as glaucoma and optic disc drusen. Optical coherence tomography (OCT) is a widely used technology to visualize the lamellar structures of retina. Accurate segmentation of retinal lamellar structures is crucial for diagnosis, treatment, and related research of ocular diseases. However, existing studies have focused on improving the segmentation accuracy, they cannot achieve consistent segmentation performance on different types of datasets, such as retinal OCT images with optic disc and interference of diseases. To this end, a general retinal layer segmentation method is presented in this paper. To obtain more continuous and smoother boundaries, feature enhanced decoding module with reinforcement constraint is proposed, fusing boundary prior and distribution prior, and correcting bias in learning process simultaneously. To enhance the model's perception of the slender retinal structure, position channel attention is introduced, obtaining global dependencies of both space and channel. To handle the imbalanced distribution of retinal OCT images, focal loss is introduced, guiding the model to pay more attention to retinal layers with a smaller proportion. The designed method achieves the state-of-the-art (SOTA) overall performance on five datasets (i.e., MGU, DUKE, NR206, OCTA500 and private dataset).

A weak edge estimation based multi-task neural network for OCT segmentation

Optical Coherence Tomography (OCT) offers high-resolution images of the eye's fundus. This enables thorough analysis of retinal health by doctors, providing a solid basis for diagnosis and treatment. With the development of deep learning, deep learning-based methods are becoming more popular for fundus OCT image segmentation. Yet, these methods still encounter two primary challenges. Firstly, deep learning methods are sensitive to weak edges. Secondly, the high cost of annotating medical image data results in a lack of labeled data, leading to overfitting during model training. To tackle these challenges, we introduce the Multi-Task Attention Mechanism Network with Pruning (MTAMNP), consisting of a segmentation branch and a boundary regression branch. The boundary regression branch utilizes an adaptive weighted loss function derived from the Truncated Signed Distance Function(TSDF), improving the model's capacity to preserve weak edge details. The Spatial Attention Based Dual-Branch Information Fusion Block links these branches, enabling mutual benefit. Furthermore, we present a structured pruning method grounded in channel attention to decrease parameter count, mitigate overfitting, and uphold segmentation accuracy. Our method surpasses other cutting-edge segmentation networks on two widely accessible datasets, achieving Dice scores of 84.09% and 93.84% on the HCMS and Duke datasets.

Fully automatic deep convolutional approaches for the screening of neurodegeneratives diseases using multi-view OCT images

The prevalence of neurodegenerative diseases (NDDs) such as Alzheimer's (AD), Parkinson's (PD), Essential tremor (ET), and Multiple Sclerosis (MS) is increasing alongside the aging population. Recent studies suggest that these disorders can be identified through retinal imaging, allowing for early detection and monitoring via Optical Coherence Tomography (OCT) scans. This study is at the forefront of research, pioneering the application of multi-view OCT and 3D information to the neurological diseases domain. Our methodology consists of two main steps. In the first one, we focus on the segmentation of the retinal nerve fiber layer (RNFL) and a class layer grouping between the ganglion cell layer and Bruch's membrane (GCL-BM) in both macular and optic disc OCT scans. These are the areas where changes in thickness serve as a potential indicator of NDDs. The second phase is to select patients based on information about the retinal layers. We explore how the integration of both views (macula and optic disc) improves each screening scenario: Healthy Controls (HC) vs. NDD, AD vs. NDD, ET vs. NDD, MS vs. NDD, PD vs. NDD, and a final multi-class approach considering all four NDDs. For the segmentation task, we obtained satisfactory results for both 2D and 3D approaches in macular segmentation, in which 3D performed better due to the inclusion of depth and cross-sectional information. As for the optic disc view, transfer learning did not improve the metrics over training from scratch, but it did provide a faster training. As for screening, 3D computational biomarkers provided better results than 2D ones, and multi-view methods were usually better than the single-view ones. Regarding separability among diseases, MS and PD were the ones that provided better results in their screening approaches, being also the most represented classes. In conclusion, our methodology has been successfully validated with an extensive experimentation of configurations, techniques and OCT views, becoming the first multi-view analysis that merges data from both macula-centered and optic disc-centered perspectives. Besides, it is also the first effort to examine key retinal layers across four major NDDs within the framework of pathological screening.

Boundary-Repairing Dual-Path Network for Retinal Layer Segmentation in OCT Image with Pigment Epithelial Detachment

Automatic retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis of ocular diseases. Currently, automatic retinal layer segmentation works well with normal OCT images. However, pigment epithelial detachment (PED) dramatically alters the retinal structure, causing blurred boundaries and partial disappearance of the Bruch's Membrane (BM), thus posing challenges to the segmentation. To tackle these problems, we propose a novel dual-path U-shaped network for simultaneous layer segmentation and boundary regression. This network first designs a feature interaction fusion (FIF) module to strengthen the boundary shape constraints in the layer path. To address the challenge posed by partial BM disappearance and boundary-blurring, we propose a layer boundary repair (LBR) module. This module aims to use contrastive loss to enhance the confidence of blurred boundary regions and refine the segmentation of layer boundaries through the re-prediction head. In addition, we introduce a novel bilateral threshold distance map (BTDM) designed for the boundary path. The BTDM serves to emphasize information within boundary regions. This map, combined with the updated probability map, culminates in topology-guaranteed segmentation results achieved through a topology correction (TC) module. We investigated the proposed network on two severely deformed datasets (i.e., OCTA-500 and Aier-PED) and one slightly deformed dataset (i.e., DUKE). The proposed method achieves an average Dice score of 94.26% on the OCTA-500 dataset, which was 1.5% higher than BAU-Net and outperformed other methods. In the DUKE and Aier-PED datasets, the proposed method achieved average Dice scores of 91.65% and 95.75%, respectively.

Deep learning based highly accurate transplanted bioengineered corneal equivalent thickness measurement using optical coherence tomography

Corneal transplantation is the primary treatment for irreversible corneal diseases, but due to limited donor availability, bioengineered corneal equivalents are being developed as a solution, with biocompatibility, structural integrity, and physical function considered key factors. Since conventional evaluation methods may not fully capture the complex properties of the cornea, there is a need for advanced imaging and assessment techniques. In this study, we proposed a deep learning-based automatic segmentation method for transplanted bioengineered corneal equivalents using optical coherence tomography to achieve a highly accurate evaluation of graft integrity and biocompatibility. Our method provides quantitative individual thickness values, detailed maps, and volume measurements of the bioengineered corneal equivalents, and has been validated through 14 days of monitoring. Based on the results, it is expected to have high clinical utility as a quantitative assessment method for human keratoplasties, including automatic opacity area segmentation and implanted graft part extraction, beyond animal studies.

Diverse Data Generation for Retinal Layer Segmentation With Potential Structure Modeling

Accurate retinal layer segmentation on optical coherence tomography (OCT) images is hampered by the challenges of collecting OCT images with diverse pathological characterization and balanced distribution. Current generative models can produce high-realistic images and corresponding labels without quantitative limitations by fitting distributions of real collected data. Nevertheless, the diversity of their generated data is still limited due to the inherent imbalance of training data. To address these issues, we propose an image-label pair generation framework that generates diverse and balanced potential data from imbalanced real samples. Specifically, the framework first generates diverse layer masks, and then generates plausible OCT images corresponding to these layer masks using two customized diffusion probabilistic models respectively. To learn from imbalanced data and facilitate balanced generation, we introduce pathological-related conditions to guide the generation processes. To enhance the diversity of the generated image-label pairs, we propose a potential structure modeling technique that transfers the knowledge of diverse sub-structures from lowly- or non-pathological samples to highly pathological samples. We conducted extensive experiments on two public datasets for retinal layer segmentation. Firstly, our method generates OCT images with higher image quality and diversity compared to other generative methods. Furthermore, based on the extensive training with the generated OCT images, downstream retinal layer segmentation tasks demonstrate improved results. The code is publicly available at: https://github.com/nicetomeetu21/GenPSM.

HDB-Net: hierarchical dual-branch network for retinal layer segmentation in diseased OCT images

Optical coherence tomography (OCT) retinal layer segmentation is a critical procedure of the modern ophthalmic process, which can be used for diagnosis and treatment of diseases such as diabetic macular edema (DME) and multiple sclerosis (MS). Due to the difficulties of low OCT image quality, highly similar retinal interlayer morphology, and the uncertain presence, shape and size of lesions, the existing algorithms do not perform well. In this work, we design an HDB-Net network for retinal layer segmentation in diseased OCT images, which solves this problem by combining global and detailed features. First, the proposed network uses a Swin transformer and Res50 as a parallel backbone network, combined with the pyramid structure in UperNet, to extract global context and aggregate multi-scale information from images. Secondly, a feature aggregation module (FAM) is designed to extract global context information from the Swin transformer and local feature information from ResNet by introducing mixed attention mechanism. Finally, the boundary awareness and feature enhancement module (BA-FEM) is used to extract the retinal layer boundary information and topological order from the low-resolution features of the shallow layer. Our approach has been validated on two public datasets, and Dice scores were 87.61% and 92.44, respectively, both outperforming other state-of-the-art technologies.

ScLNet: A cornea with scleral lens OCT layers segmentation dataset and new multi-task model

Objective

To develop deep learning methods with high accuracy for segmenting irregular corneas and detecting the tear fluid reservoir (TFR) boundary under the scleral lens. Additionally, this study aims to provide a publicly available cornea with scleral lens OCT dataset, including manually labeled layer masks for training and validation of segmentation algorithms. This study introduces ScLNet, a dataset comprising cornea with Scleral Lens (ScL) optical coherence tomography (OCT) images with layer annotations, and a multi-task network designed to achieve rapid, accurate, automated segmentation of scleral lens with regular and irregular corneas.

Methods

We created a dataset comprising 31,360 OCT images with scleral lens annotations. The network architecture includes an encoder with multi-scale input and a context coding layer, along with two decoders for specific tasks. The primary task focuses on predicting ScL, TFR, and cornea regions, while the auxiliary task, aimed at predicting the boundaries of ScL, TFR, and cornea, enhances feature extraction for the main task. Segmentation results were compared with state-of-the-art methods and evaluated using Dice similarity coefficient (DSC), intersection over union (IoU), Matthews correlation coefficient (MCC), Precision, and Hausdorff distance (HD).

Results

ScLNet achieves 98.22 % DSC, 96.50 % IoU, 98.13 % MCC, 98.35 % Precision, and 3.6840 HD (in pixels) in segmenting ScL; 97.78 % DSC, 95.66 % IoU, 97.71 % MCC, 97.70 % Precision, and 3.7838 HD (in pixels) in segmenting TFR; and 99.22 % DSC, 98.45 % IoU, 99.15 % MCC, 99.14 % Precision, and 3.5355 HD (in pixels) in segmenting cornea. The layer interfaces recognized by ScLNet closely align with expert annotations, as evidenced by high IoU scores. Boundary metrics further confirm its effectiveness.

Conclusion

We constructed a dataset of corneal OCT images with ScL wearing, which includes regular and irregular cornea patients. The proposed ScLNet achieves high accuracy in extracting ScL, TFR, and corneal layer masks and boundaries from OCT images of the dataset.

Lightweight Retinal Layer Segmentation With Global Reasoning

Automatic retinal layer segmentation with medical images, such as optical coherence tomography (OCT) images, serves as an important tool for diagnosing ophthalmic diseases. However, it is challenging to achieve accurate segmentation due to low contrast and blood flow noises presented in the images. In addition, the algorithm should be light-weight to be deployed for practical clinical applications. Therefore, it is desired to design a light-weight network with high performance for retinal layer segmentation. In this paper, we propose LightReSeg for retinal layer segmentation which can be applied to OCT images. Specifically, our approach follows an encoder-decoder structure, where the encoder part employs multi-scale feature extraction and a Transformer block for fully exploiting the semantic information of feature maps at all scales and making the features have better global reasoning capabilities, while the decoder part, we design a multi-scale asymmetric attention (MAA) module for preserving the semantic information at each encoder scale. The experiments show that our approach achieves a better segmentation performance compared to the current state-of-the-art method TransUnet with 105.7M parameters on both our collected dataset and two other public datasets, with only 3.3M parameters.

Advancing Ocular Imaging: A Hybrid Attention Mechanism-Based U-Net Model for Precise Segmentation of Sub-Retinal Layers in OCT Images

This paper presents a novel U-Net model incorporating a hybrid attention mechanism for automating the segmentation of sub-retinal layers in Optical Coherence Tomography (OCT) images. OCT is an ophthalmology tool that provides detailed insights into retinal structures. Manual segmentation of these layers is time-consuming and subjective, calling for automated solutions. Our proposed model combines edge and spatial attention mechanisms with the U-Net architecture to improve segmentation accuracy. By leveraging attention mechanisms, the U-Net focuses selectively on image features. Extensive evaluations using datasets demonstrate that our model outperforms existing approaches, making it a valuable tool for medical professionals. The study also highlights the model's robustness through performance metrics such as an average Dice score of 94.99%, Adjusted Rand Index (ARI) of 97.00%, and Strength of Agreement (SOA) classifications like "Almost Perfect", "Excellent", and "Very Strong". This advanced predictive model shows promise in expediting processes and enhancing the precision of ocular imaging in real-world applications.

Retinal Layer Segmentation in OCT Images With Boundary Regression and Feature Polarization

The geometry of retinal layers is an important imaging feature for the diagnosis of some ophthalmic diseases. In recent years, retinal layer segmentation methods for optical coherence tomography (OCT) images have emerged one after another, and huge progress has been achieved. However, challenges due to interference factors such as noise, blurring, fundus effusion, and tissue artifacts remain in existing methods, primarily manifesting as intra-layer false positives and inter-layer boundary deviation. To solve these problems, we propose a method called Tightly combined Cross-Convolution and Transformer with Boundary regression and feature Polarization (TCCT-BP). This method uses a hybrid architecture of CNN and lightweight Transformer to improve the perception of retinal layers. In addition, a feature grouping and sampling method and the corresponding polarization loss function are designed to maximize the differentiation of the feature vectors of different retinal layers, and a boundary regression loss function is devised to constrain the retinal boundary distribution for a better fit to the ground truth. Extensive experiments on four benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance in dealing with problems of false positives and boundary distortion. The proposed method ranked first in the OCT Layer Segmentation task of GOALS challenge held by MICCAI 2022. The source code is available at https://www.github.com/tyb311/TCCT.

Graph Attention U-Net for Retinal Layer Surface Detection and Choroid Neovascularization Segmentation in OCT Images

Choroidal neovascularization (CNV) is a typical symptom of age-related macular degeneration (AMD) and is one of the leading causes for blindness. Accurate segmentation of CNV and detection of retinal layers are critical for eye disease diagnosis and monitoring. In this paper, we propose a novel graph attention U-Net (GA-UNet) for retinal layer surface detection and CNV segmentation in optical coherence tomography (OCT) images. Due to retinal layer deformation caused by CNV, it is challenging for existing models to segment CNV and detect retinal layer surfaces with the correct topological order. We propose two novel modules to address the challenge. The first module is a graph attention encoder (GAE) in a U-Net model that automatically integrates topological and pathological knowledge of retinal layers into the U-Net structure to achieve effective feature embedding. The second module is a graph decorrelation module (GDM) that takes reconstructed features by the decoder of the U-Net as inputs, it then decorrelates and removes information unrelated to retinal layer for improved retinal layer surface detection. In addition, we propose a new loss function to maintain the correct topological order of retinal layers and the continuity of their boundaries. The proposed model learns graph attention maps automatically during training and performs retinal layer surface detection and CNV segmentation simultaneously with the attention maps during inference. We evaluated the proposed model on our private AMD dataset and another public dataset. Experiment results show that the proposed model outperformed the competing methods for retinal layer surface detection and CNV segmentation and achieved new state of the arts on the datasets.

Automatic Segmentation of Retinal Layers in Multiple Neurodegenerative Disorder Scenarios

Retinal Optical Coherence Tomography (OCT) allows the non-invasive direct observation of the central nervous system, enabling the measurement and extraction of biomarkers from neural tissue that can be helpful in the assessment of ocular, systemic and Neurological Disorders (ND). Deep learning models can be trained to segment the retinal layers for biomarker extraction. However, the onset of ND can have an impact on the neural tissue, which can lead to the degraded performance of models not exposed to images displaying signs of disease during training. We present a fully automatic approach for the retinal layer segmentation in multiple neurodegenerative disorder scenarios, using an annotated dataset of patients of the most prevalent NDs: Alzheimer's disease, Parkinson's disease, multiple sclerosis and essential tremor, along with healthy control patients. Furthermore, we present a two-part, comprehensive study on the effects of ND on the performance of these models. The results show that images of healthy patients may not be sufficient for the robust training of automated segmentation models intended for the analysis of ND patients, and that using images representative of different NDs can increase the model performance. These results indicate that the presence or absence of patients of ND in datasets should be taken into account when training deep learning models for retinal layer segmentation, and that the proposed approach can provide a valuable tool for the robust and reliable diagnosis in multiple scenarios of ND.

A collaborative multi-task learning method for BI-RADS category 4 breast lesion segmentation and classification of MRI images

BACKGROUND AND OBJECTIVE

The diagnosis of BI-RADS category 4 breast lesion is difficult because its probability of malignancy ranges from 2% to 95%. For BI-RADS category 4 breast lesions, MRI is one of the prominent noninvasive imaging techniques. In this paper, we research computer algorithms to segment lesions and classify the benign or malignant lesions in MRI images. However, this task is challenging because the BI-RADS category 4 lesions are characterized by irregular shape, imbalanced class, and low contrast.

METHODS

We fully utilize the intrinsic correlation between segmentation and classification tasks, where accurate segmentation will yield accurate classification results, and classification results will promote better segmentation. Therefore, we propose a collaborative multi-task algorithm (CMTL-SC). Specifically, a preliminary segmentation subnet is designed to identify the boundaries, locations and segmentation masks of lesions; a classification subnet, which combines the information provided by the preliminary segmentation, is designed to achieve benign or malignant classification; a repartition segmentation subnet which aggregates the benign or malignant results, is designed to refine the lesion segment. The three subnets work cooperatively so that the CMTL-SC can identify the lesions better which solves the three challenges.

RESULTS AND CONCLUSION

We collect MRI data from 248 patients in the Second Hospital of Dalian Medical University. The results show that the lesion boundaries delineated by the CMTL-SC are close to the boundaries delineated by the physicians. Moreover, the CMTL-SC yields better results than the single-task and multi-task state-of-the-art algorithms. Therefore, CMTL-SC can help doctors make precise diagnoses and refine treatments for patients.

Deep learning segmentation of the tear fluid reservoir under the sclera lens in optical coherence tomography images

The tear fluid reservoir (TFR) under the sclera lens is a unique characteristic providing optical neutralization of any aberrations from corneal irregularities. Anterior segment optical coherence tomography (AS-OCT) has become an important imaging modality for sclera lens fitting and visual rehabilitation therapy in both optometry and ophthalmology. Herein, we aimed to investigate whether deep learning can be used to segment the TFR from healthy and keratoconus eyes, with irregular corneal surfaces, in OCT images. Using AS-OCT, a dataset of 31850 images from 52 healthy and 46 keratoconus eyes, during sclera lens wear, was obtained and labeled with our previously developed algorithm of semi-automatic segmentation. A custom-improved U-shape network architecture with a full-range multi-scale feature-enhanced module (FMFE-Unet) was designed and trained. A hybrid loss function was designed to focus training on the TFR, to tackle the class imbalance problem. The experiments on our database showed an IoU, precision, specificity, and recall of 0.9426, 0.9678, 0.9965, and 0.9731, respectively. Furthermore, FMFE-Unet was found to outperform the other two state-of-the-art methods and ablation models, suggesting its strength in segmenting the TFR under the sclera lens depicted on OCT images. The application of deep learning for TFR segmentation in OCT images provides a powerful tool to assess changes in the dynamic tear film under the sclera lens, improving the efficiency and accuracy of lens fitting, and thus supporting the promotion of sclera lenses in clinical practice.

Boundary-Sensitive Loss Function With Location Constraint for Hard Region Segmentation

In computer-aided diagnosis and treatment planning, accurate segmentation of medical images plays an essential role, especially for some hard regions including boundaries, small objects and background interference. However, existing segmentation loss functions including distribution-, region- and boundary-based losses cannot achieve satisfactory performances on these hard regions. In this paper, a boundary-sensitive loss function with location constraint is proposed for hard region segmentation in medical images, which provides three advantages: i) our Boundary-Sensitive loss (BS-loss) can automatically pay more attention to the hard-to-segment boundaries (e.g., thin structures and blurred boundaries), thus obtaining finer object boundaries; ii) BS-loss also can adjust its attention to small objects during training to segment them more accurately; and iii) our location constraint can alleviate the negative impact of the background interference, through the distribution matching of pixels between prediction and Ground Truth (GT) along each axis. By resorting to the proposed BS-loss and location constraint, the hard regions in both foreground and background are considered. Experimental results on three public datasets demonstrate the superiority of our method. Specifically, compared to the second-best method tested in this study, our method improves performance on hard regions in terms of Dice similarity coefficient (DSC) and 95% Hausdorff distance (95%HD) of up to 4.17% and 73% respectively. In addition, it also achieves the best overall segmentation performance. Hence, we can conclude that our method can accurately segment these hard regions and improve the overall segmentation performance in medical images.

Deep learning based diagnostic quality assessment of choroidal OCT features with expert-evaluated explainability

Various vision-threatening eye diseases including age-related macular degeneration (AMD) and central serous chorioretinopathy (CSCR) are caused due to the dysfunctions manifested in the highly vascular choroid layer of the posterior segment of the eye. In the current clinical practice, screening choroidal structural changes is widely based on optical coherence tomography (OCT) images. Accordingly, to assist clinicians, several automated choroidal biomarker detection methods using OCT images are developed. However, the performance of these algorithms is largely constrained by the quality of the OCT scan. Consequently, determining the quality of choroidal features in OCT scans is significant in building standardized quantification tools and hence constitutes our main objective. This study includes a dataset of 1593 good and 2581 bad quality Spectralis OCT images graded by an expert. Noting the efficacy of deep-learning (DL) in medical image analysis, we propose to train three state-of-the-art DL models: ResNet18, EfficientNet-B0 and EfficientNet-B3 to detect the quality of OCT images. The choice of these models was inspired by their ability to preserve the salient features across all the layers without information loss. To evaluate the attention of DL models on the choroid, we introduced color transparency maps (CTMs) based on GradCAM explanations. Further, we proposed two subjective grading scores: overall choroid coverage (OCC) and choroid coverage in the visible region(CCVR) based on CTMs to objectively correlate visual explanations vis-à-vis DL model attentions. We observed that the average accuracy and F-scores for the three DL models are greater than 96%. Further, the OCC and CCVR scores achieved for the three DL models under consideration substantiate that they mostly focus on the choroid layer in making the decision. In particular, of the three DL models, EfficientNet-B3 is in close agreement with the clinician's inference. The proposed DL-based framework demonstrated high detection accuracy as well as attention on the choroid layer, where EfficientNet-B3 reported superior performance. Our work assumes significance in bench-marking the automated choroid biomarker detection tools and facilitating high-throughput screening. Further, the methods proposed in this work can be adopted for evaluating the attention of DL-based approaches developed for other region-specific quality assessment tasks.

Learning to detect boundary information for brain image segmentation

MRI brain images are always of low contrast, which makes it difficult to identify to which area the information at the boundary of brain images belongs. This can make the extraction of features at the boundary more challenging, since those features can be misleading as they might mix properties of different brain regions. Hence, to alleviate such a problem, image boundary detection plays a vital role in medical image segmentation, and brain segmentation in particular, as unclear boundaries can worsen brain segmentation results. Yet, given the low quality of brain images, boundary detection in the context of brain image segmentation remains challenging. Despite the research invested to improve boundary detection and brain segmentation, these two problems were addressed independently, i.e., little attention was paid to applying boundary detection to brain segmentation tasks. Therefore, in this paper, we propose a boundary detection-based model for brain image segmentation. To this end, we first design a boundary segmentation network for detecting and segmenting images brain tissues. Then, we design a boundary information module (BIM) to distinguish boundaries from the three different brain tissues. After that, we add a boundary attention gate (BAG) to the encoder output layers of our transformer to capture more informative local details. We evaluate our proposed model on two datasets of brain tissue images, including infant and adult brains. The extensive evaluation experiments of our model show better performance (a Dice Coefficient (DC) accuracy of up to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5.3\%$$\end{document}5.3% compared to the state-of-the-art models) in detecting and segmenting brain tissue images.

A single-step regression method based on transformer for retinal layer segmentation

The shape and structure of retinal layers are basic characteristics for the diagnosis of many ophthalmological diseases. Based on B-Scans of optical coherence tomography, most of retinal layer segmentation methods are composed of two-steps: classifying pixels and extracting retinal layers, in which the optimization of two independent steps decreases the accuracy. Although the methods based on deep learning are highly accurate, they require a large amount of labeled data. This paper proposes a single-step method based on transformer for retinal layer segmentation, which is trained by axial data (A-Scans), to obtain the boundary of each layer. The proposed method was evaluated on two public data sets. The first one contains eight retinal layer boundaries for diabetic macular edema, and the second one contains nine retinal layer boundaries for healthy controls and subjects with multiple sclerosis. Its absolute average distance errors are 0.99 pixels and 3.67 pixels, respectively, for the two sets, and its root mean square error is 1.29 pixels for the latter set. In addition, its accuracy is acceptable even if the training data is reduced to 0.3. The proposed method achieves state-of-the-art performance while maintaining the correct topology and requires less labeled data.

Convolutional neural network-based common-path optical coherence tomography A-scan boundary-tracking training and validation using a parallel Monte Carlo synthetic dataset

We present a parallel Monte Carlo (MC) simulation platform for rapidly generating synthetic common-path optical coherence tomography (CP-OCT) A-scan image dataset for image-guided needle insertion. The computation time of the method has been evaluated on different configurations and 100000 A-scan images are generated based on 50 different eye models. The synthetic dataset is used to train an end-to-end convolutional neural network (Ascan-Net) to localize the Descemet's membrane (DM) during the needle insertion. The trained Ascan-Net has been tested on the A-scan images collected from the ex-vivo human and porcine cornea as well as simulated data and shows improved tracking accuracy compared to the result by using the Canny-edge detector.

Self-Supervised Sequence Recovery for Semi-Supervised Retinal Layer Segmentation

Automated layer segmentation plays an important role for retinal disease diagnosis in optical coherence tomography (OCT) images. However, the severe retinal diseases result in the performance degeneration of automated layer segmentation approaches. In this paper, we present a robust semi-supervised retinal layer segmentation network to relieve the model failures on abnormal retinas, in which we obtain the lesion features from the labeled images with disease-balanced distribution, and utilize the unlabeled images to supplement the layer structure information. Specifically, in our proposed method, the cross-consistency training is utilized over the predictions of the different decoders, and we enforce a consistency between different decoder predictions to improve the encoders representation. Then, we proposed a sequence prediction branch based on self-supervised manner, which is designed to predict the position of each jigsaw puzzle to obtain sensory perception of the retinal layer structure. To this task, a layer spatial pyramid pooling (LSPP) module is designed to extract multi-scale layer spatial features. Furthermore, we use the optical coherence tomography angiography (OCTA) to supplement the information damaged by diseases. The experimental results validate that our method achieves more robust results compared with current supervised segmentation methods. Meanwhile, advanced segmentation performance can be obtained compared with state-of-the-art semi-supervised segmentation methods.

Graph-Based Region and Boundary Aggregation for Biomedical Image Segmentation

Segmentation is a fundamental task in biomedical image analysis. Unlike the existing region-based dense pixel classification methods or boundary-based polygon regression methods, we build a novel graph neural network (GNN) based deep learning framework with multiple graph reasoning modules to explicitly leverage both region and boundary features in an end-to-end manner. The mechanism extracts discriminative region and boundary features, referred to as initialized region and boundary node embeddings, using a proposed Attention Enhancement Module (AEM). The weighted links between cross-domain nodes (region and boundary feature domains) in each graph are defined in a data-dependent way, which retains both global and local cross-node relationships. The iterative message aggregation and node update mechanism can enhance the interaction between each graph reasoning module's global semantic information and local spatial characteristics. Our model, in particular, is capable of concurrently addressing region and boundary feature reasoning and aggregation at several different feature levels due to the proposed multi-level feature node embeddings in different parallel graph reasoning modules. Experiments on two types of challenging datasets demonstrate that our method outperforms state-of-the-art approaches for segmentation of polyps in colonoscopy images and of the optic disc and optic cup in colour fundus images. The trained models will be made available at: https://github.com/smallmax00/Graph_Region_Boudnary.

Robust region encoding and layer attribute protection for the segmentation of retina with multifarious abnormalities

PURPOSE

To robustly segment retinal layers that are affected by complex variety of retinal diseases for optical coherence tomography angiography (OCTA) en face projection generation.

METHODS

In this paper, we propose a robust retinal layer segmentation model to reduce the impact of multifarious abnormalities on model performance. OCTA vascular distribution that is regarded as the supplements of spectral domain optical coherence tomography (SD-OCT) structural information is introduced to improve the robustness of layer region encoding. To further reduce the sensitivity of region encoding to retinal abnormalities, we propose a multitask layer-wise refinement (MLR) module that can refine the initial layer region segmentation results layer-by-layer. Finally, we design a region-to-surface transformation (RtST) module without additional training parameters to convert the encoding layer regions to their corresponding layer surfaces. This transformation from layer regions to layer surfaces can remove the inaccurate segmentation regions, and the layer surfaces are easier to be used to protect the retinal layer natures than layer regions.

RESULTS

Experimental data includes 273 eyes, where 95 eyes are normal and 178 eyes contain complex retinal diseases, including age-related macular degeneration (AMD), diabetic retinopathy (DR), central serous chorioretinopathy (CSC), choroidal neovascularization (CNV), and so forth. The dice similarity coefficient (DSC: %) of superficial, deep and outer retina achieves 98.92, 97.48, and 98.87 on normal eyes and 98.35, 95.33, and 98.17 on abnormal eyes. Compared with other commonly used layer segmentation models, our model achieves the state-of-the-art layer segmentation performance.

CONCLUSIONS

The final results prove that our proposed model obtains outstanding performance and has enough ability to resist retinal abnormalities. Besides, OCTA modality is helpful for retinal layer segmentation.

A category attention instance segmentation network for four cardiac chambers segmentation in fetal echocardiography

Fetal echocardiography is an essential and comprehensive examination technique for the detection of fetal heart anomalies. Accurate cardiac chambers segmentation can assist cardiologists to analyze cardiac morphology and facilitate heart disease diagnosis. Previous research mainly focused on the segmentation of single cardiac chambers, such as left ventricle (LV) segmentation or left atrium (LA) segmentation. We propose a generic framework based on instance segmentation to segment the four cardiac chambers accurately and simultaneously. The proposed Category Attention Instance Segmentation Network (CA-ISNet) has three branches: a category branch for predicting the semantic category, a mask branch for segmenting the cardiac chambers, and a category attention branch for learning category information of instances. The category attention branch is used to correct instance misclassification of the category branch. In our collected dataset, which contains echocardiography images with four-chamber views of 319 fetuses, experimental results show our method can achieve superior segmentation performance against state-of-the-art methods. Specifically, using fivefold cross-validation, our model achieves Dice coefficients of 0.7956, 0.7619, 0.8199, 0.7470 for the four cardiac chambers, and with an average precision of 45.64%.