MFI-Net: Multiscale Feature Interaction Network for Retinal Vessel Segmentation

Retina Blood Vessels Segmentation and Classification with the Multi-featured Approach

Segmenting retinal blood vessels poses a significant challenge due to the irregularities inherent in small vessels. The complexity arises from the intricate task of effectively merging features at multiple levels, coupled with potential spatial information loss during successive down-sampling steps. This particularly affects the identification of small and faintly contrasting vessels. To address these challenges, we present a model tailored for automated arterial and venous (A/V) classification, complementing blood vessel segmentation. This paper presents an advanced methodology for segmenting and classifying retinal vessels using a series of sophisticated pre-processing and feature extraction techniques. The ensemble filter approach, incorporating Bilateral and Laplacian edge detectors, enhances image contrast and preserves edges. The proposed algorithm further refines the image by generating an orientation map. During the vessel extraction step, a complete convolution network processes the input image to create a detailed vessel map, enhanced by attention operations that improve modeling perception and resilience. The encoder extracts semantic features, while the Attention Module refines blood vessel depiction, resulting in highly accurate segmentation outcomes. The model was verified using the STARE dataset, which includes 400 images; the DRIVE dataset with 40 images; the HRF dataset with 45 images; and the INSPIRE-AVR dataset containing 40 images. The proposed model demonstrated superior performance across all datasets, achieving an accuracy of 97.5% on the DRIVE dataset, 99.25% on the STARE dataset, 98.33% on the INSPIREAVR dataset, and 98.67% on the HRF dataset. These results highlight the method's effectiveness in accurately segmenting and classifying retinal vessels.

A cascaded FAS-UNet+ framework with iterative optimization strategy for segmentation of organs at risk

Segmentation of organs at risks (OARs) in the thorax plays a critical role in radiation therapy for lung and esophageal cancer. Although automatic segmentation of OARs has been extensively studied, it remains challenging due to the varying sizes and shapes of organs, as well as the low contrast between the target and background. This paper proposes a cascaded FAS-UNet+ framework, which integrates convolutional neural networks and nonlinear multi-grid theory to solve a modified Mumford-shah model for segmenting OARs. This framework is equipped with an enhanced iteration block, a coarse-to-fine multiscale architecture, an iterative optimization strategy, and a model ensemble technique. The enhanced iteration block aims to extract multiscale features, while the cascade module is used to refine coarse segmentation predictions. The iterative optimization strategy improves the network parameters to avoid unfavorable local minima. An efficient data augmentation method is also developed to train the network, which significantly improves its performance. During the prediction stage, a weighted ensemble technique combines predictions from multiple models to refine the final segmentation. The proposed cascaded FAS-UNet+ framework was evaluated on the SegTHOR dataset, and the results demonstrate significant improvements in Dice score and Hausdorff Distance (HD). The Dice scores were 95.22%, 95.68%, and HD values were 0.1024, and 0.1194 for the segmentations of the aorta and heart in the official unlabeled dataset, respectively. Our code and trained models are available at https://github.com/zhuhui100/C-FASUNet-plus .

GKE-TUNet: Geometry-Knowledge Embedded TransUNet Model for Retinal Vessel Segmentation Considering Anatomical Topology

Automated retinal vessel segmentation is crucial for computer-aided clinical diagnosis and retinopathy screening. However, deep learning faces challenges in extracting complex intertwined structures and subtle small vessels from densely vascularized regions. To address these issues, we propose a novel segmentation model, called Geometry-Knowledge Embedded TransUNet (GKE-TUNet), which incorporates explicit embedding of topological features of retinal vessel anatomy. In the proposed GKE-TUNet model, a skeleton extraction network is pre-trained to extract the anatomical topology of retinal vessels from refined segmentation labels. During vessel segmentation, the dense skeleton graph is sampled as a graph of key-points and connections and is incorporated into the skip connection layer of TransUNet. The graph vertices are used as node features and correspond to positions in the low-level feature maps. The graph attention network (GAT) is used as the graph convolution backbone network to capture the shape semantics of vessels and the interaction of key locations along the topological direction. Finally, the node features obtained by graph convolution are read out as a sparse feature map based on their corresponding spatial coordinates. To address the problem of sparse feature maps, we employ convolution operators to fuse sparse feature maps with low-level dense feature maps. This fusion is weighted and connected to deep feature maps. Experimental results on the DRIVE, CHASE-DB1, and STARE datasets demonstrate the competitiveness of our proposed method compared to existing ones.

ScLNet: A cornea with scleral lens OCT layers segmentation dataset and new multi-task model

Objective

To develop deep learning methods with high accuracy for segmenting irregular corneas and detecting the tear fluid reservoir (TFR) boundary under the scleral lens. Additionally, this study aims to provide a publicly available cornea with scleral lens OCT dataset, including manually labeled layer masks for training and validation of segmentation algorithms. This study introduces ScLNet, a dataset comprising cornea with Scleral Lens (ScL) optical coherence tomography (OCT) images with layer annotations, and a multi-task network designed to achieve rapid, accurate, automated segmentation of scleral lens with regular and irregular corneas.

Methods

We created a dataset comprising 31,360 OCT images with scleral lens annotations. The network architecture includes an encoder with multi-scale input and a context coding layer, along with two decoders for specific tasks. The primary task focuses on predicting ScL, TFR, and cornea regions, while the auxiliary task, aimed at predicting the boundaries of ScL, TFR, and cornea, enhances feature extraction for the main task. Segmentation results were compared with state-of-the-art methods and evaluated using Dice similarity coefficient (DSC), intersection over union (IoU), Matthews correlation coefficient (MCC), Precision, and Hausdorff distance (HD).

Results

ScLNet achieves 98.22 % DSC, 96.50 % IoU, 98.13 % MCC, 98.35 % Precision, and 3.6840 HD (in pixels) in segmenting ScL; 97.78 % DSC, 95.66 % IoU, 97.71 % MCC, 97.70 % Precision, and 3.7838 HD (in pixels) in segmenting TFR; and 99.22 % DSC, 98.45 % IoU, 99.15 % MCC, 99.14 % Precision, and 3.5355 HD (in pixels) in segmenting cornea. The layer interfaces recognized by ScLNet closely align with expert annotations, as evidenced by high IoU scores. Boundary metrics further confirm its effectiveness.

Conclusion

We constructed a dataset of corneal OCT images with ScL wearing, which includes regular and irregular cornea patients. The proposed ScLNet achieves high accuracy in extracting ScL, TFR, and corneal layer masks and boundaries from OCT images of the dataset.

Cascade spatial and channel-wise multifusion network with criss cross augmentation for corneal segmentation and reconstruction

High-quality 3D corneal reconstruction from AS-OCT images has demonstrated significant potential in computer-aided diagnosis, enabling comprehensive observation of corneal thickness, precise assessment of morphological characteristics, as well as location and quantification of keratitis-affected regions. However, it faces two main challenges: (1) prevalent medical image segmentation networks often struggle to accurately process low-contrast corneal regions, which is a vital pre-processing step for 3D corneal reconstruction, and (2) there are no reconstruction methods that can be directly applied to AS-OCT sequences with 180-degree scanning. To combat these, we propose CSCM-CCA-Net, a simple yet efficient network for accurate corneal segmentation. This network incorporates two key techniques: cascade spatial and channel-wise multifusion (CSCM), which captures intricate contextual interdependencies and effectively extracts low-contrast and obscure corneal features; and criss cross augmentation (CCA), which enhances shape-preserved feature representation to improve segmentation accuracy. Based on the obtained corneal segmentation results, we reconstruct the 3D volume data and generate a topographic map of corneal thickness through corneal image alignment. Additionally, we design a transfer function based on the analysis of intensity histogram and gradient histogram to explore more internal cues for better visualization results. Experimental results on CORNEA benchmark demonstrate the impressive performance of our proposed method in terms of both corneal segmentation and 3D reconstruction. Furthermore, we compare CSCM-CCA-Net with state-of-the-art medical image segmentation approaches using three challenging medical fundus segmentation datasets (DRIVE, CHASEDB1, FIVES), highlighting its superiority in terms of segmentation accuracy. The code and models will be made available at https://github.com/qianguiping/CSCM-CCA-Net.

Integrating image and gene-data with a semi-supervised attention model for prediction of KRAS gene mutation status in non-small cell lung cancer

KRAS is a pathogenic gene frequently implicated in non-small cell lung cancer (NSCLC). However, biopsy as a diagnostic method has practical limitations. Therefore, it is important to accurately determine the mutation status of the KRAS gene non-invasively by combining NSCLC CT images and genetic data for early diagnosis and subsequent targeted therapy of patients. This paper proposes a Semi-supervised Multimodal Multiscale Attention Model (S2MMAM). S2MMAM comprises a Supervised Multilevel Fusion Segmentation Network (SMF-SN) and a Semi-supervised Multimodal Fusion Classification Network (S2MF-CN). S2MMAM facilitates the execution of the classification task by transferring the useful information captured in SMF-SN to the S2MF-CN to improve the model prediction accuracy. In SMF-SN, we propose a Triple Attention-guided Feature Aggregation module for obtaining segmentation features that incorporate high-level semantic abstract features and low-level semantic detail features. Segmentation features provide pre-guidance and key information expansion for S2MF-CN. S2MF-CN shares the encoder and decoder parameters of SMF-SN, which enables S2MF-CN to obtain rich classification features. S2MF-CN uses the proposed Intra and Inter Mutual Guidance Attention Fusion (I2MGAF) module to first guide segmentation and classification feature fusion to extract hidden multi-scale contextual information. I2MGAF then guides the multidimensional fusion of genetic data and CT image data to compensate for the lack of information in single modality data. S2MMAM achieved 83.27% AUC and 81.67% accuracy in predicting KRAS gene mutation status in NSCLC. This method uses medical image CT and genetic data to effectively improve the accuracy of predicting KRAS gene mutation status in NSCLC.

ERNet: Edge Regularization Network for Cerebral Vessel Segmentation in Digital Subtraction Angiography Images

Stroke is a leading cause of disability and fatality in the world, with ischemic stroke being the most common type. Digital Subtraction Angiography images, the gold standard in the operation process, can accurately show the contours and blood flow of cerebral vessels. The segmentation of cerebral vessels in DSA images can effectively help physicians assess the lesions. However, due to the disturbances in imaging parameters and changes in imaging scale, accurate cerebral vessel segmentation in DSA images is still a challenging task. In this paper, we propose a novel Edge Regularization Network (ERNet) to segment cerebral vessels in DSA images. Specifically, ERNet employs the erosion and dilation processes on the original binary vessel annotation to generate pseudo-ground truths of False Negative and False Positive, which serve as constraints to refine the coarse predictions based on their mapping relationship with the original vessels. In addition, we exploit a Hybrid Fusion Module based on convolution and transformers to extract local features and build long-range dependencies. Moreover, to support and advance the open research in the field of ischemic stroke, we introduce FPDSA, the first pixel-level semantic segmentation dataset for cerebral vessels. Extensive experiments on FPDSA illustrate the leading performance of our ERNet.

Semi-supervised point consistency network for retinal artery/vein classification

Recent deep learning methods with convolutional neural networks (CNNs) have boosted advance prosperity of medical image analysis and expedited the automatic retinal artery/vein (A/V) classification. However, it is challenging for these CNN-based approaches in two aspects: (1) specific tubular structures and subtle variations in appearance, contrast, and geometry, which tend to be ignored in CNNs with network layer increasing; (2) limited well-labeled data for supervised segmentation of retinal vessels, which may hinder the effectiveness of deep learning methods. To address these issues, we propose a novel semi-supervised point consistency network (SPC-Net) for retinal A/V classification. SPC-Net consists of an A/V classification (AVC) module and a multi-class point consistency (MPC) module. The AVC module adopts an encoder-decoder segmentation network to generate the prediction probability map of A/V for supervised learning. The MPC module introduces point set representations to adaptively generate point set classification maps of the arteriovenous skeleton, which enjoys its prediction flexibility and consistency (i.e. point consistency) to effectively alleviate arteriovenous confusion. In addition, we propose a consistency regularization between the predicted A/V classification probability maps and point set representations maps for unlabeled data to explore the inherent segmentation perturbation of the point consistency, reducing the need for annotated data. We validate our method on two typical public datasets (DRIVE, HRF) and a private dataset (TR280) with different resolutions. Extensive qualitative and quantitative experimental results demonstrate the effectiveness of our proposed method for supervised and semi-supervised learning.

A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation

Accurate segmentation of retinal vessels is an essential prerequisite for the subsequent analysis of fundus images. Recently, a number of methods based on deep learning have been proposed and shown to demonstrate promising segmentation performance, especially U-Net and its variants. However, tiny vessels and low-contrast vessels are hard to detect due to the issues of a loss of spatial details caused by consecutive down-sample operations and inadequate fusion of multi-level features caused by vanilla skip connections. To address these issues and enhance the segmentation precision of retinal vessels, we propose a novel high-resolution network with strip attention. Instead of the U-Net-shaped architecture, the proposed network follows an HRNet-shaped architecture as the basic network, learning high-resolution representations throughout the training process. In addition, a strip attention module including a horizontal attention mechanism and a vertical attention mechanism is designed to obtain long-range dependencies in the horizontal and vertical directions by calculating the similarity between each pixel and all pixels in the same row and the same column, respectively. For effective multi-layer feature fusion, we incorporate the strip attention module into the basic network to dynamically guide adjacent hierarchical features. Experimental results on the DRIVE and STARE datasets show that the proposed method can extract more tiny vessels and low-contrast vessels compared with existing mainstream methods, achieving accuracies of 96.16% and 97.08% and sensitivities of 82.68% and 89.36%, respectively. The proposed method has the potential to aid in the analysis of fundus images.

OCT2Former: A retinal OCT-angiography vessel segmentation transformer

BACKGROUND AND OBJECTIVE

Retinal vessel segmentation plays an important role in the automatic retinal disease screening and diagnosis. How to segment thin vessels and maintain the connectivity of vessels are the key challenges of the retinal vessel segmentation task. Optical coherence tomography angiography (OCTA) is a noninvasive imaging technique that can reveal high-resolution retinal vessels. Aiming at make full use of its characteristic of high resolution, a new end-to-end transformer based network named as OCT²Former (OCT-a Transformer) is proposed to segment retinal vessel accurately in OCTA images.

METHODS

The proposed OCT²Former is based on encoder-decoder structure, which mainly includes dynamic transformer encoder and lightweight decoder. Dynamic transformer encoder consists of dynamic token aggregation transformer and auxiliary convolution branch, in which the multi-head dynamic token aggregation attention based dynamic token aggregation transformer is designed to capture the global retinal vessel context information from the first layer throughout the network and the auxiliary convolution branch is proposed to compensate for the lack of inductive bias of the transformer and assist in the efficient feature extraction. A convolution based lightweight decoder is proposed to decode features efficiently and reduce the complexity of the proposed OCT²Former.

RESULTS

The proposed OCT²Former is validated on three publicly available datasets i.e. OCTA-SS, ROSE-1, OCTA-500 (subset OCTA-6M and OCTA-3M). The Jaccard indexes of the proposed OCT²Former on these datasets are 0.8344, 0.7855, 0.8099 and 0.8513, respectively, outperforming the best convolution based network 1.43, 1.32, 0.75 and 1.46%, respectively.

CONCLUSION

The experimental results have demonstrated that the proposed OCT²Former can achieve competitive performance on retinal OCTA vessel segmentation tasks.

MINet: Multi-scale input network for fundus microvascular segmentation

Vessel segmentation in fundus images is a key procedure in the diagnosis of ophthalmic diseases, which can play a role in assisting doctors in diagnosis. Although current deep learning-based methods can achieve high accuracy in segmenting fundus vessel images, the results are not satisfactory in segmenting microscopic vessels that are close to the background region. The reason for this problem is that thin blood vessels contain very little information, with the convolution operation of each layer in the deep network, this part of the information will be randomly lost. To improve the segmentation ability of the small blood vessel region, a multi-input network (MINet) was proposed to segment vascular regions more accurately. We designed a multi-input fusion module (MIF) in the encoder, which is proposed to acquire multi-scale features in the encoder stage while preserving the microvessel feature information. In addition, to further aggregate multi-scale information from adjacent regions, a multi-scale atrous spatial pyramid (MASP) module is proposed. This module can enhance the extraction of vascular information without reducing the resolution loss. In order to better recover segmentation results with details, we designed a refinement module, which acts on the last layer of the network output to refine the results of the last layer of the network to get more accurate segmentation results. We use the HRF, CHASE_DB1 public dataset to validate the fundus vessel segmentation performance of the MINet model. Also, we merged these two public datasets with our collected Ultra-widefield fundus image (UWF) data as one dataset to test the generalization ability of the model. Experimental results show that MINet achieves an F₁ score of 0.8324 on the microvessel segmentation task, achieving a high accuracy compared to the current mainstream models.

MG-Net: Multi-level global-aware network for thymoma segmentation

BACKGROUND AND OBJECTIVE

Automatic thymoma segmentation in preoperative contrast-enhanced computed tomography (CECT) images makes great sense for diagnosis. Although convolutional neural networks (CNNs) are distinguished in medical image segmentation, they are challenged by thymomas with various shapes, scales and textures, owing to the intrinsic locality of convolution operations. In order to overcome this deficit, we built a deep learning network with enhanced global-awareness for thymoma segmentation.

METHODS

We propose a multi-level global-aware network (MG-Net) for thymoma segmentation, in which the multi-level feature interaction and integration are jointly designed to enhance the global-awareness of CNNs. Particularly, we design the cross-attention block (CAB) to calculate pixel-wise interactions of multi-level features, resulting in the Global Enhanced Convolution Block, which can enable the network to handle various thymomas by strengthening the global-awareness of the encoder. We further devise the Global Spatial Attention Module to integrate coarse- and fine-grain information for enhancing the semantic consistency between the encoder and decoder with CABs. We also develop an Adaptive Attention Fusion Module to adaptively aggregate different semantic-scale features in the decoder to preserve comprehensive details.

RESULTS

The MG-Net has been evaluated against several state-of-the-art models on the self-collected CECT dataset and NIH Pancreas-CT dataset. Results suggest that all designed components are effective, and MG-Net has superior segmentation performance and generalization ability over existing models.

CONCLUSION

Both the qualitative and quantitative experimental results indicate that our MG-Net with global-aware ability can achieve accurate thymoma segmentation and has generalization ability in different tasks. The code is available at: https://github.com/Leejyuan/MGNet.