A Three-Stage Deep Learning Model for Accurate Retinal Vessel Segmentation

(DA-U)2Net: double attention U2Net for retinal vessel segmentation

BACKGROUND

Morphological changes in the retina are crucial and serve as valuable references in the clinical diagnosis of ophthalmic and cardiovascular diseases. However, the retinal vascular structure is complex, making manual segmentation time-consuming and labor-intensive.

METHODS

This paper proposes a retinal segmentation network that integrates feature channel attention and the Convolutional Block Attention Module (CBAM) attention within the U2Net model. First, a feature channel attention module is introduced into the RSU (Residual Spatial Unit) block of U2Net, forming an Attention-RSU block, which focuses more on significant areas during feature extraction and suppresses the influence of noise; Second, a Spatial Attention Module (SAM) is introduced into the high-resolution module of Attention-RSU to enrich feature extraction from both spatial and channel dimensions, and a Channel Attention Module (CAM) is integrated into the lowresolution module of Attention-RSU, which uses dual channel attention to reduce detail loss.Finally, dilated convolution is applied during the upscaling and downscaling processes to expand the receptive field in low-resolution states, allowing the model to better integrate contextual information.

RESULTS

The evaluation across multiple clinical datasets demonstrated excellent performance on various metrics, with an accuracy (ACC) of 98.71%.

CONCLUSION

The proposed Network is general enough and we believe it can be easily extended to other medical image segmentation tasks where large scale variation and complicated features are the main challenges.

Multi scale multi attention network for blood vessel segmentation in fundus images

Precise segmentation of retinal vasculature is crucial for the early detection, diagnosis, and treatment of vision-threatening ailments. However, this task is challenging due to limited contextual information, variations in vessel thicknesses, the complexity of vessel structures, and the potential for confusion with lesions. In this paper, we introduce a novel approach, the MSMA Net model, which overcomes these challenges by replacing traditional convolution blocks and skip connections with an improved multi-scale squeeze and excitation block (MSSE Block) and Bottleneck residual paths (B-Res paths) with spatial attention blocks (SAB). Our experimental findings on publicly available datasets of fundus images, specifically DRIVE, STARE, CHASE_DB1, HRF and DR HAGIS consistently demonstrate that our approach outperforms other segmentation techniques, achieving higher accuracy, sensitivity, Dice score, and area under the receiver operator characteristic (AUC) in the segmentation of blood vessels with different thicknesses, even in situations involving diverse contextual information, the presence of coexisting lesions, and intricate vessel morphologies.

A Novel Ensemble Meta-Model for Enhanced Retinal Blood Vessel Segmentation Using Deep Learning Architectures

Background: Retinal blood vessel segmentation plays an important role in diagnosing retinal diseases such as diabetic retinopathy, glaucoma, and hypertensive retinopathy. Accurate segmentation of blood vessels in retinal images presents a challenging task due to noise, low contrast, and the complex morphology of blood vessel structures. Methods: In this study, we propose a novel ensemble learning framework combining four deep learning architectures: U-Net, ResNet50, U-Net with a ResNet50 backbone, and U-Net with a transformer block. Each architecture is customized to enhance feature extraction and segmentation performance. The models are trained on the DRIVE and STARE datasets to improve the degree of generalization and evaluated using the performance metric accuracy, F1-Score, sensitivity, specificity, and AUC. Results: The ensemble meta-model integrates predictions from these architectures using a stacking approach, achieving state-of-the-art performance with an accuracy of 0.9778, an AUC of 0.9912, and an F1-Score of 0.8231. These results demonstrate the performance of the proposed technique in identifying thin retinal blood vessels. Conclusions: A comparative analysis using qualitative and quantitative results with individual models highlights the robustness of the ensemble framework, especially under conditions of noise and poor visibility.

A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs

Background

Vessel segmentation in fundus photography has become a cornerstone technique for disease analysis. Within this field, Ultra-WideField (UWF) fundus images offer distinct advantages, including an expansive imaging range, detailed lesion data, and minimal adverse effects. However, the high resolution and low contrast inherent to UWF fundus images present significant challenges for accurate segmentation using deep learning methods, thereby complicating disease analysis in this context.

Methods

To address these issues, this study introduces M3B-Net, a novel multi-modal, multi-branch framework that leverages fundus fluorescence angiography (FFA) images to improve retinal vessel segmentation in UWF fundus images. Specifically, M3B-Net tackles the low segmentation accuracy caused by the inherently low contrast of UWF fundus images. Additionally, we propose an enhanced UWF-based segmentation network in M3B-Net, specifically designed to improve the segmentation of fine retinal vessels. The segmentation network includes the Selective Fusion Module (SFM), which enhances feature extraction within the segmentation network by integrating features generated during the FFA imaging process. To further address the challenges of high-resolution UWF fundus images, we introduce a Local Perception Fusion Module (LPFM) to mitigate context loss during the segmentation cut-patch process. Complementing this, the Attention-Guided Upsampling Module (AUM) enhances segmentation performance through convolution operations guided by attention mechanisms.

Results

Extensive experimental evaluations demonstrate that our approach significantly outperforms existing state-of-the-art methods for UWF fundus image segmentation.

AI-based 3D analysis of retinal vasculature associated with retinal diseases using OCT angiography

Retinal vasculature is the only vascular system in the human body that can be observed in a non-invasive manner, with a phenotype associated with a wide range of ocular, cerebral, and cardiovascular diseases. OCT and OCT angiography (OCTA) provide powerful imaging methods to visualize the three-dimensional morphological and functional information of the retina. In this study, based on OCT and OCTA multimodal inputs, a multitask convolutional neural network model was built to realize 3D segmentation of retinal blood vessels and disease classification for different retinal diseases, overcoming the limitations of existing methods that can only perform 2D analysis of OCTA. Two hundred thirty sets of OCT and OCTA data from 109 patients, including 138,000 cross-sectional images in normal and diseased eyes (age-related macular degeneration, retinal vein occlusion, and central serous chorioretinopathy), were collected from four commercial OCT systems for model training, validation, and testing. Experimental results verified that the proposed method was able to achieve a DICE coefficient of 0.956 for 3D segmentation of blood vessels and an accuracy of 91.49% for disease classification, and further enabled us to evaluate the 3D reconstruction of retinal vessels, explore the interlayer connections of superficial and deep vasculatures, and reveal the 3D quantitative vessel characteristics in different retinal diseases.

TCDDU-Net: combining transformer and convolutional dual-path decoding U-Net for retinal vessel segmentation

Accurate segmentation of retinal blood vessels is crucial for enhancing diagnostic efficiency and preventing disease progression. However, the small size and complex structure of retinal blood vessels, coupled with low contrast in corresponding fundus images, pose significant challenges for this task. We propose a novel approach for retinal vessel segmentation, which combines the transformer and convolutional dual-path decoding U-Net (TCDDU-Net). We propose the selective dense connection swin transformer block, which converts the input feature map into patches, introduces MLPs to generate probabilities, and performs selective fusion at different stages. This structure forms a dense connection framework, enabling the capture of long-distance dependencies and effective fusion of features across different stages. The subsequent stage involves the design of the background decoder, which utilizes deformable convolution to learn the background information of retinal vessels by treating them as segmentation objects. This is then combined with the foreground decoder to form a dual-path decoding U-Net. Finally, the foreground segmentation results and the processed background segmentation results are fused to obtain the final retinal vessel segmentation map. To evaluate the effectiveness of our method, we performed experiments on the DRIVE, STARE, and CHASE datasets for retinal vessel segmentation. Experimental results show that the segmentation accuracies of our algorithms are 96.98, 97.40, and 97.23, and the AUC metrics are 98.68, 98.56, and 98.50, respectively.In addition, we evaluated our methods using F1 score, specificity, and sensitivity metrics. Through a comparative analysis, we found that our proposed TCDDU-Net method effectively improves retinal vessel segmentation performance and achieves impressive results on multiple datasets compared to existing methods.

A fundus vessel segmentation method based on double skip connections combined with deep supervision

Background

Fundus vessel segmentation is vital for diagnosing ophthalmic diseases like central serous chorioretinopathy (CSC), diabetic retinopathy, and glaucoma. Accurate segmentation provides crucial vessel morphology details, aiding the early detection and intervention of ophthalmic diseases. However, current algorithms struggle with fine vessel segmentation and maintaining sensitivity in complex regions. Challenges also stem from imaging variability and poor generalization across multimodal datasets, highlighting the need for more advanced algorithms in clinical practice.

Methods

This paper aims to explore a new vessel segmentation method to alleviate the above problems. We propose a fundus vessel segmentation model based on a combination of double skip connections, deep supervision, and TransUNet, namely DS2TUNet. Initially, the original fundus images are improved through grayscale conversion, normalization, histogram equalization, gamma correction, and other preprocessing techniques. Subsequently, by utilizing the U-Net architecture, the preprocessed fundus images are segmented to obtain the final vessel information. Specifically, the encoder firstly incorporates the ResNetV1 downsampling, dilated convolution downsampling, and Transformer to capture both local and global features, which upgrades its vessel feature extraction ability. Then, the decoder introduces the double skip connections to facilitate upsampling and refine segmentation outcomes. Finally, the deep supervision module introduces multiple upsampling vessel features from the decoder into the loss function, so that the model can learn vessel feature representations more effectively and alleviate gradient vanishing during the training phase.

Results

Extensive experiments on publicly available multimodal fundus datasets such as DRIVE, CHASE_DB1, and ROSE-1 demonstrate that the DS2TUNet model attains F1-scores of 0.8195, 0.8362, and 0.8425, with Accuracy of 0.9664, 0.9741, and 0.9557, Sensitivity of 0.8071, 0.8101, and 0.8586, and Specificity of 0.9823, 0.9869, and 0.9713, respectively. Additionally, the model also exhibits excellent test performance on the clinical fundus dataset CSC, with F1-score of 0.7757, Accuracy of 0.9688, Sensitivity of 0.8141, and Specificity of 0.9801 based on the weight trained on the CHASE_DB1 dataset. These results comprehensively validate that the proposed method obtains good performance in fundus vessel segmentation, thereby aiding clinicians in the further diagnosis and treatment of fundus diseases in terms of effectiveness and feasibility.

A Comparative Study of AI-Based Automated and Manual Postprocessing of Head and Neck CT Angiography: An Independent External Validation with Multi-Vendor and Multi-Center Data

PURPOSE

To externally validate the performance of automated postprocessing (AP) on head and neck CT Angiography (CTA) and compare it with manual postprocessing (MP).

METHODS

This retrospective study included head and neck CTA-exams of patients from three tertiary hospitals acquired on CT scanners from five manufacturers. AP was performed by CerebralDoc. The image quality was assessed using Likert scales, and the qualitative and quantitative diagnostic performance of arterial stenosis and aneurysm, postprocessing time, and scanning radiation dose were also evaluated.

RESULTS

A total of 250 patients were included. Among these, 55 patients exhibited significant stenosis (≥ 50%), and 33 patients had aneurysms, diagnosed using original CTA datasets and corresponding multiplanar reconstructions as the reference. While the scores of the V4 segment and the edge of the M1 segment on volume rendering (VR), as well as the C4 segment on maximum intensity projection (MIP), were significantly lower with AP compared to MP across vendors (all P < 0.05), most scores in AP demonstrated image quality that was either superior to or comparable with that of MP. Furthermore, the diagnostic performance of AP was either superior to or comparable with that of MP. Moreover, AP also exhibited advantages in terms of postprocessing time and radiation dose when compared to MP (P < 0.001).

CONCLUSION

The AP of CerebralDoc presents clear advantages over MP and holds significant clinical value. However, further optimization is required in the image quality of the V4 and M1 segments on VR as well as the C4 segment on MIP.

Skeleton-guided multi-scale dual-coordinate attention aggregation network for retinal blood vessel segmentation

Deep learning plays a pivotal role in retinal blood vessel segmentation for medical diagnosis. Despite their significant efficacy, these techniques face two major challenges. Firstly, they often neglect the severe class imbalance in fundus images, where thin vessels in the foreground are proportionally minimal. Secondly, they are susceptible to poor image quality and blurred vessel edges, resulting in discontinuities or breaks in vascular structures. In response, this paper proposes the Skeleton-guided Multi-scale Dual-coordinate Attention Aggregation (SMDAA) network for retinal vessel segmentation. SMDAA comprises three innovative modules: Dual-coordinate Attention (DCA), Unbalanced Pixel Amplifier (UPA), and Vessel Skeleton Guidance (VSG). DCA, integrating Multi-scale Coordinate Feature Aggregation (MCFA) and Scale Coordinate Attention Decoding (SCAD), meticulously analyzes vessel structures across various scales, adept at capturing intricate details, thereby significantly enhancing segmentation accuracy. To address class imbalance, we introduce UPA, dynamically allocating more attention to misclassified pixels, ensuring precise extraction of thin and small blood vessels. Moreover, to preserve vessel structure continuity, we integrate vessel anatomy and develop the VSG module to connect fragmented vessel segments. Additionally, a Feature-level Contrast (FCL) loss is introduced to capture subtle nuances within the same category, enhancing the fidelity of retinal blood vessel segmentation. Extensive experiments on three public datasets (DRIVE, STARE, and CHASE_DB1) demonstrate superior performance in comparison to current methods. The code is available at https://github.com/wangwxr/SMDAA_NET.

Retinal Vessel Segmentation by a Transformer-U-Net Hybrid Model With Dual-Path Decoder

This paper introduces an effective and efficient framework for retinal vessel segmentation. First, we design a Transformer-CNN hybrid model in which a Transformer module is inserted inside the U-Net to capture long-range interactions. Second, we design a dual-path decoder in the U-Net framework, which contains two decoding paths for multi-task outputs. Specifically, we train the extra decoder to predict vessel skeletons as an auxiliary task which helps the model learn balanced features. The proposed framework, named as TSNet, not only achieves good performances in a fully supervised learning manner but also enables a rough skeleton annotation process. The annotators only need to roughly delineate vessel skeletons instead of giving precise pixel-wise vessel annotations. To learn with rough skeleton annotations plus a few precise vessel annotations, we propose a skeleton semi-supervised learning scheme. We adopt a mean teacher model to produce pseudo vessel annotations and conduct annotation correction for roughly labeled skeletons annotations. This learning scheme can achieve promising performance with fewer annotation efforts. We have evaluated TSNet through extensive experiments on five benchmarking datasets. Experimental results show that TSNet yields state-of-the-art performances on retinal vessel segmentation and provides an efficient training scheme in practice.

Systematic Review of Retinal Blood Vessels Segmentation Based on AI-driven Technique

Image segmentation is a crucial task in computer vision and image processing, with numerous segmentation algorithms being found in the literature. It has important applications in scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, image compression, among others. In light of this, the widespread popularity of deep learning (DL) and machine learning has inspired the creation of fresh methods for segmenting images using DL and ML models respectively. We offer a thorough analysis of this recent literature, encompassing the range of ground-breaking initiatives in semantic and instance segmentation, including convolutional pixel-labeling networks, encoder-decoder architectures, multi-scale and pyramid-based methods, recurrent networks, visual attention models, and generative models in adversarial settings. We study the connections, benefits, and importance of various DL- and ML-based segmentation models; look at the most popular datasets; and evaluate results in this Literature.

Enhancing vascular network visualization in 3D photoacoustic imaging: in vivo experiments with a vasculature filter

Filter-based vessel enhancement algorithms facilitate the extraction of vascular networks from medical images. Traditional filter-based algorithms struggle with high noise levels in images with false vessel extraction, and a low standard deviation (σ) value may introduce gaps at the centers of wide vessels. In this paper, a robust technique with less sensitivity to parameter tuning and better noise suppression than other filter-based methods for two-dimensional and three-dimensional images is implemented. In this study, we propose a filter that employs non-local means (NLM) for denoising, applying the vesselness function to suppress blob-like structures and filling the gaps in wide vessels without compromising edge quality or details. Acoustic resolution photoacoustic microscopy (AR-PAM) systems generate high-resolution volumetric photoacoustic images, but their vascular structure imaging suffers from out-of-focal signal-to-noise ratio (SNR) and lateral resolution loss. Implementing a synthetic aperture focusing technique (SAFT) based on a virtual detector (VD) improves out-of-focal region resolution and SNR. Combining the proposed filter with the SAFT algorithm enhances vascular structural imaging in AR-PAM systems. The proposed method is robust and applicable for animal tissues with less error of vasculature structure extraction in comparison to traditional fliter-based methods like Frangi and Sato filter. Also, the method is faster in terms of processing speed and less tuning parameters. We applied the method to a digital phantom to validate our approach and conducted in vivo experiments to demonstrate its superiority for real volumetric tissue imaging.

LMBiS-Net: A lightweight bidirectional skip connection based multipath CNN for retinal blood vessel segmentation

Blinding eye diseases are often related to changes in retinal structure, which can be detected by analysing retinal blood vessels in fundus images. However, existing techniques struggle to accurately segment these delicate vessels. Although deep learning has shown promise in medical image segmentation, its reliance on specific operations can limit its ability to capture crucial details such as the edges of the vessel. This paper introduces LMBiS-Net, a lightweight convolutional neural network designed for the segmentation of retinal vessels. LMBiS-Net achieves exceptional performance with a remarkably low number of learnable parameters (only 0.172 million). The network used multipath feature extraction blocks and incorporates bidirectional skip connections for the information flow between the encoder and decoder. In addition, we have optimised the efficiency of the model by carefully selecting the number of filters to avoid filter overlap. This optimisation significantly reduces training time and improves computational efficiency. To assess LMBiS-Net's robustness and ability to generalise to unseen data, we conducted comprehensive evaluations on four publicly available datasets: DRIVE, STARE, CHASE_DB1, and HRF The proposed LMBiS-Net achieves significant performance metrics in various datasets. It obtains sensitivity values of 83.60%, 84.37%, 86.05%, and 83.48%, specificity values of 98.83%, 98.77%, 98.96%, and 98.77%, accuracy (acc) scores of 97.08%, 97.69%, 97.75%, and 96.90%, and AUC values of 98.80%, 98.82%, 98.71%, and 88.77% on the DRIVE, STARE, CHEASE_DB, and HRF datasets, respectively. In addition, it records F1 scores of 83.43%, 84.44%, 83.54%, and 78.73% on the same datasets. Our evaluations demonstrate that LMBiS-Net achieves high segmentation accuracy (acc) while exhibiting both robustness and generalisability across various retinal image datasets. This combination of qualities makes LMBiS-Net a promising tool for various clinical applications.

LDMRes-Net: A Lightweight Neural Network for Efficient Medical Image Segmentation on IoT and Edge Devices

In this study, we propose LDMRes-Net, a lightweight dual-multiscale residual block-based convolutional neural network tailored for medical image segmentation on IoT and edge platforms. Conventional U-Net-based models face challenges in meeting the speed and efficiency demands of real-time clinical applications, such as disease monitoring, radiation therapy, and image-guided surgery. In this study, we present the Lightweight Dual Multiscale Residual Block-based Convolutional Neural Network (LDMRes-Net), which is specifically designed to overcome these difficulties. LDMRes-Net overcomes these limitations with its remarkably low number of learnable parameters (0.072 M), making it highly suitable for resource-constrained devices. The model's key innovation lies in its dual multiscale residual block architecture, which enables the extraction of refined features on multiple scales, enhancing overall segmentation performance. To further optimize efficiency, the number of filters is carefully selected to prevent overlap, reduce training time, and improve computational efficiency. The study includes comprehensive evaluations, focusing on the segmentation of the retinal image of vessels and hard exudates crucial for the diagnosis and treatment of ophthalmology. The results demonstrate the robustness, generalizability, and high segmentation accuracy of LDMRes-Net, positioning it as an efficient tool for accurate and rapid medical image segmentation in diverse clinical applications, particularly on IoT and edge platforms. Such advances hold significant promise for improving healthcare outcomes and enabling real-time medical image analysis in resource-limited settings.

WSA-MP-Net: Weak-signal-attention and multi-scale perception network for microvascular extraction in optical-resolution photoacoustic microcopy

The unique advantage of optical-resolution photoacoustic microscopy (OR-PAM) is its ability to achieve high-resolution microvascular imaging without exogenous agents. This ability has excellent potential in the study of tissue microcirculation. However, tracing and monitoring microvascular morphology and hemodynamics in tissues is challenging because the segmentation of microvascular in OR-PAM images is complex due to the high density, structure complexity, and low contrast of vascular structures. Various microvasculature extraction techniques have been developed over the years but have many limitations: they cannot consider both thick and thin blood vessel segmentation simultaneously, they cannot address incompleteness and discontinuity in microvasculature, there is a lack of open-access datasets for DL-based algorithms. We have developed a novel segmentation approach to extract vascularity in OR-PAM images using a deep learning network incorporating a weak signal attention mechanism and multi-scale perception (WSA-MP-Net) model. The proposed WSA network focuses on weak and tiny vessels, while the MP module extracts features from different vessel sizes. In addition, Hessian-matrix enhancement is incorporated into the pre-and post-processing of the input and output data of the network to enhance vessel continuity. We constructed normal vessel (NV-ORPAM, 660 data pairs) and tumor vessel (TV-ORPAM, 1168 data pairs) datasets to verify the performance of the proposed method. We developed a semi-automatic annotation algorithm to obtain the ground truth for our network optimization. We applied our optimized model successfully to monitor glioma angiogenesis in mouse brains, thus demonstrating the feasibility and excellent generalization ability of our model. Compared to previous works, our proposed WSA-MP-Net extracts a significant number of microvascular while maintaining vessel continuity and signal fidelity. In quantitative analysis, the indicator values of our method improved by about 1.3% to 25.9%. We believe our proposed approach provides a promising way to extract a complete and continuous microvascular network of OR-PAM and enables its use in many microvascular-related biological studies and medical diagnoses.

ConvMedSegNet: A multi-receptive field depthwise convolutional neural network for medical image segmentation

In order to achieve highly precise medical image segmentation, this paper presents ConvMedSegNet, a novel convolutional neural network designed with a U-shaped architecture that seamlessly integrates two crucial modules: the multi-receptive field depthwise convolution module (MRDC) and the guided fusion module (GF). The MRDC module's primary function is to capture texture information of varying sizes through multi-scale convolutional layers. This information is subsequently utilized to enhance the correlation of global feature data by expanding the network's width. This strategy adeptly preserves the inherent inductive biases of convolution while concurrently amplifying the network's ability to establish dependencies on global information. Conversely, the GF module assumes responsibility for implementing multi-scale feature fusion by connecting the encoder and decoder components. It facilitates the transfer of information between features that are separated over substantial distances through guided fusion, effectively minimizing the loss of critical data. In experiments conducted on public medical image datasets such as BUSI and ISIC2018, ConvMedSegNet outperforms several advanced competing methods, yielding superior results. Additionally, the code can be accessed at https://github.com/csust-yixin/ConvMedSegNet.

Deep Learning based Retinal Vessel Caliber Measurement and the Association with Hypertension

PURPOSE

To develop a highly efficient and fully automated method that measures retinal vessel caliber using digital retinal photographs and evaluate the association between retinal vessel caliber and hypertension.

METHODS

The subjects of this study were from two sources in Beijing, China, a hypertension case-control study from Tongren Hospital (Tongren study) and a community-based atherosclerosis cohort from Peking University First Hospital (Shougang study). Retinal vessel segmentation and arteriovenous classification were achieved simultaneously by a customized deep learning model. Two experienced ophthalmologists evaluated whether retinal vessels were correctly segmented and classified. The ratio of incorrectly segmented and classified retinal vessels was used to measure the accuracy of the model's recognition. Central retinal artery equivalents, central retinal vein equivalents and arteriolar-to-venular diameter ratio were computed to analyze the association between retinal vessel caliber and the risk of hypertension. The association was then compared to that derived from the widely used semi-automated software (Integrative Vessel Analysis).

RESULTS

The deep learning model achieved an arterial recognition error rate of 1.26%, a vein recognition error rate of 0.79%, and a total error rate of 1.03%. Central retinal artery equivalents and arteriolar-to-venular diameter ratio measured by both Integrative Vessel Analysis and deep learning methods were inversely associated with the odds of hypertension in both Tongren and Shougang studies. The comparisons of areas under the receiver operating characteristic curves from the proposed deep learning method and Integrative Vessel Analysis were all not significantly different (p > .05).

CONCLUSION

The proposed deep learning method showed a comparable diagnostic value to Integrative Vessel Analysis software. Compared with semi-automatic software, our deep learning model has significant advantage in efficiency and can be applied to population screening and risk evaluation.

Recurrent attention U-Net for segmentation and quantification of breast arterial calcifications on synthesized 2D mammograms

Breast arterial calcifications (BAC) are a type of calcification commonly observed on mammograms and are generally considered benign and not associated with breast cancer. However, there is accumulating observational evidence of an association between BAC and cardiovascular disease, the leading cause of death in women. We present a deep learning method that could assist radiologists in detecting and quantifying BAC in synthesized 2D mammograms. We present a recurrent attention U-Net model consisting of encoder and decoder modules that include multiple blocks that each use a recurrent mechanism, a recurrent mechanism, and an attention module between them. The model also includes a skip connection between the encoder and the decoder, similar to a U-shaped network. The attention module was used to enhance the capture of long-range dependencies and enable the network to effectively classify BAC from the background, whereas the recurrent blocks ensured better feature representation. The model was evaluated using a dataset containing 2,000 synthesized 2D mammogram images. We obtained 99.8861% overall accuracy, 69.6107% sensitivity, 66.5758% F-1 score, and 59.5498% Jaccard coefficient, respectively. The presented model achieved promising performance compared with related models.

CS-UNet: Cross-scale U-Net with Semantic-position dependencies for retinal vessel segmentation

Accurate retinal vessel segmentation is the prerequisite for early recognition and treatment of retina-related diseases. However, segmenting retinal vessels is still challenging due to the intricate vessel tree in fundus images, which has a significant number of tiny vessels, low contrast, and lesion interference. For this task, the u-shaped architecture (U-Net) has become the de-facto standard and has achieved considerable success. However, U-Net is a pure convolutional network, which usually shows limitations in global modelling. In this paper, we propose a novel Cross-scale U-Net with Semantic-position Dependencies (CS-UNet) for retinal vessel segmentation. In particular, we first designed a Semantic-position Dependencies Aggregator (SPDA) and incorporate it into each layer of the encoder to better focus on global contextual information by integrating the relationship of semantic and position. To endow the model with the capability of cross-scale interaction, the Cross-scale Relation Refine Module (CSRR) is designed to dynamically select the information associated with the vessels, which helps guide the up-sampling operation. Finally, we have evaluated CS-UNet on three public datasets: DRIVE, CHASE_DB1, and STARE. Compared to most existing state-of-the-art methods, CS-UNet demonstrated better performance.

A retinal vessel segmentation network with multiple-dimension attention and adaptive feature fusion

The incidence of blinding eye diseases is highly correlated with changes in retinal morphology, and is clinically detected by segmenting retinal structures in fundus images. However, some existing methods have limitations in accurately segmenting thin vessels. In recent years, deep learning has made a splash in the medical image segmentation, but the lack of edge information representation due to repetitive convolution and pooling, limits the final segmentation accuracy. To this end, this paper proposes a pixel-level retinal vessel segmentation network with multiple-dimension attention and adaptive feature fusion. Here, a multiple dimension attention enhancement (MDAE) block is proposed to acquire more local edge information. Meanwhile, a deep guidance fusion (DGF) block and a cross-pooling semantic enhancement (CPSE) block are proposed simultaneously to acquire more global contexts. Further, the predictions of different decoding stages are learned and aggregated by an adaptive weight learner (AWL) unit to obtain the best weights for effective feature fusion. The experimental results on three public fundus image datasets show that proposed network could effectively enhance the segmentation performance on retinal blood vessels. In particular, the proposed method achieves AUC of 98.30%, 98.75%, and 98.71% on the DRIVE, CHASE_DB1, and STARE datasets, respectively, while the F1 score on all three datasets exceeded 83%. The source code of the proposed model is available at https://github.com/gegao310/VesselSeg-Pytorch-master.

High-precision retinal blood vessel segmentation based on a multi-stage and dual-channel deep learning network

Objective.The high-precision segmentation of retinal vessels in fundus images is important for the early diagnosis of ophthalmic diseases. However, the extraction for microvessels is challenging due to their characteristics of low contrast and high structural complexity. Although some works have been developed to improve the segmentation ability in thin vessels, they have only been successful in recognizing small vessels with relatively high contrast.Approach.Therefore, we develop a deep learning (DL) framework with a multi-stage and dual-channel network model (MSDC_NET) to further improve the thin-vessel segmentation with low contrast. Specifically, an adaptive image enhancement strategy combining multiple preprocessing and the DL method is firstly proposed to elevate the contrast of thin vessels; then, a two-channel model with multi-scale perception is developed to implement whole- and thin-vessel segmentation; and finally, a series of post-processing operations are designed to extract more small vessels in the predicted maps from thin-vessel channels.Main results.Experiments on DRIVE, STARE and CHASE_DB1 demonstrate the superiorities of the proposed MSDC_NET in extracting more thin vessels in fundus images, and quantitative evaluations on several parameters based on the advanced ground truth further verify the advantages of our proposed DL model. Compared with the previous multi-branch method, the specificity and F1score are improved by about 2.18%, 0.68%, 1.73% and 2.91%, 0.24%, 8.38% on the three datasets, respectively.Significance.This work may provide richer information to ophthalmologists for the diagnosis and treatment of vascular-related ophthalmic diseases.

MRL-Seg: Overcoming Imbalance in Medical Image Segmentation With Multi-Step Reinforcement Learning

Medical image segmentation is a critical task for clinical diagnosis and research. However, dealing with highly imbalanced data remains a significant challenge in this domain, where the region of interest (ROI) may exhibit substantial variations across different slices. This presents a significant hurdle to medical image segmentation, as conventional segmentation methods may either overlook the minority class or overly emphasize the majority class, ultimately leading to a decrease in the overall generalization ability of the segmentation results. To overcome this, we propose a novel approach based on multi-step reinforcement learning, which integrates prior knowledge of medical images and pixel-wise segmentation difficulty into the reward function. Our method treats each pixel as an individual agent, utilizing diverse actions to evaluate its relevance for segmentation. To validate the effectiveness of our approach, we conduct experiments on four imbalanced medical datasets, and the results show that our approach surpasses other state-of-the-art methods in highly imbalanced scenarios. These findings hold substantial implications for clinical diagnosis and research.

An improved supervised and attention mechanism-based U-Net algorithm for retinal vessel segmentation

The segmentation results of retinal blood vessels are crucial for automatically diagnosing ophthalmic diseases such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular diseases. To improve the accuracy of vessel segmentation and better extract information about small vessels and edges, we introduce the U-Net algorithm with a supervised attention mechanism for retinal vessel segmentation. We achieve this by introducing a decoder fusion module (DFM) in the encoding part, effectively combining different convolutional blocks to extract features comprehensively. Additionally, in the decoding part of U-Net, we propose the context squeeze and excitation (CSE) decoding module to enhance important contextual feature information and the detection of tiny blood vessels. For the final output, we introduce the supervised fusion mechanism (SFM), which combines multiple branches from shallow to deep layers, effectively fusing multi-scale features and capturing information from different levels, fully integrating low-level and high-level features to improve segmentation performance. Our experimental results on the public datasets of DRIVE, STARE, and CHASED_B1 demonstrate the excellent performance of our proposed network.

GAN-Based Approach for Diabetic Retinopathy Retinal Vasculature Segmentation

Most diabetes patients develop a condition known as diabetic retinopathy after having diabetes for a prolonged period. Due to this ailment, damaged blood vessels may occur behind the retina, which can even progress to a stage of losing vision. Hence, doctors advise diabetes patients to screen their retinas regularly. Examining the fundus for this requires a long time and there are few ophthalmologists available to check the ever-increasing number of diabetes patients. To address this issue, several computer-aided automated systems are being developed with the help of many techniques like deep learning. Extracting the retinal vasculature is a significant step that aids in developing such systems. This paper presents a GAN-based model to perform retinal vasculature segmentation. The model achieves good results on the ARIA, DRIVE, and HRF datasets.

Evolutionary Architecture Optimization for Retinal Vessel Segmentation

Retinal vessel segmentation (RVS) is crucial in medical image analysis as it helps identify and monitor retinal diseases. Deep learning approaches have shown promising results for RVS, but designing optimal neural network architecture is challenging and time-consuming. Neural architecture search (NAS) is a recent technique that automates the design of neural network architectures within a predefined search space. This study proposes a new NAS method for U-shaped networks, MedUNAS, that discovers deep neural networks with high segmentation performance and lower inference time for RVS problem. We perform opposition-based differential evolution (ODE) and genetic algorithm (GA) to search for the best network structure and compare discrete and continuous encoding strategies on the proposed search space. To the best of our knowledge, this is the first NAS study that performs ODE for RVS problems. The results show that the MedUNAS ODE and GA yield the best and second-best results regarding segmentation performance with less than 50% of the parameters of U-shaped state-of-the-art methods on most of the compared datasets. In addition, the proposed methods outperform the baseline U-Net on four datasets with networks with up to 15 times fewer parameters. Furthermore, ablation studies are performed to evaluate the generalizability of the generated networks to medical image segmentation problems that differ from the trained domain, revealing that such networks can be effectively adapted to new tasks with fine-tuning. The MedUNAS can be a valuable tool for automated and efficient RVS in clinical practice.

Unfolded deep kernel estimation-attention UNet-based retinal image segmentation

Retinal vessel segmentation is a critical process in the automated inquiry of fundus images to screen and diagnose diabetic retinopathy. It is a widespread complication of diabetes that causes sudden vision loss. Automated retinal vessel segmentation can help to detect these changes more accurately and quickly than manual evaluation by an ophthalmologist. The proposed approach aims to precisely segregate blood vessels in retinal images while shortening the complication and computational value of the segmentation procedure. This can help to improve the accuracy and reliability of retinal image analysis and assist in diagnosing various eye diseases. Attention U-Net is an essential architecture in retinal image segmentation in diabetic retinopathy that obtained promising results in improving the segmentation accuracy especially in the situation where the training data and ground truth are limited. This approach involves U-Net with an attention mechanism to mainly focus on applicable regions of the input image along with the unfolded deep kernel estimation (UDKE) method to enhance the effective performance of semantic segmentation models. Extensive experiments were carried out on STARE, DRIVE, and CHASE_DB datasets, and the proposed method achieved good performance compared to existing methods.

Multi-level spatial-temporal and attentional information deep fusion network for retinal vessel segmentation

Objective.Automatic segmentation of fundus vessels has the potential to enhance the judgment ability of intelligent disease diagnosis systems. Even though various methods have been proposed, it is still a demanding task to accurately segment the fundus vessels. The purpose of our study is to develop a robust and effective method to segment the vessels in human color retinal fundus images.Approach.We present a novel multi-level spatial-temporal and attentional information deep fusion network for the segmentation of retinal vessels, called MSAFNet, which enhances segmentation performance and robustness. Our method utilizes the multi-level spatial-temporal encoding module to obtain spatial-temporal information and the Self-Attention module to capture feature correlations in different levels of our network. Based on the encoder and decoder structure, we combine these features to get the final segmentation results.Main results.Through abundant experiments on four public datasets, our method achieves preferable performance compared with other SOTA retinal vessel segmentation methods. Our Accuracy and Area Under Curve achieve the highest scores of 96.96%, 96.57%, 96.48% and 98.78%, 98.54%, 98.27% on DRIVE, CHASE_DB1, and HRF datasets. Our Specificity achieves the highest score of 98.58% and 99.08% on DRIVE and STARE datasets.Significance.The experimental results demonstrate that our method has strong learning and representation capabilities and can accurately detect retinal blood vessels, thereby serving as a potential tool for assisting in diagnosis.

CCS-UNet: a cross-channel spatial attention model for accurate retinal vessel segmentation

Precise segmentation of retinal vessels plays an important role in computer-assisted diagnosis. Deep learning models have been applied to retinal vessel segmentation, but the efficacy is limited by the significant scale variation of vascular structures and the intricate background of retinal images. This paper supposes a cross-channel spatial attention U-Net (CCS-UNet) for accurate retinal vessel segmentation. In comparison to other models based on U-Net, our model employes a ResNeSt block for the encoder-decoder architecture. The block has a multi-branch structure that enables the model to extract more diverse vascular features. It facilitates weight distribution across channels through the incorporation of soft attention, which effectively aggregates contextual information in vascular images. Furthermore, we suppose an attention mechanism within the skip connection. This mechanism serves to enhance feature integration across various layers, thereby mitigating the degradation of effective information. It helps acquire cross-channel information and enhance the localization of regions of interest, ultimately leading to improved recognition of vascular structures. In addition, the feature fusion module (FFM) module is used to provide semantic information for a more refined vascular segmentation map. We evaluated CCS-UNet based on five benchmark retinal image datasets, DRIVE, CHASEDB1, STARE, IOSTAR and HRF. Our proposed method exhibits superior segmentation efficacy compared to other state-of-the-art techniques with a global accuracy of 0.9617/0.9806/0.9766/0.9786/0.9834 and AUC of 0.9863/0.9894/0.9938/0.9902/0.9855 on DRIVE, CHASEDB1, STARE, IOSTAR and HRF respectively. Ablation studies are also performed to evaluate the the relative contributions of different architectural components. Our proposed model is potential for diagnostic aid of retinal diseases.

Dual-path multi-scale context dense aggregation network for retinal vessel segmentation

There has been steady progress in the field of deep learning-based blood vessel segmentation. However, several challenging issues still continue to limit its progress, including inadequate sample sizes, the neglect of contextual information, and the loss of microvascular details. To address these limitations, we propose a dual-path deep learning framework for blood vessel segmentation. In our framework, the fundus images are divided into concentric patches with different scales to alleviate the overfitting problem. Then, a Multi-scale Context Dense Aggregation Network (MCDAU-Net) is proposed to accurately extract the blood vessel boundaries from these patches. In MCDAU-Net, a Cascaded Dilated Spatial Pyramid Pooling (CDSPP) module is designed and incorporated into intermediate layers of the model, enhancing the receptive field and producing feature maps enriched with contextual information. To improve segmentation performance for low-contrast vessels, we propose an InceptionConv (IConv) module, which can explore deeper semantic features and suppress the propagation of non-vessel information. Furthermore, we design a Multi-scale Adaptive Feature Aggregation (MAFA) module to fuse the multi-scale feature by assigning adaptive weight coefficients to different feature maps through skip connections. Finally, to explore the complementary contextual information and enhance the continuity of microvascular structures, a fusion module is designed to combine the segmentation results obtained from patches of different sizes, achieving fine microvascular segmentation performance. In order to assess the effectiveness of our approach, we conducted evaluations on three widely-used public datasets: DRIVE, CHASE-DB1, and STARE. Our findings reveal a remarkable advancement over the current state-of-the-art (SOTA) techniques, with the mean values of Se and F1 scores being an increase of 7.9% and 4.7%, respectively. The code is available at https://github.com/bai101315/MCDAU-Net.

Feature-guided attention network for medical image segmentation

BACKGROUND

U-Net and its variations have achieved remarkable performances in medical image segmentation. However, they have two limitations. First, the shallow layer feature of the encoder always contains background noise. Second, semantic gaps exist between the features of the encoder and the decoder. Skip-connections directly connect the encoder to the decoder, which will lead to the fusion of semantically dissimilar feature maps.

PURPOSE

To overcome these two limitations, this paper proposes a novel medical image segmentation algorithm, called feature-guided attention network, which consists of U-Net, the cross-level attention filtering module (CAFM), and the attention-guided upsampling module (AUM).

METHODS

In the proposed method, the AUM and the CAFM were introduced into the U-Net, where the AUM learns to filter the background noise in the low-level feature map of the encoder and the CAFM tries to eliminate the semantic gap between the encoder and the decoder. Specifically, the AUM adopts a top-down pathway to use the high-level feature map so as to filter the background noise in the low-level feature map of the encoder. The AUM uses the encoder features to guide the upsampling of the corresponding decoder features, thus eliminating the semantic gap between them. Four medical image segmentation tasks, including coronary atherosclerotic plaque segmentation (Dataset A), retinal vessel segmentation (Dataset B), skin lesion segmentation (Dataset C), and multiclass retinal edema lesions segmentation (Dataset D), were used to validate the proposed method.

RESULTS

For Dataset A, the proposed method achieved higher Intersection over Union (IoU) (67.91 ± 3.82 % $67.91\pm 3.82\%$ ), dice (79.39 ± 3.37 % $79.39\pm 3.37\%$ ), accuracy (98.39 ± 0.34 % $98.39\pm 0.34\%$ ), and sensitivity (85.10 ± 3.74 % $85.10\pm 3.74\%$ ) than the previous best method: CA-Net. For Dataset B, the proposed method achieved higher sensitivity (83.50%) and accuracy (97.55%) than the previous best method: SCS-Net. For Dataset C, the proposed method had highest IoU (83.47 ± 0.41 % $83.47\pm 0.41\%$ ) and dice (90.81 ± 0.34 % $90.81\pm 0.34\%$ ) than those of all compared previous methods. For Dataset D, the proposed method had highest dice (average: 81.53%; retina edema area [REA]: 83.78%; pigment epithelial detachment [PED] 77.13%), sensitivity (REA: 89.01%; SRF: 85.50%), specificity (REA: 99.35%; PED: 100.00), and accuracy (98.73%) among all compared previous networks. In addition, the number of parameters of the proposed method was 2.43 M, which is less than CA-Net (3.21 M) and CPF-Net (3.07 M).

CONCLUSIONS

The proposed method demonstrated state-of-the-art performance, outperforming other top-notch medical image segmentation algorithms. The CAFM filtered the background noise in the low-level feature map of the encoder, while the AUM eliminated the semantic gap between the encoder and the decoder. Furthermore, the proposed method was of high computational efficiency.

Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation

Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. However, achieving high pixel-wise accuracy, complete topology structure and robustness to various contrast variations are critical and challenging, and most existing methods focus only on achieving one or two of these aspects. In this paper, we present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach. Specifically, we compute a multiscale affinity field for each pixel, capturing its semantic relationships with neighboring pixels in the predicted mask image. This field represents the local geometry of vessel segments of different sizes, allowing us to learn spatial- and scale-aware adaptive weights to strengthen vessel features. We evaluate our AFN on four different types of vascular datasets: X-ray angiography coronary vessel dataset (XCAD), portal vein dataset (PV), digital subtraction angiography cerebrovascular vessel dataset (DSA) and retinal vessel dataset (DRIVE). Extensive experimental results demonstrate that our AFN outperforms the state-of-the-art methods in terms of both higher accuracy and topological metrics, while also being more robust to various contrast changes.

TBENet:A two-branch boundary enhancement Network for cerebrovascular segmentation

Cerebrovascular segmentation in digital subtraction angiography (DSA) images is the gold standard for clinical diagnosis. However, owing to the complexity of cerebrovascular, automatic cerebrovascular segmentation in DSA images is a challenging task. In this paper, we propose a CNN-based Two-branch Boundary Enhancement Network (TBENet) for automatic segmentation of cerebrovascular in DSA images. The TBENet is inspired by U-Net and designed as an encoder-decoder architecture. We propose an additional boundary branch to segment the boundary of cerebrovascular and a Main and Boundary branches Fusion Module (MBFM) to integrate the boundary branch outcome with the main branch outcome to achieve better segmentation performance. The TBENet was evaluated on HMCDSA (an in-house DSA cerebrovascular dataset), and reaches 0.9611, 0.7486, 0.7152, 0.9860 and 0.9556 in Accuracy, F1 score, Sensitivity, Specificity, and AUC, respectively. Meanwhile, we tested our TBENet on the public vessel segmentation benchmark DRIVE, and the results show that our TBENet can be extended to diverse vessel segmentation tasks.

A lightweight dual-path cascaded network for vessel segmentation in fundus image

Automatic and fast segmentation of retinal vessels in fundus images is a prerequisite in clinical ophthalmic diseases; however, the high model complexity and low segmentation accuracy still limit its application. This paper proposes a lightweight dual-path cascaded network (LDPC-Net) for automatic and fast vessel segmentation. We designed a dual-path cascaded network via two U-shaped structures. Firstly, we employed a structured discarding (SD) convolution module to alleviate the over-fitting problem in both codec parts. Secondly, we introduced the depthwise separable convolution (DSC) technique to reduce the parameter amount of the model. Thirdly, a residual atrous spatial pyramid pooling (ResASPP) model is constructed in the connection layer to aggregate multi-scale information effectively. Finally, we performed comparative experiments on three public datasets. Experimental results show that the proposed method achieved superior performance on the accuracy, connectivity, and parameter quantity, thus proving that it can be a promising lightweight assisted tool for ophthalmic diseases.

Using Artificial Intelligence to Analyse the Retinal Vascular Network: The Future of Cardiovascular Risk Assessment Based on Oculomics? A Narrative Review

The healthcare burden of cardiovascular diseases remains a major issue worldwide. Understanding the underlying mechanisms and improving identification of people with a higher risk profile of systemic vascular disease through noninvasive examinations is crucial. In ophthalmology, retinal vascular network imaging is simple and noninvasive and can provide in vivo information of the microstructure and vascular health. For more than 10 years, different research teams have been working on developing software to enable automatic analysis of the retinal vascular network from different imaging techniques (retinal fundus photographs, OCT angiography, adaptive optics, etc.) and to provide a description of the geometric characteristics of its arterial and venous components. Thus, the structure of retinal vessels could be considered a witness of the systemic vascular status. A new approach called "oculomics" using retinal image datasets and artificial intelligence algorithms recently increased the interest in retinal microvascular biomarkers. Despite the large volume of associated research, the role of retinal biomarkers in the screening, monitoring, or prediction of systemic vascular disease remains uncertain. A PubMed search was conducted until August 2022 and yielded relevant peer-reviewed articles based on a set of inclusion criteria. This literature review is intended to summarize the state of the art in oculomics and cardiovascular disease research.

Separable Paravector Orientation Tensors for Enhancing Retinal Vessels

Robust detection of retinal vessels remains an unsolved research problem, particularly in handling the intrinsic real-world challenges of highly imbalanced contrast between thick vessels and thin ones, inhomogeneous background regions, uneven illumination, and complex geometries of crossing/bifurcations. This paper presents a new separable paravector orientation tensor that addresses these difficulties by characterizing the enhancement of retinal vessels to be dependent on a nonlinear scale representation, invariant to changes in contrast and lighting, responsive for symmetric patterns, and fitted with elliptical cross-sections. The proposed method is built on projecting vessels as a 3D paravector valued function rotated in an alpha quarter domain, providing geometrical, structural, symmetric, and energetic features. We introduce an innovative symmetrical inhibitory scheme that incorporates paravector features for producing a set of directional contrast-independent elongated-like patterns reconstructing vessel tree in orientation tensors. By fitting constraint elliptical volumes via eigensystem analysis, the final vessel tree is produced with a strong and uniform response preserving various vessel features. The validation of proposed method on clinically relevant retinal images with high-quality results, shows its excellent performance compared to the state-of-the-art benchmarks and the second human observers.

Synthesizing multi-frame high-resolution fluorescein angiography images from retinal fundus images using generative adversarial networks

BACKGROUND

Fundus fluorescein angiography (FA) can be used to diagnose fundus diseases by observing dynamic fluorescein changes that reflect vascular circulation in the fundus. As FA may pose a risk to patients, generative adversarial networks have been used to convert retinal fundus images into fluorescein angiography images. However, the available methods focus on generating FA images of a single phase, and the resolution of the generated FA images is low, being unsuitable for accurately diagnosing fundus diseases.

METHODS

We propose a network that generates multi-frame high-resolution FA images. This network consists of a low-resolution GAN (LrGAN) and a high-resolution GAN (HrGAN), where LrGAN generates low-resolution and full-size FA images with global intensity information, HrGAN takes the FA images generated by LrGAN as input to generate multi-frame high-resolution FA patches. Finally, the FA patches are merged into full-size FA images.

RESULTS

Our approach combines supervised and unsupervised learning methods and achieves better quantitative and qualitative results than using either method alone. Structural similarity (SSIM), normalized cross-correlation (NCC) and peak signal-to-noise ratio (PSNR) were used as quantitative metrics to evaluate the performance of the proposed method. The experimental results show that our method achieves better quantitative results with structural similarity of 0.7126, normalized cross-correlation of 0.6799, and peak signal-to-noise ratio of 15.77. In addition, ablation experiments also demonstrate that using a shared encoder and residual channel attention module in HrGAN is helpful for the generation of high-resolution images.

CONCLUSIONS

Overall, our method has higher performance for generating retinal vessel details and leaky structures in multiple critical phases, showing a promising clinical diagnostic value.

CNTSeg: A multimodal deep-learning-based network for cranial nerves tract segmentation

The segmentation of cranial nerves (CNs) tracts based on diffusion magnetic resonance imaging (dMRI) provides a valuable quantitative tool for the analysis of the morphology and course of individual CNs. Tractography-based approaches can describe and analyze the anatomical area of CNs by selecting the reference streamlines in combination with ROIs-based (regions-of-interests) or clustering-based. However, due to the slender structure of CNs and the complex anatomical environment, single-modality data based on dMRI cannot provide a complete and accurate description, resulting in low accuracy or even failure of current algorithms in performing individualized CNs segmentation. In this work, we propose a novel multimodal deep-learning-based multi-class network for automated cranial nerves tract segmentation without using tractography, ROI placement or clustering, called CNTSeg. Specifically, we introduced T1w images, fractional anisotropy (FA) images, and fiber orientation distribution function (fODF) peaks into the training data set, and design the back-end fusion module which uses the complementary information of the interphase feature fusion to improve the segmentation performance. CNTSeg has achieved the segmentation of 5 pairs of CNs (i.e. optic nerve CN II, oculomotor nerve CN III, trigeminal nerve CN V, and facial-vestibulocochlear nerve CN VII/VIII). Extensive comparisons and ablation experiments show promising results and are anatomically convincing even for difficult tracts. The code will be openly available at https://github.com/IPIS-XieLei/CNTSeg.

MIC-Net: multi-scale integrated context network for automatic retinal vessel segmentation in fundus image

PURPOSE

Accurate retinal vessel segmentation is of great value in the auxiliary screening of various diseases. However, due to the low contrast between the ends of the branches of the fundus blood vessels and the background, and the variable morphology of the optic disc and cup in the retinal image, the task of high-precision retinal blood vessel segmentation still faces difficulties.

METHOD

This paper proposes a multi-scale integrated context network, MIC-Net, which fully fuses the encoder-decoder features, and extracts multi-scale information. First, a hybrid stride sampling (HSS) block was designed in the encoder to minimize the loss of helpful information caused by the downsampling operation. Second, a dense hybrid dilated convolution (DHDC) was employed in the connection layer. On the premise of preserving feature resolution, it can perceive richer contextual information. Third, a squeeze-and-excitation with residual connections (SERC) was introduced in the decoder to adjust the channel attention adaptively. Finally, we utilized a multi-layer feature fusion mechanism in the skip connection part, which enables the network to consider both low-level details and high-level semantic information.

RESULTS

We evaluated the proposed method on three public datasets DRIVE, STARE and CHASE. In the experimental results, the Area under the receiver operating characteristic (ROC) and the accuracy rate (Acc) achieved high performances of 98.62%/97.02%, 98.60%/97.76% and 98.73%/97.38%, respectively.

CONCLUSIONS

Experimental results show that the proposed method can obtain comparable segmentation performance compared with the state-of-the-art (SOTA) methods. Specifically, the proposed method can effectively reduce the small blood vessel segmentation error, thus proving it a promising tool for auxiliary diagnosis of ophthalmic diseases.

Retinal blood vessel segmentation by using the MS-LSDNet network and geometric skeleton reconnection method

Automatic retinal blood vessel segmentation is a key link in the diagnosis of ophthalmic diseases. Recent deep learning methods have achieved high accuracy in vessel segmentation but still face challenges in maintaining vascular structural connectivity. Therefore, this paper proposes a novel retinal blood vessel segmentation strategy that includes three stages: vessel structure detection, vessel branch extraction and broken vessel segment reconnection. First, we propose a multiscale linear structure detection network (MS-LSDNet), which improves the detection ability of fine blood vessels by learning the types of rich hierarchical features. In addition, to maintain the connectivity of the vascular structure in the process of binarization of the vascular probability map, an adaptive hysteresis threshold method for vascular extraction is proposed. Finally, we propose a vascular tree structure reconstruction algorithm based on a geometric skeleton to connect the broken vessel segments. Experimental results on three publicly available datasets show that compared with current state-of-the-art algorithms, our strategy effectively maintains the connectivity of retinal vascular tree structure.

Retinal Blood-Vessel Extraction Using Weighted Kernel Fuzzy C-Means Clustering and Dilation-Based Functions

Automated blood-vessel extraction is essential in diagnosing Diabetic Retinopathy (DR) and other eye-related diseases. However, the traditional methods for extracting blood vessels tend to provide low accuracy when dealing with difficult situations, such as extracting both micro and large blood vessels simultaneously with low-intensity images and blood vessels with DR. This paper proposes a complete preprocessing method to enhance original retinal images before transferring the enhanced images to a novel blood-vessel extraction method by a combined three extraction stages. The first stage focuses on the fast extraction of retinal blood vessels using Weighted Kernel Fuzzy C-Means (WKFCM) Clustering to draw the vessel feature from the retinal background. The second stage focuses on the accuracy of full-size images to achieve regional vessel feature recognition of large and micro blood vessels and to minimize false extraction. This stage implements the mathematical dilation operator from a trained model called Dilation-Based Function (DBF). Finally, an optimal parameter threshold is empirically determined in the third stage to remove non-vessel features in the binary image and improve the overall vessel extraction results. According to evaluations of the method via the datasets DRIVE, STARE, and DiaretDB0, the proposed WKFCM-DBF method achieved sensitivities, specificities, and accuracy performances of 98.12%, 98.20%, and 98.16%, 98.42%, 98.80%, and 98.51%, and 98.89%, 98.10%, and 98.09%, respectively.

Wave-Net: A lightweight deep network for retinal vessel segmentation from fundus images

Accurate segmentation of retinal vessels from fundus images is fundamental for the diagnosis of numerous diseases of eye, and an automated vessel segmentation method can effectively help clinicians to make accurate diagnosis for the patients and provide the appropriate treatment schemes. It is important to note that both thick and thin vessels play the key role for disease judgements. Because of complex factors, the precise segmentation of thin vessels is still a great challenge, such as the presence of various lesions, image noise, complex backgrounds and poor contrast in the fundus images. Recently, because of the advantage of context feature representation learning capabilities, deep learning has shown a remarkable segmentation performance on retinal vessels. However, it still has some shortcomings on high-precision retinal vessel extraction due to some factors, such as semantic information loss caused by pooling operations, limited receptive field, etc. To address these problems, this paper proposes a new lightweight segmentation network for precise retinal vessel segmentation, which is called as Wave-Net model on account of the whole shape. To alleviate the influence of semantic information loss problem to thin vessels, to acquire more contexts about micro structures and details, a detail enhancement and denoising block (DED) is proposed to improve the segmentation precision on thin vessels, which replaces the simple skip connections of original U-Net. On the other hand, it could well alleviate the influence of the semantic gap problem. Further, faced with limited receptive field, for multi-scale vessel detection, a multi-scale feature fusion block (MFF) is proposed to fuse cross-scale contexts to achieve higher segmentation accuracy and realize effective processing of local feature maps. Experiments indicate that proposed Wave-Net achieves an excellent performance on retinal vessel segmentation while maintaining a lightweight network design compared to other advanced segmentation methods, and it also has shown a better segmentation ability to thin vessels.

An efficient registration-based approach for retinal blood vessel segmentation using generalized Pareto and fatigue pdf

Segmentation of Retinal Blood Vessel (RBV) extraction in the retina images and Registration of segmented RBV structure is implemented to identify changes in vessel structure by ophthalmologists in diagnosis of various illnesses like Glaucoma, Diabetes, and Hypertension's. The Retinal Blood Vessel provides blood to the inner retinal neurons, RBV are located mainly in internal retina but it may partly in the ganglion cell layer, following network failure haven't been identified with past methods. Classifications of accurate RBV and Registration of segmented blood vessels are challenging tasks in the low intensity background of Retinal Image. So, we projected a novel approach of segmentation of RBV extraction used matched filter of Generalized Pareto Probability Distribution Function (pdf) and Registration approach on feature-based segmented retinal blood vessel of Binary Robust Invariant Scalable Key point (BRISK). The BRISK provides the predefined sampling pattern as compared to Pdf. The BRISK feature is implemented for attention point recognition & matching approach for change in vessel structure. The proposed approaches contain 3 levels: pre-processing, matched filter-based Generalized Pareto pdf as a source along with the novel approach of fatigue pdf as a target, and BRISK framework is used for Registration on segmented retinal images of supply & intention images. This implemented system's performance is estimated in experimental analysis by the Average accuracy, Normalized Cross-Correlation (NCC), and computation time process of the segmented retinal source and target image. The NCC is main element to give more statistical information about retinal image segmentation. The proposed approach of Generalized Pareto value pdf has Average Accuracy of 95.21%, NCC of both image pairs is 93%, and Average accuracy of Registration of segmented source images and the target image is 98.51% respectively. The proposed approach of average computational time taken is around 1.4 s, which has been identified on boundary condition of Pdf function.

Recent trends and advances in fundus image analysis: A review

Automated retinal image analysis holds prime significance in the accurate diagnosis of various critical eye diseases that include diabetic retinopathy (DR), age-related macular degeneration (AMD), atherosclerosis, and glaucoma. Manual diagnosis of retinal diseases by ophthalmologists takes time, effort, and financial resources, and is prone to error, in comparison to computer-aided diagnosis systems. In this context, robust classification and segmentation of retinal images are primary operations that aid clinicians in the early screening of patients to ensure the prevention and/or treatment of these diseases. This paper conducts an extensive review of the state-of-the-art methods for the detection and segmentation of retinal image features. Existing notable techniques for the detection of retinal features are categorized into essential groups and compared in depth. Additionally, a summary of quantifiable performance measures for various important stages of retinal image analysis, such as image acquisition and preprocessing, is provided. Finally, the widely used in the literature datasets for analyzing retinal images are described and their significance is emphasized.

Do you need sharpened details? Asking MMDC-Net: Multi-layer multi-scale dilated convolution network for retinal vessel segmentation

Convolutional neural networks (CNN), especially numerous U-shaped models, have achieved great progress in retinal vessel segmentation. However, a great quantity of global information in fundus images has not been fully explored. And the class imbalance problem of background and blood vessels is still serious. To alleviate these issues, we design a novel multi-layer multi-scale dilated convolution network (MMDC-Net) based on U-Net. We propose an MMDC module to capture sufficient global information under diverse receptive fields through a cascaded mode. Then, we place a new multi-layer fusion (MLF) module behind the decoder, which can not only fuse complementary features but filter noisy information. This enables MMDC-Net to capture the blood vessel details after continuous up-sampling. Finally, we employ a recall loss to resolve the class imbalance problem. Extensive experiments have been done on diverse fundus color image datasets, including STARE, CHASEDB1, DRIVE, and HRF. HRF has a large resolution of 3504 × 2336 whereas others have a small resolution of slightly more than 512 × 512. Qualitative and quantitative results verify the superiority of MMDC-Net. Notably, satisfactory accuracy and sensitivity are acquired by our model. Hence, some key blood vessel details are sharpened. In addition, a large number of further validations and discussions prove the effectiveness and generalization of the proposed MMDC-Net.