Full-Resolution Network and Dual-Threshold Iteration for Retinal Vessel and Coronary Angiograph Segmentation

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

Screening chronic kidney disease through deep learning utilizing ultra-wide-field fundus images

To address challenges in screening for chronic kidney disease (CKD), we devised a deep learning-based CKD screening model named UWF-CKDS. It utilizes ultra-wide-field (UWF) fundus images to predict the presence of CKD. We validated the model with data from 23 tertiary hospitals across China. Retinal vessels and retinal microvascular parameters (RMPs) were extracted to enhance model interpretability, which revealed a significant correlation between renal function and RMPs. UWF-CKDS, utilizing UWF images, RMPs, and relevant medical history, can accurately determine CKD status. Importantly, UWF-CKDS exhibited superior performance compared to CTR-CKDS, a model developed using the central region (CTR) cropped from UWF images, underscoring the contribution of the peripheral retina in predicting renal function. The study presents UWF-CKDS as a highly implementable method for large-scale and accurate CKD screening at the population level.

DIAS: A dataset and benchmark for intracranial artery segmentation in DSA sequences

The automated segmentation of Intracranial Arteries (IA) in Digital Subtraction Angiography (DSA) plays a crucial role in the quantification of vascular morphology, significantly contributing to computer-assisted stroke research and clinical practice. Current research primarily focuses on the segmentation of single-frame DSA using proprietary datasets. However, these methods face challenges due to the inherent limitation of single-frame DSA, which only partially displays vascular contrast, thereby hindering accurate vascular structure representation. In this work, we introduce DIAS, a dataset specifically developed for IA segmentation in DSA sequences. We establish a comprehensive benchmark for evaluating DIAS, covering full, weak, and semi-supervised segmentation methods. Specifically, we propose the vessel sequence segmentation network, in which the sequence feature extraction module effectively captures spatiotemporal representations of intravascular contrast, achieving intracranial artery segmentation in 2D+Time DSA sequences. For weakly-supervised IA segmentation, we propose a novel scribble learning-based image segmentation framework, which, under the guidance of scribble labels, employs cross pseudo-supervision and consistency regularization to improve the performance of the segmentation network. Furthermore, we introduce the random patch-based self-training framework, aimed at alleviating the performance constraints encountered in IA segmentation due to the limited availability of annotated DSA data. Our extensive experiments on the DIAS dataset demonstrate the effectiveness of these methods as potential baselines for future research and clinical applications. The dataset and code are publicly available at https://doi.org/10.5281/zenodo.11401368 and https://github.com/lseventeen/DIAS.

An improved semi-supervised segmentation of the retinal vasculature using curvelet-based contrast adjustment and generalized linear model

Diagnosis of most ophthalmic conditions, such as diabetic retinopathy, generally relies on an effective analysis of retinal blood vessels. Techniques that depend solely on the visual observation of clinicians can be tedious and prone to numerous errors. In this article, we propose a semi-supervised automated approach for segmenting blood vessels in retinal color images. Our method effectively combines some classical filters with a Generalized Linear Model (GLM). We first apply the Curvelet Transform along with the Contrast-Limited Histogram Adaptive Equalization (CLAHE) technique to significantly enhance the contrast of vessels in the retinal image during the preprocessing phase. We then use Gabor transform to extract features from the enhanced image. For retinal vasculature identification, we use a GLM learning model with a simple link identity function. Binarization is then performed using an automatic optimal threshold based on the maximum Youden index. A morphological cleaning operation is applied to remove isolated or unwanted segments from the final segmented image. The proposed model is evaluated using statistical parameters on images from three publicly available databases. We achieve average accuracies of 0.9593, 0.9553 and 0.9643, with Receiver Operating Characteristic (ROC) analysis yielding Area Under Curve (AUC) values of 0.9722, 0.9682 and 0.9767 for the CHASE_DB1, STARE and DRIVE databases, respectively. Compared to some of the best results from similar approaches published recently, our results exceed their performance on several datasets.

RetVes segmentation: A pseudo-labeling and feature knowledge distillation optimization technique for retinal vessel channel enhancement

Recent advancements in retinal vessel segmentation, which employ transformer-based and domain-adaptive approaches, show promise in addressing the complexity of ocular diseases such as diabetic retinopathy. However, current algorithms face challenges in effectively accommodating domain-specific variations and limitations of training datasets, which fail to represent real-world conditions comprehensively. Manual inspection by specialists remains time-consuming despite technological progress in medical imaging, underscoring the pressing need for automated and robust segmentation techniques. Additionally, these methods have deficiencies in handling unlabeled target sets, requiring extra preprocessing steps and manual intervention, which hinders their scalability and practical application in clinical settings. This research introduces a novel framework that employs semi-supervised domain adaptation and contrastive pre-training to address these limitations. The proposed model effectively learns from target data by implementing a novel pseudo-labeling approach and feature-based knowledge distillation within a temporal convolutional network (TCN) and extracts robust, domain-independent features. This approach enhances cross-domain adaptation, significantly enhancing the model's versatility and performance in clinical settings. The semi-supervised domain adaptation component overcomes the challenges posed by domain shifts, while pseudo-labeling utilizes the data's inherent structure for enhanced learning, which is particularly beneficial when labeled data is scarce. Evaluated on the DRIVE and CHASE_DB1 datasets, which contain clinical fundus images, the proposed model achieves outstanding performance, with accuracy, sensitivity, specificity, and AUC values of 0.9792, 0.8640, 0.9901, and 0.9868 on DRIVE, and 0.9830, 0.9058, 0.9888, and 0.9950 on CHASE_DB1, respectively, outperforming current state-of-the-art vessel segmentation methods. The partitioning of datasets into training and testing sets ensures thorough validation, while extensive ablation studies with thorough sensitivity analysis of the model's parameters and different percentages of labeled data further validate its robustness.

Effective descriptor extraction strategies for correspondence matching in coronary angiography images

The importance of 3D reconstruction of coronary arteries using multiple coronary angiography (CAG) images has been increasingly recognized in the field of cardiovascular disease management. This process relies on the camera matrix's optimization, needing correspondence info for identical point positions across two images. Therefore, an automatic method for determining correspondence between two CAG images is highly desirable. Despite this need, there is a paucity of research focusing on image matching in the CAG images. Additionally, standard deep learning image matching techniques often degrade due to unique features and noise in CAG images. This study aims to fill this gap by applying a deep learning-based image matching method specifically tailored for the CAG images. We have improved the structure of our point detector and redesigned loss function to better handle sparse labeling and indistinct local features specific to CAG images. Our method include changes to training loss and introduction of a multi-head descriptor structure leading to an approximate 6% improvement. We anticipate that our work will provide valuable insights into adapting techniques from general domains to more specialized ones like medical imaging and serve as an improved benchmark for future endeavors in X-ray image-based correspondence matching.

Unsupervised domain adaptation multi-level adversarial learning-based crossing-domain retinal vessel segmentation

BACKGROUND

The retinal vasculature, a crucial component of the human body, mirrors various illnesses such as cardiovascular disease, glaucoma, and retinopathy. Accurate segmentation of retinal vessels in funduscopic images is essential for diagnosing and understanding these conditions. However, existing segmentation models often struggle with images from different sources, making accurate segmentation in crossing-source fundus images challenging.

METHODS

To address the crossing-source segmentation issues, this paper proposes a novel Multi-level Adversarial Learning and Pseudo-label Denoising-based Self-training Framework (MLAL&PDSF). Expanding on our previously proposed Multiscale Context Gating with Breakpoint and Spatial Dual Attention Network (MCG&BSA-Net), MLAL&PDSF introduces a multi-level adversarial network that operates at both the feature and image layers to align distributions between the target and source domains. Additionally, it employs a distance comparison technique to refine pseudo-labels generated during the self-training process. By comparing the distance between the pseudo-labels and the network predictions, the framework identifies and corrects inaccuracies, thus enhancing the accuracy of the fine vessel segmentation.

RESULTS

We have conducted extensive validation and comparative experiments on the CHASEDB1, STARE, and HRF datasets to evaluate the efficacy of the MLAL&PDSF. The evaluation metrics included the area under the operating characteristic curve (AUC), sensitivity (SE), specificity (SP), accuracy (ACC), and balanced F-score (F1). The performance results from unsupervised domain adaptive segmentation are remarkable: for DRIVE to CHASEDB1, results are AUC: 0.9806, SE: 0.7400, SP: 0.9737, ACC: 0.9874, and F1: 0.8851; for DRIVE to STARE, results are AUC: 0.9827, SE: 0.7944, SP: 0.9651, ACC: 0.9826, and F1: 0.8326.

CONCLUSION

These results demonstrate the effectiveness and robustness of MLAL&PDSF in achieving accurate segmentation results from crossing-domain retinal vessel datasets. The framework lays a solid foundation for further advancements in cross-domain segmentation and enhances the diagnosis and understanding of related diseases.

G2ViT: Graph Neural Network-Guided Vision Transformer Enhanced Network for retinal vessel and coronary angiograph segmentation

Blood vessel segmentation is a crucial stage in extracting morphological characteristics of vessels for the clinical diagnosis of fundus and coronary artery disease. However, traditional convolutional neural networks (CNNs) are confined to learning local vessel features, making it challenging to capture the graph structural information and fail to perceive the global context of vessels. Therefore, we propose a novel graph neural network-guided vision transformer enhanced network (G2ViT) for vessel segmentation. G2ViT skillfully orchestrates the Convolutional Neural Network, Graph Neural Network, and Vision Transformer to enhance comprehension of the entire graphical structure of blood vessels. To achieve deeper insights into the global graph structure and higher-level global context cognizance, we investigate a graph neural network-guided vision transformer module. This module constructs graph-structured representation in an unprecedented manner using the high-level features extracted by CNNs for graph reasoning. To increase the receptive field while ensuring minimal loss of edge information, G2ViT introduces a multi-scale edge feature attention module (MEFA), leveraging dilated convolutions with different dilation rates and the Sobel edge detection algorithm to obtain multi-scale edge information of vessels. To avoid critical information loss during upsampling and downsampling, we design a multi-level feature fusion module (MLF2) to fuse complementary information between coarse and fine features. Experiments on retinal vessel datasets (DRIVE, STARE, CHASE_DB1, and HRF) and coronary angiography datasets (DCA1 and CHUAC) indicate that the G2ViT excels in robustness, generality, and applicability. Furthermore, it has acceptable inference time and computational complexity and presents a new solution for blood vessel segmentation.

Partial class activation mapping guided graph convolution cascaded U-Net for retinal vessel segmentation

Accurate segmentation of retinal vessels in fundus images is of great importance for the diagnosis of numerous ocular diseases. However, due to the complex characteristics of fundus images, such as various lesions, image noise and complex background, the pixel features of some vessels have significant differences, which makes it easy for the segmentation networks to misjudge these vessels as noise, thus affecting the accuracy of the overall segmentation. Therefore, accurately segment retinal vessels in complex situations is still a great challenge. To address the problem, a partial class activation mapping guided graph convolution cascaded U-Net for retinal vessel segmentation is proposed. The core idea of the proposed network is first to use the partial class activation mapping guided graph convolutional network to eliminate the differences of local vessels and generate feature maps with global consistency, and subsequently these feature maps are further refined by segmentation network U-Net to achieve better segmentation results. Specifically, a new neural network block, called EdgeConv, is stacked multiple layers to form a graph convolutional network to realize the transfer an update of information from local to global, so as gradually enhance the feature consistency of graph nodes. Simultaneously, in an effort to suppress the noise information that may be transferred in graph convolution and thus reduce adverse effects of noise on segmentation results, the partial class activation mapping is introduced. The partial class activation mapping can guide the information transmission between graph nodes and effectively activate vessel feature through classification labels, thereby improving the accuracy of segmentation. The performance of the proposed method is validated on four different fundus image datasets. Compared with existing state-of-the-art methods, the proposed method can improve the integrity of vessel to a certain extent when the pixel features of local vessels are significantly different, caused by objective factors such as inappropriate illumination and exudates. Moreover, the proposed method shows robustness when segmenting complex retinal vessels.

Real-time coronary artery segmentation in CAG images: A semi-supervised deep learning strategy

BACKGROUND

When treating patients with coronary artery disease and concurrent renal concerns, we often encounter a conundrum: how to achieve a clearer view of vascular details while minimizing the contrast and radiation doses during percutaneous coronary intervention (PCI). Our goal is to use deep learning (DL) to create a real-time roadmap for guiding PCI. To this end, segmentation, a critical first step, paves the way for detailed vascular analysis. Unlike traditional supervised learning, which demands extensive labeling time and manpower, our strategy leans toward semi-supervised learning. This method not only economizes on labeling efforts but also aims at reducing contrast and radiation exposure.

METHODS AND RESULTS

CAG data sourced from eight tertiary centers in Taiwan, comprising 500 labeled and 8952 unlabeled images. Employing 400 labels for training and reserving 100 for validation, we built a U-Net based network within a teacher-student architecture. The initial teacher model was updated with 8952 unlabeled images inputted, employing a quality control strategy involving consistency regularization and RandAugment. The optimized teacher model produced pseudo-labels for label expansion, which were then utilized to train the final student model. We attained an average dice similarity coefficient of 0.9003 for segmentation, outperforming supervised learning methods with the same label count. Even with only 5 % labels for semi-supervised training, the results surpassed a supervised method with 100 % labels inputted. This semi-supervised approach's advantage extends beyond single-frame prediction, yielding consistently superior results in continuous angiography films.

CONCLUSIONS

High labeling cost hinders DL training. Semi-supervised learning, quality control, and pseudo-label expansion can overcome this. DL-assisted segmentation potentially provides a real-time PCI roadmap and further diminishes radiation and contrast doses.

A Retinal Vessel Segmentation Method Based on the Sharpness-Aware Minimization Model

Retinal vessel segmentation is crucial for diagnosing and monitoring various eye diseases such as diabetic retinopathy, glaucoma, and hypertension. In this study, we examine how sharpness-aware minimization (SAM) can improve RF-UNet's generalization performance. RF-UNet is a novel model for retinal vessel segmentation. We focused our experiments on the digital retinal images for vessel extraction (DRIVE) dataset, which is a benchmark for retinal vessel segmentation, and our test results show that adding SAM to the training procedure leads to notable improvements. Compared to the non-SAM model (training loss of 0.45709 and validation loss of 0.40266), the SAM-trained RF-UNet model achieved a significant reduction in both training loss (0.094225) and validation loss (0.08053). Furthermore, compared to the non-SAM model (training accuracy of 0.90169 and validation accuracy of 0.93999), the SAM-trained model demonstrated higher training accuracy (0.96225) and validation accuracy (0.96821). Additionally, the model performed better in terms of sensitivity, specificity, AUC, and F1 score, indicating improved generalization to unseen data. Our results corroborate the notion that SAM facilitates the learning of flatter minima, thereby improving generalization, and are consistent with other research highlighting the advantages of advanced optimization methods. With wider implications for other medical imaging tasks, these results imply that SAM can successfully reduce overfitting and enhance the robustness of retinal vessel segmentation models. Prospective research avenues encompass verifying the model on vaster and more diverse datasets and investigating its practical implementation in real-world clinical situations.

A Microvascular Segmentation Network Based on Pyramidal Attention Mechanism

The precise segmentation of retinal vasculature is crucial for the early screening of various eye diseases, such as diabetic retinopathy and hypertensive retinopathy. Given the complex and variable overall structure of retinal vessels and their delicate, minute local features, the accurate extraction of fine vessels and edge pixels remains a technical challenge in the current research. To enhance the ability to extract thin vessels, this paper incorporates a pyramid channel attention module into a U-shaped network. This allows for more effective capture of information at different levels and increased attention to vessel-related channels, thereby improving model performance. Simultaneously, to prevent overfitting, this paper optimizes the standard convolutional block in the U-Net with the pre-activated residual discard convolution block, thus improving the model's generalization ability. The model is evaluated on three benchmark retinal datasets: DRIVE, CHASE_DB1, and STARE. Experimental results demonstrate that, compared to the baseline model, the proposed model achieves improvements in sensitivity (Sen) scores of 7.12%, 9.65%, and 5.36% on these three datasets, respectively, proving its strong ability to extract fine vessels.

Hi-gMISnet: generalized medical image segmentation using DWT based multilayer fusion and dual mode attention into high resolutionpGAN

Objective.Automatic medical image segmentation is crucial for accurately isolating target tissue areas in the image from background tissues, facilitating precise diagnoses and procedures. While the proliferation of publicly available clinical datasets led to the development of deep learning-based medical image segmentation methods, a generalized, accurate, robust, and reliable approach across diverse imaging modalities remains elusive.Approach.This paper proposes a novel high-resolution parallel generative adversarial network (pGAN)-based generalized deep learning method for automatic segmentation of medical images from diverse imaging modalities. The proposed method showcases better performance and generalizability by incorporating novel components such as partial hybrid transfer learning, discrete wavelet transform (DWT)-based multilayer and multiresolution feature fusion in the encoder, and a dual mode attention gate in the decoder of the multi-resolution U-Net-based GAN. With multi-objective adversarial training loss functions including a unique reciprocal loss for enforcing cooperative learning inpGANs, it further enhances the robustness and accuracy of the segmentation map.Main results.Experimental evaluations conducted on nine diverse publicly available medical image segmentation datasets, including PhysioNet ICH, BUSI, CVC-ClinicDB, MoNuSeg, GLAS, ISIC-2018, DRIVE, Montgomery, and PROMISE12, demonstrate the proposed method's superior performance. The proposed method achieves mean F1 scores of 79.53%, 88.68%, 82.50%, 93.25%, 90.40%, 94.19%, 81.65%, 98.48%, and 90.79%, respectively, on the above datasets, surpass state-of-the-art segmentation methods. Furthermore, our proposed method demonstrates robust multi-domain segmentation capabilities, exhibiting consistent and reliable performance. The assessment of the model's proficiency in accurately identifying small details indicates that the high-resolution generalized medical image segmentation network (Hi-gMISnet) is more precise in segmenting even when the target area is very small.Significance.The proposed method provides robust and reliable segmentation performance on medical images, and thus it has the potential to be used in a clinical setting for the diagnosis of patients.

3D/2D Vessel Registration Based on Monte Carlo Tree Search and Manifold Regularization

The augmented intra-operative real-time imaging in vascular interventional surgery, which is generally performed by projecting preoperative computed tomography angiography images onto intraoperative digital subtraction angiography (DSA) images, can compensate for the deficiencies of DSA-based navigation, such as lack of depth information and excessive use of toxic contrast agents. 3D/2D vessel registration is the critical step in image augmentation. A 3D/2D registration method based on vessel graph matching is proposed in this study. For rigid registration, the matching of vessel graphs can be decomposed into continuous states, thus 3D/2D vascular registration is formulated as a search tree problem. The Monte Carlo tree search method is applied to find the optimal vessel matching associated with the highest rigid registration score. For nonrigid registration, we propose a novel vessel deformation model based on manifold regularization. This model incorporates the smoothness constraint of vessel topology into the objective function. Furthermore, we derive simplified gradient formulas that enable fast registration. The proposed technique undergoes evaluation against seven rigid and three nonrigid methods using a variety of data - simulated, algorithmically generated, and manually annotated - across three vascular anatomies: the hepatic artery, coronary artery, and aorta. Our findings show the proposed method's resistance to pose variations, noise, and deformations, outperforming existing methods in terms of registration accuracy and computational efficiency. The proposed method demonstrates average registration errors of 2.14 mm and 0.34 mm for rigid and nonrigid registration, and an average computation time of 0.51 s.

Cross-patch feature interactive net with edge refinement for retinal vessel segmentation

Retinal vessel segmentation based on deep learning is an important auxiliary method for assisting clinical doctors in diagnosing retinal diseases. However, existing methods often produce mis-segmentation when dealing with low contrast images and thin blood vessels, which affects the continuity and integrity of the vessel skeleton. In addition, existing deep learning methods tend to lose a lot of detailed information during training, which affects the accuracy of segmentation. To address these issues, we propose a novel dual-decoder based Cross-patch Feature Interactive Net with Edge Refinement (CFI-Net) for end-to-end retinal vessel segmentation. In the encoder part, a joint refinement down-sampling method (JRDM) is proposed to compress feature information in the process of reducing image size, so as to reduce the loss of thin vessels and vessel edge information during the encoding process. In the decoder part, we adopt a dual-path model based on edge detection, and propose a Cross-patch Interactive Attention Mechanism (CIAM) in the main path to enhancing multi-scale spatial channel features and transferring cross-spatial information. Consequently, it improve the network's ability to segment complete and continuous vessel skeletons, reducing vessel segmentation fractures. Finally, the Adaptive Spatial Context Guide Method (ASCGM) is proposed to fuse the prediction results of the two decoder paths, which enhances segmentation details while removing part of the background noise. We evaluated our model on two retinal image datasets and one coronary angiography dataset, achieving outstanding performance in segmentation comprehensive assessment metrics such as AUC and CAL. The experimental results showed that the proposed CFI-Net has superior segmentation performance compared with other existing methods, especially for thin vessels and vessel edges. The code is available at https://github.com/kita0420/CFI-Net.

FRD-Net: a full-resolution dilated convolution network for retinal vessel segmentation

Accurate and automated retinal vessel segmentation is essential for performing diagnosis and surgical planning of retinal diseases. However, conventional U-shaped networks often suffer from segmentation errors when dealing with fine and low-contrast blood vessels due to the loss of continuous resolution in the encoding stage and the inability to recover the lost information in the decoding stage. To address this issue, this paper introduces an effective full-resolution retinal vessel segmentation network, namely FRD-Net, which consists of two core components: the backbone network and the multi-scale feature fusion module (MFFM). The backbone network achieves horizontal and vertical expansion through the interaction mechanism of multi-resolution dilated convolutions while preserving the complete image resolution. In the backbone network, the effective application of dilated convolutions with varying dilation rates, coupled with the utilization of dilated residual modules for integrating multi-scale feature maps from adjacent stages, facilitates continuous learning of multi-scale features to enhance high-level contextual information. Moreover, MFFM further enhances segmentation by fusing deeper multi-scale features with the original image, facilitating edge detail recovery for accurate vessel segmentation. In tests on multiple classical datasets,compared to state-of-the-art segmentation algorithms, FRD-Net achieves superior performance and generalization with fewer model parameters.

PKSEA-Net: A prior knowledge supervised edge-aware multi-task network for retinal arteriolar morphometry

Retinal fundus images serve as a non-invasive modality to obtain information pertaining to retinal vessels through fundus photography, thereby offering insights into cardiovascular and cerebrovascular diseases. Retinal arteriolar morphometry has emerged as the most convenient and fundamental clinical methodology in the realm of patient screening and diagnosis. Nevertheless, the analysis of retinal arterioles is challenging attributable to imaging noise, stochastic fuzzy characteristics, and blurred boundaries proximal to blood vessels. In response to these limitations, we introduce an innovative methodology, named PKSEA-Net, which aims to improve segmentation accuracy by enhancing the perception of edge information in retinal fundus images. PKSEA-Net employs the universal architecture PVT-v2 as the encoder, complemented by a novel decoder architecture consisting of an Edge-Aware Block (EAB) and a Pyramid Feature Fusion Module (PFFM). The EAB block incorporates prior knowledge for supervision and multi-query for multi-task learning, with supervision information derived from an enhanced Full Width at Half Maximum (FWHM) algorithm and gradient map. Moreover, PFFM efficiently integrates multi-scale features through a novel attention fusion method. Additionally, we have collected a Retinal Cross-Sectional Vessel (RCSV) dataset derived from approximately 200 patients in Quzhou People's Hospital to serve as the benchmark dataset. Comparative evaluations with several state-of-the-art (SOTA) networks confirm that PKSEA-Net achieves exceptional experimental performance, thereby establishing its status as a SOTA approach for precise boundary delineation and retinal vessel segmentation.

Segment anything model for medical images?

The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM's performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: (1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. (2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. (3) SAM performed better with manual hints, especially box, than the Everything mode. (4) SAM could help human annotation with high labeling quality and less time. (5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. (6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. (7) SAM's performance correlated to different factors, including boundary complexity, intensity differences, etc. (8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. Codes and models are available at: https://github.com/yuhoo0302/Segment-Anything-Model-for-Medical-Images. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM.

Retina-TransNet: A Gradient-Guided Few-Shot Retinal Vessel Segmentation Net

Due to the high labor cost of physicians, it is difficult to collect a rich amount of manually-labeled medical images for developing learning-based computer-aided diagnosis (CADx) systems or segmentation algorithms. To tackle this issue, we reshape the image segmentation task as an image-to-image (I2I) translation problem and propose a retinal vascular segmentation network, which can achieve good cross-domain generalizability even with a small amount of training data. We devise primarily two components to facilitate this I2I-based segmentation method. The first is the constraints provided by the proposed gradient-vector-flow (GVF) loss, and, the second is a two-stage Unet (2Unet) generator with a skip connection. This configuration makes 2Unet's first-stage play a role similar to conventional Unet, but forces 2Unet's second stage to learn to be a refinement module. Extensive experiments show that by re-casting retinal vessel segmentation as an image-to-image translation problem, our I2I translator-based segmentation subnetwork achieves better cross-domain generalizability than existing segmentation methods. Our model, trained on one dataset, e.g., DRIVE, can produce segmentation results stably on datasets of other domains, e.g., CHASE-DB1, STARE, HRF, and DIARETDB1, even in low-shot circumstances.

Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation

Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. However, achieving high pixel-wise accuracy, complete topology structure and robustness to various contrast variations are critical and challenging, and most existing methods focus only on achieving one or two of these aspects. In this paper, we present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach. Specifically, we compute a multiscale affinity field for each pixel, capturing its semantic relationships with neighboring pixels in the predicted mask image. This field represents the local geometry of vessel segments of different sizes, allowing us to learn spatial- and scale-aware adaptive weights to strengthen vessel features. We evaluate our AFN on four different types of vascular datasets: X-ray angiography coronary vessel dataset (XCAD), portal vein dataset (PV), digital subtraction angiography cerebrovascular vessel dataset (DSA) and retinal vessel dataset (DRIVE). Extensive experimental results demonstrate that our AFN outperforms the state-of-the-art methods in terms of both higher accuracy and topological metrics, while also being more robust to various contrast changes.

Attention-guided cascaded network with pixel-importance-balance loss for retinal vessel segmentation

Accurate retinal vessel segmentation from fundus images is essential for eye disease diagnosis. Many deep learning methods have shown great performance in this task but still struggle with limited annotated data. To alleviate this issue, we propose an Attention-Guided Cascaded Network (AGC-Net) that learns more valuable vessel features from a few fundus images. Attention-guided cascaded network consists of two stages: the coarse stage produces a rough vessel prediction map from the fundus image, and the fine stage refines the missing vessel details from this map. In attention-guided cascaded network, we incorporate an inter-stage attention module (ISAM) to cascade the backbone of these two stages, which helps the fine stage focus on vessel regions for better refinement. We also propose Pixel-Importance-Balance Loss (PIB Loss) to train the model, which avoids gradient domination by non-vascular pixels during backpropagation. We evaluate our methods on two mainstream fundus image datasets (i.e., DRIVE and CHASE-DB1) and achieve AUCs of 0.9882 and 0.9914, respectively. Experimental results show that our method outperforms other state-of-the-art methods in performance.

Neural Networks Application for Accurate Retina Vessel Segmentation from OCT Fundus Reconstruction

The use of neural networks for retinal vessel segmentation has gained significant attention in recent years. Most of the research related to the segmentation of retinal blood vessels is based on fundus images. In this study, we examine five neural network architectures to accurately segment vessels in fundus images reconstructed from 3D OCT scan data. OCT-based fundus reconstructions are of much lower quality compared to color fundus photographs due to noise and lower and disproportionate resolutions. The fundus image reconstruction process was performed based on the segmentation of the retinal layers in B-scans. Three reconstruction variants were proposed, which were then used in the process of detecting blood vessels using neural networks. We evaluated performance using a custom dataset of 24 3D OCT scans (with manual annotations performed by an ophthalmologist) using 6-fold cross-validation and demonstrated segmentation accuracy up to 98%. Our results indicate that the use of neural networks is a promising approach to segmenting the retinal vessel from a properly reconstructed fundus.