Recurrent residual U-Net for medical image segmentation

VascuConNet: an enhanced connectivity network for vascular segmentation

Medical image segmentation commonly involves diverse tissue types and structures, including tasks such as blood vessel segmentation and nerve fiber bundle segmentation. Enhancing the continuity of segmentation outcomes represents a pivotal challenge in medical image segmentation, driven by the demands of clinical applications, focusing on disease localization and quantification. In this study, a novel segmentation model is specifically designed for retinal vessel segmentation, leveraging vessel orientation information, boundary constraints, and continuity constraints to improve segmentation accuracy. To achieve this, we cascade U-Net with a long-short-term memory network (LSTM). U-Net is characterized by a small number of parameters and high segmentation efficiency, while LSTM offers a parameter-sharing capability. Additionally, we introduce an orientation information enhancement module inserted into the model's bottom layer to obtain feature maps containing orientation information through an orientation convolution operator. Furthermore, we design a new hybrid loss function that consists of connectivity loss, boundary loss, and cross-entropy loss. Experimental results demonstrate that the model achieves excellent segmentation outcomes across three widely recognized retinal vessel segmentation datasets, CHASE_DB1, DRIVE, and ARIA.

Vein segmentation and visualization of upper and lower extremities using convolution neural network

OBJECTIVES

The study focused on developing a reliable real-time venous localization, identification, and visualization framework based upon deep learning (DL) self-parametrized Convolution Neural Network (CNN) algorithm for segmentation of the venous map for both lower and upper limb dataset acquired under unconstrained conditions using near-infrared (NIR) imaging setup, specifically to assist vascular surgeons during venipuncture, vascular surgeries, or Chronic Venous Disease (CVD) treatments.

METHODS

A portable image acquisition setup has been designed to collect venous data (upper and lower extremities) from 72 subjects. A manually annotated image dataset was used to train and compare the performance of existing well-known CNN-based architectures such as ResNet and VGGNet with self-parameterized U-Net, improving automated vein segmentation and visualization.

RESULTS

Experimental results indicated that self-parameterized U-Net performs better at segmenting the unconstrained dataset in comparison with conventional CNN feature-based learning models, with a Dice score of 0.58 and displaying 96.7 % accuracy for real-time vein visualization, making it appropriate to locate veins in real-time under unconstrained conditions.

CONCLUSIONS

Self-parameterized U-Net for vein segmentation and visualization has the potential to reduce risks associated with traditional venipuncture or CVD treatments by outperforming conventional CNN architectures, providing vascular assistance, and improving patient care and treatment outcomes.

USCT-UNet: Rethinking the Semantic Gap in U-Net Network From U-Shaped Skip Connections With Multichannel Fusion Transformer

Medical image segmentation is a crucial component of computer-aided clinical diagnosis, with state-of-the-art models often being variants of U-Net. Despite their success, these models' skip connections introduce an unnecessary semantic gap between the encoder and decoder, which hinders their ability to achieve the high precision required for clinical applications. Awareness of this semantic gap and its detrimental influences have increased over time. However, a quantitative understanding of how this semantic gap compromises accuracy and reliability remains lacking, emphasizing the need for effective mitigation strategies. In response, we present the first quantitative evaluation of the semantic gap between corresponding layers of U-Net and identify two key characteristics: 1) The direct skip connection (DSC) exhibits a semantic gap that negatively impacts models' performance; 2) The magnitude of the semantic gap varies across different layers. Based on these findings, we re-examine this issue through the lens of skip connections. We introduce a Multichannel Fusion Transformer (MCFT) and propose a novel USCT-UNet architecture, which incorporates U-shaped skip connections (USC) to replace DSC, allocates varying numbers of MCFT blocks based on the semantic gap magnitude at different layers, and employs a spatial channel cross-attention (SCCA) module to facilitate the fusion of features between the decoder and USC. We evaluate USCT-UNet on four challenging datasets, and the results demonstrate that it effectively eliminates the semantic gap. Compared to using DSC, our USC and SCCA strategies achieve maximum improvements of 4.79% in the Dice coefficient, 5.70% in mean intersection over union (MIoU), and 3.26 in Hausdorff distance.

Exploiting Temporal Features in Calculating Automated Morphological Properties of Spiky Nanoparticles Using Deep Learning

Object segmentation in images is typically spatial and focuses on the spatial coherence of pixels. Nanoparticles in electron microscopy images are also segmented frame by frame, with subsequent morphological analysis. However, morphological analysis is inherently sequential, and a temporal regularity is evident in the process. In this study, we extend the spatially focused morphological analysis by incorporating a fusion of hard and soft inductive bias from sequential machine learning techniques to account for temporal relationships. Previously, spiky Au nanoparticles (Au-SNPs) in electron microscopy images were analyzed, and their morphological properties were automatically generated using a hourglass convolutional neural network architecture. In this study, recurrent layers are integrated to capture the natural, sequential growth of the particles. The network is trained with a spike-focused loss function. Continuous segmentation of the images explores the regressive relationships among natural growth features, generating morphological statistics of the nanoparticles. This study comprehensively evaluates the proposed approach by comparing the results of segmentation and morphological properties analysis, demonstrating its superiority over earlier methods.

PET radiomics in lung cancer: advances and translational challenges

Radiomics is an emerging field of medical imaging that aims at improving the accuracy of diagnosis, prognosis, treatment planning and monitoring non-invasively through the automated or semi-automated quantitative analysis of high-dimensional image features. Specifically in the field of nuclear medicine, radiomics utilizes imaging methods such as positron emission tomography (PET) and single photon emission computed tomography (SPECT) to evaluate biomarkers related to metabolism, blood flow, cellular activity and some biological pathways. Lung cancer ranks among the leading causes of cancer-related deaths globally, and radiomics analysis has shown great potential in guiding individualized therapy, assessing treatment response, and predicting clinical outcomes. In this review, we summarize the current state-of-the-art radiomics progress in lung cancer, highlighting the potential benefits and existing limitations of this approach. The radiomics workflow was introduced first including image acquisition, segmentation, feature extraction, and model building. Then the published literatures were described about radiomics-based prediction models for lung cancer diagnosis, differentiation, prognosis and efficacy evaluation. Finally, we discuss current challenges and provide insights into future directions and potential opportunities for integrating radiomics into routine clinical practice.

A fundus vessel segmentation method based on double skip connections combined with deep supervision

Background

Fundus vessel segmentation is vital for diagnosing ophthalmic diseases like central serous chorioretinopathy (CSC), diabetic retinopathy, and glaucoma. Accurate segmentation provides crucial vessel morphology details, aiding the early detection and intervention of ophthalmic diseases. However, current algorithms struggle with fine vessel segmentation and maintaining sensitivity in complex regions. Challenges also stem from imaging variability and poor generalization across multimodal datasets, highlighting the need for more advanced algorithms in clinical practice.

Methods

This paper aims to explore a new vessel segmentation method to alleviate the above problems. We propose a fundus vessel segmentation model based on a combination of double skip connections, deep supervision, and TransUNet, namely DS2TUNet. Initially, the original fundus images are improved through grayscale conversion, normalization, histogram equalization, gamma correction, and other preprocessing techniques. Subsequently, by utilizing the U-Net architecture, the preprocessed fundus images are segmented to obtain the final vessel information. Specifically, the encoder firstly incorporates the ResNetV1 downsampling, dilated convolution downsampling, and Transformer to capture both local and global features, which upgrades its vessel feature extraction ability. Then, the decoder introduces the double skip connections to facilitate upsampling and refine segmentation outcomes. Finally, the deep supervision module introduces multiple upsampling vessel features from the decoder into the loss function, so that the model can learn vessel feature representations more effectively and alleviate gradient vanishing during the training phase.

Results

Extensive experiments on publicly available multimodal fundus datasets such as DRIVE, CHASE_DB1, and ROSE-1 demonstrate that the DS2TUNet model attains F1-scores of 0.8195, 0.8362, and 0.8425, with Accuracy of 0.9664, 0.9741, and 0.9557, Sensitivity of 0.8071, 0.8101, and 0.8586, and Specificity of 0.9823, 0.9869, and 0.9713, respectively. Additionally, the model also exhibits excellent test performance on the clinical fundus dataset CSC, with F1-score of 0.7757, Accuracy of 0.9688, Sensitivity of 0.8141, and Specificity of 0.9801 based on the weight trained on the CHASE_DB1 dataset. These results comprehensively validate that the proposed method obtains good performance in fundus vessel segmentation, thereby aiding clinicians in the further diagnosis and treatment of fundus diseases in terms of effectiveness and feasibility.

Dilated multi-scale residual attention (DMRA) U-Net: three-dimensional (3D) dilated multi-scale residual attention U-Net for brain tumor segmentation

Background

The precise identification of the position and form of a tumor mass can improve early diagnosis and treatment. However, due to the complicated tumor categories and varying sizes and forms, the segregation of brain gliomas and their internal sub-regions is still very challenging. This study sought to design a new deep-learning network based on three-dimensional (3D) U-Net to address its shortcomings in brain tumor segmentation (BraTS) tasks.

Methods

We developed a 3D dilated multi-scale residual attention U-Net (DMRA-U-Net) model for magnetic resonance imaging (MRI) BraTS. It used dilated convolution residual (DCR) modules to better process shallow features, multi-scale convolution residual (MCR) modules in the bottom encoding path to create richer and more comprehensive feature expression while reducing overall information loss or blurring, and a channel attention (CA) module between the encoding and decoding paths to address the problem of retrieving and preserving important features during the processing of deep feature maps.

Results

The BraTS 2018-2021 datasets served as the training and evaluation datasets for this study. Further, the proposed architecture was assessed using metrics such as the dice similarity coefficient (DSC), Hausdorff distance (HD), and sensitivity (Sens). The DMRA U-Net model segments the whole tumor (WT), and the tumor core (TC), and the enhancing tumor (ET) regions of brain tumors. Using the suggested architecture, the DSCs were 0.9012, 0.8867, and 0.8813, the HDs were 28.86, 13.34, and 10.88 mm, and the Sens was 0.9429, 0.9452, and 0.9303 for the WT, TC, and ET regions, respectively. Compared to the traditional 3D U-Net, the DSC of the DMRA U-Net increased by 4.5%, 2.5%, and 0.8%, the HD of the DMRA U-Net decreased by 21.83, 16.42, and 10.00, the Sens of the DMRA U-Net increased by 0.4%, 0.7%, and 1.4% for the WT, TC, and ET regions, respectively. Further, the results of the statistical comparison of the performance indicators revealed that our model performed well generally in the segmentation of the WT, TC, and ET regions.

Conclusions

We developed a promising tumor segmentation model. Our solution is open sourced and is available at: https://github.com/Gold3nk/dmra-unet.

Evaluation of the clinical application value of artificial intelligence in diagnosing head and neck aneurysms

OBJECTIVE

To evaluate the performance of a semi-automated artificial intelligence (AI) software program (CerebralDoc® system) in aneurysm detection and morphological measurement.

METHODS

In this study, 354 cases of computed tomographic angiography (CTA) were retrospectively collected in our hospital. Among them, 280 cases were diagnosed with aneurysms by either digital subtraction angiography (DSA) and CTA (DSA group, n = 102), or CTA-only (non-DSA group, n = 178). The presence or absence of aneurysms, as well as their location and related morphological features determined by AI were evaluated using DSA and radiologist findings. Besides, post-processing image quality from AI and radiologists were also rated and compared.

RESULTS

In the DSA group, AI achieved a sensitivity of 88.24% and an accuracy of 81.97%, whereas radiologists achieved a sensitivity of 95.10% and an accuracy of 84.43%, using DSA results as the gold standard. The AI in the non-DSA group achieved 81.46% sensitivity and 76.29% accuracy, as per the radiologists' findings. The comparison of position consistency results showed better performance under loose criteria than strict criteria. In terms of morphological characteristics, both the DSA and the non-DSA groups agreed well with the diagnostic results for neck width and maximum diameter, demonstrating excellent ICC reliability exceeding 0.80. The AI-generated images exhibited superior quality compared to the standard software for post-processing, while also demonstrating a significantly reduced processing time.

CONCLUSIONS

The AI-based aneurysm detection rate demonstrates a commendable performance, while the extracted morphological parameters exhibit a remarkable consistency with those assessed by radiologists, thereby showcasing significant potential for clinical application.

Skeleton-guided multi-scale dual-coordinate attention aggregation network for retinal blood vessel segmentation

Deep learning plays a pivotal role in retinal blood vessel segmentation for medical diagnosis. Despite their significant efficacy, these techniques face two major challenges. Firstly, they often neglect the severe class imbalance in fundus images, where thin vessels in the foreground are proportionally minimal. Secondly, they are susceptible to poor image quality and blurred vessel edges, resulting in discontinuities or breaks in vascular structures. In response, this paper proposes the Skeleton-guided Multi-scale Dual-coordinate Attention Aggregation (SMDAA) network for retinal vessel segmentation. SMDAA comprises three innovative modules: Dual-coordinate Attention (DCA), Unbalanced Pixel Amplifier (UPA), and Vessel Skeleton Guidance (VSG). DCA, integrating Multi-scale Coordinate Feature Aggregation (MCFA) and Scale Coordinate Attention Decoding (SCAD), meticulously analyzes vessel structures across various scales, adept at capturing intricate details, thereby significantly enhancing segmentation accuracy. To address class imbalance, we introduce UPA, dynamically allocating more attention to misclassified pixels, ensuring precise extraction of thin and small blood vessels. Moreover, to preserve vessel structure continuity, we integrate vessel anatomy and develop the VSG module to connect fragmented vessel segments. Additionally, a Feature-level Contrast (FCL) loss is introduced to capture subtle nuances within the same category, enhancing the fidelity of retinal blood vessel segmentation. Extensive experiments on three public datasets (DRIVE, STARE, and CHASE_DB1) demonstrate superior performance in comparison to current methods. The code is available at https://github.com/wangwxr/SMDAA_NET.

Comparing Deep Learning Performance for Chronic Lymphocytic Leukaemia Cell Segmentation in Brightfield Microscopy Images

Objectives

This article focuses on the detection of cells in low-contrast brightfield microscopy images; in our case, it is chronic lymphocytic leukaemia cells. The automatic detection of cells from brightfield time-lapse microscopic images brings new opportunities in cell morphology and migration studies; to achieve the desired results, it is advisable to use state-of-the-art image segmentation methods that not only detect the cell but also detect its boundaries with the highest possible accuracy, thus defining its shape and dimensions.

Methods

We compared eight state-of-the-art neural network architectures with different backbone encoders for image data segmentation, namely U-net, U-net++, the Pyramid Attention Network, the Multi-Attention Network, LinkNet, the Feature Pyramid Network, DeepLabV3, and DeepLabV3+. The training process involved training each of these networks for 1000 epochs using the PyTorch and PyTorch Lightning libraries. For instance segmentation, the watershed algorithm and three-class image semantic segmentation were used. We also used StarDist, a deep learning-based tool for object detection with star-convex shapes.

Results

The optimal combination for semantic segmentation was the U-net++ architecture with a ResNeSt-269 background with a data set intersection over a union score of 0.8902. For the cell characteristics examined (area, circularity, solidity, perimeter, radius, and shape index), the difference in mean value using different chronic lymphocytic leukaemia cell segmentation approaches appeared to be statistically significant (Mann-Whitney U test, P < .0001).

Conclusion

We found that overall, the algorithms demonstrate equal agreement with ground truth, but with the comparison, it can be seen that the different approaches prefer different morphological features of the cells. Consequently, choosing the most suitable method for instance-based cell segmentation depends on the particular application, namely, the specific cellular traits being investigated.

Semantic segmentation in skin surface microscopic images with artifacts removal

Skin surface imaging has been used to examine skin lesions with a microscope for over a century and is commonly known as epiluminescence microscopy, dermatoscopy, or dermoscopy. Skin surface microscopy has been recommended to reduce the necessity of biopsy. This imaging technique could improve the clinical diagnostic performance of pigmented skin lesions. Different imaging techniques are employed in dermatology to find diseases. Segmentation and classification are the two main steps in the examination. The classification performance is influenced by the algorithm employed in the segmentation procedure. The most difficult aspect of segmentation is getting rid of the unwanted artifacts. Many deep-learning models are being created to segment skin lesions. In this paper, an analysis of common artifacts is proposed to investigate the segmentation performance of deep learning models with skin surface microscopic images. The most prevalent artifacts in skin images are hair and dark corners. These artifacts can be observed in the majority of dermoscopy images captured through various imaging techniques. While hair detection and removal methods are common, the introduction of dark corner detection and removal represents a novel approach to skin lesion segmentation. A comprehensive analysis of this segmentation performance is assessed using the surface density of artifacts. Assessment of the PH2, ISIC 2017, and ISIC 2018 datasets demonstrates significant enhancements, as reflected by Dice coefficients rising to 93.49 (86.81), 85.86 (79.91), and 75.38 (51.28) respectively, upon artifact removal. These results underscore the pivotal significance of artifact removal techniques in amplifying the efficacy of deep-learning models for skin lesion segmentation.

FMD-UNet: fine-grained feature squeeze and multiscale cascade dilated semantic aggregation dual-decoder UNet for COVID-19 lung infection segmentation from CT images

With the advancement of computer-aided diagnosis, the automatic segmentation of COVID-19 infection areas holds great promise for assisting in the timely diagnosis and recovery of patients in clinical practice. Currently, methods relying on U-Net face challenges in effectively utilizing fine-grained semantic information from input images and bridging the semantic gap between the encoder and decoder. To address these issues, we propose an FMD-UNet dual-decoder U-Net network for COVID-19 infection segmentation, which integrates a Fine-grained Feature Squeezing (FGFS) decoder and a Multi-scale Dilated Semantic Aggregation (MDSA) decoder. The FGFS decoder produces fine feature maps through the compression of fine-grained features and a weighted attention mechanism, guiding the model to capture detailed semantic information. The MDSA decoder consists of three hierarchical MDSA modules designed for different stages of input information. These modules progressively fuse different scales of dilated convolutions to process the shallow and deep semantic information from the encoder, and use the extracted feature information to bridge the semantic gaps at various stages, this design captures extensive contextual information while decoding and predicting segmentation, thereby suppressing the increase in model parameters. To better validate the robustness and generalizability of the FMD-UNet, we conducted comprehensive performance evaluations and ablation experiments on three public datasets, and achieved leading Dice Similarity Coefficient (DSC) scores of 84.76, 78.56 and 61.99% in COVID-19 infection segmentation, respectively. Compared to previous methods, the FMD-UNet has fewer parameters and shorter inference time, which also demonstrates its competitiveness.

A Review of Advancements and Challenges in Liver Segmentation

Liver segmentation technologies play vital roles in clinical diagnosis, disease monitoring, and surgical planning due to the complex anatomical structure and physiological functions of the liver. This paper provides a comprehensive review of the developments, challenges, and future directions in liver segmentation technology. We systematically analyzed high-quality research published between 2014 and 2024, focusing on liver segmentation methods, public datasets, and evaluation metrics. This review highlights the transition from manual to semi-automatic and fully automatic segmentation methods, describes the capabilities and limitations of available technologies, and provides future outlooks.

EAAC-Net: An Efficient Adaptive Attention and Convolution Fusion Network for Skin Lesion Segmentation

Accurate segmentation of skin lesions in dermoscopic images is of key importance for quantitative analysis of melanoma. Although existing medical image segmentation methods significantly improve skin lesion segmentation, they still have limitations in extracting local features with global information, do not handle challenging lesions well, and usually have a large number of parameters and high computational complexity. To address these issues, this paper proposes an efficient adaptive attention and convolutional fusion network for skin lesion segmentation (EAAC-Net). We designed two parallel encoders, where the efficient adaptive attention feature extraction module (EAAM) adaptively establishes global spatial dependence and global channel dependence by constructing the adjacency matrix of the directed graph and can adaptively filter out the least relevant tokens at the coarse-grained region level, thus reducing the computational complexity of the self-attention mechanism. The efficient multiscale attention-based convolution module (EMA⋅C) utilizes multiscale attention for cross-space learning of local features extracted from the convolutional layer to enhance the representation of richly detailed local features. In addition, we designed a reverse attention feature fusion module (RAFM) to enhance the effective boundary information gradually. To validate the performance of our proposed network, we compared it with other methods on ISIC 2016, ISIC 2018, and PH2 public datasets, and the experimental results show that EAAC-Net has superior segmentation performance under commonly used evaluation metrics.

Improving diffuse optical tomography imaging quality using APU-Net: an attention-based physical U-Net model

Significance

Traditional diffuse optical tomography (DOT) reconstructions are hampered by image artifacts arising from factors such as DOT sources being closer to shallow lesions, poor optode-tissue coupling, tissue heterogeneity, and large high-contrast lesions lacking information in deeper regions (known as shadowing effect). Addressing these challenges is crucial for improving the quality of DOT images and obtaining robust lesion diagnosis.

Aim

We address the limitations of current DOT imaging reconstruction by introducing an attention-based U-Net (APU-Net) model to enhance the image quality of DOT reconstruction, ultimately improving lesion diagnostic accuracy.

Approach

We designed an APU-Net model incorporating a contextual transformer attention module to enhance DOT reconstruction. The model was trained on simulation and phantom data, focusing on challenges such as artifact-induced distortions and lesion-shadowing effects. The model was then evaluated by the clinical data.

Results

Transitioning from simulation and phantom data to clinical patients' data, our APU-Net model effectively reduced artifacts with an average artifact contrast decrease of 26.83% and improved image quality. In addition, statistical analyses revealed significant contrast improvements in depth profile with an average contrast increase of 20.28% and 45.31% for the second and third target layers, respectively. These results highlighted the efficacy of our approach in breast cancer diagnosis.

Conclusions

The APU-Net model improves the image quality of DOT reconstruction by reducing DOT image artifacts and improving the target depth profile.

Bifurcation detection in intravascular optical coherence tomography using vision transformer based deep learning

Objective. Bifurcation detection in intravascular optical coherence tomography (IVOCT) images plays a significant role in guiding optimal revascularization strategies for percutaneous coronary intervention (PCI). We propose a bifurcation detection method using vision transformer (ViT) based deep learning in IVOCT.Approach. Instead of relying on lumen segmentation, the proposed method identifies the bifurcation image using a ViT-based classification model and then estimate bifurcation ostium points by a ViT-based landmark detection model.Main results. By processing 8640 clinical images, the Accuracy and F1-score of bifurcation identification by the proposed ViT-based model are 2.54% and 16.08% higher than that of traditional non-deep learning methods, are similar to the best performance of convolutional neural networks (CNNs) based methods, respectively. The ostium distance error of the ViT-based model is 0.305 mm, which is reduced 68.5% compared with the traditional non-deep learning method and reduced 24.81% compared with the best performance of CNNs based methods. The results also show that the proposed ViT-based method achieves the highest success detection rate are 11.3% and 29.2% higher than the non-deep learning method, and 4.6% and 2.5% higher than the best performance of CNNs based methods when the distance section is 0.1 and 0.2 mm, respectively.Significance. The proposed ViT-based method enhances the performance of bifurcation detection of IVOCT images, which maintains a high correlation and consistency between the automatic detection results and the expert manual results. It is of great significance in guiding the selection of PCI treatment strategies.

LMBiS-Net: A lightweight bidirectional skip connection based multipath CNN for retinal blood vessel segmentation

Blinding eye diseases are often related to changes in retinal structure, which can be detected by analysing retinal blood vessels in fundus images. However, existing techniques struggle to accurately segment these delicate vessels. Although deep learning has shown promise in medical image segmentation, its reliance on specific operations can limit its ability to capture crucial details such as the edges of the vessel. This paper introduces LMBiS-Net, a lightweight convolutional neural network designed for the segmentation of retinal vessels. LMBiS-Net achieves exceptional performance with a remarkably low number of learnable parameters (only 0.172 million). The network used multipath feature extraction blocks and incorporates bidirectional skip connections for the information flow between the encoder and decoder. In addition, we have optimised the efficiency of the model by carefully selecting the number of filters to avoid filter overlap. This optimisation significantly reduces training time and improves computational efficiency. To assess LMBiS-Net's robustness and ability to generalise to unseen data, we conducted comprehensive evaluations on four publicly available datasets: DRIVE, STARE, CHASE_DB1, and HRF The proposed LMBiS-Net achieves significant performance metrics in various datasets. It obtains sensitivity values of 83.60%, 84.37%, 86.05%, and 83.48%, specificity values of 98.83%, 98.77%, 98.96%, and 98.77%, accuracy (acc) scores of 97.08%, 97.69%, 97.75%, and 96.90%, and AUC values of 98.80%, 98.82%, 98.71%, and 88.77% on the DRIVE, STARE, CHEASE_DB, and HRF datasets, respectively. In addition, it records F1 scores of 83.43%, 84.44%, 83.54%, and 78.73% on the same datasets. Our evaluations demonstrate that LMBiS-Net achieves high segmentation accuracy (acc) while exhibiting both robustness and generalisability across various retinal image datasets. This combination of qualities makes LMBiS-Net a promising tool for various clinical applications.

SaraNet: Semantic aggregation reverse attention network for pulmonary nodule segmentation

Accurate segmentation of pulmonary nodule is essential for subsequent pathological analysis and diagnosis. However, current U-Net architectures often rely on a simple skip connection scheme, leading to the fusion of feature maps with different semantic information, which can have a negative impact on the segmentation model. In response to this challenge, this study introduces a novel U-shaped model specifically designed for pulmonary nodule segmentation. The proposed model incorporates features such as the U-Net backbone, semantic aggregation feature pyramid module, and reverse attention module. The semantic aggregation module combines semantic information with multi-scale features, addressing the semantic gap between the encoder and decoder. The reverse attention module explores missing object parts and captures intricate details by erasing the currently predicted salient regions from side-output features. The proposed model is evaluated using the LIDC-IDRI dataset. Experimental results reveal that the proposed method achieves a dice similarity coefficient of 89.11%and a sensitivity of 90.73 %, outperforming state-of-the-art approaches comprehensively.

PL-Net: progressive learning network for medical image segmentation

In recent years, deep convolutional neural network-based segmentation methods have achieved state-of-the-art performance for many medical analysis tasks. However, most of these approaches rely on optimizing the U-Net structure or adding new functional modules, which overlooks the complementation and fusion of coarse-grained and fine-grained semantic information. To address these issues, we propose a 2D medical image segmentation framework called Progressive Learning Network (PL-Net), which comprises Internal Progressive Learning (IPL) and External Progressive Learning (EPL). PL-Net offers the following advantages: 1) IPL divides feature extraction into two steps, allowing for the mixing of different size receptive fields and capturing semantic information from coarse to fine granularity without introducing additional parameters; 2) EPL divides the training process into two stages to optimize parameters and facilitate the fusion of coarse-grained information in the first stage and fine-grained information in the second stage. We conducted comprehensive evaluations of our proposed method on five medical image segmentation datasets, and the experimental results demonstrate that PL-Net achieves competitive segmentation performance. It is worth noting that PL-Net does not introduce any additional learnable parameters compared to other U-Net variants.

Robust PCA with Lw,∗ and L2,1 Norms: A Novel Method for Low-Quality Retinal Image Enhancement

Nonmydriatic retinal fundus images often suffer from quality issues and artifacts due to ocular or systemic comorbidities, leading to potential inaccuracies in clinical diagnoses. In recent times, deep learning methods have been widely employed to improve retinal image quality. However, these methods often require large datasets and lack robustness in clinical settings. Conversely, the inherent stability and adaptability of traditional unsupervised learning methods, coupled with their reduced reliance on extensive data, render them more suitable for real-world clinical applications, particularly in the limited data context of high noise levels or a significant presence of artifacts. However, existing unsupervised learning methods encounter challenges such as sensitivity to noise and outliers, reliance on assumptions like cluster shapes, and difficulties with scalability and interpretability, particularly when utilized for retinal image enhancement. To tackle these challenges, we propose a novel robust PCA (RPCA) method with low-rank sparse decomposition that also integrates affine transformations τi, weighted nuclear norm, and the L2,1 norms, aiming to overcome existing method limitations and to achieve image quality improvement unseen by these methods. We employ the weighted nuclear norm (Lw,∗) to assign weights to singular values to each retinal images and utilize the L2,1 norm to eliminate correlated samples and outliers in the retinal images. Moreover, τi is employed to enhance retinal image alignment, making the new method more robust to variations, outliers, noise, and image blurring. The Alternating Direction Method of Multipliers (ADMM) method is used to optimally determine parameters, including τi, by solving an optimization problem. Each parameter is addressed separately, harnessing the benefits of ADMM. Our method introduces a novel parameter update approach and significantly improves retinal image quality, detecting cataracts, and diabetic retinopathy. Simulation results confirm our method's superiority over existing state-of-the-art methods across various datasets.

Gaussian filter facilitated deep learning-based architecture for accurate and efficient liver tumor segmentation for radiation therapy

Purpose

Addressing the challenges of unclear tumor boundaries and the confusion between cysts and tumors in liver tumor segmentation, this study aims to develop an auto-segmentation method utilizing Gaussian filter with the nnUNet architecture to effectively distinguish between tumors and cysts, enhancing the accuracy of liver tumor auto-segmentation.

Methods

Firstly, 130 cases of liver tumorsegmentation challenge 2017 (LiTS2017) were used for training and validating nnU-Net-based auto-segmentation model. Then, 14 cases of 3D-IRCADb dataset and 25 liver cancer cases retrospectively collected in our hospital were used for testing. The dice similarity coefficient (DSC) was used to evaluate the accuracy of auto-segmentation model by comparing with manual contours.

Results

The nnU-Net achieved an average DSC value of 0.86 for validation set (20 LiTS cases) and 0.82 for public testing set (14 3D-IRCADb cases). For clinical testing set, the standalone nnU-Net model achieved an average DSC value of 0.75, which increased to 0.81 after post-processing with the Gaussian filter (P<0.05), demonstrating its effectiveness in mitigating the influence of liver cysts on liver tumor segmentation.

Conclusion

Experiments show that Gaussian filter is beneficial to improve the accuracy of liver tumor segmentation in clinic.

Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentation

Accurate segmentation of multiple organs in the head, neck, chest, and abdomen from medical images is an essential step in computer-aided diagnosis, surgical navigation, and radiation therapy. In the past few years, with a data-driven feature extraction approach and end-to-end training, automatic deep learning-based multi-organ segmentation methods have far outperformed traditional methods and become a new research topic. This review systematically summarizes the latest research in this field. We searched Google Scholar for papers published from January 1, 2016 to December 31, 2023, using keywords "multi-organ segmentation" and "deep learning", resulting in 327 papers. We followed the PRISMA guidelines for paper selection, and 195 studies were deemed to be within the scope of this review. We summarized the two main aspects involved in multi-organ segmentation: datasets and methods. Regarding datasets, we provided an overview of existing public datasets and conducted an in-depth analysis. Concerning methods, we categorized existing approaches into three major classes: fully supervised, weakly supervised and semi-supervised, based on whether they require complete label information. We summarized the achievements of these methods in terms of segmentation accuracy. In the discussion and conclusion section, we outlined and summarized the current trends in multi-organ segmentation.

Automated assessment of cardiac dynamics in aging and dilated cardiomyopathy Drosophila models using machine learning

The Drosophila model is pivotal in deciphering the pathophysiological underpinnings of various human ailments, notably aging and cardiovascular diseases. Cutting-edge imaging techniques and physiology yield vast high-resolution videos, demanding advanced analysis methods. Our platform leverages deep learning to segment optical microscopy images of Drosophila hearts, enabling the quantification of cardiac parameters in aging and dilated cardiomyopathy (DCM). Validation using experimental datasets confirms the efficacy of our aging model. We employ two innovative approaches deep-learning video classification and machine-learning based on cardiac parameters to predict fly aging, achieving accuracies of 83.3% (AUC 0.90) and 79.1%, (AUC 0.87) respectively. Moreover, we extend our deep-learning methodology to assess cardiac dysfunction associated with the knock-down of oxoglutarate dehydrogenase (OGDH), revealing its potential in studying DCM. This versatile approach promises accelerated cardiac assays for modeling various human diseases in Drosophila and holds promise for application in animal and human cardiac physiology under diverse conditions.

Multi-Modality Deep Infarct: Non-invasive identification of infarcted myocardium using composite in-silico-human data learning

Myocardial infarction (MI) continues to be a leading cause of death worldwide. The precise quantification of infarcted tissue is crucial to diagnosis, therapeutic management, and post-MI care. Late gadolinium enhancement-cardiac magnetic resonance (LGE-CMR) is regarded as the gold standard for precise infarct tissue localization in MI patients. A fundamental limitation of LGE-CMR is the invasive intravenous introduction of gadolinium-based contrast agents that present potential high-risk toxicity, particularly for individuals with underlying chronic kidney diseases. Herein, we develop a completely non-invasive methodology that identifies the location and extent of an infarct region in the left ventricle via a machine learning (ML) model using only cardiac strains as inputs. In this transformative approach, we demonstrate the remarkable performance of a multi-fidelity ML model that combines rodent-based in-silico-generated training data (low-fidelity) with very limited patient-specific human data (high-fidelity) in predicting LGE ground truth. Our results offer a new paradigm for developing feasible prognostic tools by augmenting synthetic simulation-based data with very small amounts of in-vivo human data. More broadly, the proposed approach can significantly assist with addressing biomedical challenges in healthcare where human data are limited.

HRU-Net: A high-resolution convolutional neural network for esophageal cancer radiotherapy target segmentation

BACKGROUND AND OBJECTIVE

The effective segmentation of esophageal squamous carcinoma lesions in CT scans is significant for auxiliary diagnosis and treatment. However, accurate lesion segmentation is still a challenging task due to the irregular form of the esophagus and small size, the inconsistency of spatio-temporal structure, and low contrast of esophagus and its peripheral tissues in medical images. The objective of this study is to improve the segmentation effect of esophageal squamous cell carcinoma lesions.

METHODS

It is critical for a segmentation network to effectively extract 3D discriminative features to distinguish esophageal cancers from some visually closed adjacent esophageal tissues and organs. In this work, an efficient HRU-Net architecture (High-Resolution U-Net) was exploited for esophageal cancer and esophageal carcinoma segmentation in CT slices. Based on the idea of localization first and segmentation later, the HRU-Net locates the esophageal region before segmentation. In addition, an Resolution Fusion Module (RFM) was designed to integrate the information of adjacent resolution feature maps to obtain strong semantic information, as well as preserve the high-resolution features.

RESULTS

Compared with the other five typical methods, the devised HRU-Net is capable of generating superior segmentation results.

CONCLUSIONS

Our proposed HRU-NET improves the accuracy of segmentation for squamous esophageal cancer. Compared to other models, our model performs the best. The designed method may improve the efficiency of clinical diagnosis of esophageal squamous cell carcinoma lesions.

Multi-scale nested UNet with transformer for colorectal polyp segmentation

BACKGROUND

Polyp detection and localization are essential tasks for colonoscopy. U-shape network based convolutional neural networks have achieved remarkable segmentation performance for biomedical images, but lack of long-range dependencies modeling limits their receptive fields.

PURPOSE

Our goal was to develop and test a novel architecture for polyp segmentation, which takes advantage of learning local information with long-range dependencies modeling.

METHODS

A novel architecture combining with multi-scale nested UNet structure integrated transformer for polyp segmentation was developed. The proposed network takes advantage of both CNN and transformer to extract distinct feature information. The transformer layer is embedded between the encoder and decoder of a U-shape net to learn explicit global context and long-range semantic information. To address the challenging of variant polyp sizes, a MSFF unit was proposed to fuse features with multiple resolution.

RESULTS

Four public datasets and one in-house dataset were used to train and test the model performance. Ablation study was also conducted to verify each component of the model. For dataset Kvasir-SEG and CVC-ClinicDB, the proposed model achieved mean dice score of 0.942 and 0.950 respectively, which were more accurate than the other methods. To show the generalization of different methods, we processed two cross dataset validations, the proposed model achieved the highest mean dice score. The results demonstrate that the proposed network has powerful learning and generalization capability, significantly improving segmentation accuracy and outperforming state-of-the-art methods.

CONCLUSIONS

The proposed model produced more accurate polyp segmentation than current methods on four different public and one in-house datasets. Its capability of polyps segmentation in different sizes shows the potential clinical application.

MOTC: Abdominal Multi-objective Segmentation Model with Parallel Fusion of Global and Local Information

Convolutional Neural Networks have been widely applied in medical image segmentation. However, the existence of local inductive bias in convolutional operations restricts the modeling of long-term dependencies. The introduction of Transformer enables the modeling of long-term dependencies and partially eliminates the local inductive bias in convolutional operations, thereby improving the accuracy of tasks such as segmentation and classification. Researchers have proposed various hybrid structures combining Transformer and Convolutional Neural Networks. One strategy is to stack Transformer blocks and convolutional blocks to concentrate on eliminating the accumulated local bias of convolutional operations. Another strategy is to nest convolutional blocks and Transformer blocks to eliminate bias within each nested block. However, due to the granularity of bias elimination operations, these two strategies cannot fully exploit the potential of Transformer. In this paper, a parallel hybrid model is proposed for segmentation, which includes a Transformer branch and a Convolutional Neural Network branch in encoder. After parallel feature extraction, inter-layer information fusion and exchange of complementary information are performed between the two branches, simultaneously extracting local and global features while eliminating the local bias generated by convolutional operations within the current layer. A pure convolutional operation is used in decoder to obtain final segmentation results. To validate the impact of the granularity of bias elimination operations on the effectiveness of local bias elimination, the experiments in this paper were conducted on Flare21 dataset and Amos22 dataset. The average Dice coefficient reached 92.65% on Flare21 dataset, and 91.61% on Amos22 dataset, surpassing comparative methods. The experimental results demonstrate that smaller granularity of bias elimination operations leads to better performance.

INSTRAS: INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation

Infrared (IR) spectroscopic imaging is of potentially wide use in medical imaging applications due to its ability to capture both chemical and spatial information. This complexity of the data both necessitates using machine intelligence as well as presents an opportunity to harness a high-dimensionality data set that offers far more information than today's manually-interpreted images. While convolutional neural networks (CNNs), including the well-known U-Net model, have demonstrated impressive performance in image segmentation, the inherent locality of convolution limits the effectiveness of these models for encoding IR data, resulting in suboptimal performance. In this work, we propose an INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation (INSTRAS). This novel model leverages the strength of the transformer encoders to segment IR breast images effectively. Incorporating skip-connection and transformer encoders, INSTRAS overcomes the issue of pure convolution models, such as the difficulty of capturing long-range dependencies. To evaluate the performance of our model and existing convolutional models, we conducted training on various encoder-decoder models using a breast dataset of IR images. INSTRAS, utilizing 9 spectral bands for segmentation, achieved a remarkable AUC score of 0.9788, underscoring its superior capabilities compared to purely convolutional models. These experimental results attest to INSTRAS's advanced and improved segmentation abilities for IR imaging.

CS-UNet: Cross-scale U-Net with Semantic-position dependencies for retinal vessel segmentation

Accurate retinal vessel segmentation is the prerequisite for early recognition and treatment of retina-related diseases. However, segmenting retinal vessels is still challenging due to the intricate vessel tree in fundus images, which has a significant number of tiny vessels, low contrast, and lesion interference. For this task, the u-shaped architecture (U-Net) has become the de-facto standard and has achieved considerable success. However, U-Net is a pure convolutional network, which usually shows limitations in global modelling. In this paper, we propose a novel Cross-scale U-Net with Semantic-position Dependencies (CS-UNet) for retinal vessel segmentation. In particular, we first designed a Semantic-position Dependencies Aggregator (SPDA) and incorporate it into each layer of the encoder to better focus on global contextual information by integrating the relationship of semantic and position. To endow the model with the capability of cross-scale interaction, the Cross-scale Relation Refine Module (CSRR) is designed to dynamically select the information associated with the vessels, which helps guide the up-sampling operation. Finally, we have evaluated CS-UNet on three public datasets: DRIVE, CHASE_DB1, and STARE. Compared to most existing state-of-the-art methods, CS-UNet demonstrated better performance.

UGLS: an uncertainty guided deep learning strategy for accurate image segmentation

Accurate image segmentation plays a crucial role in computer vision and medical image analysis. In this study, we developed a novel uncertainty guided deep learning strategy (UGLS) to enhance the performance of an existing neural network (i.e., U-Net) in segmenting multiple objects of interest from images with varying modalities. In the developed UGLS, a boundary uncertainty map was introduced for each object based on its coarse segmentation (obtained by the U-Net) and then combined with input images for the fine segmentation of the objects. We validated the developed method by segmenting optic cup (OC) regions from color fundus images and left and right lung regions from Xray images. Experiments on public fundus and Xray image datasets showed that the developed method achieved a average Dice Score (DS) of 0.8791 and a sensitivity (SEN) of 0.8858 for the OC segmentation, and 0.9605, 0.9607, 0.9621, and 0.9668 for the left and right lung segmentation, respectively. Our method significantly improved the segmentation performance of the U-Net, making it comparable or superior to five sophisticated networks (i.e., AU-Net, BiO-Net, AS-Net, Swin-Unet, and TransUNet).

MCAFNet: multiscale cross-layer attention fusion network for honeycomb lung lesion segmentation

Accurate segmentation of honeycomb lung lesions from lung CT images plays a crucial role in the diagnosis and treatment of various lung diseases. However, the availability of algorithms for automatic segmentation of honeycomb lung lesions remains limited. In this study, we propose a novel multi-scale cross-layer attention fusion network (MCAFNet) specifically designed for the segmentation of honeycomb lung lesions, taking into account their shape specificity and similarity to surrounding vascular shadows. The MCAFNet incorporates several key modules to enhance the segmentation performance. Firstly, a multiscale aggregation (MIA) module is introduced in the input part to preserve spatial information during downsampling. Secondly, a cross-layer attention fusion (CAF) module is proposed to capture multiscale features by integrating channel information and spatial information from different layers of the feature maps. Lastly, a bidirectional attention gate (BAG) module is constructed within the skip connection to enhance the model's ability to filter out background information and focus on the segmentation target. Experimental results demonstrate the effectiveness of the proposed MCAFNet. On the honeycomb lung segmentation dataset, the network achieves an Intersection over Union (IoU) of 0.895, mean IoU (mIoU) of 0.921, and mean Dice coefficient (mDice) of 0.949, outperforming existing medical image segmentation algorithms. Furthermore, experiments conducted on additional datasets confirm the generalizability and robustness of the proposed model. The contribution of this study lies in the development of the MCAFNet, which addresses the lack of automated segmentation algorithms for honeycomb lung lesions. The proposed network demonstrates superior performance in accurately segmenting honeycomb lung lesions, thereby facilitating the diagnosis and treatment of lung diseases. This work contributes to the existing literature by presenting a novel approach that effectively combines multi-scale features and attention mechanisms for lung lesion segmentation. The code is available at https://github.com/Oran9er/MCAFNet .

SG-MIAN: Self-guided multiple information aggregation network for image-level weakly supervised skin lesion segmentation

Nowadays, skin disease is becoming one of the most malignant diseases that threaten people's health. Computer aided diagnosis based on deep learning has become a widely used technology to assist medical professionals in diagnosis, and segmentation of lesion areas is one of the most important steps in it. However, traditional medical image segmentation methods rely on numerous pixel-level labels for fully supervised training, and such labeling process is time-consuming and requires professional competence. In order to reduce the costs of pixel-level labeling, we proposed a method only using image-level label to segment skin lesion areas. Due to the lack of lesion's spatial and intensity information in image-level labels, and the wide distribution range of irregular shape and different texture on skin lesions, the algorithm must pay great attention to the automatic lesion localization and perception of lesion boundary. In this paper, we proposed a Self-Guided Multiple Information Aggregation Network (SG-MIAN). Our backbone network MIAN utilizes the Multiple Spatial Perceptron (MSP) solely using classification information as guidance to discriminate the key classification features of lesion areas, and thereby performing more accurate localization and activation of lesion areas. Additionally, adjunct to MSP, we also proposed an Auxiliary Activation Structure (AAS) and two auxiliary loss functions to further self-guided boundary correction, achieving the goal of accurate boundary activation. To verify the effectiveness of the proposed method, we conducted extensive experiments using the HAM10000 dataset and the PH2dataset, which demonstrated superior performance compared to most existing weakly supervised segmentation methods.

SFE-Net: Spatial-Frequency Enhancement Network for robust nuclei segmentation in histopathology images

Morphological features of individual nuclei serve as a dependable foundation for pathologists in making accurate diagnoses. Existing methods that rely on spatial information for feature extraction have achieved commendable results in nuclei segmentation tasks. However, these approaches are not sufficient to extract edge information of nuclei with small sizes and blurred outlines. Moreover, the lack of attention to the interior of the nuclei leads to significant internal inconsistencies. To address these challenges, we introduce a novel Spatial-Frequency Enhancement Network (SFE-Net) to incorporate spatial-frequency features and promote intra-nuclei consistency for robust nuclei segmentation. Specifically, SFE-Net incorporates a distinctive Spatial-Frequency Feature Extraction module and a Spatial-Guided Feature Enhancement module, which are designed to preserve spatial-frequency information and enhance feature representation respectively, to achieve comprehensive extraction of edge information. Furthermore, we introduce the Label-Guided Distillation method, which utilizes semantic features to guide the segmentation network in strengthening boundary constraints and learning the intra-nuclei consistency of individual nuclei, to improve the robustness of nuclei segmentation. Extensive experiments on three publicly available histopathology image datasets (MoNuSeg, TNBC and CryoNuSeg) demonstrate the superiority of our proposed method, which achieves 79.23%, 81.96% and 73.26% Aggregated Jaccard Index, respectively. The proposed model is available at https://github.com/jinshachen/SFE-Net.

EAMR-Net: A multiscale effective spatial and cross-channel attention network for retinal vessel segmentation

Delineation of retinal vessels in fundus images is essential for detecting a range of eye disorders. An automated technique for vessel segmentation can assist clinicians and enhance the efficiency of the diagnostic process. Traditional methods fail to extract multiscale information, discard unnecessary information, and delineate thin vessels. In this paper, a novel residual U-Net architecture that incorporates multi-scale feature learning and effective attention is proposed to delineate the retinal vessels precisely. Since drop block regularization performs better than drop out in preventing overfitting, drop block was used in this study. A multi-scale feature learning module was added instead of a skip connection to learn multi-scale features. A novel effective attention block was proposed and integrated with the decoder block to obtain precise spatial and channel information. Experimental findings indicated that the proposed model exhibited outstanding performance in retinal vessel delineation. The sensitivities achieved for DRIVE, STARE, and CHASE_DB datasets were 0.8293, 0.8151 and 0.8084, respectively.

Learning with limited annotations: A survey on deep semi-supervised learning for medical image segmentation

Medical image segmentation is a fundamental and critical step in many image-guided clinical approaches. Recent success of deep learning-based segmentation methods usually relies on a large amount of labeled data, which is particularly difficult and costly to obtain, especially in the medical imaging domain where only experts can provide reliable and accurate annotations. Semi-supervised learning has emerged as an appealing strategy and been widely applied to medical image segmentation tasks to train deep models with limited annotations. In this paper, we present a comprehensive review of recently proposed semi-supervised learning methods for medical image segmentation and summarize both the technical novelties and empirical results. Furthermore, we analyze and discuss the limitations and several unsolved problems of existing approaches. We hope this review can inspire the research community to explore solutions to this challenge and further advance the field of medical image segmentation.

Multimodal deep learning for personalized renal cell carcinoma prognosis: Integrating CT imaging and clinical data

BACKGROUND AND OBJECTIVE

Renal cell carcinoma represents a significant global health challenge with a low survival rate. The aim of this research was to devise a comprehensive deep-learning model capable of predicting survival probabilities in patients with renal cell carcinoma by integrating CT imaging and clinical data and addressing the limitations observed in prior studies. The aim is to facilitate the identification of patients requiring urgent treatment.

METHODS

The proposed framework comprises three modules: a 3D image feature extractor, clinical variable selection, and survival prediction. Based on the 3D CNN architecture, the feature extractor module predicts the ISUP grade of renal cell carcinoma tumors linked to mortality rates from CT images. Clinical variables are systematically selected using the Spearman score and random forest importance score as criteria. A deep learning-based network, trained with discrete LogisticHazard-based loss, performs the survival prediction. Nine distinct experiments are performed, with varying numbers of clinical variables determined by different thresholds of the Spearman and importance scores.

RESULTS

Our findings demonstrate that the proposed strategy surpasses the current literature on renal cancer prognosis based on CT scans and clinical factors. The best-performing experiment yielded a concordance index of 0.84 and an area under the curve value of 0.8 on the test cohort, which suggests strong predictive power.

CONCLUSIONS

The multimodal deep-learning approach developed in this study shows promising results in estimating survival probabilities for renal cell carcinoma patients using CT imaging and clinical data. This may have potential implications in identifying patients who require urgent treatment, potentially improving patient outcomes. The code created for this project is available for the public on: GitHub.

Class key feature extraction and fusion for 2D medical image segmentation

BACKGROUND

The size variation, complex semantic environment and high similarity in medical images often prevent deep learning models from achieving good performance.

PURPOSE

To overcome these problems and improve the model segmentation performance and generalizability.

METHODS

We propose the key class feature reconstruction module (KCRM), which ranks channel weights and selects key features (KFs) that contribute more to the segmentation results for each class. Meanwhile, KCRM reconstructs all local features to establish the dependence relationship from local features to KFs. In addition, we propose the spatial gating module (SGM), which employs KFs to generate two spatial maps to suppress irrelevant regions, strengthening the ability to locate semantic objects. Finally, we enable the model to adapt to size variations by diversifying the receptive field.

RESULTS

We integrate these modules into class key feature extraction and fusion network (CKFFNet) and validate its performance on three public medical datasets: CHAOS, UW-Madison, and ISIC2017. The experimental results show that our method achieves better segmentation results and generalizability than those of mainstream methods.

CONCLUSION

Through quantitative and qualitative research, the proposed module improves the segmentation results and enhances the model generalizability, making it suitable for application and expansion.

Retinal vessel segmentation method based on RSP-SA Unet network

Segmenting retinal vessels plays a significant role in the diagnosis of fundus disorders. However, there are two problems in the retinal vessel segmentation methods. First, fine-grained features of fine blood vessels are difficult to be extracted. Second, it is easy to lose track of the details of blood vessel edges. To solve the problems above, the Residual SimAM Pyramid-Spatial Attention Unet (RSP-SA Unet) is proposed, in which the encoding, decoding, and upsampling layers of the Unet are mainly improved. Firstly, the RSP structure proposed in this paper approximates a residual structure combined with SimAM and Pyramid Segmentation Attention (PSA), which is applied to the encoding and decoding parts to extract multi-scale spatial information and important features across dimensions at a finer level. Secondly, the spatial attention (SA) is used in the upsampling layer to perform multi-attention mapping on the input feature map, which could enhance the segmentation effect of small blood vessels with low contrast. Finally, the RSP-SA Unet is verified on the CHASE_DB1, DRIVE, and STARE datasets, and the segmentation accuracy (ACC) of the RSP-SA Unet could reach 0.9763, 0.9704, and 0.9724, respectively. Area under the ROC curve (AUC) could reach 0.9896, 0.9858, and 0.9906, respectively. The RSP-SA Unet overall performance is better than the comparison methods.

Segmentation of Low-Light Optical Coherence Tomography Angiography Images under the Constraints of Vascular Network Topology

Optical coherence tomography angiography (OCTA) offers critical insights into the retinal vascular system, yet its full potential is hindered by challenges in precise image segmentation. Current methodologies struggle with imaging artifacts and clarity issues, particularly under low-light conditions and when using various high-speed CMOS sensors. These challenges are particularly pronounced when diagnosing and classifying diseases such as branch vein occlusion (BVO). To address these issues, we have developed a novel network based on topological structure generation, which transitions from superficial to deep retinal layers to enhance OCTA segmentation accuracy. Our approach not only demonstrates improved performance through qualitative visual comparisons and quantitative metric analyses but also effectively mitigates artifacts caused by low-light OCTA, resulting in reduced noise and enhanced clarity of the images. Furthermore, our system introduces a structured methodology for classifying BVO diseases, bridging a critical gap in this field. The primary aim of these advancements is to elevate the quality of OCTA images and bolster the reliability of their segmentation. Initial evaluations suggest that our method holds promise for establishing robust, fine-grained standards in OCTA vascular segmentation and analysis.

MAG-Net : Multi-fusion network with grouped attention for retinal vessel segmentation

Retinal vessel segmentation plays a vital role in the clinical diagnosis of ophthalmic diseases. Despite convolutional neural networks (CNNs) excelling in this task, challenges persist, such as restricted receptive fields and information loss from downsampling. To address these issues, we propose a new multi-fusion network with grouped attention (MAG-Net). First, we introduce a hybrid convolutional fusion module instead of the original encoding block to learn more feature information by expanding the receptive field. Additionally, the grouped attention enhancement module uses high-level features to guide low-level features and facilitates detailed information transmission through skip connections. Finally, the multi-scale feature fusion module aggregates features at different scales, effectively reducing information loss during decoder upsampling. To evaluate the performance of the MAG-Net, we conducted experiments on three widely used retinal datasets: DRIVE, CHASE and STARE. The results demonstrate remarkable segmentation accuracy, specificity and Dice coefficients. Specifically, the MAG-Net achieved segmentation accuracy values of 0.9708, 0.9773 and 0.9743, specificity values of 0.9836, 0.9875 and 0.9906 and Dice coefficients of 0.8576, 0.8069 and 0.8228, respectively. The experimental results demonstrate that our method outperforms existing segmentation methods exhibiting superior performance and segmentation outcomes.

Multi-scale context UNet-like network with redesigned skip connections for medical image segmentation

BACKGROUND AND OBJECTIVE

Medical image segmentation has garnered significant research attention in the neural network community as a fundamental requirement for developing intelligent medical assistant systems. A series of UNet-like networks with an encoder-decoder architecture have achieved remarkable success in medical image segmentation. Among these networks, UNet2+ (UNet++) and UNet3+ (UNet+++) have introduced redesigned skip connections, dense skip connections, and full-scale skip connections, respectively, surpassing the performance of the original UNet. However, UNet2+ lacks comprehensive information obtained from the entire scale, which hampers its ability to learn organ placement and boundaries. Similarly, due to the limited number of neurons in its structure, UNet3+ fails to effectively segment small objects when trained with a small number of samples.

METHOD

In this study, we propose UNet_sharp (UNet#), a novel network topology named after the "#" symbol, which combines dense skip connections and full-scale skip connections. In the decoder sub-network, UNet# can effectively integrate feature maps of different scales and capture fine-grained features and coarse-grained semantics from the entire scale. This approach enhances the understanding of organ and lesion positions and enables accurate boundary segmentation. We employ deep supervision for model pruning to accelerate testing and enable mobile device deployment. Additionally, we construct two classification-guided modules to reduce false positives and improve segmentation accuracy.

RESULTS

Compared to current UNet-like networks, our proposed method achieves the highest Intersection over Union (IoU) values ((92.67±0.96)%, (92.38±1.29)%, (95.36±1.22)%, (74.01±2.03)%) and F1 scores ((91.64±1.86)%, (95.70±2.16)%, (97.34±2.76)%, (84.77±2.65)%) on the semantic segmentation tasks of nuclei, brain tumors, liver, and lung nodules, respectively.

CONCLUSIONS

The experimental results demonstrate that the reconstructed skip connections in UNet successfully incorporate multi-scale contextual semantic information. Compared to most state-of-the-art medical image segmentation models, our proposed method more accurately locates organs and lesions and precisely segments boundaries.

An improved supervised and attention mechanism-based U-Net algorithm for retinal vessel segmentation

The segmentation results of retinal blood vessels are crucial for automatically diagnosing ophthalmic diseases such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular diseases. To improve the accuracy of vessel segmentation and better extract information about small vessels and edges, we introduce the U-Net algorithm with a supervised attention mechanism for retinal vessel segmentation. We achieve this by introducing a decoder fusion module (DFM) in the encoding part, effectively combining different convolutional blocks to extract features comprehensively. Additionally, in the decoding part of U-Net, we propose the context squeeze and excitation (CSE) decoding module to enhance important contextual feature information and the detection of tiny blood vessels. For the final output, we introduce the supervised fusion mechanism (SFM), which combines multiple branches from shallow to deep layers, effectively fusing multi-scale features and capturing information from different levels, fully integrating low-level and high-level features to improve segmentation performance. Our experimental results on the public datasets of DRIVE, STARE, and CHASED_B1 demonstrate the excellent performance of our proposed network.

Expanded tube attention for tubular structure segmentation

PURPOSE

Semantic segmentation of tubular structures, such as blood vessels and cell membranes, is a very difficult task, and it tends to break many predicted regions in the middle. This problem is due to the fact that tubular ground truth is very thin, and the number of pixels is extremely unbalanced compared to the background.

METHODS

We present a novel training method using pseudo-labels generated by morphological transformation. Furthermore, we present an attention module using thickened pseudo-labels, called the expanded tube attention (ETA) module. By using the ETA module, the network learns thickened regions based on pseudo-labels at first and then gradually learns thinned original regions while transferring information in the thickened regions as an attention map.

RESULTS

Through experiments conducted on retina vessel image datasets using various evaluation measures, we confirmed that the proposed method using ETA modules improved the clDice metric accuracy in comparison with the conventional methods.

CONCLUSIONS

We demonstrated that the proposed novel expanded tube attention module using thickened pseudo-labels can achieve easy-to-hard learning.

Retina image segmentation using the three-path Unet model

Unsupervised image segmentation is a technique that divides an image into distinct regions or objects without prior labeling. This approach offers flexibility and adaptability to various types of image data. Particularly for large datasets, it eliminates the need for manual labeling, thereby it presents advantages in terms of time and labor costs. However, when applied to retinal image segmentation, challenges arise due to variations in data, presence of noise, and manual threshold adjustments, which can lead to over-segmentation or under-segmentation of small blood vessel boundaries and endpoints. In order to enhance the precision and accuracy of retinal image segmentation, we propose a novel image supervised segmentation network based on three-path Unet model.Firstly, the Haar wavelet transform is employed to extract high-frequency image information, which forms the foundation for the proposed HaarNet, a Unet-inspired architecture. Next, the HaarNet is integrated with the Unet and SegNet frameworks to develop a three-path Unet model, referred to as TP-Unet. Finally, the model is further refined into TP-Unet+AE+DSL by incorporating the advantages of auto-encoding (AE) and deep supervised learning (DSL) techniques, thereby enhancing the overall performance of the system. To evaluate the effectiveness of our proposed model, we conduct experiments using the DRIVE and CHASE public datasets. On the DRIVE dataset, our recommended model achieves a Dice coefficient of 0.8291 and a sensitivity index of 0.8184. These results significantly outperform the Unet model by [Formula: see text] and [Formula: see text], respectively. Furthermore, our model demonstrates excellent performance on the CHASE dataset, with a Dice coefficient of 0.8162, a sensitivity of 0.8242, and an accuracy of 0.9664. These metrics surpass the Unet model by [Formula: see text], [Formula: see text], and [Formula: see text], respectively. Our proposed model provides more accurate and reliable results for retinal vessel segmentation, which holds significant potential for assisting doctors in their diagnosis.

A deep supervised transformer U-shaped full-resolution residual network for the segmentation of breast ultrasound image

PURPOSE

Breast ultrasound (BUS) is an important breast imaging tool. Automatic BUS image segmentation can measure the breast tumor size objectively and reduce doctors' workload. In this article, we proposed a deep supervised transformer U-shaped full-resolution residual network (DSTransUFRRN) to segment BUS images.

METHODS

In the proposed method, a full-resolution residual stream and a deep supervision mechanism were introduced into TransU-Net. The residual stream can keep full resolution features from different levels and enhance features fusion. Then, the deep supervision can suppress gradient dispersion. Moreover, the transformer module can suppress irrelevant features and improve feature extraction process. Two datasets (dataset A and B) were used for training and evaluation. The dataset A included 980 BUS image samples and the dataset B had 163 BUS image samples.

RESULTS

Cross-validation was conducted. For the dataset A, the proposed DSTransUFRRN achieved significantly higher Dice (91.04 ± 0.86%) than all compared methods (p < 0.05). For the dataset B, the Dice was lower than that for the dataset A due to the small number of samples, but the Dice of DSTransUFRRN (88.15% ± 2.11%) was significantly higher than that of other compared methods (p < 0.05).

CONCLUSIONS

In this study, we proposed DSTransUFRRN for BUS image segmentation. The proposed methods achieved significantly higher accuracy than the compared previous methods.

Follow-up of liver metastases: a comparison of deep learning and RECIST 1.1

OBJECTIVES

To compare liver metastases changes in CT assessed by radiologists using RECIST 1.1 and with aided simultaneous deep learning-based volumetric lesion changes analysis.

METHODS

A total of 86 abdominal CT studies from 43 patients (prior and current scans) of abdominal CT scans of patients with 1041 liver metastases (mean = 12.1, std = 11.9, range 1-49) were analyzed. Two radiologists performed readings of all pairs; conventional with RECIST 1.1 and with computer-aided assessment. For computer-aided reading, we used a novel simultaneous multi-channel 3D R2U-Net classifier trained and validated on other scans. The reference was established by having an expert radiologist validate the computed lesion detection and segmentation. The results were then verified and modified as needed by another independent radiologist. The primary outcome measure was the disease status assessment with the conventional and the computer-aided readings with respect to the reference.

RESULTS

For conventional and computer-aided reading, there was a difference in disease status classification in 40 out of 86 (46.51%) and 10 out of 86 (27.9%) CT studies with respect to the reference, respectively. Computer-aided reading improved conventional reading in 30 CT studies by 34.5% for two readers (23.2% and 46.51%) with respect to the reference standard. The main reason for the difference between the two readings was lesion volume differences (p = 0.01).

CONCLUSIONS

AI-based computer-aided analysis of liver metastases may improve the accuracy of the evaluation of neoplastic liver disease status.

CLINICAL RELEVANCE STATEMENT

AI may aid radiologists to improve the accuracy of evaluating changes over time in metastasis of the liver.

KEY POINTS

• Classification of liver metastasis changes improved significantly in one-third of the cases with an automatically generated comprehensive lesion and lesion changes report. • Simultaneous deep learning changes detection and volumetric assessment may improve the evaluation of liver metastases temporal changes potentially improving disease management.

Utilizing deep learning via the 3D U-net neural network for the delineation of brain stroke lesions in MRI image

The segmentation of acute stroke lesions plays a vital role in healthcare by assisting doctors in making prompt and well-informed treatment choices. Although Magnetic Resonance Imaging (MRI) is a time-intensive procedure, it produces high-fidelity images widely regarded as the most reliable diagnostic tool available. Employing deep learning techniques for automated stroke lesion segmentation can offer valuable insights into the precise location and extent of affected tissue, enabling medical professionals to effectively evaluate treatment risks and make informed assessments. In this research, a deep learning approach is introduced for segmenting acute and sub-acute stroke lesions from MRI images. To enhance feature learning through brain hemisphere symmetry, pre-processing techniques are applied to the data. To tackle the class imbalance challenge, we employed a strategy of using small patches with balanced sampling during training, along with a dynamically weighted loss function that incorporates f1-score and IOU-score (Intersection over Union). Furthermore, the 3D U-Net architecture is used to generate predictions for complete patches, employing a high degree of overlap between patches to minimize the requirement for subsequent post-processing steps. The 3D U-Net model, utilizing ResnetV2 as the pre-trained encoder for IOU-score and Seresnext101 for f1-score, stands as the leading state-of-the-art (SOTA) model for segmentation tasks. However, recent research has introduced a novel model that surpasses these metrics and demonstrates superior performance compared to other backbone architectures. The f1-score and IOU-score were computed for various backbones, with Seresnext101 achieving the highest f1-score and ResnetV2 performing the highest IOU-score. These calculations were conducted using a threshold value of 0.5. This research proposes a valuable model based on transfer learning for the classification of brain diseases in MRI scans. The achieved f1-score using the recommended classifiers demonstrates the effectiveness of the approach employed in this study. The findings indicate that Seresnext101 attains the highest f1-score of 0.94226, while ResnetV2 achieves the best IOU-score of 0.88342, making it the preferred architecture for segmentation methods. Furthermore, the study presents experimental results of the 3D U-Net model applied to brain stroke lesion segmentation, suggesting prospects for researchers interested in segmenting brain strokes and enhancing 3D U-Net models.

S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications

With the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-level features are learned, and the receptive field size of the network increases, which results in unsatisfactory performance for detecting low-level small/thin structures such as atrial walls and small arteries. To address this issue, we propose to keep the different encoding layer features at their original sizes to constrain the receptive field from increasing as the network goes deeper. Accordingly, we develop a novel S-shaped multiple cross-aggregation segmentation architecture named S-Net, which has two branches in the encoding stage, i.e., a resampling branch to capture low-level fine-grained details and thin/small structures and a downsampling branch to learn high-level discriminative knowledge. In particular, these two branches learn complementary features by residual cross-aggregation; the fusion of the complementary features from different decoding layers can be effectively accomplished through lateral connections. Meanwhile, we perform supervised prediction at all decoding layers to incorporate coarse-level features with high semantic meaning and fine-level features with high localization capability to detect multi-scale structures, especially for small/thin volumes fully. To validate the effectiveness of our S-Net, we conducted extensive experiments on the segmentation of cardiac wall and intracranial aneurysm (IA) vasculature, and quantitative and qualitative evaluations demonstrated the superior performance of our method for predicting small/thin structures in medical images.

Beyond CNNs: Exploiting Further Inherent Symmetries in Medical Image Segmentation

Automatic tumor or lesion segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because, although the human visual system can detect symmetries in 2-D images effectively, regular CNNs can only exploit translation invariance, overlooking further inherent symmetries existing in medical images, such as rotations and reflections. To solve this problem, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning more precise representations. First, kernel-based equivariant operations are devised on each orientation, which allows it to effectively address the gaps of learning symmetries in existing approaches. Then, to keep segmentation networks globally equivariant, we design distinctive group layers with layer-wise symmetry constraints. Finally, based on our novel framework, extensive experiments conducted on real-world clinical data demonstrate that a group equivariant Res-UNet (called GER-UNet) outperforms its regular CNN-based counterpart and the state-of-the-art segmentation methods in the tasks of hepatic tumor segmentation, COVID-19 lung infection segmentation, and retinal vessel detection. More importantly, the newly built GER-UNet also shows potential in reducing the sample complexity and the redundancy of filters, upgrading current segmentation CNNs, and delineating organs on other medical imaging modalities.

Nuclei instance segmentation from histopathology images using Bayesian dropout based deep learning

BACKGROUND

The deterministic deep learning models have achieved state-of-the-art performance in various medical image analysis tasks, including nuclei segmentation from histopathology images. The deterministic models focus on improving the model prediction accuracy without assessing the confidence in the predictions.

METHODS

We propose a semantic segmentation model using Bayesian representation to segment nuclei from the histopathology images and to further quantify the epistemic uncertainty. We employ Bayesian approximation with Monte-Carlo (MC) dropout during the inference time to estimate the model's prediction uncertainty.

RESULTS

We evaluate the performance of the proposed approach on the PanNuke dataset, which consists of 312 visual fields from 19 organ types. We compare the nuclei segmentation accuracy of our approach with that of a fully convolutional neural network, U-Net, SegNet, and the state-of-the-art Hover-net. We use F1-score and intersection over union (IoU) as the evaluation metrics. The proposed approach achieves a mean F1-score of 0.893 ± 0.008 and an IoU value of 0.868 ± 0.003 on the test set of the PanNuke dataset. These results outperform the Hover-net, which has a mean F1-score of 0.871 ± 0.010 and an IoU value of 0.840 ± 0.032.

CONCLUSIONS

The proposed approach, which incorporates Bayesian representation and Monte-Carlo dropout, demonstrates superior performance in segmenting nuclei from histopathology images compared to existing models such as U-Net, SegNet, and Hover-net. By considering the epistemic uncertainty, our model provides a more reliable estimation of the prediction confidence. These findings highlight the potential of Bayesian deep learning for improving medical image analysis tasks and can contribute to the development of more accurate and reliable computer-aided diagnostic systems.