MetricUNet: Synergistic image- and voxel-level learning for precise prostate segmentation via online sampling

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

Narrowing the semantic gaps in U-Net with learnable skip connections: The case of medical image segmentation

Current state-of-the-art medical image segmentation techniques predominantly employ the encoder-decoder architecture. Despite its widespread use, this U-shaped framework exhibits limitations in effectively capturing multi-scale features through simple skip connections. In this study, we made a thorough analysis to investigate the potential weaknesses of connections across various segmentation tasks, and suggest two key aspects of potential semantic gaps crucial to be considered: the semantic gap among multi-scale features in different encoding stages and the semantic gap between the encoder and the decoder. To bridge these semantic gaps, we introduce a novel segmentation framework, which incorporates a Dual Attention Transformer module for capturing channel-wise and spatial-wise relationships, and a Decoder-guided Recalibration Attention module for fusing DAT tokens and decoder features. These modules establish a principle of learnable connection that resolves the semantic gaps, leading to a high-performance segmentation model for medical images. Furthermore, it provides a new paradigm for effectively incorporating the attention mechanism into the traditional convolution-based architecture. Comprehensive experimental results demonstrate that our model achieves consistent, significant gains and outperforms state-of-the-art methods with relatively fewer parameters. This study contributes to the advancement of medical image segmentation by offering a more effective and efficient framework for addressing the limitations of current encoder-decoder architectures. Code: https://github.com/McGregorWwww/UDTransNet.

Automated contouring of CTV and OARs in planning CT scans using novel hybrid convolution-transformer networks for prostate cancer radiotherapy

PURPOSE OBJECTIVE(S)

Manual contouring of the prostate region in planning computed tomography (CT) images is a challenging task due to factors such as low contrast in soft tissues, inter- and intra-observer variability, and variations in organ size and shape. Consequently, the use of automated contouring methods can offer significant advantages. In this study, we aimed to investigate automated male pelvic multi-organ contouring in multi-center planning CT images using a hybrid convolutional neural network-vision transformer (CNN-ViT) that combines convolutional and ViT techniques.

MATERIALS/METHODS

We used retrospective data from 104 localized prostate cancer patients, with delineations of the clinical target volume (CTV) and critical organs at risk (OAR) for external beam radiotherapy. We introduced a novel attention-based fusion module that merges detailed features extracted through convolution with the global features obtained through the ViT.

RESULTS

The average dice similarity coefficients (DSCs) achieved by VGG16-UNet-ViT for the prostate, bladder, rectum, right femoral head (RFH), and left femoral head (LFH) were 91.75%, 95.32%, 87.00%, 96.30%, and 96.34%, respectively. Experiments conducted on multi-center planning CT images indicate that combining the ViT structure with the CNN network resulted in superior performance for all organs compared to pure CNN and transformer architectures. Furthermore, the proposed method achieves more precise contours compared to state-of-the-art techniques.

CONCLUSION

Results demonstrate that integrating ViT into CNN architectures significantly improves segmentation performance. These results show promise as a reliable and efficient tool to facilitate prostate radiotherapy treatment planning.

The use of deep learning in interventional radiotherapy (brachytherapy): A review with a focus on open source and open data

Deep learning advanced to one of the most important technologies in almost all medical fields. Especially in areas, related to medical imaging it plays a big role. However, in interventional radiotherapy (brachytherapy) deep learning is still in an early phase. In this review, first, we investigated and scrutinised the role of deep learning in all processes of interventional radiotherapy and directly related fields. Additionally, we summarised the most recent developments. For better understanding, we provide explanations of key terms and approaches to solving common deep learning problems. To reproduce results of deep learning algorithms both source code and training data must be available. Therefore, a second focus of this work is on the analysis of the availability of open source, open data and open models. In our analysis, we were able to show that deep learning plays already a major role in some areas of interventional radiotherapy, but is still hardly present in others. Nevertheless, its impact is increasing with the years, partly self-propelled but also influenced by closely related fields. Open source, data and models are growing in number but are still scarce and unevenly distributed among different research groups. The reluctance in publishing code, data and models limits reproducibility and restricts evaluation to mono-institutional datasets. The conclusion of our analysis is that deep learning can positively change the workflow of interventional radiotherapy but there is still room for improvements when it comes to reproducible results and standardised evaluation methods.

Lightweight semantic segmentation network for tumor cell nuclei and skin lesion

In the field of medical image segmentation, achieving fast and accurate semantic segmentation of tumor cell nuclei and skin lesions is of significant importance. However, the considerable variations in skin lesion forms and cell types pose challenges to attaining high network accuracy and robustness. Additionally, as network depth increases, the growing parameter size and computational complexity make practical implementation difficult. To address these issues, this paper proposes MD-UNet, a fast cell nucleus segmentation network that integrates Tokenized Multi-Layer Perceptron modules, attention mechanisms, and Inception structures. Firstly, tokenized MLP modules are employed to label and project convolutional features, reducing computational complexity. Secondly, the paper introduces Depthwise Attention blocks and Multi-layer Feature Extraction modules. The Depthwise Attention blocks eliminate irrelevant and noisy responses from coarse-scale extracted information, serving as alternatives to skip connections in the UNet architecture. The Multi-layer Feature Extraction modules capture a wider range of high-level and low-level semantic features during decoding and facilitate feature fusion. The proposed MD-UNet approach is evaluated on two datasets: the International Skin Imaging Collaboration (ISIC2018) dataset and the PanNuke dataset. The experimental results demonstrate that MD-UNet achieves the best performance on both datasets.

FAFS-UNet: Redesigning skip connections in UNet with feature aggregation and feature selection

In recent years, the encoder-decoder U-shaped network architecture has become a mainstream structure for medical image segmentation. Its biggest advantage lies in the incorporation of shallow features into deeper layers of the network through skip connections. However, according to our research, there are still some limitations in the skip connection part of the network: (1) The information from the encoder stage is not completely and effectively supplemented to the decoder stage; (2) The decoder receives the supplemented feature information from the encoder indiscriminately, which sometimes leads to the poor performance of the model. Therefore, to effectively address these limitations, we have redesigned the skip connections in UNet using a feature aggregation and feature selection approach. We firstly design the FA module to aggregate all encoder features and perform local multi-scale information extraction to obtain the complete multi-scale aggregated features. Further, we design the FS module to actively perform specific selection of these aggregated features through the decoder, thus effectively guiding the semantic recovery of the decoder. Finally, we conduct experiments on several medical image datasets, and the results show that our method has higher segmentation accuracy compared with other methods.

A survey on cancer detection via convolutional neural networks: Current challenges and future directions

Cancer is a condition in which abnormal cells uncontrollably split and damage the body tissues. Hence, detecting cancer at an early stage is highly essential. Currently, medical images play an indispensable role in detecting various cancers; however, manual interpretation of these images by radiologists is observer-dependent, time-consuming, and tedious. An automatic decision-making process is thus an essential need for cancer detection and diagnosis. This paper presents a comprehensive survey on automated cancer detection in various human body organs, namely, the breast, lung, liver, prostate, brain, skin, and colon, using convolutional neural networks (CNN) and medical imaging techniques. It also includes a brief discussion about deep learning based on state-of-the-art cancer detection methods, their outcomes, and the possible medical imaging data used. Eventually, the description of the dataset used for cancer detection, the limitations of the existing solutions, future trends, and challenges in this domain are discussed. The utmost goal of this paper is to provide a piece of comprehensive and insightful information to researchers who have a keen interest in developing CNN-based models for cancer detection.

Intelligent contour extraction approach for accurate segmentation of medical ultrasound images

Introduction: Accurate contour extraction in ultrasound images is of great interest for image-guided organ interventions and disease diagnosis. Nevertheless, it remains a problematic issue owing to the missing or ambiguous outline between organs (i.e., prostate and kidney) and surrounding tissues, the appearance of shadow artifacts, and the large variability in the shape of organs. Methods: To address these issues, we devised a method that includes four stages. In the first stage, the data sequence is acquired using an improved adaptive selection principal curve method, in which a limited number of radiologist defined data points are adopted as the prior. The second stage then uses an enhanced quantum evolution network to help acquire the optimal neural network. The third stage involves increasing the precision of the experimental outcomes after training the neural network, while using the data sequence as the input. In the final stage, the contour is smoothed using an explicable mathematical formula explained by the model parameters of the neural network. Results: Our experiments showed that our approach outperformed other current methods, including hybrid and Transformer-based deep-learning methods, achieving an average Dice similarity coefficient, Jaccard similarity coefficient, and accuracy of 95.7 ± 2.4%, 94.6 ± 2.6%, and 95.3 ± 2.6%, respectively. Discussion: This work develops an intelligent contour extraction approach on ultrasound images. Our approach obtained more satisfactory outcome compared with recent state-of-the-art approaches . The knowledge of precise boundaries of the organ is significant for the conservation of risk structures. Our developed approach has the potential to enhance disease diagnosis and therapeutic outcomes.

Automatic segmentation of prostate MRI based on 3D pyramid pooling Unet

PURPOSE

Automatic segmentation of prostate magnetic resonance (MR) images is crucial for the diagnosis, evaluation, and prognosis of prostate diseases (including prostate cancer). In recent years, the mainstream segmentation method for the prostate has been converted to convolutional neural networks. However, owing to the complexity of the tissue structure in MR images and the limitations of existing methods in spatial context modeling, the segmentation performance should be improved further.

METHODS

In this study, we proposed a novel 3D pyramid pool Unet that benefits from the pyramid pooling structure embedded in the skip connection (SC) and the deep supervision (DS) in the up-sampling of the 3D Unet. The parallel SC of the conventional 3D Unet network causes low-resolution information to be sent to the feature map repeatedly, resulting in blurred image features. To overcome the shortcomings of the conventional 3D Unet, we merge each decoder layer with the feature map of the same scale as the encoder and the smaller scale feature map of the pyramid pooling encoder. This SC combines the low-level details and high-level semantics at two different levels of feature maps. In addition, pyramid pooling performs multifaceted feature extraction on each image behind the convolutional layer, and DS learns hierarchical representations from comprehensive aggregated feature maps, which can improve the accuracy of the task.

RESULTS

Experiments on 3D prostate MR images of 78 patients demonstrated that our results were highly correlated with expert manual segmentation. The average relative volume difference and Dice similarity coefficient of the prostate volume area were 2.32% and 91.03%, respectively.

CONCLUSION

Quantitative experiments demonstrate that, compared with other methods, the results of our method are highly consistent with the expert manual segmentation.

Automatic prostate and peri-prostatic fat segmentation based on pyramid mechanism fusion network for T2-weighted MRI

BACKGROUND AND OBJECTIVE

Automatic and accurate segmentation of prostate and peri-prostatic fat in male pelvic MRI images is a critical step in the diagnosis and prognosis of prostate cancer. The boundary of prostate tissue is not clear, which makes the task of automatic segmentation very challenging. The main issues, especially for the peri-prostatic fat, which is being offered for the first time, are hazy boundaries and a large form variation.

METHODS

We propose a pyramid mechanism fusion network (PMF-Net) to learn global features and more comprehensive context information. In the proposed PMF-Net, we devised two pyramid techniques in particular. A pyramid mechanism module made of dilated convolutions of varying rates is inserted before each down sample of the fundamental network architecture encoder. The module is intended to address the issue of information loss during the feature coding process, particularly in the case of segmentation object boundary information. In the transition stage from encoder to decoder, pyramid fusion module is designed to extract global features. The features of the decoder not only integrate the features of the previous stage after up sampling and the output features of pyramid mechanism, but also include the features of skipping connection transmission under the same scale of the encoder.

RESULTS

The segmentation results of prostate and peri-prostatic fat on numerous diverse male pelvic MRI datasets show that our proposed PMF-Net has higher performance than existing methods. The average surface distance (ASD) and Dice similarity coefficient (DSC) of prostate segmentation results reached 10.06 and 90.21%, respectively. The ASD and DSC of the peri-prostatic fat segmentation results reached 50.96 and 82.41%.

CONCLUSIONS

The results of our segmentation are substantially connected and consistent with those of expert manual segmentation. Furthermore, peri-prostatic fat segmentation is a new issue, and good automatic segmentation has substantial therapeutic implications.

Technology for Position Correction of Satellite Precipitation and Contributions to Error Reduction-A Case of the '720' Rainstorm in Henan, China

In July 2021, an extreme precipitation event occurred in Henan, China, causing tremendous damage and deaths; so, it is very important to study the observation technology of extreme precipitation. Surface rain gauge precipitation observations have high accuracy but low resolution and coverage. Satellite remote sensing has high spatial resolution and wide coverage, but has large precipitation accuracy and distribution errors. Therefore, how to merge the above two kinds of precipitation observations effectively to obtain heavy precipitation products with more accurate geographic distributions has become an important but difficult scientific problem. In this paper, a new information fusion method for improving the position accuracy of satellite precipitation estimations is used based on the idea of registration and warping in image processing. The key point is constructing a loss function that includes a term for measuring two information field differences and a term for a warping field constraint. By minimizing the loss function, the purpose of position error correction of quantitative precipitation estimation from FY-4A and Integrated Multisatellite Retrievals of GPM are achieved, respectively, using observations from surface rain gauge stations. The errors of different satellite precipitation products relative to ground stations are compared and analyzed before and after position correction, using the ‘720’ extreme precipitation in Henan, China, as an example. The experimental results show that the final run has the best performance and FY-4A has the worse performance. After position corrections, the precipitation products of the three satellites are improved, among which FY-4A has the largest improvement, IMERG final run has the smallest improvement, and IMERG late run has the best performance and the smallest error. Their mean absolute errors are reduced by 23%, 14%, and 16%, respectively, and their correlation coefficients with rain gauge stations are improved by 63%, 9%, and 16%, respectively. The error decomposition model is used to examine the contributions of each error component to the total error. The results show that the new method improves the precipitation products of GPM primarily in terms of hit bias. However, it does not significantly reduce the hit bias of precipitation products of FY-4A while it reduces the total error by reducing the number of false alarms.

BMNet: A New Region-Based Metric Learning Method for Early Alzheimer's Disease Identification With FDG-PET Images

18F-fluorodeoxyglucose (FDG)-positron emission tomography (PET) reveals altered brain metabolism in individuals with mild cognitive impairment (MCI) and Alzheimer's disease (AD). Some biomarkers derived from FDG-PET by computer-aided-diagnosis (CAD) technologies have been proved that they can accurately diagnosis normal control (NC), MCI, and AD. However, existing FDG-PET-based researches are still insufficient for the identification of early MCI (EMCI) and late MCI (LMCI). Compared with methods based other modalities, current methods with FDG-PET are also inadequate in using the inter-region-based features for the diagnosis of early AD. Moreover, considering the variability in different individuals, some hard samples which are very similar with both two classes limit the classification performance. To tackle these problems, in this paper, we propose a novel bilinear pooling and metric learning network (BMNet), which can extract the inter-region representation features and distinguish hard samples by constructing the embedding space. To validate the proposed method, we collect 898 FDG-PET images from Alzheimer's disease neuroimaging initiative (ADNI) including 263 normal control (NC) patients, 290 EMCI patients, 147 LMCI patients, and 198 AD patients. Following the common preprocessing steps, 90 features are extracted from each FDG-PET image according to the automatic anatomical landmark (AAL) template and then sent into the proposed network. Extensive fivefold cross-validation experiments are performed for multiple two-class classifications. Experiments show that most metrics are improved after adding the bilinear pooling module and metric losses to the Baseline model respectively. Specifically, in the classification task between EMCI and LMCI, the specificity improves 6.38% after adding the triple metric loss, and the negative predictive value (NPV) improves 3.45% after using the bilinear pooling module. In addition, the accuracy of classification between EMCI and LMCI achieves 79.64% using imbalanced FDG-PET images, which illustrates that the proposed method yields a state-of-the-art result of the classification accuracy between EMCI and LMCI based on PET images.

Deep Learning for Automated Analysis of Cellular and Extracellular Components of the Foreign Body Response in Multiphoton Microscopy Images

The Foreign body response (FBR) is a major unresolved challenge that compromises medical implant integration and function by inflammation and fibrotic encapsulation. Mice implanted with polymeric scaffolds coupled to intravital non-linear multiphoton microscopy acquisition enable multiparametric, longitudinal investigation of the FBR evolution and interference strategies. However, follow-up analyses based on visual localization and manual segmentation are extremely time-consuming, subject to human error, and do not allow for automated parameter extraction. We developed an integrated computational pipeline based on an innovative and versatile variant of the U-Net neural network to segment and quantify cellular and extracellular structures of interest, which is maintained across different objectives without impairing accuracy. This software for automatically detecting the elements of the FBR shows promise to unravel the complexity of this pathophysiological process.

The dilated dense U-net for spinal fracture lesions segmentation

With the development of computer technology, more and more deep learning algorithms are widely used in medical image processing. Viewing CT images is a very usual and important way in diagnosing spinal fracture diseases, but correctly reading CT images and effectively segmenting spinal lesions or not is deeply depended on doctors’ clinical experiences. In this paper, we present a method of combining U-net, dense blocks and dilated convolution to segment lesions objectively, so as to give a help in diagnosing spinal diseases and provide a reference clinically. First, we preprocess and augment CT images of spinal lesions. Second, we present the DenseU-net network model consists of dense blocks and U-net to raise the depth of training network. Third, we introduce dilated convolution into DenseU-net to construct proposed DDU-net(Dilated Dense U-net), in order to raise receptive field of CT images for getting more lesions information. The experiments show that DDU-net has a good segmentation performance of spinal lesions, which can build a solid foundation for both doctors and patients.