1
|
He J, Luo Z, Lian S, Su S, Li S. Towards accurate abdominal tumor segmentation: A 2D model with Position-Aware and Key Slice Feature Sharing. Comput Biol Med 2024; 179:108743. [PMID: 38964246 DOI: 10.1016/j.compbiomed.2024.108743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 05/24/2024] [Accepted: 06/08/2024] [Indexed: 07/06/2024]
Abstract
Abdominal tumor segmentation is a crucial yet challenging step during the screening and diagnosis of tumors. While 3D segmentation models provide powerful performance, they demand substantial computational resources. Additionally, in 3D data, tumors often represent a small portion, leading to imbalanced data and potentially overlooking crucial information. Conversely, 2D segmentation models have a lightweight structure, but disregard the inter-slice correlation, risking the loss of tumor in edge slices. To address these challenges, this paper proposes a novel Position-Aware and Key Slice Feature Sharing 2D tumor segmentation model (PAKS-Net). Leveraging the Swin-Transformer, we effectively model the global features within each slice, facilitating essential information extraction. Furthermore, we introduce a Position-Aware module to capture the spatial relationship between tumors and their corresponding organs, mitigating noise and interference from surrounding organ tissues. To enhance the edge slice segmentation accuracy, we employ key slices to assist in the segmentation of other slices to prioritize tumor regions. Through extensive experiments on three abdominal tumor segmentation CT datasets and a lung tumor segmentation CT dataset, PAKS-Net demonstrates superior performance, reaching 0.893, 0.769, 0.598 and 0.738 tumor DSC on the KiTS19, LiTS17, pancreas and LOTUS datasets, surpassing 3D segmentation models, while remaining computationally efficient with fewer parameters.
Collapse
Affiliation(s)
- Jiezhou He
- Institute of Artificial Intelligence, Xiamen University Xiamen, Xiamen, 361005, China.
| | - Zhiming Luo
- Department of Artificial Intelligence, Xiamen University, Xiamen, 361005, China.
| | - Sheng Lian
- Fuzhou University, Xiamen, 361005, China.
| | - Songzhi Su
- Department of Artificial Intelligence, Xiamen University, Xiamen, 361005, China.
| | - Shaozi Li
- Department of Artificial Intelligence, Xiamen University, Xiamen, 361005, China.
| |
Collapse
|
2
|
Xu C, Wu X, Wang B, Chen J, Gao Z, Liu X, Zhang H. Accurate segmentation of liver tumor from multi-modality non-contrast images using a dual-stream multi-level fusion framework. Comput Med Imaging Graph 2024; 116:102414. [PMID: 38981250 DOI: 10.1016/j.compmedimag.2024.102414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 05/10/2024] [Accepted: 06/28/2024] [Indexed: 07/11/2024]
Abstract
The use of multi-modality non-contrast images (i.e., T1FS, T2FS and DWI) for segmenting liver tumors provides a solution by eliminating the use of contrast agents and is crucial for clinical diagnosis. However, this remains a challenging task to discover the most useful information to fuse multi-modality images for accurate segmentation due to inter-modal interference. In this paper, we propose a dual-stream multi-level fusion framework (DM-FF) to, for the first time, accurately segment liver tumors from non-contrast multi-modality images directly. Our DM-FF first designs an attention-based encoder-decoder to effectively extract multi-level feature maps corresponding to a specified representation of each modality. Then, DM-FF creates two types of fusion modules, in which a module fuses learned features to obtain a shared representation across multi-modality images to exploit commonalities and improve the performance, and a module fuses the decision evidence of segment to discover differences between modalities to prevent interference caused by modality's conflict. By integrating these three components, DM-FF enables multi-modality non-contrast images to cooperate with each other and enables an accurate segmentation. Evaluation on 250 patients including different types of tumors from two MRI scanners, DM-FF achieves a Dice of 81.20%, and improves performance (Dice by at least 11%) when comparing the eight state-of-the-art segmentation architectures. The results indicate that our DM-FF significantly promotes the development and deployment of non-contrast liver tumor technology.
Collapse
Affiliation(s)
- Chenchu Xu
- Artificial Intelligence Institute, Anhui University, Hefei, China; Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
| | - Xue Wu
- Artificial Intelligence Institute, Anhui University, Hefei, China
| | - Boyan Wang
- Department of Computer Science and Technology, Tsinghua University, Beijing, China.
| | - Jie Chen
- Artificial Intelligence Institute, Anhui University, Hefei, China
| | - Zhifan Gao
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen, China
| | - Xiujian Liu
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen, China.
| | - Heye Zhang
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen, China
| |
Collapse
|
3
|
d'Albenzio G, Kamkova Y, Naseem R, Ullah M, Colonnese S, Cheikh FA, Kumar RP. A dual-encoder double concatenation Y-shape network for precise volumetric liver and lesion segmentation. Comput Biol Med 2024; 179:108870. [PMID: 39024904 DOI: 10.1016/j.compbiomed.2024.108870] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 07/07/2024] [Accepted: 07/08/2024] [Indexed: 07/20/2024]
Abstract
Accurate segmentation of the liver and tumors from CT volumes is crucial for hepatocellular carcinoma diagnosis and pre-operative resection planning. Despite advances in deep learning-based methods for abdominal CT images, fully-automated segmentation remains challenging due to class imbalance and structural variations, often requiring cascaded approaches that incur significant computational costs. In this paper, we present the Dual-Encoder Double Concatenation Network (DEDC-Net) for simultaneous segmentation of the liver and its tumors. DEDC-Net leverages both residual and skip connections to enhance feature reuse and optimize performance in liver and tumor segmentation tasks. Extensive qualitative and quantitative experiments on the LiTS dataset demonstrate that DEDC-Net outperforms existing state-of-the-art liver segmentation methods. An ablation study was conducted to evaluate different encoder backbones - specifically VGG19 and ResNet - and the impact of incorporating an attention mechanism. Our results indicate that DEDC-Net, without any additional attention gates, achieves a superior mean Dice Score (DS) of 0.898 for liver segmentation. Moreover, integrating residual connections into one encoder yielded the highest DS for tumor segmentation tasks. The robustness of our proposed network was further validated on two additional, unseen CT datasets: IDCARDb-01 and COMET. Our model demonstrated superior lesion segmentation capabilities, particularly on IRCADb-01, achieving a DS of 0.629. The code implementation is publicly available at this website.
Collapse
Affiliation(s)
- Gabriella d'Albenzio
- The Intervention Center, Oslo University Hospital, 0slo, Norway; Department of Informatics, University of Oslo, Oslo, Norway.
| | - Yuliia Kamkova
- Department of Informatics, University of Oslo, Oslo, Norway; Department of Research and Development, Division of Emergencies and Critical Care, Oslo University Hospital, Oslo, Norway
| | - Rabia Naseem
- COMSATS, University Islamabad, Islamabad, Pakistan
| | - Mohib Ullah
- Department of Computer Science, Norwegian University of Science and Technology, Gjøvik, Norway
| | - Stefania Colonnese
- Department of Information Engineering, Electronics and Telecommunications (DIET), La Sapienza University of Rome, Rome, Italy
| | - Faouzi Alaya Cheikh
- Department of Computer Science, Norwegian University of Science and Technology, Gjøvik, Norway
| | | |
Collapse
|
4
|
Lai J, Luo Z, Liu J, Hu H, Jiang H, Liu P, He L, Cheng W, Ren W, Wu Y, Piao JG, Wu Z. Charged Gold Nanoparticles for Target Identification-Alignment and Automatic Segmentation of CT Image-Guided Adaptive Radiotherapy in Small Hepatocellular Carcinoma. NANO LETTERS 2024; 24:10614-10623. [PMID: 39046153 PMCID: PMC11363118 DOI: 10.1021/acs.nanolett.4c02823] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/17/2024] [Revised: 07/19/2024] [Accepted: 07/22/2024] [Indexed: 07/25/2024]
Abstract
Because of the challenges posed by anatomical uncertainties and the low resolution of plain computed tomography (CT) scans, implementing adaptive radiotherapy (ART) for small hepatocellular carcinoma (sHCC) using artificial intelligence (AI) faces obstacles in tumor identification-alignment and automatic segmentation. The current study aims to improve sHCC imaging for ART using a gold nanoparticle (Au NP)-based CT contrast agent to enhance AI-driven automated image processing. The synthesized charged Au NPs demonstrated notable in vitro aggregation, low cytotoxicity, and minimal organ toxicity. Over time, an in situ sHCC mouse model was established for in vivo CT imaging at multiple time points. The enhanced CT images processed using 3D U-Net and 3D Trans U-Net AI models demonstrated high geometric and dosimetric accuracy. Therefore, charged Au NPs enable accurate and automatic sHCC segmentation in CT images using classical AI models, potentially addressing the technical challenges related to tumor identification, alignment, and automatic segmentation in CT-guided online ART.
Collapse
Affiliation(s)
- Jianjun Lai
- Department
of Radiation Oncology, Zhejiang Hospital, Hangzhou 310013, China
- Instiute
of Intelligent Control and Robotics, Hangzhou
Dianzi University, Hangzhou 310018, China
| | - Zhizeng Luo
- Instiute
of Intelligent Control and Robotics, Hangzhou
Dianzi University, Hangzhou 310018, China
| | - Jiping Liu
- Department
of Radiation Physics, Zhejiang Cancer Hospital, Hangzhou 310022, China
| | - Haili Hu
- Department
of Radiation Oncology, Zhejiang Hospital, Hangzhou 310013, China
| | - Hao Jiang
- Department
of Radiation Oncology, Zhejiang Hospital, Hangzhou 310013, China
| | - Pengyuan Liu
- Department
of Radiation Oncology, Zhejiang Hospital, Hangzhou 310013, China
| | - Li He
- School
of Pharmaceutical Sciences, Zhejiang Chinese
Medical University, Hangzhou 310053, China
| | - Weiyi Cheng
- School
of Pharmaceutical Sciences, Zhejiang Chinese
Medical University, Hangzhou 310053, China
| | - Weiye Ren
- School
of Pharmaceutical Sciences, Zhejiang Chinese
Medical University, Hangzhou 310053, China
| | - Yajun Wu
- Department
of Pharmacy, Zhejiang Hospital, Hangzhou 310013, China
| | - Ji-Gang Piao
- School
of Pharmaceutical Sciences, Zhejiang Chinese
Medical University, Hangzhou 310053, China
| | - Zhibing Wu
- Department
of Radiation Oncology, Zhejiang Hospital, Hangzhou 310013, China
- Department
of Radiation Oncology, Affiliated Zhejiang
Hospital, Zhejiang University School of Medicine, Hangzhou 310013, China
| |
Collapse
|
5
|
Arjmandi N, Momennezhad M, Arastouei S, Mosleh-Shirazi MA, Albawi A, Pishevar Z, Nasseri S. Deep learning-based automated liver contouring using a small sample of radiotherapy planning computed tomography images. Radiography (Lond) 2024:S1078-8174(24)00203-7. [PMID: 39179459 DOI: 10.1016/j.radi.2024.08.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Revised: 07/11/2024] [Accepted: 08/06/2024] [Indexed: 08/26/2024]
Abstract
INTRODUCTION No study has yet investigated the minimum amount of data required for deep learning-based liver contouring. Therefore, this study aimed to investigate the feasibility of automated liver contouring using limited data. METHODS Radiotherapy planning Computed tomography (CT) images were subjected to various preprocessing methods, such as denoising and windowing. Segmentation was conducted using the modified Attention U-Net and Residual U-Net networks. Two different modified networks were trained separately for different training sizes. For each architecture, the model trained with the training set size that achieved the highest dice similarity coefficient (DSC) score was selected for further evaluation. Two unseen external datasets with different distributions from the training set were also used to examine the generalizability of the proposed method. RESULTS The modified Residual U-Net and Attention U-Net networks achieved average DSCs of 97.62% and 96.48%, respectively, on the test set, using 62 training cases. The average Hausdorff distances (AHDs) for the modified Residual U-Net and Attention U-Net networks were 0.57 mm and 0.71 mm, respectively. Also, the modified Residual U-Net and Attention U-Net networks were tested on two unseen external datasets, achieving DSCs of 95.35% and 95.82% for data from another center and 95.16% and 94.93% for the AbdomenCT-1K dataset, respectively. CONCLUSION This study demonstrates that deep learning models can accurately segment livers using a small training set. The method, utilizing simple preprocessing and modified network architectures, shows strong performance on unseen datasets, indicating its generalizability. IMPLICATIONS FOR PRACTICE This promising result suggests its potential for automated liver contouring in radiotherapy planning.
Collapse
Affiliation(s)
- N Arjmandi
- Department of Medical Physics, School of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran; Student research committee, Mashhad University of medical sciences, Mashhad, Iran.
| | - M Momennezhad
- Department of Medical Physics, School of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran; Medical Physics Research Center, School of Medicine, Mashhad University of Medical Sciences, Mashhad.
| | - S Arastouei
- Department of Radiation Oncology, Mashhad University of Medical Sciences, Mashhad, Iran.
| | - M A Mosleh-Shirazi
- Physics Unit, Department of Radio-Oncology, Shiraz University of Medical Sciences, Shiraz, Iran; Ionizing and Non-Ionizing Radiation Protection Research Center, School of Paramedical Sciences, Shiraz University of Medical Sciences, Shiraz, Iran.
| | - A Albawi
- Radiology Techniques Department, College of Medical Technology, The Islamic University, Najaf, Iraq; Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran.
| | - Z Pishevar
- Department of Radiation Oncology, Mashhad University of Medical Sciences, Mashhad, Iran.
| | - S Nasseri
- Department of Medical Physics, School of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran.
| |
Collapse
|
6
|
Zhang B, Qiu S, Liang T. Dual Attention-Based 3D U-Net Liver Segmentation Algorithm on CT Images. Bioengineering (Basel) 2024; 11:737. [PMID: 39061819 PMCID: PMC11273630 DOI: 10.3390/bioengineering11070737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Revised: 07/11/2024] [Accepted: 07/17/2024] [Indexed: 07/28/2024] Open
Abstract
The liver is a vital organ in the human body, and CT images can intuitively display its morphology. Physicians rely on liver CT images to observe its anatomical structure and areas of pathology, providing evidence for clinical diagnosis and treatment planning. To assist physicians in making accurate judgments, artificial intelligence techniques are adopted. Addressing the limitations of existing methods in liver CT image segmentation, such as weak contextual analysis and semantic information loss, we propose a novel Dual Attention-Based 3D U-Net liver segmentation algorithm on CT images. The innovations of our approach are summarized as follows: (1) We improve the 3D U-Net network by introducing residual connections to better capture multi-scale information and alleviate semantic information loss. (2) We propose the DA-Block encoder structure to enhance feature extraction capability. (3) We introduce the CBAM module into skip connections to optimize feature transmission in the encoder, reducing semantic gaps and achieving accurate liver segmentation. To validate the effectiveness of the algorithm, experiments were conducted on the LiTS dataset. The results showed that the Dice coefficient and HD95 index for liver images were 92.56% and 28.09 mm, respectively, representing an improvement of 0.84% and a reduction of 2.45 mm compared to 3D Res-UNet.
Collapse
Affiliation(s)
- Benyue Zhang
- Key Laboratory of Spectral Imaging Technology CAS, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China;
- School of Optoelectronics, University of Chinese Academy of Sciences, Beijing 100408, China
| | - Shi Qiu
- Key Laboratory of Spectral Imaging Technology CAS, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China;
| | - Ting Liang
- Department of Radiology, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an 710119, China
| |
Collapse
|
7
|
Ou J, Jiang L, Bai T, Zhan P, Liu R, Xiao H. ResTransUnet: An effective network combined with Transformer and U-Net for liver segmentation in CT scans. Comput Biol Med 2024; 177:108625. [PMID: 38823365 DOI: 10.1016/j.compbiomed.2024.108625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 04/15/2024] [Accepted: 05/18/2024] [Indexed: 06/03/2024]
Abstract
Liver segmentation is a fundamental prerequisite for the diagnosis and surgical planning of hepatocellular carcinoma. Traditionally, the liver contour is drawn manually by radiologists using a slice-by-slice method. However, this process is time-consuming and error-prone, depending on the radiologist's experience. In this paper, we propose a new end-to-end automatic liver segmentation framework, named ResTransUNet, which exploits the transformer's ability to capture global context for remote interactions and spatial relationships, as well as the excellent performance of the original U-Net architecture. The main contribution of this paper lies in proposing a novel fusion network that combines Unet and Transformer architectures. In the encoding structure, a dual-path approach is utilized, where features are extracted separately using both convolutional neural networks (CNNs) and Transformer networks. Additionally, an effective feature enhancement unit is designed to transfer the global features extracted by the Transformer network to the CNN for feature enhancement. This model aims to address the drawbacks of traditional Unet-based methods, such as feature loss during encoding and poor capture of global features. Moreover, it avoids the disadvantages of pure Transformer models, which suffer from large parameter sizes and high computational complexity. The experimental results on the LiTS2017 dataset demonstrate remarkable performance for our proposed model, with Dice coefficients, volumetric overlap error (VOE), and relative volume difference (RVD) values for liver segmentation reaching 0.9535, 0.0804, and -0.0007, respectively. Furthermore, to further validate the model's generalization capability, we conducted tests on the 3Dircadb, Chaos, and Sliver07 datasets. The experimental results demonstrate that the proposed method outperforms other closely related models with higher liver segmentation accuracy. In addition, significant improvements can be achieved by applying our method when handling liver segmentation with small and discontinuous liver regions, as well as blurred liver boundaries. The code is available at the website: https://github.com/Jouiry/ResTransUNet.
Collapse
Affiliation(s)
- Jiajie Ou
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China
| | - Linfeng Jiang
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China; School of Computing and College of Design and Engineering, National University of Singapore, Singapore.
| | - Ting Bai
- School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
| | - Peidong Zhan
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China
| | - Ruihua Liu
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China
| | - Hanguang Xiao
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China
| |
Collapse
|
8
|
Liu H, Yang J, Jiang C, He S, Fu Y, Zhang S, Hu X, Fang J, Ji W. S2DA-Net: Spatial and spectral-learning double-branch aggregation network for liver tumor segmentation in CT images. Comput Biol Med 2024; 174:108400. [PMID: 38613888 DOI: 10.1016/j.compbiomed.2024.108400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 03/10/2024] [Accepted: 04/01/2024] [Indexed: 04/15/2024]
Abstract
Accurate liver tumor segmentation is crucial for aiding radiologists in hepatocellular carcinoma evaluation and surgical planning. While convolutional neural networks (CNNs) have been successful in medical image segmentation, they face challenges in capturing long-term dependencies among pixels. On the other hand, Transformer-based models demand a high number of parameters and involve significant computational costs. To address these issues, we propose the Spatial and Spectral-learning Double-branched Aggregation Network (S2DA-Net) for liver tumor segmentation. S2DA-Net consists of a double-branched encoder and a decoder with a Group Multi-Head Cross-Attention Aggregation (GMCA) module, Two branches in the encoder consist of a Fourier Spectral-learning Multi-scale Fusion (FSMF) branch and a Multi-axis Aggregation Hadamard Attention (MAHA) branch. The FSMF branch employs a Fourier-based network to learn amplitude and phase information, capturing richer features and detailed information without introducing an excessive number of parameters. The FSMF branch utilizes a Fourier-based network to capture amplitude and phase information, enriching features without introducing excessive parameters. The MAHA branch incorporates spatial information, enhancing discriminative features while minimizing computational costs. In the decoding path, a GMCA module extracts local information and establishes long-term dependencies, improving localization capabilities by amalgamating features from diverse branches. Experimental results on the public LiTS2017 liver tumor datasets show that the proposed segmentation model achieves significant improvements compared to the state-of-the-art methods, obtaining dice per case (DPC) 69.4 % and global dice (DG) 80.0 % for liver tumor segmentation on the LiTS2017 dataset. Meanwhile, the pre-trained model based on the LiTS2017 datasets obtain, DPC 73.4 % and an DG 82.2 % on the 3DIRCADb dataset.
Collapse
Affiliation(s)
- Huaxiang Liu
- Department Radiology of Taizhou Hospital, Zhejiang University, Taizhou, 318000, Zhejiang, China; Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China; Key Laboratory of Evidence-based Radiology of Taizhou, Taizhou, 317000, Zhejiang, China
| | - Jie Yang
- School of Geophysics and Measurement and Control Technology, East China University of Technology, Nanchang, 330013, China
| | - Chao Jiang
- School of Geophysics and Measurement and Control Technology, East China University of Technology, Nanchang, 330013, China
| | - Sailing He
- Department Radiology of Taizhou Hospital, Zhejiang University, Taizhou, 318000, Zhejiang, China
| | - Youyao Fu
- Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China
| | - Shiqing Zhang
- Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China
| | - Xudong Hu
- Key Laboratory of Evidence-based Radiology of Taizhou, Taizhou, 317000, Zhejiang, China
| | - Jiangxiong Fang
- Institute of Intelligent Information Processing, Taizhou University, Taizhou, 318000, Zhejiang, China.
| | - Wenbin Ji
- Department Radiology of Taizhou Hospital, Zhejiang University, Taizhou, 318000, Zhejiang, China; Key Laboratory of Evidence-based Radiology of Taizhou, Taizhou, 317000, Zhejiang, China.
| |
Collapse
|
9
|
Li G, Xie J, Zhang L, Sun M, Li Z, Sun Y. MCAFNet: multiscale cross-layer attention fusion network for honeycomb lung lesion segmentation. Med Biol Eng Comput 2024; 62:1121-1137. [PMID: 38150110 DOI: 10.1007/s11517-023-02995-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/07/2023] [Indexed: 12/28/2023]
Abstract
Accurate segmentation of honeycomb lung lesions from lung CT images plays a crucial role in the diagnosis and treatment of various lung diseases. However, the availability of algorithms for automatic segmentation of honeycomb lung lesions remains limited. In this study, we propose a novel multi-scale cross-layer attention fusion network (MCAFNet) specifically designed for the segmentation of honeycomb lung lesions, taking into account their shape specificity and similarity to surrounding vascular shadows. The MCAFNet incorporates several key modules to enhance the segmentation performance. Firstly, a multiscale aggregation (MIA) module is introduced in the input part to preserve spatial information during downsampling. Secondly, a cross-layer attention fusion (CAF) module is proposed to capture multiscale features by integrating channel information and spatial information from different layers of the feature maps. Lastly, a bidirectional attention gate (BAG) module is constructed within the skip connection to enhance the model's ability to filter out background information and focus on the segmentation target. Experimental results demonstrate the effectiveness of the proposed MCAFNet. On the honeycomb lung segmentation dataset, the network achieves an Intersection over Union (IoU) of 0.895, mean IoU (mIoU) of 0.921, and mean Dice coefficient (mDice) of 0.949, outperforming existing medical image segmentation algorithms. Furthermore, experiments conducted on additional datasets confirm the generalizability and robustness of the proposed model. The contribution of this study lies in the development of the MCAFNet, which addresses the lack of automated segmentation algorithms for honeycomb lung lesions. The proposed network demonstrates superior performance in accurately segmenting honeycomb lung lesions, thereby facilitating the diagnosis and treatment of lung diseases. This work contributes to the existing literature by presenting a novel approach that effectively combines multi-scale features and attention mechanisms for lung lesion segmentation. The code is available at https://github.com/Oran9er/MCAFNet .
Collapse
Affiliation(s)
- Gang Li
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Jinjie Xie
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Ling Zhang
- Taiyuan University of Technology Software College, Taiyuan, China.
| | - Mengxia Sun
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Zhichao Li
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Yuanjin Sun
- Taiyuan University of Technology Software College, Taiyuan, China
| |
Collapse
|
10
|
He J, Zhang M, Li W, Peng Y, Fu B, Liu C, Wang J, Wang R. SaB-Net: Self-attention backward network for gastric tumor segmentation in CT images. Comput Biol Med 2024; 169:107866. [PMID: 38134751 DOI: 10.1016/j.compbiomed.2023.107866] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 11/04/2023] [Accepted: 12/17/2023] [Indexed: 12/24/2023]
Abstract
Gastric cancer is a significant contributor to cancer-related fatalities globally. The automated segmentation of gastric tumors has the potential to analyze the medical condition of patients and enhance the likelihood of surgical treatment success. However, the development of an automatic solution is challenged by the heterogeneous intensity distribution of gastric tumors in computed tomography (CT) images, the low-intensity contrast between organs, and the high variability in the stomach shapes and gastric tumors in different patients. To address these challenges, we propose a self-attention backward network (SaB-Net) for gastric tumor segmentation (GTS) in CT images by introducing a self-attention backward layer (SaB-Layer) to feed the self-attention information learned at the deep layer back to the shallow layers. The SaB-Layer efficiently extracts tumor information from CT images and integrates the information into the network, thereby enhancing the network's tumor segmentation ability. We employed datasets from two centers, one for model training and testing and the other for external validation. The model achieved dice scores of 0.8456 on the test set and 0.8068 on the external verification set. Moreover, we validated the model's transfer learning ability on a publicly available liver cancer dataset, achieving results comparable to state-of-the-art liver cancer segmentation models recently developed. SaB-Net has strong potential for assisting in the clinical diagnosis of and therapy for gastric cancer. Our implementation is available at https://github.com/TyrionJ/SaB-Net.
Collapse
Affiliation(s)
- Junjie He
- Engineering Research Center of Text Computing & Cognitive Intelligence, Ministry of Education, Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, No. 2288, Huaxi Avenue, Guiyang, 550025, Guizhou, China; Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, No. 83, Zhongshan Dong Road, Guiyang, 550002, Guizhou, China
| | - Mudan Zhang
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, No. 83, Zhongshan Dong Road, Guiyang, 550002, Guizhou, China
| | - Wuchao Li
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, No. 83, Zhongshan Dong Road, Guiyang, 550002, Guizhou, China
| | - Yunsong Peng
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, No. 83, Zhongshan Dong Road, Guiyang, 550002, Guizhou, China
| | - Bangkang Fu
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, No. 83, Zhongshan Dong Road, Guiyang, 550002, Guizhou, China
| | - Chen Liu
- Department of Radiology, Southwest Hospital, Army Medical University (Third Military Medical University), No. 30, Gao Tan Yan Street, 400038, Chongqing, China
| | - Jian Wang
- Department of Radiology, Southwest Hospital, Army Medical University (Third Military Medical University), No. 30, Gao Tan Yan Street, 400038, Chongqing, China
| | - Rongpin Wang
- Department of Radiology, International Exemplary Cooperation Base of Precision Imaging for Diagnosis and Treatment, Guizhou Provincial People's Hospital, No. 83, Zhongshan Dong Road, Guiyang, 550002, Guizhou, China.
| |
Collapse
|
11
|
Yang S, Liang Y, Wu S, Sun P, Chen Z. SADSNet: A robust 3D synchronous segmentation network for liver and liver tumors based on spatial attention mechanism and deep supervision. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024; 32:707-723. [PMID: 38552134 DOI: 10.3233/xst-230312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2024]
Abstract
Highlights • Introduce a data augmentation strategy to expand the required different morphological data during the training and learning phase, and improve the algorithm's feature learning ability for complex and diverse tumor morphology CT images.• Design attention mechanisms for encoding and decoding paths to extract fine pixel level features, improve feature extraction capabilities, and achieve efficient spatial channel feature fusion.• The deep supervision layer is used to correct and decode the final image data to provide high accuracy of results.• The effectiveness of this method has been affirmed through validation on the LITS, 3DIRCADb, and SLIVER datasets. BACKGROUND Accurately extracting liver and liver tumors from medical images is an important step in lesion localization and diagnosis, surgical planning, and postoperative monitoring. However, the limited number of radiation therapists and a great number of images make this work time-consuming. OBJECTIVE This study designs a spatial attention deep supervised network (SADSNet) for simultaneous automatic segmentation of liver and tumors. METHOD Firstly, self-designed spatial attention modules are introduced at each layer of the encoder and decoder to extract image features at different scales and resolutions, helping the model better capture liver tumors and fine structures. The designed spatial attention module is implemented through two gate signals related to liver and tumors, as well as changing the size of convolutional kernels; Secondly, deep supervision is added behind the three layers of the decoder to assist the backbone network in feature learning and improve gradient propagation, enhancing robustness. RESULTS The method was testing on LITS, 3DIRCADb, and SLIVER datasets. For the liver, it obtained dice similarity coefficients of 97.03%, 96.11%, and 97.40%, surface dice of 81.98%, 82.53%, and 86.29%, 95% hausdorff distances of 8.96 mm, 8.26 mm, and 3.79 mm, and average surface distances of 1.54 mm, 1.19 mm, and 0.81 mm. Additionally, it also achieved precise tumor segmentation, which with dice scores of 87.81% and 87.50%, surface dice of 89.63% and 84.26%, 95% hausdorff distance of 12.96 mm and 16.55 mm, and average surface distances of 1.11 mm and 3.04 mm on LITS and 3DIRCADb, respectively. CONCLUSION The experimental results show that the proposed method is effective and superior to some other methods. Therefore, this method can provide technical support for liver and liver tumor segmentation in clinical practice.
Collapse
Affiliation(s)
- Sijing Yang
- School of Life and Environmental Science, Guilin University of Electronic Technology, Guilin, China
| | - Yongbo Liang
- School of Life and Environmental Science, Guilin University of Electronic Technology, Guilin, China
| | - Shang Wu
- School of Life and Environmental Science, Guilin University of Electronic Technology, Guilin, China
| | - Peng Sun
- School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, China
| | - Zhencheng Chen
- School of Life and Environmental Science, Guilin University of Electronic Technology, Guilin, China
- School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, China
- Guangxi Colleges and Universities Key Laboratory of Biomedical Sensors and Intelligent Instruments, Guilin, China
- Guangxi Engineering Technology Research Center of Human Physiological Information Noninvasive Detection, Guilin, China
| |
Collapse
|