1
|
Yang S, Li J, Li Y, Nie J, Ercisli S, Khan MA. Imbalanced segmentation for abnormal cotton fiber based on GAN and multiscale residual U-Net. ALEXANDRIA ENGINEERING JOURNAL 2024; 106:25-41. [DOI: 10.1016/j.aej.2024.07.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/25/2024]
|
2
|
Delmoral JC, R S Tavares JM. Semantic Segmentation of CT Liver Structures: A Systematic Review of Recent Trends and Bibliometric Analysis : Neural Network-based Methods for Liver Semantic Segmentation. J Med Syst 2024; 48:97. [PMID: 39400739 DOI: 10.1007/s10916-024-02115-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2024] [Accepted: 10/02/2024] [Indexed: 10/15/2024]
Abstract
The use of artificial intelligence (AI) in the segmentation of liver structures in medical images has become a popular research focus in the past half-decade. The performance of AI tools in screening for this task may vary widely and has been tested in the literature in various datasets. However, no scientometric report has provided a systematic overview of this scientific area. This article presents a systematic and bibliometric review of recent advances in neuronal network modeling approaches, mainly of deep learning, to outline the multiple research directions of the field in terms of algorithmic features. Therefore, a detailed systematic review of the most relevant publications addressing fully automatic semantic segmenting liver structures in Computed Tomography (CT) images in terms of algorithm modeling objective, performance benchmark, and model complexity is provided. The review suggests that fully automatic hybrid 2D and 3D networks are the top performers in the semantic segmentation of the liver. In the case of liver tumor and vasculature segmentation, fully automatic generative approaches perform best. However, the reported performance benchmark indicates that there is still much to be improved in segmenting such small structures in high-resolution abdominal CT scans.
Collapse
Affiliation(s)
- Jessica C Delmoral
- Instituto de Ciência e Inovação em Engenharia Mecânica e Engenharia Industrial, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias, s/n, 4200-465, Porto, Portugal
| | - João Manuel R S Tavares
- Instituto de Ciência e Inovação em Engenharia Mecânica e Engenharia Industrial, Departamento de Engenharia Mecânica, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias, s/n, 4200-465, Porto, Portugal.
| |
Collapse
|
3
|
Zhang S, Chen L, Tan Y, Wu S, Guo P, Jiang X, Pan H. Deep learning assisted quantitative detection of cardiac troponin I in hierarchical dendritic copper-nickel nanostructure lateral flow immunoassay. ANALYTICAL METHODS : ADVANCING METHODS AND APPLICATIONS 2024; 16:6715-6725. [PMID: 39259228 DOI: 10.1039/d4ay01187b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/12/2024]
Abstract
The rising demand for point-of-care testing (POCT) in disease diagnosis has made LFIA sensors based on dendritic metal thin film (HD-nanometal) and background fluorescence technology essential for rapid and accurate disease marker detection, thanks to their integrated design, high sensitivity, and cost-effectiveness. However, their unique 3D nanostructures cause significant fluorescence variation, challenging traditional image processing methods in segmenting weak fluorescence regions. This paper develops a deep learning method to efficiently segment target regions in HD-nanometal LFIA sensor images, improving quantitative detection accuracy. We propose an improved UNet++ network with attention and residual modules, accurately segmenting varying fluorescence intensities, especially weak ones. We evaluated the method using IoU and Dice coefficients, comparing it with UNet, Deeplabv3, and UNet++. We used an HD-nanoCu-Ni LFIA sensor for cardiac troponin I (cTnI) as a case study to validate the method's practicality. The proposed method achieved a 96.3% IoU, outperforming other networks. The R2 between characteristic quantity and cTnI concentration reached 0.994, confirming the method's accuracy and reliability. This enhances POCT accuracy and provides a reference for future fluorescence immunochromatography expansion.
Collapse
Affiliation(s)
- Shenglan Zhang
- Key Laboratory of Advanced Manufacturing and Automation Technology(Guilin University of Technology), Education Department of Guangxi Zhuang Autonomous Region, Guilin, 541006, China
- College of Mechanical and Control Engineering, Guilin University of Technology, Guilin, 541006, China
- College of Environment and Science, Guilin University of Technology, Guilin 541006, China.
| | - Liqiang Chen
- Key Laboratory of Advanced Manufacturing and Automation Technology(Guilin University of Technology), Education Department of Guangxi Zhuang Autonomous Region, Guilin, 541006, China
- College of Mechanical and Control Engineering, Guilin University of Technology, Guilin, 541006, China
| | - YuXin Tan
- College of Environment and Science, Guilin University of Technology, Guilin 541006, China.
| | - Shaojie Wu
- Key Laboratory of Advanced Manufacturing and Automation Technology(Guilin University of Technology), Education Department of Guangxi Zhuang Autonomous Region, Guilin, 541006, China
- College of Mechanical and Control Engineering, Guilin University of Technology, Guilin, 541006, China
| | - Pengxin Guo
- Key Laboratory of Advanced Manufacturing and Automation Technology(Guilin University of Technology), Education Department of Guangxi Zhuang Autonomous Region, Guilin, 541006, China
- College of Mechanical and Control Engineering, Guilin University of Technology, Guilin, 541006, China
| | - Xincheng Jiang
- Key Laboratory of Advanced Manufacturing and Automation Technology(Guilin University of Technology), Education Department of Guangxi Zhuang Autonomous Region, Guilin, 541006, China
- College of Mechanical and Control Engineering, Guilin University of Technology, Guilin, 541006, China
| | - Hongcheng Pan
- College of Environment and Science, Guilin University of Technology, Guilin 541006, China.
| |
Collapse
|
4
|
Bao F, Zhao Y, Zhang X, Zhang Y, Ning Y. SARC-UNet: A coronary artery segmentation method based on spatial attention and residual convolution. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 255:108353. [PMID: 39096572 DOI: 10.1016/j.cmpb.2024.108353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 07/18/2024] [Accepted: 07/22/2024] [Indexed: 08/05/2024]
Abstract
BACKGROUND AND OBJECTIVE Coronary artery segmentation is a pivotal field that has received increasing attention in recent years. However, this task remains challenging because of the inhomogeneous distributions of the contrast agent and dim light, resulting in noise, vascular breakages and small vessel losses in the obtained segmentation results. METHODS To acquire better automatic blood vessel segmentation results for coronary angiography images, a UNet-based segmentation network (SARC-UNet) is constructed for coronary artery segmentation; this approach is based on residual convolution and spatial attention. First, we use the low-light image enhancement (LIME) approach to increase the contrast and clarity levels of coronary angiography images. Then, we design two residual convolution fusion modules (RCFM1 and RCFM2) that can successfully fuse the local and global information of coronary images while also capturing the characteristics of finer-grained blood vessels, hence preventing the loss of tiny blood vessels in the segmentation findings. Finally, using a cascaded waterfall structure, we create a new location-enhanced spatial attention (LESA) mechanism that can efficiently improve the long-distance dependencies between coronary vascular pixel features, eradicating vascular ruptures and noise in the segmentation results. RESULTS This article subjectively and objectively evaluates the experimental results. This method has performed well on five general indicators. Furthermore, it outperforms the connectivity indicators proposed in this article. This method can effectively segment blood vessels and obtain higher accuracy results. CONCLUSIONS Numerous experiments have shown that the suggested method outperforms the state-of-the-art approaches, particularly in terms of vessel connectivity and small blood vessel segmentation.
Collapse
Affiliation(s)
- Fangxun Bao
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China.
| | - Yongqi Zhao
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China
| | - Xinyue Zhang
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China
| | - Yunfeng Zhang
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, Shandong, 250014, China
| | - Yang Ning
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, Shandong, 250101, China
| |
Collapse
|
5
|
Qu Y, Lu T, Zhang S, Wang G. ScribSD+: Scribble-supervised medical image segmentation based on simultaneous multi-scale knowledge distillation and class-wise contrastive regularization. Comput Med Imaging Graph 2024; 116:102416. [PMID: 39018640 DOI: 10.1016/j.compmedimag.2024.102416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 06/16/2024] [Accepted: 07/04/2024] [Indexed: 07/19/2024]
Abstract
Despite that deep learning has achieved state-of-the-art performance for automatic medical image segmentation, it often requires a large amount of pixel-level manual annotations for training. Obtaining these high-quality annotations is time-consuming and requires specialized knowledge, which hinders the widespread application that relies on such annotations to train a model with good segmentation performance. Using scribble annotations can substantially reduce the annotation cost, but often leads to poor segmentation performance due to insufficient supervision. In this work, we propose a novel framework named as ScribSD+ that is based on multi-scale knowledge distillation and class-wise contrastive regularization for learning from scribble annotations. For a student network supervised by scribbles and the teacher based on Exponential Moving Average (EMA), we first introduce multi-scale prediction-level Knowledge Distillation (KD) that leverages soft predictions of the teacher network to supervise the student at multiple scales, and then propose class-wise contrastive regularization which encourages feature similarity within the same class and dissimilarity across different classes, thereby effectively improving the segmentation performance of the student network. Experimental results on the ACDC dataset for heart structure segmentation and a fetal MRI dataset for placenta and fetal brain segmentation demonstrate that our method significantly improves the student's performance and outperforms five state-of-the-art scribble-supervised learning methods. Consequently, the method has a potential for reducing the annotation cost in developing deep learning models for clinical diagnosis.
Collapse
Affiliation(s)
- Yijie Qu
- University of Electronic Science and Technology of China, Chengdu, China
| | - Tao Lu
- Sichuan Provincial People's Hospital, Chengdu, China
| | - Shaoting Zhang
- University of Electronic Science and Technology of China, Chengdu, China; Shanghai AI lab, Shanghai, China
| | - Guotai Wang
- University of Electronic Science and Technology of China, Chengdu, China; Shanghai AI lab, Shanghai, China.
| |
Collapse
|
6
|
Li S, Li X, Zhou F, Zhang Y, Bie Z, Cheng L, Peng J, Li B. Automated segmentation of liver and hepatic vessels on portal venous phase computed tomography images using a deep learning algorithm. J Appl Clin Med Phys 2024; 25:e14397. [PMID: 38773719 PMCID: PMC11302809 DOI: 10.1002/acm2.14397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Revised: 04/20/2024] [Accepted: 04/29/2024] [Indexed: 05/24/2024] Open
Abstract
BACKGROUND CT-image segmentation for liver and hepatic vessels can facilitate liver surgical planning. However, time-consuming process and inter-observer variations of manual segmentation have limited wider application in clinical practice. PURPOSE Our study aimed to propose an automated deep learning (DL) segmentation algorithm for liver and hepatic vessels on portal venous phase CT images. METHODS This retrospective study was performed to develop a coarse-to-fine DL-based algorithm that was trained, validated, and tested using private 413, 52, and 50 portal venous phase CT images, respectively. Additionally, the performance of the DL algorithm was extensively evaluated and compared with manual segmentation using an independent clinical dataset of preoperative contrast-enhanced CT images from 44 patients with hepatic focal lesions. The accuracy of DL-based segmentation was quantitatively evaluated using the Dice Similarity Coefficient (DSC) and complementary metrics [Normalized Surface Dice (NSD) and Hausdorff distance_95 (HD95) for liver segmentation, Recall and Precision for hepatic vessel segmentation]. The processing time for DL and manual segmentation was also compared. RESULTS Our DL algorithm achieved accurate liver segmentation with DSC of 0.98, NSD of 0.92, and HD95 of 1.52 mm. DL-segmentation of hepatic veins, portal veins, and inferior vena cava attained DSC of 0.86, 0.89, and 0.94, respectively. Compared with the manual approach, the DL algorithm significantly outperformed with better segmentation results for both liver and hepatic vessels, with higher accuracy of liver and hepatic vessel segmentation (all p < 0.001) in independent 44 clinical data. In addition, the DL method significantly reduced the manual processing time of clinical postprocessing (p < 0.001). CONCLUSIONS The proposed DL algorithm potentially enabled accurate and rapid segmentation for liver and hepatic vessels using portal venous phase contrast CT images.
Collapse
Affiliation(s)
- Shengwei Li
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Xiao‐Guang Li
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Fanyu Zhou
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Yumeng Zhang
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Zhixin Bie
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Lin Cheng
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Jinzhao Peng
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| | - Bin Li
- Minimally Invasive Tumor Therapy CenterBeijing HospitalPeking Union Medical CollegeBeijingChina
| |
Collapse
|
7
|
Dickson AJ, Linsely JA, Daniel VAA, Rahul K. Sparse deep belief network coupled with extended local fuzzy active contour model-based liver cancer segmentation from abdomen CT images. Med Biol Eng Comput 2024; 62:1361-1374. [PMID: 38189903 DOI: 10.1007/s11517-023-03001-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Accepted: 12/13/2023] [Indexed: 01/09/2024]
Abstract
Liver cancer from abdominal CT images must be accurately segmented for the purpose of diagnosis with treatment planning. But, the similarity in gray values between the liver and the surrounding tissues poses a challenge. To address this, a novel sparse deep belief network coupled with extended local fuzzy active contour model-based liver cancer segmentation from abdomen CT images (SDBN-ELFAC-LCS-CT) is proposed. This method incorporates dynamic adaptive pooling and residual modules in SDBN to improve the feature selection and generalization ability. Additionally, the 3D reconstruction is performed to refine segmentation results. The proposed SDBN-ELFAC-LCS-CT approach is implemented in MATLAB. The performance of the proposed SDBN-ELFAC-LCS-CT achieves dice coefficients that were up to 96.16% higher and 75.88%, 88.75%, and 71.16% lower. Volumetric overlap error compared with existing models, like basic ensembles of vanilla-style deep learning modes, increases liver segmentation from CT imageries (BEVS-LCS-CT), an incorporated 3 dimensional sparse deep belief network along enriched seagull optimization approach for liver segmentation (3DBN-ESOA-LCS-CT) and iterative convolutional encoder-decoder network and multiple scale context learning for segmenting liver (ICEDN-LCS-CT), respectively.
Collapse
Affiliation(s)
- A Joel Dickson
- Department of Electronics and Communication Engineering, Bethlahem Institute of Engineering, Karungal, Kanyakumari, 629157, Tamil Nadu, India.
| | - J Arul Linsely
- Department of Electrical and Electronics Engineering, Noorul Islam Centre for Higher Education, Kumaracoil, Kanyakumari, 629166, Tamil Nadu, India
| | - V Antony Asir Daniel
- Department of Electronics and Communication Engineering, Loyola Institute of Technology & Science, Kanyakumari, 629302, Tamil Nadu, India
| | - Kumar Rahul
- Department of Basic and Applied Science, NIFTEM, Kundli, Sonepat, 131028, India
| |
Collapse
|
8
|
Zhang W, Tao Y, Huang Z, Li Y, Chen Y, Song T, Ma X, Zhang Y. Multi-phase features interaction transformer network for liver tumor segmentation and microvascular invasion assessment in contrast-enhanced CT. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:5735-5761. [PMID: 38872556 DOI: 10.3934/mbe.2024253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2024]
Abstract
Precise segmentation of liver tumors from computed tomography (CT) scans is a prerequisite step in various clinical applications. Multi-phase CT imaging enhances tumor characterization, thereby assisting radiologists in accurate identification. However, existing automatic liver tumor segmentation models did not fully exploit multi-phase information and lacked the capability to capture global information. In this study, we developed a pioneering multi-phase feature interaction Transformer network (MI-TransSeg) for accurate liver tumor segmentation and a subsequent microvascular invasion (MVI) assessment in contrast-enhanced CT images. In the proposed network, an efficient multi-phase features interaction module was introduced to enable bi-directional feature interaction among multiple phases, thus maximally exploiting the available multi-phase information. To enhance the model's capability to extract global information, a hierarchical transformer-based encoder and decoder architecture was designed. Importantly, we devised a multi-resolution scales feature aggregation strategy (MSFA) to optimize the parameters and performance of the proposed model. Subsequent to segmentation, the liver tumor masks generated by MI-TransSeg were applied to extract radiomic features for the clinical applications of the MVI assessment. With Institutional Review Board (IRB) approval, a clinical multi-phase contrast-enhanced CT abdominal dataset was collected that included 164 patients with liver tumors. The experimental results demonstrated that the proposed MI-TransSeg was superior to various state-of-the-art methods. Additionally, we found that the tumor mask predicted by our method showed promising potential in the assessment of microvascular invasion. In conclusion, MI-TransSeg presents an innovative paradigm for the segmentation of complex liver tumors, thus underscoring the significance of multi-phase CT data exploitation. The proposed MI-TransSeg network has the potential to assist radiologists in diagnosing liver tumors and assessing microvascular invasion.
Collapse
Affiliation(s)
- Wencong Zhang
- Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou, China
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Yuxi Tao
- Department of Radiology, The Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China
| | - Zhanyao Huang
- Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou, China
| | - Yue Li
- School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
| | - Yingjia Chen
- Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou, China
| | - Tengfei Song
- Department of Radiology, The Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China
| | - Xiangyuan Ma
- Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou, China
| | - Yaqin Zhang
- Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou, China
| |
Collapse
|
9
|
Xu K, Zhang F, Huang Y, Huang X. 2.5D UNet with context-aware feature sequence fusion for accurate esophageal tumor semantic segmentation. Phys Med Biol 2024; 69:085002. [PMID: 38484399 DOI: 10.1088/1361-6560/ad3419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Accepted: 03/14/2024] [Indexed: 04/04/2024]
Abstract
Segmenting esophageal tumor from computed tomography (CT) sequence images can assist doctors in diagnosing and treating patients with this malignancy. However, accurately extracting esophageal tumor features from CT images often present challenges due to their small area, variable position, and shape, as well as the low contrast with surrounding tissues. This results in not achieving the level of accuracy required for practical applications in current methods. To address this problem, we propose a 2.5D context-aware feature sequence fusion UNet (2.5D CFSF-UNet) model for esophageal tumor segmentation in CT sequence images. Specifically, we embed intra-slice multiscale attention feature fusion (Intra-slice MAFF) in each skip connection of UNet to improve feature learning capabilities, better expressing the differences between anatomical structures within CT sequence images. Additionally, the inter-slice context fusion block (Inter-slice CFB) is utilized in the center bridge of UNet to enhance the depiction of context features between CT slices, thereby preventing the loss of structural information between slices. Experiments are conducted on a dataset of 430 esophageal tumor patients. The results show an 87.13% dice similarity coefficient, a 79.71% intersection over union and a 2.4758 mm Hausdorff distance, which demonstrates that our approach can improve contouring consistency and can be applied to clinical applications.
Collapse
Affiliation(s)
- Kai Xu
- Scholl of the Internet, Anhui university, Anhui, 230039, People's Republic of China
| | - Feixiang Zhang
- Scholl of the Internet, Anhui university, Anhui, 230039, People's Republic of China
| | - Yong Huang
- Department of Medical Oncology, The Second People's Hospital of Hefei, Hefei, 230011, People's Republic of China
| | - Xiaoyu Huang
- Department of Chinese Integrative Medicine Oncology, The First Affiliated Hospital of Anhui Medical University, Hefei, 230022, People's Republic of China
| |
Collapse
|
10
|
Li G, Xie J, Zhang L, Sun M, Li Z, Sun Y. MCAFNet: multiscale cross-layer attention fusion network for honeycomb lung lesion segmentation. Med Biol Eng Comput 2024; 62:1121-1137. [PMID: 38150110 DOI: 10.1007/s11517-023-02995-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/07/2023] [Indexed: 12/28/2023]
Abstract
Accurate segmentation of honeycomb lung lesions from lung CT images plays a crucial role in the diagnosis and treatment of various lung diseases. However, the availability of algorithms for automatic segmentation of honeycomb lung lesions remains limited. In this study, we propose a novel multi-scale cross-layer attention fusion network (MCAFNet) specifically designed for the segmentation of honeycomb lung lesions, taking into account their shape specificity and similarity to surrounding vascular shadows. The MCAFNet incorporates several key modules to enhance the segmentation performance. Firstly, a multiscale aggregation (MIA) module is introduced in the input part to preserve spatial information during downsampling. Secondly, a cross-layer attention fusion (CAF) module is proposed to capture multiscale features by integrating channel information and spatial information from different layers of the feature maps. Lastly, a bidirectional attention gate (BAG) module is constructed within the skip connection to enhance the model's ability to filter out background information and focus on the segmentation target. Experimental results demonstrate the effectiveness of the proposed MCAFNet. On the honeycomb lung segmentation dataset, the network achieves an Intersection over Union (IoU) of 0.895, mean IoU (mIoU) of 0.921, and mean Dice coefficient (mDice) of 0.949, outperforming existing medical image segmentation algorithms. Furthermore, experiments conducted on additional datasets confirm the generalizability and robustness of the proposed model. The contribution of this study lies in the development of the MCAFNet, which addresses the lack of automated segmentation algorithms for honeycomb lung lesions. The proposed network demonstrates superior performance in accurately segmenting honeycomb lung lesions, thereby facilitating the diagnosis and treatment of lung diseases. This work contributes to the existing literature by presenting a novel approach that effectively combines multi-scale features and attention mechanisms for lung lesion segmentation. The code is available at https://github.com/Oran9er/MCAFNet .
Collapse
Affiliation(s)
- Gang Li
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Jinjie Xie
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Ling Zhang
- Taiyuan University of Technology Software College, Taiyuan, China.
| | - Mengxia Sun
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Zhichao Li
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Yuanjin Sun
- Taiyuan University of Technology Software College, Taiyuan, China
| |
Collapse
|
11
|
Wang S, Wang X, Yin X, Lv X, Cai J. Differentiating HCC from ICC and prediction of ICC grade based on MRI deep-radiomics: Using lesions and their extended regions. Phys Med 2024; 120:103322. [PMID: 38452430 DOI: 10.1016/j.ejmp.2024.103322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 01/29/2024] [Accepted: 03/03/2024] [Indexed: 03/09/2024] Open
Abstract
PURPOSE This study aimed to evaluate the ability of MRI-based intratumoral and peritumoral radiomics features of liver tumors to differentiate between hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC) and to predict ICC differentiation. METHODS This study retrospectively collected 87 HCC patients and 75 ICC patients who were confirmed pathologically. The standard region of interest (ROI) of the lesion drawn by the radiologist manually shrank inward and expanded outward to form multiple ROI extended regions. A three-step feature selection method was used to select important radiomics features and convolution features from extended regions. The predictive performance of several machine learning classifiers on dominant feature sets was compared. The extended region performance was assessed by area under the curve (AUC), specificity, sensitivity, F1-score and accuracy. RESULTS The performance of the model is further improved by incorporating convolution features. Compared with the standard ROI, the extended region obtained better prediction performance, among which 6 mm extended region had the best prediction ability (Classification: AUC = 0.96, F1-score = 0.94, Accuracy: 0.94; Grading: AUC = 0.94, F1-score = 0.93, Accuracy = 0.89). CONCLUSION Larger extended region and fusion features can improve tumor predictive performance and have potential value in tumor radiology.
Collapse
Affiliation(s)
- Shuping Wang
- College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China; College of Electronic and Information Engineering, Hebei University, Baoding 071002, China
| | - Xuehu Wang
- College of Electronic and Information Engineering, Hebei University, Baoding 071002, China; Research Center of Machine Vision Engineering & Technology of Hebei Province, Baoding 071002, China; Key Laboratory of Digital Medical Engineering of Hebei Province, Baoding 071002, China.
| | - Xiaoping Yin
- Affiliated Hospital of Hebei University, Baoding 071000, China
| | - Xiaoyan Lv
- Fifth Medical Center of Chinese PLA General Hospital, Beijing 100039, China
| | - Jianming Cai
- Fifth Medical Center of Chinese PLA General Hospital, Beijing 100039, China.
| |
Collapse
|
12
|
Lin Z, Henson WH, Dowling L, Walsh J, Dall’Ara E, Guo L. Automatic segmentation of skeletal muscles from MR images using modified U-Net and a novel data augmentation approach. Front Bioeng Biotechnol 2024; 12:1355735. [PMID: 38456001 PMCID: PMC10919285 DOI: 10.3389/fbioe.2024.1355735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 02/05/2024] [Indexed: 03/09/2024] Open
Abstract
Rapid and accurate muscle segmentation is essential for the diagnosis and monitoring of many musculoskeletal diseases. As gold standard, manual annotation suffers from intensive labor and high inter-operator reproducibility errors. In this study, deep learning (DL) based automatic muscle segmentation from MR scans is investigated for post-menopausal women, who normally experience a decline in muscle volume. The performance of four Deep Learning (DL) models was evaluated: U-Net and UNet++ and two modified U-Net networks, which combined feature fusion and attention mechanisms (Feature-Fusion-UNet, FFU, and Attention-Feature-Fusion-UNet, AFFU). The models were tested for automatic segmentation of 16-lower limb muscles from MRI scans of two cohorts of post-menopausal women (11 subjects in PMW-1, 8 subjects in PMW-2; from two different studies so considered independent datasets) and 10 obese post-menopausal women (PMW-OB). Furthermore, a novel data augmentation approach is proposed to enlarge the training dataset. The results were assessed and compared by using the Dice similarity coefficient (DSC), relative volume error (RVE), and Hausdorff distance (HD). The best performance among all four DL models was achieved by AFFU (PMW-1: DSC 0.828 ± 0.079, 1-RVE 0.859 ± 0.122, HD 29.9 mm ± 26.5 mm; PMW-2: DSC 0.833 ± 0.065, 1-RVE 0.873 ± 0.105, HD 25.9 mm ± 27.9 mm; PMW-OB: DSC 0.862 ± 0.048, 1-RVE 0.919 ± 0.076, HD 34.8 mm ± 46.8 mm). Furthermore, the augmentation of data significantly improved the DSC scores of U-Net and AFFU for all 16 tested muscles (between 0.23% and 2.17% (DSC), 1.6%-1.93% (1-RVE), and 9.6%-19.8% (HD) improvement). These findings highlight the feasibility of utilizing DL models for automatic segmentation of muscles in post-menopausal women and indicate that the proposed augmentation method can enhance the performance of models trained on small datasets.
Collapse
Affiliation(s)
- Zhicheng Lin
- Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, United Kingdom
| | - William H. Henson
- Department of Mechanical Engineering, University of Sheffield, Sheffield, United Kingdom
| | - Lisa Dowling
- Faculty of Health, University of Sheffield, Sheffield, United Kingdom
| | - Jennifer Walsh
- Faculty of Health, University of Sheffield, Sheffield, United Kingdom
| | - Enrico Dall’Ara
- Department of Oncology and Metabolism, University of Sheffield, Sheffield, United Kingdom
- Insigneo, University of Sheffield, Sheffield, United Kingdom
| | - Lingzhong Guo
- Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, United Kingdom
- Insigneo, University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|
13
|
Wang Z, Zhu J, Fu S, Ye Y. Context fusion network with multi-scale-aware skip connection and twin-split attention for liver tumor segmentation. Med Biol Eng Comput 2023; 61:3167-3180. [PMID: 37470963 DOI: 10.1007/s11517-023-02876-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 06/20/2023] [Indexed: 07/21/2023]
Abstract
Manually annotating liver tumor contours is a time-consuming and labor-intensive task for clinicians. Therefore, automated segmentation is urgently needed in clinical diagnosis. However, automatic segmentation methods face certain challenges due to heterogeneity, fuzzy boundaries, and irregularity of tumor tissue. In this paper, a novel deep learning-based approach with multi-scale-aware (MSA) module and twin-split attention (TSA) module is proposed for tumor segmentation. The MSA module can bridge the semantic gap and reduce the loss of detailed information. The TSA module can recalibrate the channel response of the feature map. Eventually, we can count tumors based on the segmentation results from a 3D perspective for cancer grading. Extensive experiments conducted on the LiTS2017 dataset show the effectiveness of the proposed method by achieving a Dice index of 85.97% and a Jaccard index of 81.56% over the state of the art. In addition, the proposed method also achieved a Dice index of 83.67% and a Jaccard index of 80.11% in 3Dircadb dataset verification, which further reflects its robustness and generalization ability.
Collapse
Affiliation(s)
- Zhendong Wang
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China
| | - Jiehua Zhu
- Department of Mathematical Sciences, Georgia Southern University, Statesboro, GA, 30460, USA
| | - Shujun Fu
- School of Mathematics, Shandong University, Jinan, Shandong, 250100, China
| | - Yangbo Ye
- Department of Mathematics, The University of Iowa, Iowa City, IA, 52242, USA.
| |
Collapse
|
14
|
Fang J, Wang J, Li A, Yan Y, Liu H, Li J, Yang H, Hou Y, Yang X, Yang M, Liu J. Parameterized Gompertz-Guided Morphological AutoEncoder for Predicting Pulmonary Nodule Growth. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3602-3613. [PMID: 37471191 DOI: 10.1109/tmi.2023.3297209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/22/2023]
Abstract
The growth rate of pulmonary nodules is a critical clue to the cancerous diagnosis. It is essential to monitor their dynamic progressions during pulmonary nodule management. To facilitate the prosperity of research on nodule growth prediction, we organized and published a temporal dataset called NLSTt with consecutive computed tomography (CT) scans. Based on the self-built dataset, we develop a visual learner to predict the growth for the following CT scan qualitatively and further propose a model to predict the growth rate of pulmonary nodules quantitatively, so that better diagnosis can be achieved with the help of our predicted results. To this end, in this work, we propose a parameterized Gempertz-guided morphological autoencoder (GM-AE) to generate any future-time-span high-quality visual appearances of pulmonary nodules from the baseline CT scan. Specifically, we parameterize a popular mathematical model for tumor growth kinetics, Gompertz, to predict future masses and volumes of pulmonary nodules. Then, we exploit the expected growth rate on the mass and volume to guide decoders generating future shape and texture of pulmonary nodules. We introduce two branches in an autoencoder to encourage shape-aware and textural-aware representation learning and integrate the generated shape into the textural-aware branch to simulate the future morphology of pulmonary nodules. We conduct extensive experiments on the self-built NLSTt dataset to demonstrate the superiority of our GM-AE to its competitive counterparts. Experiment results also reveal the learnable Gompertz function enjoys promising descriptive power in accounting for inter-subject variability of the growth rate for pulmonary nodules. Besides, we evaluate our GM-AE model on an in-house dataset to validate its generalizability and practicality. We make its code publicly available along with the published NLSTt dataset.
Collapse
|
15
|
Hettihewa K, Kobchaisawat T, Tanpowpong N, Chalidabhongse TH. MANet: a multi-attention network for automatic liver tumor segmentation in computed tomography (CT) imaging. Sci Rep 2023; 13:20098. [PMID: 37973987 PMCID: PMC10654423 DOI: 10.1038/s41598-023-46580-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2023] [Accepted: 11/02/2023] [Indexed: 11/19/2023] Open
Abstract
Automatic liver tumor segmentation is a paramount important application for liver tumor diagnosis and treatment planning. However, it has become a highly challenging task due to the heterogeneity of the tumor shape and intensity variation. Automatic liver tumor segmentation is capable to establish the diagnostic standard to provide relevant radiological information to all levels of expertise. Recently, deep convolutional neural networks have demonstrated superiority in feature extraction and learning in medical image segmentation. However, multi-layer dense feature stacks make the model quite inconsistent in imitating visual attention and awareness of radiological expertise for tumor recognition and segmentation task. To bridge that visual attention capability, attention mechanisms have developed for better feature selection. In this paper, we propose a novel network named Multi Attention Network (MANet) as a fusion of attention mechanisms to learn highlighting important features while suppressing irrelevant features for the tumor segmentation task. The proposed deep learning network has followed U-Net as the basic architecture. Moreover, residual mechanism is implemented in the encoder. Convolutional block attention module has split into channel attention and spatial attention modules to implement in encoder and decoder of the proposed architecture. The attention mechanism in Attention U-Net is integrated to extract low-level features to combine with high-level ones. The developed deep learning architecture is trained and evaluated on the publicly available MICCAI 2017 Liver Tumor Segmentation dataset and 3DIRCADb dataset under various evaluation metrics. MANet demonstrated promising results compared to state-of-the-art methods with comparatively small parameter overhead.
Collapse
Affiliation(s)
- Kasun Hettihewa
- Perceptual Intelligent Computing Laboratory, Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, 10330, Thailand
| | | | - Natthaporn Tanpowpong
- Department of Radiology, Faculty of Medicine, Chulalongkorn University, Bangkok, 10330, Thailand
| | - Thanarat H Chalidabhongse
- Perceptual Intelligent Computing Laboratory, Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, 10330, Thailand.
- Applied Digital Technology in Medicine (ATM) Research Group, Faculty of Engineering, Chulalongkorn University, Bangkok, 10330, Thailand.
| |
Collapse
|
16
|
Wang J, Peng Y, Jing S, Han L, Li T, Luo J. A deep-learning approach for segmentation of liver tumors in magnetic resonance imaging using UNet+. BMC Cancer 2023; 23:1060. [PMID: 37923988 PMCID: PMC10623778 DOI: 10.1186/s12885-023-11432-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Accepted: 09/21/2023] [Indexed: 11/06/2023] Open
Abstract
OBJECTIVE Radiomic and deep learning studies based on magnetic resonance imaging (MRI) of liver tumor are gradually increasing. Manual segmentation of normal hepatic tissue and tumor exhibits limitations. METHODS 105 patients diagnosed with hepatocellular carcinoma were retrospectively studied between Jan 2015 and Dec 2020. The patients were divided into three sets: training (n = 83), validation (n = 11), and internal testing (n = 11). Additionally, 9 cases were included from the Cancer Imaging Archive as the external test set. Using the arterial phase and T2WI sequences, expert radiologists manually delineated all images. Using deep learning, liver tumors and liver segments were automatically segmented. A preliminary liver segmentation was performed using the UNet + + network, and the segmented liver mask was re-input as the input end into the UNet + + network to segment liver tumors. The false positivity rate was reduced using a threshold value in the liver tumor segmentation. To evaluate the segmentation results, we calculated the Dice similarity coefficient (DSC), average false positivity rate (AFPR), and delineation time. RESULTS The average DSC of the liver in the validation and internal testing sets was 0.91 and 0.92, respectively. In the validation set, manual and automatic delineation took 182.9 and 2.2 s, respectively. On an average, manual and automatic delineation took 169.8 and 1.7 s, respectively. The average DSC of liver tumors was 0.612 and 0.687 in the validation and internal testing sets, respectively. The average time for manual and automatic delineation and AFPR in the internal testing set were 47.4 s, 2.9 s, and 1.4, respectively, and those in the external test set were 29.5 s, 4.2 s, and 1.6, respectively. CONCLUSION UNet + + can automatically segment normal hepatic tissue and liver tumors based on MR images. It provides a methodological basis for the automated segmentation of liver tumors, improves the delineation efficiency, and meets the requirement of extraction set analysis of further radiomics and deep learning.
Collapse
Affiliation(s)
- Jing Wang
- Department of General medicine, The First Medical Center Department of Chinese PLA General Hospital, Peking, 100039, China
| | - Yanyang Peng
- Department of Radiology, First Medical Center of General Hospital of People's Liberation Army, Peking, China
| | - Shi Jing
- Department of Oncology, Huaihe Hospital, Henan University, Kaifeng, 475000, China
| | - Lujun Han
- Department of Radiology, State Key Laboratory of Oncology in South China, Collaborative Innovation Cancer for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, 510030, China.
- Translational Medical Center of Huaihe Hospital, Henan University, 115 West Gate Street, Kaifeng, 475000, China.
| | - Tian Li
- School of Basic Medicine, Fourth Military Medical University, Xi'an, 710032, China.
- Translational Medical Center of Huaihe Hospital, Henan University, 115 West Gate Street, Kaifeng, 475000, China.
| | - Junpeng Luo
- Translational Medical Center of Huaihe Hospital, Henan University, 115 West Gate Street, Kaifeng, 475000, China.
- Academy for Advanced Interdisciplinary Studies, Henan University, Zhengzhou, 450046, China.
| |
Collapse
|
17
|
Saumiya S, Franklin SW. Residual Deformable Split Channel and Spatial U-Net for Automated Liver and Liver Tumour Segmentation. J Digit Imaging 2023; 36:2164-2178. [PMID: 37464213 PMCID: PMC10501969 DOI: 10.1007/s10278-023-00874-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 06/20/2023] [Accepted: 06/21/2023] [Indexed: 07/20/2023] Open
Abstract
Accurate segmentation of the liver and liver tumour (LT) is challenging due to its hazy boundaries and large shape variability. Although using U-Net for liver and LT segmentation achieves better results than manual segmentation, it loses spatial and channel features during segmentation, leading to inaccurate liver and LT segmentation. A residual deformable split depth-wise separable U-Net (RDSDSU-Net) is proposed to increase the accuracy of liver and LT segmentation. The residual deformable convolution layer (DCL) with deformable pooling (DP) is used in the encoder as an attention mechanism to adaptively extract liver and LT shape and position characteristics. Afterward, a convolutional spatial and channel features split graph network (CSCFSG-Net) is introduced in the middle processing layer to improve the expression capability of the liver and LT features by capturing spatial and channel features separately and to extract global contextual liver and LT information from spatial and channel features. Sub-pixel convolutions (SPC) are used in the decoder section to prevent the segmentation results from having a chequerboard artefact effect. Also, the residual deformable encoder features are combined with the decoder through summation to avoid increasing the number of feature maps (FM). Finally, the efficiency of the RDSDSU-Net is evaluated on the 3DIRCADb and LiTS datasets. The DICE score of the proposed RDSDSU-Net achieved 98.21% for liver segmentation and 93.25% for LT segmentation on 3DIRCADb. The experimental outcomes illustrate that the proposed RDSDSU-Net model achieved better segmentation results than the existing techniques.
Collapse
Affiliation(s)
- S Saumiya
- Department of ECE, Bethlahem Institute of Engineering, Karungal, Tamil Nadu India
| | - S Wilfred Franklin
- Department of ECE, CSI Institute of Technology, Thovalai, Tamil Nadu India
| |
Collapse
|
18
|
Li W, Jia M, Yang C, Lin Z, Yu Y, Zhang W. SPA-UNet: A liver tumor segmentation network based on fused multi-scale features. Open Life Sci 2023; 18:20220685. [PMID: 37724113 PMCID: PMC10505346 DOI: 10.1515/biol-2022-0685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 06/26/2023] [Accepted: 07/24/2023] [Indexed: 09/20/2023] Open
Abstract
Liver tumor segmentation is a critical part in the diagnosis and treatment of liver cancer. While U-shaped convolutional neural networks (UNets) have made significant strides in medical image segmentation, challenges remain in accurately segmenting tumor boundaries and detecting small tumors, resulting in low segmentation accuracy. To improve the segmentation accuracy of liver tumors, this work proposes space pyramid attention (SPA)-UNet, a novel image segmentation network with an encoder-decoder architecture. SPA-UNet consists of four modules: (1) Spatial pyramid convolution block (SPCB), extracting multi-scale features by fusing three sets of dilated convolutions with different rates. (2) Spatial pyramid pooling block (SPPB), performing downsampling to reduce image size. (3) Upsample module, integrating dense positional and semantic information. (4) Residual attention block (RA-Block), enabling precise tumor localization. The encoder incorporates 5 SPCBs and 4 SPPBs to capture contextual information. The decoder consists of the Upsample module and RA-Block, and finally a segmentation head outputs segmented images of liver and liver tumor. Experiments using the liver tumor segmentation dataset demonstrate that SPA-UNet surpasses the traditional UNet model, achieving a 1.0 and 2.0% improvement in intersection over union indicators for liver and tumors, respectively, along with increased recall rates by 1.2 and 1.8%. These advancements provide a dependable foundation for liver cancer diagnosis and treatment.
Collapse
Affiliation(s)
- Weikun Li
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Maoning Jia
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Chen Yang
- School of Business, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Zhenyuan Lin
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Yuekang Yu
- School of Information and Communication, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| | - Wenhui Zhang
- School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541000, China
| |
Collapse
|
19
|
Chen Y, Zheng C, Zhang W, Lin H, Chen W, Zhang G, Xu G, Wu F. MS-FANet: Multi-scale feature attention network for liver tumor segmentation. Comput Biol Med 2023; 163:107208. [PMID: 37421737 DOI: 10.1016/j.compbiomed.2023.107208] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 06/07/2023] [Accepted: 06/25/2023] [Indexed: 07/10/2023]
Abstract
Accurate segmentation of liver tumors is a prerequisite for early diagnosis of liver cancer. Segmentation networks extract features continuously at the same scale, which cannot adapt to the variation of liver tumor volume in computed tomography (CT). Hence, a multi-scale feature attention network (MS-FANet) for liver tumor segmentation is proposed in this paper. The novel residual attention (RA) block and multi-scale atrous downsampling (MAD) are introduced in the encoder of MS-FANet to sufficiently learn variable tumor features and extract tumor features at different scales simultaneously. The dual-path feature (DF) filter and dense upsampling (DU) are introduced in the feature reduction process to reduce effective features for the accurate segmentation of liver tumors. On the public LiTS dataset and 3DIRCADb dataset, MS-FANet achieved 74.2% and 78.0% of average Dice, respectively, outperforming most state-of-the-art networks, this strongly proves the excellent liver tumor segmentation performance and the ability to learn features at different scales.
Collapse
Affiliation(s)
- Ying Chen
- School of Software, Nanchang Hangkong University, Nanchang, 330063, PR China
| | - Cheng Zheng
- School of Software, Nanchang Hangkong University, Nanchang, 330063, PR China.
| | - Wei Zhang
- School of Software, Nanchang Hangkong University, Nanchang, 330063, PR China
| | - Hongping Lin
- School of Software, Nanchang Hangkong University, Nanchang, 330063, PR China
| | - Wang Chen
- School of Software, Nanchang Hangkong University, Nanchang, 330063, PR China
| | - Guimei Zhang
- Institute of Computer Vision, Nanchang Hangkong University, Nanchang, 330063, PR China
| | - Guohui Xu
- Department of Hepatobiliary Surgery, Jiangxi Cancer Hospital, Nanchang, 330029, PR China.
| | - Fang Wu
- Department of Gastroenterology, First Affiliated Hospital of Wenzhou Medical University, Wenzhou, 325035, PR China.
| |
Collapse
|
20
|
Chen G, Li Z, Wang J, Wang J, Du S, Zhou J, Shi J, Zhou Y. An improved 3D KiU-Net for segmentation of liver tumor. Comput Biol Med 2023; 160:107006. [PMID: 37159962 DOI: 10.1016/j.compbiomed.2023.107006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 04/08/2023] [Accepted: 05/03/2023] [Indexed: 05/11/2023]
Abstract
It is a challenging task to accurately segment liver tumors from Computed Tomography (CT) images. The widely used U-Net and its variants generally suffer from the issue to accurately segment the detailed edges of small tumors, because the progressive down sampling operations in the encoder module will gradually increase the receptive fields. These enlarged receptive filed have limited ability to learn the information about tiny structures. KiU-Net is a newly proposed dual-branch model that can effectively perform image segmentation for small targets. However, the 3D version of KiU-Net has high computational complexity, which limits its application. In this work, an improved 3D KiU-Net (named TKiU-NeXt) is proposed for liver tumor segmentation from CT images. In TKiU-NeXt, a Transformer-based Kite-Net (TK-Net) branch is proposed to build the over-complete architecture to learn more detailed features for small structures, and an extended 3D version of UNeXt is developed to replace the original U-Net branch, which can effectively reduce computational complexity but still with superior segmentation performance. Moreover, a Mutual Guided Fusion Block (MGFB) is designed to effectively learn more features from two branches and then fuse the complementary features for image segmentation. The experimental results on two public CT datasets and a private dataset demonstrate that the proposed TKiU-NeXt outperforms all the compared algorithms, and it also has less computational complexity. It suggests the effectiveness and efficiency of TKiU-NeXt.
Collapse
Affiliation(s)
- Guodong Chen
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China; Shanghai Institute for Advanced Communication and Data Science, Shanghai University, China
| | - Zheng Li
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China; Shanghai Institute for Advanced Communication and Data Science, Shanghai University, China
| | - Jian Wang
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China; Shanghai Institute for Advanced Communication and Data Science, Shanghai University, China
| | - Jun Wang
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China; Shanghai Institute for Advanced Communication and Data Science, Shanghai University, China
| | - Shisuo Du
- Department of Radiation Oncology, Zhongshan Hospital Fudan University Shanghai, China
| | - Jinghao Zhou
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - Jun Shi
- Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, China; Shanghai Institute for Advanced Communication and Data Science, Shanghai University, China.
| | - Yongkang Zhou
- Department of Radiation Oncology, Zhongshan Hospital Fudan University Shanghai, China.
| |
Collapse
|
21
|
Jiang L, Ou J, Liu R, Zou Y, Xie T, Xiao H, Bai T. RMAU-Net: Residual Multi-Scale Attention U-Net For liver and tumor segmentation in CT images. Comput Biol Med 2023; 158:106838. [PMID: 37030263 DOI: 10.1016/j.compbiomed.2023.106838] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 03/08/2023] [Accepted: 03/26/2023] [Indexed: 03/30/2023]
Abstract
Liver cancer is one of the leading causes of cancer-related deaths worldwide. Automatic liver and tumor segmentation are of great value in clinical practice as they can reduce surgeons' workload and increase the probability of success in surgery. Liver and tumor segmentation is a challenging task because of the different sizes, shapes, blurred boundaries of livers and lesions, and low-intensity contrast between organs within patients. To address the problem of fuzzy livers and small tumors, we propose a novel Residual Multi-scale Attention U-Net (RMAU-Net) for liver and tumor segmentation by introducing two modules, i.e., Res-SE-Block and MAB. The Res-SE-Block can mitigate the problem of gradient disappearance by residual connection and enhance the quality of representations by explicitly modeling the interdependencies and feature recalibration between the channels of features. The MAB can exploit rich multi-scale feature information and capture inter-channel and inter-spatial relationships of features simultaneously. In addition, a hybrid loss function, that combines focal loss and dice loss, is designed to improve segmentation accuracy and speed up convergence. We evaluated the proposed method on two publicly available datasets, i.e., LiTS and 3D-IRCADb. Our proposed method achieved better performance than the other state-of-the-art methods, with dice scores of 0.9552 and 0.9697 for LiTS and 3D-IRCABb liver segmentation, and dice scores of 0.7616 and 0.8307 for LiTS and 3D-IRCABb liver tumor segmentation.
Collapse
|
22
|
Liu J, Yan Z, Zhou C, Shao L, Han Y, Song Y. mfeeU-Net: A multi-scale feature extraction and enhancement U-Net for automatic liver segmentation from CT Images. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:7784-7801. [PMID: 37161172 DOI: 10.3934/mbe.2023336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Medical image segmentation of the liver is an important prerequisite for clinical diagnosis and evaluation of liver cancer. For automatic liver segmentation from Computed Tomography (CT) images, we proposed a Multi-scale Feature Extraction and Enhancement U-Net (mfeeU-Net), incorporating Res2Net blocks, Squeeze-and-Excitation (SE) blocks, and Edge Attention (EA) blocks. The Res2Net blocks which are conducive to extracting multi-scale features of the liver were used as the backbone of the encoder, while the SE blocks were also added to the encoder to enhance channel information. The EA blocks were introduced to skip connections between the encoder and the decoder, to facilitate the detection of blurred liver edges where the intensities of nearby organs are close to the liver. The proposed mfeeU-Net was trained and evaluated using a publicly available CT dataset of LiTS2017. The average dice similarity coefficient, intersection-over-union ratio, and sensitivity of the mfeeU-Net for liver segmentation were 95.32%, 91.67%, and 95.53%, respectively, and all these metrics were better than those of U-Net, Res-U-Net, and Attention U-Net. The experimental results demonstrate that the mfeeU-Net can compete with and even outperform recently proposed convolutional neural networks and effectively overcome challenges, such as discontinuous liver regions and fuzzy liver boundaries.
Collapse
Affiliation(s)
- Jun Liu
- Department of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
| | - Zhenhua Yan
- Department of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
| | - Chaochao Zhou
- Department of Radiology, Feinberg School of Medicine, Northwestern University, Chicago 60611, Illinois, U.S
| | - Liren Shao
- Department of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
| | - Yuanyuan Han
- Department of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
| | - Yusheng Song
- Interventional Radiology, The People's Hospital of Ganzhou, Ganzhou 34100, Jiangxi, China
| |
Collapse
|
23
|
Kushnure DT, Tyagi S, Talbar SN. LiM-Net: Lightweight multi-level multiscale network with deep residual learning for automatic liver segmentation in CT images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104305] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
24
|
Valeri F, Bartolucci M, Cantoni E, Carpi R, Cisbani E, Cupparo I, Doria S, Gori C, Grigioni M, Lasagni L, Marconi A, Mazzoni LN, Miele V, Pradella S, Risaliti G, Sanguineti V, Sona D, Vannucchi L, Taddeucci A. UNet and MobileNet CNN-based model observers for CT protocol optimization: comparative performance evaluation by means of phantom CT images. J Med Imaging (Bellingham) 2023; 10:S11904. [PMID: 36895439 PMCID: PMC9989681 DOI: 10.1117/1.jmi.10.s1.s11904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Accepted: 02/09/2023] [Indexed: 03/09/2023] Open
Abstract
Purpose The aim of this work is the development and characterization of a model observer (MO) based on convolutional neural networks (CNNs), trained to mimic human observers in image evaluation in terms of detection and localization of low-contrast objects in CT scans acquired on a reference phantom. The final goal is automatic image quality evaluation and CT protocol optimization to fulfill the ALARA principle. Approach Preliminary work was carried out to collect localization confidence ratings of human observers for signal presence/absence from a dataset of 30,000 CT images acquired on a PolyMethyl MethAcrylate phantom containing inserts filled with iodinated contrast media at different concentrations. The collected data were used to generate the labels for the training of the artificial neural networks. We developed and compared two CNN architectures based respectively on Unet and MobileNetV2, specifically adapted to achieve the double tasks of classification and localization. The CNN evaluation was performed by computing the area under localization-ROC curve (LAUC) and accuracy metrics on the test dataset. Results The mean of absolute percentage error between the LAUC of the human observer and MO was found to be below 5% for the most significative test data subsets. An elevated inter-rater agreement was achieved in terms of S-statistics and other common statistical indices. Conclusions Very good agreement was measured between the human observer and MO, as well as between the performance of the two algorithms. Therefore, this work is highly supportive of the feasibility of employing CNN-MO combined with a specifically designed phantom for CT protocol optimization programs.
Collapse
Affiliation(s)
- Federico Valeri
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
- Università degli Studi di Firenze, Scuola di Scienze della Salute Umana, Florence, Italy
| | - Maurizio Bartolucci
- Ospedale S. Stefano, Azienda USL Toscana Centro, SOC Radiodiagnostica, Prato, Italy
| | - Elena Cantoni
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
| | - Roberto Carpi
- Ospedale Santa Maria Nuova, Azienda USL Toscana Centro, SOC Radiologia, Florence, Italy
| | - Evaristo Cisbani
- Istituto Superiore di Sanità, Centro Nazionale Tecnologie Innvative in Sanità Pubblica, Rome, Italy
| | - Ilaria Cupparo
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
- Università degli Studi di Firenze, Scuola di Scienze della Salute Umana, Florence, Italy
| | - Sandra Doria
- Istituto di Chimica dei Composti OrganoMetallici, Consiglio Nazionale delle Ricerche, Florence, Italy
- Università degli Studi di Firenze, European Laboratory for Nonlinear Spectroscopy, Florence, Italy
| | - Cesare Gori
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
| | - Mauro Grigioni
- Istituto Superiore di Sanità, Centro Nazionale Tecnologie Innvative in Sanità Pubblica, Rome, Italy
| | - Lorenzo Lasagni
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
- Università degli Studi di Firenze, Scuola di Scienze della Salute Umana, Florence, Italy
| | - Alessandro Marconi
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
| | - Lorenzo Nicola Mazzoni
- Ospedale San Jacopo, Azienda USL Toscana Centro, UO Fisica Sanitaria Prato e Pistoia, Pistoia, Italy
| | - Vittorio Miele
- Azienda Ospedaliero-Universitaria Careggi, SOD Radiodiagnostica di Emergenza-Urgenza, Florence, Italy
| | - Silvia Pradella
- Azienda Ospedaliero-Universitaria Careggi, SOD Radiodiagnostica di Emergenza-Urgenza, Florence, Italy
| | - Guido Risaliti
- Università degli Studi di Firenze, Dipartimento di Fisica e Astronomia, Florence, Italy
| | - Valentina Sanguineti
- Istituto Italiano di Tecnologia, Pattern Analysis & Computer Vision, Genoa, Italy
| | - Diego Sona
- Fondazione Bruno Kessler, Data Science for Health Unit, Trento, Italy
| | - Letizia Vannucchi
- Ospedale S. Jacopo, AUSL Toscana Centro, SOC Radiodiagnostica, Pistoia, Italy
| | - Adriana Taddeucci
- Azienda Ospedaliero-Universitaria Careggi, UO Fisica Sanitaria, Florence, Italy
- Istituto Nazionale di Fisica Nucleare - Sezione di Firenze, Sesto Fiorentino, Italy
| |
Collapse
|
25
|
Wang X, Wang S, Zhang Z, Yin X, Wang T, Li N. CPAD-Net: Contextual parallel attention and dilated network for liver tumor segmentation. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
26
|
Zhao L, Jia C, Ma J, Shao Y, Liu Z, Yuan H. Medical image segmentation based on self-supervised hybrid fusion network. Front Oncol 2023; 13:1109786. [PMID: 37124508 PMCID: PMC10141651 DOI: 10.3389/fonc.2023.1109786] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 03/28/2023] [Indexed: 05/02/2023] Open
Abstract
Automatic segmentation of medical images has been a hot research topic in the field of deep learning in recent years, and achieving accurate segmentation of medical images is conducive to breakthroughs in disease diagnosis, monitoring, and treatment. In medicine, MRI imaging technology is often used to image brain tumors, and further judgment of the tumor area needs to be combined with expert analysis. If the diagnosis can be carried out by computer-aided methods, the efficiency and accuracy will be effectively improved. Therefore, this paper completes the task of brain tumor segmentation by building a self-supervised deep learning network. Specifically, it designs a multi-modal encoder-decoder network based on the extension of the residual network. Aiming at the problem of multi-modal feature extraction, the network introduces a multi-modal hybrid fusion module to fully extract the unique features of each modality and reduce the complexity of the whole framework. In addition, to better learn multi-modal complementary features and improve the robustness of the model, a pretext task to complete the masked area is set, to realize the self-supervised learning of the network. Thus, it can effectively improve the encoder's ability to extract multi-modal features and enhance the noise immunity. Experimental results present that our method is superior to the compared methods on the tested datasets.
Collapse
Affiliation(s)
- Liang Zhao
- School of Software Technology, Dalian University of Technology, Dalian, China
| | - Chaoran Jia
- School of Software Technology, Dalian University of Technology, Dalian, China
| | - Jiajun Ma
- School of Software Technology, Dalian University of Technology, Dalian, China
| | - Yu Shao
- School of Software Technology, Dalian University of Technology, Dalian, China
| | - Zhuo Liu
- The First Affiliated Hospital of Dalian Medical University, Dalian, China
- *Correspondence: Zhuo Liu, ; Hong Yuan,
| | - Hong Yuan
- The Affiliated Central Hospital, Dalian University of Technology, Dalian, China
- *Correspondence: Zhuo Liu, ; Hong Yuan,
| |
Collapse
|
27
|
Li J, Liu K, Hu Y, Zhang H, Heidari AA, Chen H, Zhang W, Algarni AD, Elmannai H. Eres-UNet++: Liver CT image segmentation based on high-efficiency channel attention and Res-UNet+. Comput Biol Med 2022; 158:106501. [PMID: 36635120 DOI: 10.1016/j.compbiomed.2022.106501] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 11/17/2022] [Accepted: 11/18/2022] [Indexed: 01/11/2023]
Abstract
Computerized tomography (CT) is of great significance for the localization and diagnosis of liver cancer. Many scholars have recently applied deep learning methods to segment CT images of liver and liver tumors. Unlike natural images, medical image segmentation is usually more challenging due to its nature. Aiming at the problem of blurry boundaries and complex gradients of liver tumor images, a deep supervision network based on the combination of high-efficiency channel attention and Res-UNet++ (ECA residual UNet++) is proposed for liver CT image segmentation, enabling fully automated end-to-end segmentation of the network. In this paper, the UNet++ structure is selected as the baseline. The residual block feature encoder based on context awareness enhances the feature extraction ability and solves the problem of deep network degradation. The introduction of an efficient attention module combines the depth of the feature map with spatial information to alleviate the uneven sample distribution impact; Use DiceLoss to replace the cross-entropy loss function to optimize network parameters. The liver and liver tumor segmentation accuracy on the LITS dataset was 95.8% and 89.3%, respectively. The results show that compared with other algorithms, the method proposed in this paper achieves a good segmentation performance, which has specific reference significance for computer-assisted diagnosis and treatment to attain fine segmentation of liver and liver tumors.
Collapse
Affiliation(s)
- Jian Li
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Kongyu Liu
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Yating Hu
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Hongchen Zhang
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Ali Asghar Heidari
- Institute of Big Data and Information Technology, Wenzhou University, Wenzhou 325000, China.
| | - Huiling Chen
- Institute of Big Data and Information Technology, Wenzhou University, Wenzhou 325000, China.
| | - Weijiang Zhang
- College of Information Technology, Jilin Agricultural University, Changchun, 130118, China.
| | - Abeer D Algarni
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia.
| | - Hela Elmannai
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia.
| |
Collapse
|
28
|
Parra-Mora E, da Silva Cruz LA. LOCTseg: A lightweight fully convolutional network for end-to-end optical coherence tomography segmentation. Comput Biol Med 2022; 150:106174. [PMID: 36252364 DOI: 10.1016/j.compbiomed.2022.106174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2022] [Revised: 08/31/2022] [Accepted: 10/01/2022] [Indexed: 11/03/2022]
Abstract
This article presents a novel end-to-end automatic solution for semantic segmentation of optical coherence tomography (OCT) images. OCT is a non-invasive imaging technology widely used in clinical practice due to its ability to acquire high-resolution cross-sectional images of the ocular fundus. Due to the large variability of the retinal structures, OCT segmentation is usually carried out manually and requires expert knowledge. This study introduces a novel fully convolutional network (FCN) architecture designated by LOCTSeg, for end-to-end automatic segmentation of diagnostic markers in OCT b-scans. LOCTSeg is a lightweight deep FCN optimized for balancing performance and efficiency. Unlike state-of-the-art FCNs used in image segmentation, LOCTSeg achieves competitive inference speed without sacrificing segmentation accuracy. The proposed LOCTSeg is evaluated on two publicly available benchmarking datasets: (1) annotated retinal OCT image database (AROI) comprising 1136 images, and (2) healthy controls and multiple sclerosis lesions (HCMS) consisting of 1715 images. Moreover, we evaluated the proposed LOCTSeg with a private dataset of 250 OCT b-scans acquired from epiretinal membrane (ERM) and healthy patients. Results of the evaluation demonstrate empirically the effectiveness of the proposed algorithm, which improves the state-of-the-art Dice score from 69% to 73% and from 91% to 92% on AROI and HCMS datasets, respectively. Furthermore, LOCTSeg outperforms comparable lightweight FCNs' Dice score by margins between 4% and 15% on ERM segmentation.
Collapse
Affiliation(s)
- Esther Parra-Mora
- Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, 3030-290, Portugal; Instituto de Telecomunicações, Coimbra, 3030-290, Portugal.
| | - Luís A da Silva Cruz
- Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, 3030-290, Portugal; Instituto de Telecomunicações, Coimbra, 3030-290, Portugal.
| |
Collapse
|
29
|
Zhang H, Chen H, Qin J, Wang B, Ma G, Wang P, Zhong D, Liu J. MC-ViT: Multi-path cross-scale vision transformer for thymoma histopathology whole slide image typing. Front Oncol 2022; 12:925903. [PMID: 36387248 PMCID: PMC9659861 DOI: 10.3389/fonc.2022.925903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Accepted: 10/11/2022] [Indexed: 08/14/2023] Open
Abstract
OBJECTIVES Accurate histological typing plays an important role in diagnosing thymoma or thymic carcinoma (TC) and predicting the corresponding prognosis. In this paper, we develop and validate a deep learning-based thymoma typing method for hematoxylin & eosin (H&E)-stained whole slide images (WSIs), which provides useful histopathology information from patients to assist doctors for better diagnosing thymoma or TC. METHODS We propose a multi-path cross-scale vision transformer (MC-ViT), which first uses the cross attentive scale-aware transformer (CAST) to classify the pathological information related to thymoma, and then uses such pathological information priors to assist the WSIs transformer (WT) for thymoma typing. To make full use of the multi-scale (10×, 20×, and 40×) information inherent in a WSI, CAST not only employs parallel multi-path to capture different receptive field features from multi-scale WSI inputs, but also introduces the cross-correlation attention module (CAM) to aggregate multi-scale features to achieve cross-scale spatial information complementarity. After that, WT can effectively convert full-scale WSIs into 1D feature matrices with pathological information labels to improve the efficiency and accuracy of thymoma typing. RESULTS We construct a large-scale thymoma histopathology WSI (THW) dataset and annotate corresponding pathological information and thymoma typing labels. The proposed MC-ViT achieves the Top-1 accuracy of 0.939 and 0.951 in pathological information classification and thymoma typing, respectively. Moreover, the quantitative and statistical experiments on the THW dataset also demonstrate that our pipeline performs favorably against the existing classical convolutional neural networks, vision transformers, and deep learning-based medical image classification methods. CONCLUSION This paper demonstrates that comprehensively utilizing the pathological information contained in multi-scale WSIs is feasible for thymoma typing and achieves clinically acceptable performance. Specifically, the proposed MC-ViT can well predict pathological information classes as well as thymoma types, which show the application potential to the diagnosis of thymoma and TC and may assist doctors in improving diagnosis efficiency and accuracy.
Collapse
Affiliation(s)
- Huaqi Zhang
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
| | - Huang Chen
- Department of Pathology, China-Japan Friendship Hospital, Beijing, China
| | - Jin Qin
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
| | - Bei Wang
- Department of Pathology, China-Japan Friendship Hospital, Beijing, China
| | - Guolin Ma
- Department of Radiology, China-Japan Friendship Hospital, Beijing, China
| | - Pengyu Wang
- School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China
| | - Dingrong Zhong
- Department of Pathology, China-Japan Friendship Hospital, Beijing, China
| | - Jie Liu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
| |
Collapse
|
30
|
Wu L, Zhuang J, Chen W, Tang Y, Hou C, Li C, Zhong Z, Luo S. Data augmentation based on multiple oversampling fusion for medical image segmentation. PLoS One 2022; 17:e0274522. [PMID: 36256637 PMCID: PMC9578635 DOI: 10.1371/journal.pone.0274522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Accepted: 08/28/2022] [Indexed: 11/18/2022] Open
Abstract
A high-performance medical image segmentation model based on deep learning depends on the availability of large amounts of annotated training data. However, it is not trivial to obtain sufficient annotated medical images. Generally, the small size of most tissue lesions, e.g., pulmonary nodules and liver tumours, could worsen the class imbalance problem in medical image segmentation. In this study, we propose a multidimensional data augmentation method combining affine transform and random oversampling. The training data is first expanded by affine transformation combined with random oversampling to improve the prior data distribution of small objects and the diversity of samples. Secondly, class weight balancing is used to avoid having biased networks since the number of background pixels is much higher than the lesion pixels. The class imbalance problem is solved by utilizing weighted cross-entropy loss function during the training of the CNN model. The LUNA16 and LiTS17 datasets were introduced to evaluate the performance of our works, where four deep neural network models, Mask-RCNN, U-Net, SegNet and DeepLabv3+, were adopted for small tissue lesion segmentation in CT images. In addition, the small tissue segmentation performance of the four different deep learning architectures on both datasets could be greatly improved by incorporating the data augmentation strategy. The best pixelwise segmentation performance for both pulmonary nodules and liver tumours was obtained by the Mask-RCNN model, with DSC values of 0.829 and 0.879, respectively, which were similar to those of state-of-the-art methods.
Collapse
Affiliation(s)
- Liangsheng Wu
- Academy of Interdisciplinary Studies, Guangdong Polytechnic Normal University, Guangzhou, China
- Academy of Contemporary Agriculture Engineering Innovations, Zhongkai University of Agriculture and Engineering, Guangzhou, China
- Institute of Intelligent Manufacturing, Guangdong Academy of Sciences, Guangzhou, China
| | - Jiajun Zhuang
- Academy of Contemporary Agriculture Engineering Innovations, Zhongkai University of Agriculture and Engineering, Guangzhou, China
| | - Weizhao Chen
- Academy of Interdisciplinary Studies, Guangdong Polytechnic Normal University, Guangzhou, China
| | - Yu Tang
- Academy of Interdisciplinary Studies, Guangdong Polytechnic Normal University, Guangzhou, China
| | - Chaojun Hou
- Academy of Contemporary Agriculture Engineering Innovations, Zhongkai University of Agriculture and Engineering, Guangzhou, China
| | - Chentong Li
- Institute of Intelligent Manufacturing, Guangdong Academy of Sciences, Guangzhou, China
| | - Zhenyu Zhong
- Institute of Intelligent Manufacturing, Guangdong Academy of Sciences, Guangzhou, China
| | - Shaoming Luo
- Academy of Interdisciplinary Studies, Guangdong Polytechnic Normal University, Guangzhou, China
| |
Collapse
|
31
|
Liao RQ, Li AW, Yan HH, Lin JT, Liu SY, Wang JW, Fang JS, Liu HB, Hou YH, Song C, Yang HF, Li B, Jiang BY, Dong S, Nie Q, Zhong WZ, Wu YL, Yang XN. Deep learning-based growth prediction for sub-solid pulmonary nodules on CT images. Front Oncol 2022; 12:1002953. [PMID: 36313666 PMCID: PMC9597322 DOI: 10.3389/fonc.2022.1002953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 09/20/2022] [Indexed: 11/13/2022] Open
Abstract
Background Estimating the growth of pulmonary sub-solid nodules (SSNs) is crucial to the successful management of them during follow-up periods. The purpose of this study is to (1) investigate the measurement sensitivity of diameter, volume, and mass of SSNs for identifying growth and (2) seek to establish a deep learning-based model to predict the growth of SSNs. Methods A total of 2,523 patients underwent at least 2-year examination records retrospectively collected with sub-solid nodules. A total of 2,358 patients with 3,120 SSNs from the NLST dataset were randomly divided into training and validation sets. Patients from the Yibicom Health Management Center and Guangdong Provincial People’s Hospital were collected as an external test set (165 patients with 213 SSN). Trained models based on LUNA16 and Lndb19 datasets were employed to automatically obtain the diameter, volume, and mass of SSNs. Then, the increase rate in measurements between cancer and non-cancer groups was studied to evaluate the most appropriate way to identify growth-associated lung cancer. Further, according to the selected measurement, all SSNs were classified into two groups: growth and non-growth. Based on the data, the deep learning-based model (SiamModel) and radiomics model were developed and verified. Results The double time of diameter, volume, and mass were 711 vs. 963 days (P = 0.20), 552 vs. 621 days (P = 0.04) and 488 vs. 623 days (P< 0.001) in the cancer and non-cancer groups, respectively. Our proposed SiamModel performed better than the radiomics model in both the NLST validation set and external test set, with an AUC of 0.858 (95% CI 0.786–0.921) and 0.760 (95% CI 0.646–0.857) in the validation set and 0.862 (95% CI 0.789–0.927) and 0.681 (95% CI 0.506–0.841) in the external test set, respectively. Furthermore, our SiamModel could use the data from first-time CT to predict the growth of SSNs, with an AUC of 0.855 (95% CI 0.793–0.908) in the NLST validation set and 0.821 (95% CI 0.725–0.904) in the external test set. Conclusion Mass increase rate can reflect more sensitively the growth of SSNs associated with lung cancer than diameter and volume increase rates. A deep learning-based model has a great potential to predict the growth of SSNs.
Collapse
Affiliation(s)
- Ri-qiang Liao
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - An-wei Li
- Guangzhou Shiyuan Electronics Co., Ltd, Guangzhou, China
| | - Hong-hong Yan
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Jun-tao Lin
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Si-yang Liu
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Jing-wen Wang
- Guangzhou Shiyuan Electronics Co., Ltd, Guangzhou, China
| | | | - Hong-bo Liu
- Guangzhou Shiyuan Electronics Co., Ltd, Guangzhou, China
| | - Yong-he Hou
- Yibicom Health Management Center, CVTE, Guangzhou, China
| | - Chao Song
- Yibicom Health Management Center, CVTE, Guangzhou, China
| | - Hui-fang Yang
- Yibicom Health Management Center, CVTE, Guangzhou, China
| | - Bin Li
- Automation Science and Engineering, South China University of Technology, Guangzhou, China
| | - Ben-yuan Jiang
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Song Dong
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Qiang Nie
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Wen-zhao Zhong
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| | - Yi-long Wu
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
- *Correspondence: Xue-ning Yang, ; Yi-long Wu,
| | - Xue-ning Yang
- Guangdong Lung Cancer Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
- *Correspondence: Xue-ning Yang, ; Yi-long Wu,
| |
Collapse
|
32
|
Lv P, Wang J, Zhang X, Shi C. Deep supervision and atrous inception-based U-Net combining CRF for automatic liver segmentation from CT. Sci Rep 2022; 12:16995. [PMID: 36216965 PMCID: PMC9550798 DOI: 10.1038/s41598-022-21562-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 09/28/2022] [Indexed: 12/29/2022] Open
Abstract
Due to low contrast and the blurred boundary between liver tissue and neighboring organs sharing similar intensity values, the problem of liver segmentation from CT images has not yet achieved satisfactory performance and remains a challenge. To alleviate these problems, we introduce deep supervision (DS) and atrous inception (AI) technologies with conditional random field (CRF) and propose three major improvements that are experimentally shown to have substantive and practical value. First, we replace the encoder's standard convolution with the residual block. Residual blocks can increase the depth of the network. Second, we provide an AI module to connect the encoder and decoder. AI allows us to obtain multi-scale features. Third, we incorporate the DS mechanism into the decoder. This helps to make full use of information of the shallow layers. In addition, we employ the Tversky loss function to balance the segmented and non-segmented regions and perform further refinement with a dense CRF. Finally, we extensively validate the proposed method on three public databases: LiTS17, 3DIRCADb, and SLiver07. Compared to the state-of-the-art methods, the proposed method achieved increased segmentation accuracy for the livers with low contrast and the fuzzy boundary between liver tissue and neighboring organs and is, therefore, more suited for automatic segmentation of these livers.
Collapse
Affiliation(s)
- Peiqing Lv
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China
| | - Jinke Wang
- Department of Software Engineering, Harbin University of Science and Technology, Rongcheng, 264300, China.
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China.
| | - Xiangyang Zhang
- School of Automation, Harbin University of Science and Technology, Harbin, 150080, China
| | - Changfa Shi
- Mobile E-business Collaborative Innovation Center of Hunan Province, Hunan University of Technology and Business, Changsha, 410205, China
| |
Collapse
|
33
|
Yang H, Chen Q, Fu K, Zhu L, Jin L, Qiu B, Ren Q, Du H, Lu Y. Boosting medical image segmentation via conditional-synergistic convolution and lesion decoupling. Comput Med Imaging Graph 2022; 101:102110. [PMID: 36057184 DOI: 10.1016/j.compmedimag.2022.102110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 06/09/2022] [Accepted: 07/28/2022] [Indexed: 01/27/2023]
Abstract
Medical image segmentation is a critical step in pathology assessment and monitoring. Extensive methods tend to utilize a deep convolutional neural network for various medical segmentation tasks, such as polyp segmentation, skin lesion segmentation, etc. However, due to the inherent difficulty of medical images and tremendous data variations, they usually perform poorly in some intractable cases. In this paper, we propose an input-specific network called conditional-synergistic convolution and lesion decoupling network (CCLDNet) to solve these issues. First, in contrast to existing CNN-based methods with stationary convolutions, we propose the conditional synergistic convolution (CSConv) that aims to generate a specialist convolution kernel for each lesion. CSConv has the ability of dynamic modeling and could be leveraged as a basic block to construct other networks in a broad range of vision tasks. Second, we devise a lesion decoupling strategy (LDS) to decouple the original lesion segmentation map into two soft labels, i.e., lesion center label and lesion boundary label, for reducing the segmentation difficulty. Besides, we use a transformer network as the backbone, further erasing the fixed structure of the standard CNN and empowering dynamic modeling capability of the whole framework. Our CCLDNet outperforms state-of-the-art approaches by a large margin on a variety of benchmarks, including polyp segmentation (89.22% dice score on EndoScene) and skin lesion segmentation (91.15% dice score on ISIC2018). Our code is available at https://github.com/QianChen98/CCLD-Net.
Collapse
Affiliation(s)
- Huakun Yang
- College of Information Science and Technology, University of Science and Technology of China, Hefei 230041, China
| | - Qian Chen
- Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing 100191, China; Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing 100871, China
| | - Keren Fu
- College of Computer Science, National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu 610065, China
| | - Lei Zhu
- Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing 100191, China; Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing 100871, China
| | - Lujia Jin
- Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing 100191, China; Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing 100871, China
| | - Bensheng Qiu
- College of Information Science and Technology, University of Science and Technology of China, Hefei 230041, China
| | - Qiushi Ren
- Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing 100191, China; Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing 100871, China
| | - Hongwei Du
- College of Information Science and Technology, University of Science and Technology of China, Hefei 230041, China.
| | - Yanye Lu
- Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing 100191, China.
| |
Collapse
|
34
|
Hao W, Zhang J, Su J, Song Y, Liu Z, Liu Y, Qiu C, Han K. HPM-Net: Hierarchical progressive multiscale network for liver vessel segmentation in CT images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 224:107003. [PMID: 35868034 DOI: 10.1016/j.cmpb.2022.107003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 06/29/2022] [Accepted: 07/03/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE The segmentation and visualization of liver vessels in 3D CT images are essential for computer-aided diagnosis and preoperative planning of liver diseases. Due to the irregular structure of liver vessels and image noise, accurate extraction of liver vessels is difficult. In particular, accurate segmentation of small vessels is always a challenge, as multiple single down-sampling usually results in a loss of information. METHODS In this paper, we propose a hierarchical progressive multiscale learning network (HPM-Net) framework for liver vessel segmentation. Firstly, the hierarchical progressive multiscale learning network combines internal and external progressive learning methods to learn semantic information about liver vessels at different scales by acquiring receptive fields of different sizes. Secondly, to better capture vessel features, we propose a dual-branch progressive 3D Unet, which uses a dual-branch progressive (DBP) down-sampling strategy to reduce the loss of detailed information in the process of network down-sampling. Finally, a deep supervision mechanism is introduced into the framework and backbone network to speed up the network convergence and achieve better training of the network. RESULTS We conducted experiments on the public dataset 3Dircadb for liver vessel segmentation. The average dice coefficient and sensitivity of the proposed method reached 75.18% and 78.84%, respectively, both higher than the original network. CONCLUSION Experimental results show that the proposed hierarchical progressive multiscale network can accurately segment the labeled liver vessels from the CT images.
Collapse
Affiliation(s)
- Wen Hao
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
| | - Jing Zhang
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
| | - Jun Su
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
| | - Yuqing Song
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
| | - Zhe Liu
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China.
| | - Yi Liu
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
| | - Chengjian Qiu
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
| | - Kai Han
- School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang 212013, China
| |
Collapse
|
35
|
Uçar M. Automatic segmentation of COVID-19 from computed tomography images using modified U-Net model-based majority voting approach. Neural Comput Appl 2022; 34:21927-21938. [PMID: 35968248 PMCID: PMC9362439 DOI: 10.1007/s00521-022-07653-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 07/18/2022] [Indexed: 12/03/2022]
Abstract
The coronavirus disease (COVID-19) is an important public health problem that has spread rapidly around the world and has caused the death of millions of people. Therefore, studies to determine the factors affecting the disease, to perform preventive actions and to find an effective treatment are at the forefront. In this study, a deep learning and segmentation-based approach is proposed for the detection of COVID-19 disease from computed tomography images. The proposed model was created by modifying the encoder part of the U-Net segmentation model. In the encoder part, VGG16, ResNet101, DenseNet121, InceptionV3 and EfficientNetB5 deep learning models were used, respectively. Then, the results obtained with each modified U-Net model were combined with the majority vote principle and a final result was reached. As a result of the experimental tests, the proposed model obtained 85.03% Dice score, 89.13% sensitivity and 99.38% specificity on the COVID-19 segmentation test dataset. The results obtained in the study show that the proposed model will especially benefit clinicians in terms of time and cost.
Collapse
Affiliation(s)
- Murat Uçar
- Department of Management Information Systems, Faculty of Business and Management Sciences, İskenderun Technical University, 31200 İskenderun, Hatay Turkey
| |
Collapse
|
36
|
Gul S, Khan MS, Bibi A, Khandakar A, Ayari MA, Chowdhury ME. Deep learning techniques for liver and liver tumor segmentation: A review. Comput Biol Med 2022; 147:105620. [DOI: 10.1016/j.compbiomed.2022.105620] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Revised: 02/18/2022] [Accepted: 03/19/2022] [Indexed: 12/29/2022]
|
37
|
CEDRNN: A Convolutional Encoder-Decoder Residual Neural Network for Liver Tumour Segmentation. Neural Process Lett 2022. [DOI: 10.1007/s11063-022-10953-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
|
38
|
Hoorali F, Khosravi H, Moradi B. Automatic microscopic diagnosis of diseases using an improved UNet++ architecture. Tissue Cell 2022; 76:101816. [DOI: 10.1016/j.tice.2022.101816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 05/02/2022] [Accepted: 05/04/2022] [Indexed: 12/01/2022]
|
39
|
Zhang G, Gao X, Zhu Z, Zhou F, Yu D. Determination of the location of the needle entry point based on an improved pruning algorithm. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2022; 19:7952-7977. [PMID: 35801452 DOI: 10.3934/mbe.2022372] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Since the emergence of new coronaviruses and their variant virus, a large number of medical resources around the world have been put into treatment. In this case, the purpose of this article is to develop a handback intravenous intelligence injection robot, which reduces the direct contact between medical staff and patients and reduces the risk of infection. The core technology of hand back intravenous intelligent robot is a handlet venous vessel detection and segmentation and the position of the needle point position decision. In this paper, an image processing algorithm based on U-Net improvement mechanism (AT-U-Net) is proposed for core technology. It is investigated using a self-built dorsal hand vein database and the results show that it performs well, with an F1-score of 93.91%. After the detection of a dorsal hand vein, this paper proposes a location decision method for the needle entry point based on an improved pruning algorithm (PT-Pruning). The extraction of the trunk line of the dorsal hand vein is realized through this algorithm. Considering the vascular cross-sectional area and bending of each vein injection point area, the optimal injection point of the dorsal hand vein is obtained via a comprehensive decision-making process. Using the self-built dorsal hand vein injection point database, the accuracy of the detection of the effective injection area reaches 96.73%. The accuracy for the detection of the injection area at the optimal needle entry point is 96.50%, which lays a foundation for subsequent mechanical automatic injection.
Collapse
Affiliation(s)
- Guangyuan Zhang
- School of Information Science and Electrical Engineering, Shan Dong Jiao Tong University, Jinan 250000, China
| | - Xiaonan Gao
- School of Information Science and Electrical Engineering, Shan Dong Jiao Tong University, Jinan 250000, China
| | - Zhenfang Zhu
- School of Information Science and Electrical Engineering, Shan Dong Jiao Tong University, Jinan 250000, China
| | - Fengyv Zhou
- School of Control Science and Engineering, Shandong University, Jinan 250000, China
| | - Dexin Yu
- Department of Radiology, Qilu Hospital of Shandong University, Jinan 250000, China
| |
Collapse
|
40
|
DAS-Net: A lung nodule segmentation method based on adaptive dual-branch attention and shadow mapping. APPL INTELL 2022. [DOI: 10.1007/s10489-021-03038-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
41
|
Zhang J, Luo S, Qiang Y, Tian Y, Xiao X, Li K, Li X. Edge Constraint and Location Mapping for Liver Tumor Segmentation from Nonenhanced Images. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:1248311. [PMID: 35309832 PMCID: PMC8926519 DOI: 10.1155/2022/1248311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 02/14/2022] [Accepted: 02/16/2022] [Indexed: 11/18/2022]
Abstract
As there is no contrast enhancement, the liver tumor area in nonenhanced MRI exists with blurred edges and low contrast, which greatly affects the speed and accuracy of liver tumor diagnosis. As a result, precise segmentation of liver tumor from nonenhanced MRI has become an urgent and challenging task. In this paper, we propose an edge constraint and localization mapping segmentation model (ECLMS) to accurately segment liver tumor from nonenhanced MRI. It consists of two parts: localization network and dual-branch segmentation network. We build the localization network, which generates prior coarse masks to provide position mapping for the segmentation network. This part enhances the ability of the model to localize liver tumor in nonenhanced images. We design a dual-branch segmentation network, where the main decoding branch focuses on the feature representation in the core region of the tumor and the edge decoding branch concentrates on capturing the edge information of the tumor. To improve the ability of the model for capturing detailed features, sSE blocks and dense upward connections are introduced into it. We design the bottleneck multiscale module to construct multiscale feature representations using kernels of different sizes while integrating the location mapping of tumor. The ECLMS model is evaluated on a private nonenhanced MRI dataset that comprises 215 different subjects. The model achieves the best Dice coefficient, precision, and accuracy of 90.23%, 92.25%, and 92.39%, correspondingly. The effectiveness of our model is demonstrated by experiment results, and our model reaches superior results in the segmentation task of nonenhanced liver tumor compared to existing segmentation methods.
Collapse
Affiliation(s)
- Jina Zhang
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, China
| | - Shichao Luo
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, China
| | - Yan Qiang
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, China
| | - Yuling Tian
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, China
| | - Xiaojiao Xiao
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, China
| | - Keqin Li
- Department of Computer Science, State University of New York, New Paltz, New York 12561, USA
| | - Xingxu Li
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, China
| |
Collapse
|
42
|
Zhang C, Lu J, Hua Q, Li C, Wang P. SAA-Net: U-shaped network with Scale-Axis-Attention for liver tumor segmentation. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103460] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023]
|
43
|
Shin H, Agyeman R, Rafiq M, Chang MC, Choi GS. Automated segmentation of chronic stroke lesion using efficient U-Net architecture. Biocybern Biomed Eng 2022. [DOI: 10.1016/j.bbe.2022.01.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
|
44
|
Kushnure DT, Talbar SN. HFRU-Net: High-Level Feature Fusion and Recalibration UNet for Automatic Liver and Tumor Segmentation in CT Images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 213:106501. [PMID: 34752959 DOI: 10.1016/j.cmpb.2021.106501] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 10/21/2021] [Indexed: 06/13/2023]
Abstract
Automatic liver and tumor segmentation are essential steps to take decisive action in hepatic disease detection, deciding therapeutic planning, and post-treatment assessment. The computed tomography (CT) scan has become the choice of medical experts to diagnose hepatic anomalies. However, due to advancements in CT image acquisition protocol, CT scan data is growing and manual delineation of the liver and tumor from the CT volume becomes cumbersome and tedious for medical experts. Thus, the outcome becomes highly reliant on the operator's proficiency. Further, automatic liver and tumor segmentation from CT images is challenging due to complicated parenchyma, highly variable shape, and fewer voxel intensity variation among the liver, tumor, neighbouring organs, and discontinuity in liver boundaries. Recently deep learning (DL) exhibited extraordinary potential in medical image interpretation. Because of its effectiveness in performance advancement, the DL-based convolutional neural networks (CNN) gained significant interest in the medical realm. The proposed HFRU-Net is derived from the UNet architecture by modifying the skip pathways using local feature reconstruction and feature fusion mechanism that represents the detailed contextual information in the high-level features. Further, the fused features are adaptively recalibrated by learning the channel-wise interdependencies to acquire the prominent details of the modified high-level features using the squeeze-and-Excitation network (SENet). Also, in the bottleneck layer, we employed the atrous spatial pyramid pooling (ASPP) module to represent the multiscale features with dissimilar receptive fields to represent the rich spatial information in the low-level features. These amendments uplift the segmentation performance and reduce the computational complexity of the model than outperforming methods. The efficacy of the proposed model is proved by widespread experimentation on two datasets available publicly (LiTS and 3DIrcadb). The experimental result analysis illustrates that the proposed model has attained a dice similarity coefficient of 0.966 and 0.972 for liver segmentation and 0.771 and 0.776 for liver tumor segmentation on LiTS and the 3DIRCADb dataset. Further, the robustness of the HFRU-Net is confirmed on the independent LiTS challenge test dataset. The proposed model attained the global dice of 95.0% for liver segmentation and 61.4% for tumor segmentation which is comparable with the state-of-the-art methods.
Collapse
Affiliation(s)
- Devidas T Kushnure
- Department of Electronics and Telecommunication Engineering, Shri Guru Gobind Singhji Institute of Engineering and Technology, Nanded, Maharashtra, India; Department of Electronics and Telecommunication Engineering, Vidya Pratishthan's Kamalnayan Bajaj Institute of Engineering and Technology, Baramati, Maharashtra, India.
| | - Sanjay N Talbar
- Department of Electronics and Telecommunication Engineering, Shri Guru Gobind Singhji Institute of Engineering and Technology, Nanded, Maharashtra, India
| |
Collapse
|
45
|
Chen CI, Lu NH, Huang YH, Liu KY, Hsu SY, Matsushima A, Wang YM, Chen TB. Segmentation of liver tumors with abdominal computed tomography using fully convolutional networks. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2022; 30:953-966. [PMID: 35754254 DOI: 10.3233/xst-221194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
BACKGROUND Dividing liver organs or lesions depicting on computed tomography (CT) images could be applied to help tumor staging and treatment. However, most existing image segmentation technologies use manual or semi-automatic analysis, making the analysis process costly and time-consuming. OBJECTIVE This research aims to develop and apply a deep learning network architecture to segment liver tumors automatically after fine tuning parameters. METHODS AND MATERIALS The medical imaging is obtained from the International Symposium on Biomedical Imaging (ISBI), which includes 3D abdominal CT scans of 131 patients diagnosed with liver tumors. From these CT scans, there are 7,190 2D CT images along with the labeled binary images. The labeled binary images are regarded as gold standard for evaluation of the segmented results by FCN (Fully Convolutional Network). The backbones of FCN are extracted from Xception, InceptionresNetv2, MobileNetv2, ResNet18, ResNet50 in this study. Meanwhile, the parameters including optimizers (SGDM and ADAM), size of epoch, and size of batch are investigated. CT images are randomly divided into training and testing sets using a ratio of 9:1. Several evaluation indices including Global Accuracy, Mean Accuracy, Mean IoU (Intersection over Union), Weighted IoU and Mean BF Score are applied to evaluate tumor segmentation results in the testing images. RESULTS The Global Accuracy, Mean Accuracy, Mean IoU, Weighted IoU, and Mean BF Scores are 0.999, 0.969, 0.954, 0.998, 0.962 using ResNet50 in FCN with optimizer SGDM, batch size 12, and epoch 9. It is important to fine tuning the parameters in FCN model. Top 20 FNC models enable to achieve higher tumor segmentation accuracy with Mean IoU over 0.900. The occurred frequency of InceptionresNetv2, MobileNetv2, ResNet18, ResNet50, and Xception are 9, 6, 3, 5, and 2 times. Therefore, the InceptionresNetv2 has higher performance than others. CONCLUSIONS This study develop and test an automated liver tumor segmentation model based on FCN. Study results demonstrate that many deep learning models including InceptionresNetv2, MobileNetv2, ResNet18, ResNet50, and Xception have high potential to segment liver tumors from CT images with accuracy exceeding 90%. However, it is still difficult to accurately segment tiny and small size tumors by FCN models.
Collapse
Affiliation(s)
- Chih-I Chen
- Division of Colon and Rectal Surgery, Department of Surgery, E-DA Hospital, Kaohsiung City, Taiwan
- Division of General Medicine Surgery, Department of Surgery, E-DA Hospital, Kaohsiung City, Taiwan
- School of Medicine, College of Medicine, I-Shou University, Kaohsiung City, Taiwan
- Department of Information Engineering, I-Shou University, Kaohsiung City, Taiwan
- The School of Chinese Medicine for Post Baccalaureate, I-Shou University, Kaohsiung City, Taiwan
| | - Nan-Han Lu
- Department of Pharmacy, Tajen University, Pingtung City, Taiwan
- Department of Radiology, E-DA Hospital, I-Shou University, Kaohsiung City, Taiwan
- Department of Medical Imaging and Radiological Science, I-Shou University, Kaohsiung City, Taiwan
| | - Yung-Hui Huang
- Department of Medical Imaging and Radiological Science, I-Shou University, Kaohsiung City, Taiwan
| | - Kuo-Ying Liu
- Department of Radiology, E-DA Hospital, I-Shou University, Kaohsiung City, Taiwan
| | - Shih-Yen Hsu
- Department of Information Engineering, I-Shou University, Kaohsiung City, Taiwan
| | - Akari Matsushima
- Department of Radiological Technology Faculty of Medical Technology, Teikyo University, Tokyo, Japan
| | - Yi-Ming Wang
- Department of Information Engineering, I-Shou University, Kaohsiung City, Taiwan
- Department of Critical Care Medicine, E-DA hospital, I-Shou University, Kaohsiung City, Taiwan
| | - Tai-Been Chen
- Department of Medical Imaging and Radiological Science, I-Shou University, Kaohsiung City, Taiwan
- Institute of Statistics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| |
Collapse
|
46
|
Wang S, Liu X, Zhao J, Liu Y, Liu S, Liu Y, Zhao J. Computer auxiliary diagnosis technique of detecting cholangiocarcinoma based on medical imaging: A review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 208:106265. [PMID: 34311415 DOI: 10.1016/j.cmpb.2021.106265] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Accepted: 06/28/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVES Cholangiocarcinoma (CCA) is one of the most aggressive human malignant tumors and is becoming one of the main factors of death and disability globally. Specifically, 60% to 70% of CCA patients were diagnosed with local invasion or distant metastasis and lost the chance of radical operation. The overall median survival time was less than 12 months. As a non-invasive diagnostic technology, medical imaging consisting of computed tomography (CT) imaging, magnetic resonance imaging (MRI), and ultrasound (US) imaging, is the most effectively and commonly used method to detect CCA. The computer auxiliary diagnosis (CAD) system based on medical imaging is helpful for rapid diagnosis and provides credible "second opinion" for specialists. The purpose of this review is to categorize and review the CAD technique of detecting CCA based on medical imaging. METHODS This work applies a four-level screening process to choose suitable publications. 125 research papers published in different academic research databases were selected and analyzed according to specific criteria. From the five steps of medical image acquisition, processing, analysis, understanding and verification of CAD combined with artificial intelligence algorithms, we obtain the most advanced insights related to CCA detection. RESULTS This work provides a comprehensive analysis and comparison analysis of the current CAD systems of detecting CCA. After careful investigation, we find that the main detection methods are traditional machine learning method and deep learning method. For the detection, the most commonly used method is semi-automatic segmentation algorithm combined with support vector machine classifier method, combination of which has good detection performance. The end-to-end training mode makes deep learning method more and more popular in CAD systems. However, due to the limited medical training data, the accuracy of deep learning method is unsatisfactory. CONCLUSIONS Based on analysis of artificial intelligence methods applied in CCA, this work is expected to be truly applied in clinical practice in the future to improve the level of clinical diagnosis and treatment of it. This work concludes by providing a prediction of future trends, which will be of great significance for researchers in the medical imaging of CCA and artificial intelligence.
Collapse
Affiliation(s)
- Shiyu Wang
- School of Electronic and Electric Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
| | - Xiang Liu
- School of Electronic and Electric Engineering, Shanghai University of Engineering Science, Shanghai 201620, China.
| | - Jingwen Zhao
- School of Electronic and Electric Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
| | - Yiwen Liu
- School of Electronic and Electric Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
| | - Shuhong Liu
- Department of Pathology and Hepatology, The Fifth Medical Centre of Chinese PLA General Hospital, Beijing 100039, China
| | - Yisi Liu
- Department of Pathology and Hepatology, The Fifth Medical Centre of Chinese PLA General Hospital, Beijing 100039, China
| | - Jingmin Zhao
- Department of Pathology and Hepatology, The Fifth Medical Centre of Chinese PLA General Hospital, Beijing 100039, China.
| |
Collapse
|
47
|
Tang Y, Zheng Y, Chen X, Wang W, Guo Q, Shu J, Wu J, Su S. Identifying Periampullary Regions in MRI Images Using Deep Learning. Front Oncol 2021; 11:674579. [PMID: 34123843 PMCID: PMC8193851 DOI: 10.3389/fonc.2021.674579] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Accepted: 04/20/2021] [Indexed: 12/15/2022] Open
Abstract
Background Development and validation of a deep learning method to automatically segment the peri-ampullary (PA) region in magnetic resonance imaging (MRI) images. Methods A group of patients with or without periampullary carcinoma (PAC) was included. The PA regions were manually annotated in MRI images by experts. Patients were randomly divided into one training set, one validation set, and one test set. Deep learning methods were developed to automatically segment the PA region in MRI images. The segmentation performance of the methods was compared in the validation set. The model with the highest intersection over union (IoU) was evaluated in the test set. Results The deep learning algorithm achieved optimal accuracies in the segmentation of the PA regions in both T1 and T2 MRI images. The value of the IoU was 0.68, 0.68, and 0.64 for T1, T2, and combination of T1 and T2 images, respectively. Conclusions Deep learning algorithm is promising with accuracies of concordance with manual human assessment in segmentation of the PA region in MRI images. This automated non-invasive method helps clinicians to identify and locate the PA region using preoperative MRI scanning.
Collapse
Affiliation(s)
- Yong Tang
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Yingjun Zheng
- Department of General Surgery (Hepatobiliary Surgery), The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Xinpei Chen
- Department of Hepatobiliary Surgery, Deyang People's Hospital, Deyang, China
| | - Weijia Wang
- School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Qingxi Guo
- Department of Pathology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Jian Shu
- Department of Radiology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Jiali Wu
- Department of Anesthesiology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Song Su
- Department of General Surgery (Hepatobiliary Surgery), The Affiliated Hospital of Southwest Medical University, Luzhou, China
| |
Collapse
|
48
|
Abstract
Accurate liver vessel segmentation is of crucial importance for the clinical diagnosis and treatment of many hepatic diseases. Recent state-of-the-art methods for liver vessel reconstruction mostly utilize deep learning methods, namely, the U-Net model and its variants. However, to the best of our knowledge, no comparative evaluation has been proposed to compare these approaches in the liver vessel segmentation task. Moreover, most research works do not consider the liver volume segmentation as a preprocessing step, in order to keep only inner hepatic vessels, for Couinaud representation for instance. For these reasons, in this work, we propose using accurate Dense U-Net liver segmentation and conducting a comparison between 3D U-Net models inside the obtained volumes. More precisely, 3D U-Net, Dense U-Net, and MultiRes U-Net are pitted against each other in the vessel segmentation task on the IRCAD dataset. For each model, three alternative setups that allow adapting the selected CNN architectures to volumetric data are tested, namely, full 3D, slab-based, and box-based setups are considered. The results showed that the most accurate setup is the full 3D process, providing the highest Dice for most of the considered models. However, concerning the particular models, the slab-based MultiRes U-Net provided the best score. With our accurate vessel segmentations, several medical applications can be investigated, such as automatic and personalized Couinaud zoning of the liver.
Collapse
|