1
|
Jian M, Wu R, Xu W, Zhi H, Tao C, Chen H, Li X. VascuConNet: an enhanced connectivity network for vascular segmentation. Med Biol Eng Comput 2024; 62:3543-3554. [PMID: 38898202 DOI: 10.1007/s11517-024-03150-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Accepted: 06/06/2024] [Indexed: 06/21/2024]
Abstract
Medical image segmentation commonly involves diverse tissue types and structures, including tasks such as blood vessel segmentation and nerve fiber bundle segmentation. Enhancing the continuity of segmentation outcomes represents a pivotal challenge in medical image segmentation, driven by the demands of clinical applications, focusing on disease localization and quantification. In this study, a novel segmentation model is specifically designed for retinal vessel segmentation, leveraging vessel orientation information, boundary constraints, and continuity constraints to improve segmentation accuracy. To achieve this, we cascade U-Net with a long-short-term memory network (LSTM). U-Net is characterized by a small number of parameters and high segmentation efficiency, while LSTM offers a parameter-sharing capability. Additionally, we introduce an orientation information enhancement module inserted into the model's bottom layer to obtain feature maps containing orientation information through an orientation convolution operator. Furthermore, we design a new hybrid loss function that consists of connectivity loss, boundary loss, and cross-entropy loss. Experimental results demonstrate that the model achieves excellent segmentation outcomes across three widely recognized retinal vessel segmentation datasets, CHASE_DB1, DRIVE, and ARIA.
Collapse
Affiliation(s)
- Muwei Jian
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China.
- School of Information Science and Technology, Linyi University, Linyi, China.
| | - Ronghua Wu
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Wenjin Xu
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Huixiang Zhi
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Chen Tao
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Hongyu Chen
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Xiaoguang Li
- Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, China.
| |
Collapse
|
2
|
Laddi A, Goyal S, Himani, Savlania A. Vein segmentation and visualization of upper and lower extremities using convolution neural network. BIOMED ENG-BIOMED TE 2024; 69:455-464. [PMID: 38651783 DOI: 10.1515/bmt-2023-0331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Accepted: 04/03/2024] [Indexed: 04/25/2024]
Abstract
OBJECTIVES The study focused on developing a reliable real-time venous localization, identification, and visualization framework based upon deep learning (DL) self-parametrized Convolution Neural Network (CNN) algorithm for segmentation of the venous map for both lower and upper limb dataset acquired under unconstrained conditions using near-infrared (NIR) imaging setup, specifically to assist vascular surgeons during venipuncture, vascular surgeries, or Chronic Venous Disease (CVD) treatments. METHODS A portable image acquisition setup has been designed to collect venous data (upper and lower extremities) from 72 subjects. A manually annotated image dataset was used to train and compare the performance of existing well-known CNN-based architectures such as ResNet and VGGNet with self-parameterized U-Net, improving automated vein segmentation and visualization. RESULTS Experimental results indicated that self-parameterized U-Net performs better at segmenting the unconstrained dataset in comparison with conventional CNN feature-based learning models, with a Dice score of 0.58 and displaying 96.7 % accuracy for real-time vein visualization, making it appropriate to locate veins in real-time under unconstrained conditions. CONCLUSIONS Self-parameterized U-Net for vein segmentation and visualization has the potential to reduce risks associated with traditional venipuncture or CVD treatments by outperforming conventional CNN architectures, providing vascular assistance, and improving patient care and treatment outcomes.
Collapse
Affiliation(s)
- Amit Laddi
- Biomedical Applications Group, CSIR-Central Scientific Instruments Organisation (CSIO), Chandigarh-160030, India
- Academy of Scientific and Innovative Research (AcSIR), Ghaziabad, Uttar Pradesh- 201 002, India
| | - Shivalika Goyal
- Biomedical Applications Group, CSIR-Central Scientific Instruments Organisation (CSIO), Chandigarh-160030, India
- Academy of Scientific and Innovative Research (AcSIR), Ghaziabad, Uttar Pradesh- 201 002, India
| | | | - Ajay Savlania
- Department of General Surgery, 29751 PGIMER , Chandigarh, India
| |
Collapse
|
3
|
Xie X, Yang M. USCT-UNet: Rethinking the Semantic Gap in U-Net Network From U-Shaped Skip Connections With Multichannel Fusion Transformer. IEEE Trans Neural Syst Rehabil Eng 2024; 32:3782-3793. [PMID: 39325601 DOI: 10.1109/tnsre.2024.3468339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/28/2024]
Abstract
Medical image segmentation is a crucial component of computer-aided clinical diagnosis, with state-of-the-art models often being variants of U-Net. Despite their success, these models' skip connections introduce an unnecessary semantic gap between the encoder and decoder, which hinders their ability to achieve the high precision required for clinical applications. Awareness of this semantic gap and its detrimental influences have increased over time. However, a quantitative understanding of how this semantic gap compromises accuracy and reliability remains lacking, emphasizing the need for effective mitigation strategies. In response, we present the first quantitative evaluation of the semantic gap between corresponding layers of U-Net and identify two key characteristics: 1) The direct skip connection (DSC) exhibits a semantic gap that negatively impacts models' performance; 2) The magnitude of the semantic gap varies across different layers. Based on these findings, we re-examine this issue through the lens of skip connections. We introduce a Multichannel Fusion Transformer (MCFT) and propose a novel USCT-UNet architecture, which incorporates U-shaped skip connections (USC) to replace DSC, allocates varying numbers of MCFT blocks based on the semantic gap magnitude at different layers, and employs a spatial channel cross-attention (SCCA) module to facilitate the fusion of features between the decoder and USC. We evaluate USCT-UNet on four challenging datasets, and the results demonstrate that it effectively eliminates the semantic gap. Compared to using DSC, our USC and SCCA strategies achieve maximum improvements of 4.79% in the Dice coefficient, 5.70% in mean intersection over union (MIoU), and 3.26 in Hausdorff distance.
Collapse
|
4
|
Rafique MA. Exploiting Temporal Features in Calculating Automated Morphological Properties of Spiky Nanoparticles Using Deep Learning. SENSORS (BASEL, SWITZERLAND) 2024; 24:6541. [PMID: 39460021 PMCID: PMC11511305 DOI: 10.3390/s24206541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/02/2024] [Revised: 10/06/2024] [Accepted: 10/09/2024] [Indexed: 10/28/2024]
Abstract
Object segmentation in images is typically spatial and focuses on the spatial coherence of pixels. Nanoparticles in electron microscopy images are also segmented frame by frame, with subsequent morphological analysis. However, morphological analysis is inherently sequential, and a temporal regularity is evident in the process. In this study, we extend the spatially focused morphological analysis by incorporating a fusion of hard and soft inductive bias from sequential machine learning techniques to account for temporal relationships. Previously, spiky Au nanoparticles (Au-SNPs) in electron microscopy images were analyzed, and their morphological properties were automatically generated using a hourglass convolutional neural network architecture. In this study, recurrent layers are integrated to capture the natural, sequential growth of the particles. The network is trained with a spike-focused loss function. Continuous segmentation of the images explores the regressive relationships among natural growth features, generating morphological statistics of the nanoparticles. This study comprehensively evaluates the proposed approach by comparing the results of segmentation and morphological properties analysis, demonstrating its superiority over earlier methods.
Collapse
Affiliation(s)
- Muhammad Aasim Rafique
- Department of Information Systems, College of Computer Sciences & Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia
| |
Collapse
|
5
|
Zhang Y, Huang W, Jiao H, Kang L. PET radiomics in lung cancer: advances and translational challenges. EJNMMI Phys 2024; 11:81. [PMID: 39361110 PMCID: PMC11450131 DOI: 10.1186/s40658-024-00685-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2023] [Accepted: 09/26/2024] [Indexed: 10/06/2024] Open
Abstract
Radiomics is an emerging field of medical imaging that aims at improving the accuracy of diagnosis, prognosis, treatment planning and monitoring non-invasively through the automated or semi-automated quantitative analysis of high-dimensional image features. Specifically in the field of nuclear medicine, radiomics utilizes imaging methods such as positron emission tomography (PET) and single photon emission computed tomography (SPECT) to evaluate biomarkers related to metabolism, blood flow, cellular activity and some biological pathways. Lung cancer ranks among the leading causes of cancer-related deaths globally, and radiomics analysis has shown great potential in guiding individualized therapy, assessing treatment response, and predicting clinical outcomes. In this review, we summarize the current state-of-the-art radiomics progress in lung cancer, highlighting the potential benefits and existing limitations of this approach. The radiomics workflow was introduced first including image acquisition, segmentation, feature extraction, and model building. Then the published literatures were described about radiomics-based prediction models for lung cancer diagnosis, differentiation, prognosis and efficacy evaluation. Finally, we discuss current challenges and provide insights into future directions and potential opportunities for integrating radiomics into routine clinical practice.
Collapse
Affiliation(s)
- Yongbai Zhang
- Department of Nuclear Medicine, Peking University First Hospital, No. 8 Xishiku Str., Xicheng Dist, Beijing, 100034, China
| | - Wenpeng Huang
- Department of Nuclear Medicine, Peking University First Hospital, No. 8 Xishiku Str., Xicheng Dist, Beijing, 100034, China
| | - Hao Jiao
- Department of Nuclear Medicine, Peking University First Hospital, No. 8 Xishiku Str., Xicheng Dist, Beijing, 100034, China
| | - Lei Kang
- Department of Nuclear Medicine, Peking University First Hospital, No. 8 Xishiku Str., Xicheng Dist, Beijing, 100034, China.
| |
Collapse
|
6
|
Liu Q, Zhou F, Shen J, Xu J, Wan C, Xu X, Yan Z, Yao J. A fundus vessel segmentation method based on double skip connections combined with deep supervision. Front Cell Dev Biol 2024; 12:1477819. [PMID: 39430046 PMCID: PMC11487527 DOI: 10.3389/fcell.2024.1477819] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2024] [Accepted: 09/20/2024] [Indexed: 10/22/2024] Open
Abstract
Background Fundus vessel segmentation is vital for diagnosing ophthalmic diseases like central serous chorioretinopathy (CSC), diabetic retinopathy, and glaucoma. Accurate segmentation provides crucial vessel morphology details, aiding the early detection and intervention of ophthalmic diseases. However, current algorithms struggle with fine vessel segmentation and maintaining sensitivity in complex regions. Challenges also stem from imaging variability and poor generalization across multimodal datasets, highlighting the need for more advanced algorithms in clinical practice. Methods This paper aims to explore a new vessel segmentation method to alleviate the above problems. We propose a fundus vessel segmentation model based on a combination of double skip connections, deep supervision, and TransUNet, namely DS2TUNet. Initially, the original fundus images are improved through grayscale conversion, normalization, histogram equalization, gamma correction, and other preprocessing techniques. Subsequently, by utilizing the U-Net architecture, the preprocessed fundus images are segmented to obtain the final vessel information. Specifically, the encoder firstly incorporates the ResNetV1 downsampling, dilated convolution downsampling, and Transformer to capture both local and global features, which upgrades its vessel feature extraction ability. Then, the decoder introduces the double skip connections to facilitate upsampling and refine segmentation outcomes. Finally, the deep supervision module introduces multiple upsampling vessel features from the decoder into the loss function, so that the model can learn vessel feature representations more effectively and alleviate gradient vanishing during the training phase. Results Extensive experiments on publicly available multimodal fundus datasets such as DRIVE, CHASE_DB1, and ROSE-1 demonstrate that the DS2TUNet model attains F1-scores of 0.8195, 0.8362, and 0.8425, with Accuracy of 0.9664, 0.9741, and 0.9557, Sensitivity of 0.8071, 0.8101, and 0.8586, and Specificity of 0.9823, 0.9869, and 0.9713, respectively. Additionally, the model also exhibits excellent test performance on the clinical fundus dataset CSC, with F1-score of 0.7757, Accuracy of 0.9688, Sensitivity of 0.8141, and Specificity of 0.9801 based on the weight trained on the CHASE_DB1 dataset. These results comprehensively validate that the proposed method obtains good performance in fundus vessel segmentation, thereby aiding clinicians in the further diagnosis and treatment of fundus diseases in terms of effectiveness and feasibility.
Collapse
Affiliation(s)
- Qingyou Liu
- College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Fen Zhou
- The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
| | - Jianxin Shen
- College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Jianguo Xu
- College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Cheng Wan
- College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Xiangzhong Xu
- The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
| | - Zhipeng Yan
- The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
| | - Jin Yao
- The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
| |
Collapse
|
7
|
Zhang L, Li Y, Liang Y, Xu C, Liu T, Sun J. Dilated multi-scale residual attention (DMRA) U-Net: three-dimensional (3D) dilated multi-scale residual attention U-Net for brain tumor segmentation. Quant Imaging Med Surg 2024; 14:7249-7264. [PMID: 39429586 PMCID: PMC11485367 DOI: 10.21037/qims-24-779] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Accepted: 08/20/2024] [Indexed: 10/22/2024]
Abstract
Background The precise identification of the position and form of a tumor mass can improve early diagnosis and treatment. However, due to the complicated tumor categories and varying sizes and forms, the segregation of brain gliomas and their internal sub-regions is still very challenging. This study sought to design a new deep-learning network based on three-dimensional (3D) U-Net to address its shortcomings in brain tumor segmentation (BraTS) tasks. Methods We developed a 3D dilated multi-scale residual attention U-Net (DMRA-U-Net) model for magnetic resonance imaging (MRI) BraTS. It used dilated convolution residual (DCR) modules to better process shallow features, multi-scale convolution residual (MCR) modules in the bottom encoding path to create richer and more comprehensive feature expression while reducing overall information loss or blurring, and a channel attention (CA) module between the encoding and decoding paths to address the problem of retrieving and preserving important features during the processing of deep feature maps. Results The BraTS 2018-2021 datasets served as the training and evaluation datasets for this study. Further, the proposed architecture was assessed using metrics such as the dice similarity coefficient (DSC), Hausdorff distance (HD), and sensitivity (Sens). The DMRA U-Net model segments the whole tumor (WT), and the tumor core (TC), and the enhancing tumor (ET) regions of brain tumors. Using the suggested architecture, the DSCs were 0.9012, 0.8867, and 0.8813, the HDs were 28.86, 13.34, and 10.88 mm, and the Sens was 0.9429, 0.9452, and 0.9303 for the WT, TC, and ET regions, respectively. Compared to the traditional 3D U-Net, the DSC of the DMRA U-Net increased by 4.5%, 2.5%, and 0.8%, the HD of the DMRA U-Net decreased by 21.83, 16.42, and 10.00, the Sens of the DMRA U-Net increased by 0.4%, 0.7%, and 1.4% for the WT, TC, and ET regions, respectively. Further, the results of the statistical comparison of the performance indicators revealed that our model performed well generally in the segmentation of the WT, TC, and ET regions. Conclusions We developed a promising tumor segmentation model. Our solution is open sourced and is available at: https://github.com/Gold3nk/dmra-unet.
Collapse
Affiliation(s)
- Lihong Zhang
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Yuzhuo Li
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Yingbo Liang
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Chongxin Xu
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Tong Liu
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Junding Sun
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| |
Collapse
|
8
|
Shen Y, Zhu C, Chu B, Song J, Geng Y, Li J, Liu B, Wu X. Evaluation of the clinical application value of artificial intelligence in diagnosing head and neck aneurysms. BMC Med Imaging 2024; 24:261. [PMID: 39354383 PMCID: PMC11446065 DOI: 10.1186/s12880-024-01436-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 09/18/2024] [Indexed: 10/03/2024] Open
Abstract
OBJECTIVE To evaluate the performance of a semi-automated artificial intelligence (AI) software program (CerebralDoc® system) in aneurysm detection and morphological measurement. METHODS In this study, 354 cases of computed tomographic angiography (CTA) were retrospectively collected in our hospital. Among them, 280 cases were diagnosed with aneurysms by either digital subtraction angiography (DSA) and CTA (DSA group, n = 102), or CTA-only (non-DSA group, n = 178). The presence or absence of aneurysms, as well as their location and related morphological features determined by AI were evaluated using DSA and radiologist findings. Besides, post-processing image quality from AI and radiologists were also rated and compared. RESULTS In the DSA group, AI achieved a sensitivity of 88.24% and an accuracy of 81.97%, whereas radiologists achieved a sensitivity of 95.10% and an accuracy of 84.43%, using DSA results as the gold standard. The AI in the non-DSA group achieved 81.46% sensitivity and 76.29% accuracy, as per the radiologists' findings. The comparison of position consistency results showed better performance under loose criteria than strict criteria. In terms of morphological characteristics, both the DSA and the non-DSA groups agreed well with the diagnostic results for neck width and maximum diameter, demonstrating excellent ICC reliability exceeding 0.80. The AI-generated images exhibited superior quality compared to the standard software for post-processing, while also demonstrating a significantly reduced processing time. CONCLUSIONS The AI-based aneurysm detection rate demonstrates a commendable performance, while the extracted morphological parameters exhibit a remarkable consistency with those assessed by radiologists, thereby showcasing significant potential for clinical application.
Collapse
Affiliation(s)
- Yi Shen
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui Province, 230022, China
| | - Chao Zhu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui Province, 230022, China
- Department of Radiology, The First Affiliated Hospital of Wannan Medical College, Wuhu, Anhui Province, 241000, China
| | - Bingqian Chu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui Province, 230022, China
| | - Jian Song
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui Province, 230022, China
| | - Yayuan Geng
- Shukun (Beijing) Network Technology Co, Ltd, Jinhui Building, Qiyang Road, Beijing, 100102, China
| | - Jianying Li
- CT Research Center, GE Healthcare China, Shanghai, 210000, China
| | - Bin Liu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui Province, 230022, China.
| | - Xingwang Wu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui Province, 230022, China.
| |
Collapse
|
9
|
Zhou W, Wang X, Yang X, Hu Y, Yi Y. Skeleton-guided multi-scale dual-coordinate attention aggregation network for retinal blood vessel segmentation. Comput Biol Med 2024; 181:109027. [PMID: 39178808 DOI: 10.1016/j.compbiomed.2024.109027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Revised: 06/06/2024] [Accepted: 08/12/2024] [Indexed: 08/26/2024]
Abstract
Deep learning plays a pivotal role in retinal blood vessel segmentation for medical diagnosis. Despite their significant efficacy, these techniques face two major challenges. Firstly, they often neglect the severe class imbalance in fundus images, where thin vessels in the foreground are proportionally minimal. Secondly, they are susceptible to poor image quality and blurred vessel edges, resulting in discontinuities or breaks in vascular structures. In response, this paper proposes the Skeleton-guided Multi-scale Dual-coordinate Attention Aggregation (SMDAA) network for retinal vessel segmentation. SMDAA comprises three innovative modules: Dual-coordinate Attention (DCA), Unbalanced Pixel Amplifier (UPA), and Vessel Skeleton Guidance (VSG). DCA, integrating Multi-scale Coordinate Feature Aggregation (MCFA) and Scale Coordinate Attention Decoding (SCAD), meticulously analyzes vessel structures across various scales, adept at capturing intricate details, thereby significantly enhancing segmentation accuracy. To address class imbalance, we introduce UPA, dynamically allocating more attention to misclassified pixels, ensuring precise extraction of thin and small blood vessels. Moreover, to preserve vessel structure continuity, we integrate vessel anatomy and develop the VSG module to connect fragmented vessel segments. Additionally, a Feature-level Contrast (FCL) loss is introduced to capture subtle nuances within the same category, enhancing the fidelity of retinal blood vessel segmentation. Extensive experiments on three public datasets (DRIVE, STARE, and CHASE_DB1) demonstrate superior performance in comparison to current methods. The code is available at https://github.com/wangwxr/SMDAA_NET.
Collapse
Affiliation(s)
- Wei Zhou
- College of Computer Science, Shenyang Aerospace University, Shenyang, China
| | - Xiaorui Wang
- College of Computer Science, Shenyang Aerospace University, Shenyang, China
| | - Xuekun Yang
- College of Computer Science, Shenyang Aerospace University, Shenyang, China
| | - Yangtao Hu
- Department of Ophthalmology, The 908th Hospital of Chinese People's Liberation Army Joint Logistic SupportForce, Nanchang, China.
| | - Yugen Yi
- School of Software, Jiangxi Normal University, Nanchang, China.
| |
Collapse
|
10
|
Vašinková M, Doleží V, Vašinek M, Gajdoš P, Kriegová E. Comparing Deep Learning Performance for Chronic Lymphocytic Leukaemia Cell Segmentation in Brightfield Microscopy Images. Bioinform Biol Insights 2024; 18:11779322241272387. [PMID: 39246684 PMCID: PMC11378236 DOI: 10.1177/11779322241272387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Accepted: 07/15/2024] [Indexed: 09/10/2024] Open
Abstract
Objectives This article focuses on the detection of cells in low-contrast brightfield microscopy images; in our case, it is chronic lymphocytic leukaemia cells. The automatic detection of cells from brightfield time-lapse microscopic images brings new opportunities in cell morphology and migration studies; to achieve the desired results, it is advisable to use state-of-the-art image segmentation methods that not only detect the cell but also detect its boundaries with the highest possible accuracy, thus defining its shape and dimensions. Methods We compared eight state-of-the-art neural network architectures with different backbone encoders for image data segmentation, namely U-net, U-net++, the Pyramid Attention Network, the Multi-Attention Network, LinkNet, the Feature Pyramid Network, DeepLabV3, and DeepLabV3+. The training process involved training each of these networks for 1000 epochs using the PyTorch and PyTorch Lightning libraries. For instance segmentation, the watershed algorithm and three-class image semantic segmentation were used. We also used StarDist, a deep learning-based tool for object detection with star-convex shapes. Results The optimal combination for semantic segmentation was the U-net++ architecture with a ResNeSt-269 background with a data set intersection over a union score of 0.8902. For the cell characteristics examined (area, circularity, solidity, perimeter, radius, and shape index), the difference in mean value using different chronic lymphocytic leukaemia cell segmentation approaches appeared to be statistically significant (Mann-Whitney U test, P < .0001). Conclusion We found that overall, the algorithms demonstrate equal agreement with ground truth, but with the comparison, it can be seen that the different approaches prefer different morphological features of the cells. Consequently, choosing the most suitable method for instance-based cell segmentation depends on the particular application, namely, the specific cellular traits being investigated.
Collapse
Affiliation(s)
- Markéta Vašinková
- Department of Computer Science, FEECS, VSB - Technical University of Ostrava, Ostrava, Czech Republic
| | - Vít Doleží
- Department of Computer Science, FEECS, VSB - Technical University of Ostrava, Ostrava, Czech Republic
| | - Michal Vašinek
- Department of Computer Science, FEECS, VSB - Technical University of Ostrava, Ostrava, Czech Republic
| | - Petr Gajdoš
- Department of Computer Science, FEECS, VSB - Technical University of Ostrava, Ostrava, Czech Republic
| | - Eva Kriegová
- Department of Immunology, Faculty of Medicine and Dentistry, Palacky University & University Hospital, Olomouc, Czech Republic
| |
Collapse
|
11
|
R P A, Zacharias J. Semantic segmentation in skin surface microscopic images with artifacts removal. Comput Biol Med 2024; 180:108975. [PMID: 39153395 DOI: 10.1016/j.compbiomed.2024.108975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 07/29/2024] [Accepted: 07/30/2024] [Indexed: 08/19/2024]
Abstract
Skin surface imaging has been used to examine skin lesions with a microscope for over a century and is commonly known as epiluminescence microscopy, dermatoscopy, or dermoscopy. Skin surface microscopy has been recommended to reduce the necessity of biopsy. This imaging technique could improve the clinical diagnostic performance of pigmented skin lesions. Different imaging techniques are employed in dermatology to find diseases. Segmentation and classification are the two main steps in the examination. The classification performance is influenced by the algorithm employed in the segmentation procedure. The most difficult aspect of segmentation is getting rid of the unwanted artifacts. Many deep-learning models are being created to segment skin lesions. In this paper, an analysis of common artifacts is proposed to investigate the segmentation performance of deep learning models with skin surface microscopic images. The most prevalent artifacts in skin images are hair and dark corners. These artifacts can be observed in the majority of dermoscopy images captured through various imaging techniques. While hair detection and removal methods are common, the introduction of dark corner detection and removal represents a novel approach to skin lesion segmentation. A comprehensive analysis of this segmentation performance is assessed using the surface density of artifacts. Assessment of the PH2, ISIC 2017, and ISIC 2018 datasets demonstrates significant enhancements, as reflected by Dice coefficients rising to 93.49 (86.81), 85.86 (79.91), and 75.38 (51.28) respectively, upon artifact removal. These results underscore the pivotal significance of artifact removal techniques in amplifying the efficacy of deep-learning models for skin lesion segmentation.
Collapse
Affiliation(s)
- Aneesh R P
- College of Engineering Trivandrum, APJ Abdul Kalam Technological University Kerala, Thiruvananthapuram, Kerala, India.
| | - Joseph Zacharias
- College of Engineering Trivandrum, APJ Abdul Kalam Technological University Kerala, Thiruvananthapuram, Kerala, India
| |
Collapse
|
12
|
Wang W, Mao Q, Tian Y, Zhang Y, Xiang Z, Ren L. FMD-UNet: fine-grained feature squeeze and multiscale cascade dilated semantic aggregation dual-decoder UNet for COVID-19 lung infection segmentation from CT images. Biomed Phys Eng Express 2024; 10:055031. [PMID: 39142295 DOI: 10.1088/2057-1976/ad6f12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2024] [Accepted: 08/14/2024] [Indexed: 08/16/2024]
Abstract
With the advancement of computer-aided diagnosis, the automatic segmentation of COVID-19 infection areas holds great promise for assisting in the timely diagnosis and recovery of patients in clinical practice. Currently, methods relying on U-Net face challenges in effectively utilizing fine-grained semantic information from input images and bridging the semantic gap between the encoder and decoder. To address these issues, we propose an FMD-UNet dual-decoder U-Net network for COVID-19 infection segmentation, which integrates a Fine-grained Feature Squeezing (FGFS) decoder and a Multi-scale Dilated Semantic Aggregation (MDSA) decoder. The FGFS decoder produces fine feature maps through the compression of fine-grained features and a weighted attention mechanism, guiding the model to capture detailed semantic information. The MDSA decoder consists of three hierarchical MDSA modules designed for different stages of input information. These modules progressively fuse different scales of dilated convolutions to process the shallow and deep semantic information from the encoder, and use the extracted feature information to bridge the semantic gaps at various stages, this design captures extensive contextual information while decoding and predicting segmentation, thereby suppressing the increase in model parameters. To better validate the robustness and generalizability of the FMD-UNet, we conducted comprehensive performance evaluations and ablation experiments on three public datasets, and achieved leading Dice Similarity Coefficient (DSC) scores of 84.76, 78.56 and 61.99% in COVID-19 infection segmentation, respectively. Compared to previous methods, the FMD-UNet has fewer parameters and shorter inference time, which also demonstrates its competitiveness.
Collapse
Affiliation(s)
- Wenfeng Wang
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, People's Republic of China
| | - Qi Mao
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, People's Republic of China
| | - Yi Tian
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, People's Republic of China
| | - Yan Zhang
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, People's Republic of China
| | - Zhenwu Xiang
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, People's Republic of China
| | - Lijia Ren
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, People's Republic of China
| |
Collapse
|
13
|
Wei D, Jiang Y, Zhou X, Wu D, Feng X. A Review of Advancements and Challenges in Liver Segmentation. J Imaging 2024; 10:202. [PMID: 39194991 DOI: 10.3390/jimaging10080202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2024] [Revised: 08/07/2024] [Accepted: 08/13/2024] [Indexed: 08/29/2024] Open
Abstract
Liver segmentation technologies play vital roles in clinical diagnosis, disease monitoring, and surgical planning due to the complex anatomical structure and physiological functions of the liver. This paper provides a comprehensive review of the developments, challenges, and future directions in liver segmentation technology. We systematically analyzed high-quality research published between 2014 and 2024, focusing on liver segmentation methods, public datasets, and evaluation metrics. This review highlights the transition from manual to semi-automatic and fully automatic segmentation methods, describes the capabilities and limitations of available technologies, and provides future outlooks.
Collapse
Affiliation(s)
- Di Wei
- Department of Radiology, The Eighth Affiliated Hospital of The Sun Yat-sen University, No. 3025, Middle Shennan Road, Shenzhen 518033, China
| | - Yundan Jiang
- Department of Radiology, The Eighth Affiliated Hospital of The Sun Yat-sen University, No. 3025, Middle Shennan Road, Shenzhen 518033, China
| | - Xuhui Zhou
- Department of Radiology, The Eighth Affiliated Hospital of The Sun Yat-sen University, No. 3025, Middle Shennan Road, Shenzhen 518033, China
| | - Di Wu
- Department of Radiology, The Eighth Affiliated Hospital of The Sun Yat-sen University, No. 3025, Middle Shennan Road, Shenzhen 518033, China
| | - Xiaorong Feng
- Department of Radiology, The Eighth Affiliated Hospital of The Sun Yat-sen University, No. 3025, Middle Shennan Road, Shenzhen 518033, China
| |
Collapse
|
14
|
Fan C, Zhu Z, Peng B, Xuan Z, Zhu X. EAAC-Net: An Efficient Adaptive Attention and Convolution Fusion Network for Skin Lesion Segmentation. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01223-6. [PMID: 39147886 DOI: 10.1007/s10278-024-01223-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 07/13/2024] [Accepted: 07/31/2024] [Indexed: 08/17/2024]
Abstract
Accurate segmentation of skin lesions in dermoscopic images is of key importance for quantitative analysis of melanoma. Although existing medical image segmentation methods significantly improve skin lesion segmentation, they still have limitations in extracting local features with global information, do not handle challenging lesions well, and usually have a large number of parameters and high computational complexity. To address these issues, this paper proposes an efficient adaptive attention and convolutional fusion network for skin lesion segmentation (EAAC-Net). We designed two parallel encoders, where the efficient adaptive attention feature extraction module (EAAM) adaptively establishes global spatial dependence and global channel dependence by constructing the adjacency matrix of the directed graph and can adaptively filter out the least relevant tokens at the coarse-grained region level, thus reducing the computational complexity of the self-attention mechanism. The efficient multiscale attention-based convolution module (EMA⋅C) utilizes multiscale attention for cross-space learning of local features extracted from the convolutional layer to enhance the representation of richly detailed local features. In addition, we designed a reverse attention feature fusion module (RAFM) to enhance the effective boundary information gradually. To validate the performance of our proposed network, we compared it with other methods on ISIC 2016, ISIC 2018, and PH2 public datasets, and the experimental results show that EAAC-Net has superior segmentation performance under commonly used evaluation metrics.
Collapse
Affiliation(s)
- Chao Fan
- School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou City, Henan Province, China
- Key Laboratory of Grain Information Processing and Control, Ministry of Education, Zhengzhou City, Henan Province, China
| | - Zhentong Zhu
- School of Information Science and Engineering, Henan University of Technology, Henan Province, Post Code, Zhengzhou City, 450001, China.
| | - Bincheng Peng
- School of Information Science and Engineering, Henan University of Technology, Henan Province, Post Code, Zhengzhou City, 450001, China
| | - Zhihui Xuan
- School of Information Science and Engineering, Henan University of Technology, Henan Province, Post Code, Zhengzhou City, 450001, China
| | - Xinru Zhu
- School of Information Science and Engineering, Henan University of Technology, Henan Province, Post Code, Zhengzhou City, 450001, China
| |
Collapse
|
15
|
Xue M, Li S, Zhu Q. Improving diffuse optical tomography imaging quality using APU-Net: an attention-based physical U-Net model. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:086001. [PMID: 39070721 PMCID: PMC11272096 DOI: 10.1117/1.jbo.29.8.086001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/26/2024] [Revised: 05/28/2024] [Accepted: 07/01/2024] [Indexed: 07/30/2024]
Abstract
Significance Traditional diffuse optical tomography (DOT) reconstructions are hampered by image artifacts arising from factors such as DOT sources being closer to shallow lesions, poor optode-tissue coupling, tissue heterogeneity, and large high-contrast lesions lacking information in deeper regions (known as shadowing effect). Addressing these challenges is crucial for improving the quality of DOT images and obtaining robust lesion diagnosis. Aim We address the limitations of current DOT imaging reconstruction by introducing an attention-based U-Net (APU-Net) model to enhance the image quality of DOT reconstruction, ultimately improving lesion diagnostic accuracy. Approach We designed an APU-Net model incorporating a contextual transformer attention module to enhance DOT reconstruction. The model was trained on simulation and phantom data, focusing on challenges such as artifact-induced distortions and lesion-shadowing effects. The model was then evaluated by the clinical data. Results Transitioning from simulation and phantom data to clinical patients' data, our APU-Net model effectively reduced artifacts with an average artifact contrast decrease of 26.83% and improved image quality. In addition, statistical analyses revealed significant contrast improvements in depth profile with an average contrast increase of 20.28% and 45.31% for the second and third target layers, respectively. These results highlighted the efficacy of our approach in breast cancer diagnosis. Conclusions The APU-Net model improves the image quality of DOT reconstruction by reducing DOT image artifacts and improving the target depth profile.
Collapse
Affiliation(s)
- Minghao Xue
- Washington University in St. Louis, Biomedical Engineering Department, St. Louis, Missouri, United States
| | - Shuying Li
- Boston University, Electrical and Computer Engineering Department, Boston, Massachusetts, United States
| | - Quing Zhu
- Washington University in St. Louis, Biomedical Engineering Department, St. Louis, Missouri, United States
- Washington University in St. Louis, Radiology Department, St. Louis, Missouri, United States
| |
Collapse
|
16
|
Zhu R, Li Q, Ding Z, Liu K, Lin Q, Yu Y, Li Y, Zhou S, Kuang H, Jiang J, Liu T. Bifurcation detection in intravascular optical coherence tomography using vision transformer based deep learning. Phys Med Biol 2024; 69:155009. [PMID: 38981596 DOI: 10.1088/1361-6560/ad611c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2024] [Accepted: 07/09/2024] [Indexed: 07/11/2024]
Abstract
Objective. Bifurcation detection in intravascular optical coherence tomography (IVOCT) images plays a significant role in guiding optimal revascularization strategies for percutaneous coronary intervention (PCI). We propose a bifurcation detection method using vision transformer (ViT) based deep learning in IVOCT.Approach. Instead of relying on lumen segmentation, the proposed method identifies the bifurcation image using a ViT-based classification model and then estimate bifurcation ostium points by a ViT-based landmark detection model.Main results. By processing 8640 clinical images, the Accuracy and F1-score of bifurcation identification by the proposed ViT-based model are 2.54% and 16.08% higher than that of traditional non-deep learning methods, are similar to the best performance of convolutional neural networks (CNNs) based methods, respectively. The ostium distance error of the ViT-based model is 0.305 mm, which is reduced 68.5% compared with the traditional non-deep learning method and reduced 24.81% compared with the best performance of CNNs based methods. The results also show that the proposed ViT-based method achieves the highest success detection rate are 11.3% and 29.2% higher than the non-deep learning method, and 4.6% and 2.5% higher than the best performance of CNNs based methods when the distance section is 0.1 and 0.2 mm, respectively.Significance. The proposed ViT-based method enhances the performance of bifurcation detection of IVOCT images, which maintains a high correlation and consistency between the automatic detection results and the expert manual results. It is of great significance in guiding the selection of PCI treatment strategies.
Collapse
Affiliation(s)
- Rongyang Zhu
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Qingrui Li
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Zhenyang Ding
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Kun Liu
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Qiutong Lin
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Yin Yu
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Yuanyao Li
- Tianjin Institute of Metrological Supervision and Testing, Tianjin 300192, People's Republic of China
| | - Shanshan Zhou
- Department of Cardiology, Chinese PLA General Hospital, Beijing 100853, People's Republic of China
| | - Hao Kuang
- Nanjing Forssmann Medical Technology Co., Nanjing, Jiangsu 210093, People's Republic of China
| | - Junfeng Jiang
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| | - Tiegen Liu
- School of Precision Instruments and Opto-Electronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Tianjin Optical Fiber Sensing Engineering Center, Institute of Optical Fiber Sensing of Tianjin University, Tianjin 300072, People's Republic of China
- Key Laboratory of Opto-electronics Information Technology (Tianjin University), Ministry of Education, Tianjin 300072, People's Republic of China
| |
Collapse
|
17
|
Matloob Abbasi M, Iqbal S, Aurangzeb K, Alhussein M, Khan TM. LMBiS-Net: A lightweight bidirectional skip connection based multipath CNN for retinal blood vessel segmentation. Sci Rep 2024; 14:15219. [PMID: 38956117 PMCID: PMC11219784 DOI: 10.1038/s41598-024-63496-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 05/29/2024] [Indexed: 07/04/2024] Open
Abstract
Blinding eye diseases are often related to changes in retinal structure, which can be detected by analysing retinal blood vessels in fundus images. However, existing techniques struggle to accurately segment these delicate vessels. Although deep learning has shown promise in medical image segmentation, its reliance on specific operations can limit its ability to capture crucial details such as the edges of the vessel. This paper introduces LMBiS-Net, a lightweight convolutional neural network designed for the segmentation of retinal vessels. LMBiS-Net achieves exceptional performance with a remarkably low number of learnable parameters (only 0.172 million). The network used multipath feature extraction blocks and incorporates bidirectional skip connections for the information flow between the encoder and decoder. In addition, we have optimised the efficiency of the model by carefully selecting the number of filters to avoid filter overlap. This optimisation significantly reduces training time and improves computational efficiency. To assess LMBiS-Net's robustness and ability to generalise to unseen data, we conducted comprehensive evaluations on four publicly available datasets: DRIVE, STARE, CHASE_DB1, and HRF The proposed LMBiS-Net achieves significant performance metrics in various datasets. It obtains sensitivity values of 83.60%, 84.37%, 86.05%, and 83.48%, specificity values of 98.83%, 98.77%, 98.96%, and 98.77%, accuracy (acc) scores of 97.08%, 97.69%, 97.75%, and 96.90%, and AUC values of 98.80%, 98.82%, 98.71%, and 88.77% on the DRIVE, STARE, CHEASE_DB, and HRF datasets, respectively. In addition, it records F1 scores of 83.43%, 84.44%, 83.54%, and 78.73% on the same datasets. Our evaluations demonstrate that LMBiS-Net achieves high segmentation accuracy (acc) while exhibiting both robustness and generalisability across various retinal image datasets. This combination of qualities makes LMBiS-Net a promising tool for various clinical applications.
Collapse
Affiliation(s)
- Mufassir Matloob Abbasi
- Department of Electrical Engineering, Abasyn University Islamabad Campus (AUIC), Islamabad, 44000, Pakistan
| | - Shahzaib Iqbal
- Department of Electrical Engineering, Abasyn University Islamabad Campus (AUIC), Islamabad, 44000, Pakistan.
| | - Khursheed Aurangzeb
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, P. O. Box 51178, 11543, Saudi Arabia
| | - Musaed Alhussein
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, P. O. Box 51178, 11543, Saudi Arabia
| | - Tariq M Khan
- School of Computer Science and Engineering, University of New South Wales, Sydney, NSW, Australia
| |
Collapse
|
18
|
Wang J, Qi M, Xiang Z, Tian Y, Tong D. SaraNet: Semantic aggregation reverse attention network for pulmonary nodule segmentation. Comput Biol Med 2024; 177:108674. [PMID: 38815486 DOI: 10.1016/j.compbiomed.2024.108674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2023] [Revised: 04/24/2024] [Accepted: 05/26/2024] [Indexed: 06/01/2024]
Abstract
Accurate segmentation of pulmonary nodule is essential for subsequent pathological analysis and diagnosis. However, current U-Net architectures often rely on a simple skip connection scheme, leading to the fusion of feature maps with different semantic information, which can have a negative impact on the segmentation model. In response to this challenge, this study introduces a novel U-shaped model specifically designed for pulmonary nodule segmentation. The proposed model incorporates features such as the U-Net backbone, semantic aggregation feature pyramid module, and reverse attention module. The semantic aggregation module combines semantic information with multi-scale features, addressing the semantic gap between the encoder and decoder. The reverse attention module explores missing object parts and captures intricate details by erasing the currently predicted salient regions from side-output features. The proposed model is evaluated using the LIDC-IDRI dataset. Experimental results reveal that the proposed method achieves a dice similarity coefficient of 89.11%and a sensitivity of 90.73 %, outperforming state-of-the-art approaches comprehensively.
Collapse
Affiliation(s)
- Jintao Wang
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China
| | - Mao Qi
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China.
| | - Zhenwu Xiang
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China
| | - Yi Tian
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China
| | - Dongbing Tong
- School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China
| |
Collapse
|
19
|
Mao K, Li R, Cheng J, Huang D, Song Z, Liu Z. PL-Net: progressive learning network for medical image segmentation. Front Bioeng Biotechnol 2024; 12:1414605. [PMID: 38994123 PMCID: PMC11236745 DOI: 10.3389/fbioe.2024.1414605] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2024] [Accepted: 05/30/2024] [Indexed: 07/13/2024] Open
Abstract
In recent years, deep convolutional neural network-based segmentation methods have achieved state-of-the-art performance for many medical analysis tasks. However, most of these approaches rely on optimizing the U-Net structure or adding new functional modules, which overlooks the complementation and fusion of coarse-grained and fine-grained semantic information. To address these issues, we propose a 2D medical image segmentation framework called Progressive Learning Network (PL-Net), which comprises Internal Progressive Learning (IPL) and External Progressive Learning (EPL). PL-Net offers the following advantages: 1) IPL divides feature extraction into two steps, allowing for the mixing of different size receptive fields and capturing semantic information from coarse to fine granularity without introducing additional parameters; 2) EPL divides the training process into two stages to optimize parameters and facilitate the fusion of coarse-grained information in the first stage and fine-grained information in the second stage. We conducted comprehensive evaluations of our proposed method on five medical image segmentation datasets, and the experimental results demonstrate that PL-Net achieves competitive segmentation performance. It is worth noting that PL-Net does not introduce any additional learnable parameters compared to other U-Net variants.
Collapse
Affiliation(s)
- Kunpeng Mao
- Chongqing City Management College, Chongqing, China
| | - Ruoyu Li
- College of Computer Science, Sichuan University, Chengdu, China
| | - Junlong Cheng
- College of Computer Science, Sichuan University, Chengdu, China
| | - Danmei Huang
- Chongqing City Management College, Chongqing, China
| | - Zhiping Song
- Chongqing University of Engineering, Chongqing, China
| | - ZeKui Liu
- Chongqing University of Engineering, Chongqing, China
| |
Collapse
|
20
|
Likassa HT, Chen DG, Chen K, Wang Y, Zhu W. Robust PCA with Lw,∗ and L2,1 Norms: A Novel Method for Low-Quality Retinal Image Enhancement. J Imaging 2024; 10:151. [PMID: 39057722 PMCID: PMC11277667 DOI: 10.3390/jimaging10070151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2024] [Revised: 06/12/2024] [Accepted: 06/17/2024] [Indexed: 07/28/2024] Open
Abstract
Nonmydriatic retinal fundus images often suffer from quality issues and artifacts due to ocular or systemic comorbidities, leading to potential inaccuracies in clinical diagnoses. In recent times, deep learning methods have been widely employed to improve retinal image quality. However, these methods often require large datasets and lack robustness in clinical settings. Conversely, the inherent stability and adaptability of traditional unsupervised learning methods, coupled with their reduced reliance on extensive data, render them more suitable for real-world clinical applications, particularly in the limited data context of high noise levels or a significant presence of artifacts. However, existing unsupervised learning methods encounter challenges such as sensitivity to noise and outliers, reliance on assumptions like cluster shapes, and difficulties with scalability and interpretability, particularly when utilized for retinal image enhancement. To tackle these challenges, we propose a novel robust PCA (RPCA) method with low-rank sparse decomposition that also integrates affine transformations τi, weighted nuclear norm, and the L2,1 norms, aiming to overcome existing method limitations and to achieve image quality improvement unseen by these methods. We employ the weighted nuclear norm (Lw,∗) to assign weights to singular values to each retinal images and utilize the L2,1 norm to eliminate correlated samples and outliers in the retinal images. Moreover, τi is employed to enhance retinal image alignment, making the new method more robust to variations, outliers, noise, and image blurring. The Alternating Direction Method of Multipliers (ADMM) method is used to optimally determine parameters, including τi, by solving an optimization problem. Each parameter is addressed separately, harnessing the benefits of ADMM. Our method introduces a novel parameter update approach and significantly improves retinal image quality, detecting cataracts, and diabetic retinopathy. Simulation results confirm our method's superiority over existing state-of-the-art methods across various datasets.
Collapse
Affiliation(s)
- Habte Tadesse Likassa
- Department of Biostatistics, College of Health Solutions, Arizona State University, Phoenix, AZ 85004, USA
| | - Ding-Geng Chen
- Department of Biostatistics, College of Health Solutions, Arizona State University, Phoenix, AZ 85004, USA
- Department of Statistics, University of Pretoria, Pretoria 0028, South Africa
| | - Kewei Chen
- Department of Biostatistics, College of Health Solutions, Arizona State University, Phoenix, AZ 85004, USA
| | - Yalin Wang
- Computer Science and Engineering, School of Computing and Augmented Intelligence, Arizona State University, Phoenix, AZ 85287-8809, USA
| | - Wenhui Zhu
- Computer Science and Engineering, School of Computing and Augmented Intelligence, Arizona State University, Phoenix, AZ 85287-8809, USA
| |
Collapse
|
21
|
Lin H, Zhao M, Zhu L, Pei X, Wu H, Zhang L, Li Y. Gaussian filter facilitated deep learning-based architecture for accurate and efficient liver tumor segmentation for radiation therapy. Front Oncol 2024; 14:1423774. [PMID: 38966060 PMCID: PMC11222586 DOI: 10.3389/fonc.2024.1423774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Accepted: 06/06/2024] [Indexed: 07/06/2024] Open
Abstract
Purpose Addressing the challenges of unclear tumor boundaries and the confusion between cysts and tumors in liver tumor segmentation, this study aims to develop an auto-segmentation method utilizing Gaussian filter with the nnUNet architecture to effectively distinguish between tumors and cysts, enhancing the accuracy of liver tumor auto-segmentation. Methods Firstly, 130 cases of liver tumorsegmentation challenge 2017 (LiTS2017) were used for training and validating nnU-Net-based auto-segmentation model. Then, 14 cases of 3D-IRCADb dataset and 25 liver cancer cases retrospectively collected in our hospital were used for testing. The dice similarity coefficient (DSC) was used to evaluate the accuracy of auto-segmentation model by comparing with manual contours. Results The nnU-Net achieved an average DSC value of 0.86 for validation set (20 LiTS cases) and 0.82 for public testing set (14 3D-IRCADb cases). For clinical testing set, the standalone nnU-Net model achieved an average DSC value of 0.75, which increased to 0.81 after post-processing with the Gaussian filter (P<0.05), demonstrating its effectiveness in mitigating the influence of liver cysts on liver tumor segmentation. Conclusion Experiments show that Gaussian filter is beneficial to improve the accuracy of liver tumor segmentation in clinic.
Collapse
Affiliation(s)
- Hongyu Lin
- Department of Oncology, First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Min Zhao
- Department of Oncology, First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Lingling Zhu
- Department of Oncology, First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Xi Pei
- Technology Development Department, Anhui Wisdom Technology Co., Ltd., Hefei, China
| | - Haotian Wu
- Technology Development Department, Anhui Wisdom Technology Co., Ltd., Hefei, China
| | - Lian Zhang
- Department of Oncology, First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Ying Li
- Department of Oncology, First Hospital of Hebei Medical University, Shijiazhuang, China
| |
Collapse
|
22
|
Liu X, Qu L, Xie Z, Zhao J, Shi Y, Song Z. Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentation. Biomed Eng Online 2024; 23:52. [PMID: 38851691 PMCID: PMC11162022 DOI: 10.1186/s12938-024-01238-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 04/11/2024] [Indexed: 06/10/2024] Open
Abstract
Accurate segmentation of multiple organs in the head, neck, chest, and abdomen from medical images is an essential step in computer-aided diagnosis, surgical navigation, and radiation therapy. In the past few years, with a data-driven feature extraction approach and end-to-end training, automatic deep learning-based multi-organ segmentation methods have far outperformed traditional methods and become a new research topic. This review systematically summarizes the latest research in this field. We searched Google Scholar for papers published from January 1, 2016 to December 31, 2023, using keywords "multi-organ segmentation" and "deep learning", resulting in 327 papers. We followed the PRISMA guidelines for paper selection, and 195 studies were deemed to be within the scope of this review. We summarized the two main aspects involved in multi-organ segmentation: datasets and methods. Regarding datasets, we provided an overview of existing public datasets and conducted an in-depth analysis. Concerning methods, we categorized existing approaches into three major classes: fully supervised, weakly supervised and semi-supervised, based on whether they require complete label information. We summarized the achievements of these methods in terms of segmentation accuracy. In the discussion and conclusion section, we outlined and summarized the current trends in multi-organ segmentation.
Collapse
Affiliation(s)
- Xiaoyu Liu
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, 138 Yixueyuan Road, Shanghai, 200032, People's Republic of China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, 200032, China
| | - Linhao Qu
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, 138 Yixueyuan Road, Shanghai, 200032, People's Republic of China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, 200032, China
| | - Ziyue Xie
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, 138 Yixueyuan Road, Shanghai, 200032, People's Republic of China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, 200032, China
| | - Jiayue Zhao
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, 138 Yixueyuan Road, Shanghai, 200032, People's Republic of China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, 200032, China
| | - Yonghong Shi
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, 138 Yixueyuan Road, Shanghai, 200032, People's Republic of China.
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, 200032, China.
| | - Zhijian Song
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, 138 Yixueyuan Road, Shanghai, 200032, People's Republic of China.
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai, 200032, China.
| |
Collapse
|
23
|
Melkani Y, Pant A, Guo Y, Melkani GC. Automated assessment of cardiac dynamics in aging and dilated cardiomyopathy Drosophila models using machine learning. Commun Biol 2024; 7:702. [PMID: 38849449 PMCID: PMC11161577 DOI: 10.1038/s42003-024-06371-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Accepted: 05/22/2024] [Indexed: 06/09/2024] Open
Abstract
The Drosophila model is pivotal in deciphering the pathophysiological underpinnings of various human ailments, notably aging and cardiovascular diseases. Cutting-edge imaging techniques and physiology yield vast high-resolution videos, demanding advanced analysis methods. Our platform leverages deep learning to segment optical microscopy images of Drosophila hearts, enabling the quantification of cardiac parameters in aging and dilated cardiomyopathy (DCM). Validation using experimental datasets confirms the efficacy of our aging model. We employ two innovative approaches deep-learning video classification and machine-learning based on cardiac parameters to predict fly aging, achieving accuracies of 83.3% (AUC 0.90) and 79.1%, (AUC 0.87) respectively. Moreover, we extend our deep-learning methodology to assess cardiac dysfunction associated with the knock-down of oxoglutarate dehydrogenase (OGDH), revealing its potential in studying DCM. This versatile approach promises accelerated cardiac assays for modeling various human diseases in Drosophila and holds promise for application in animal and human cardiac physiology under diverse conditions.
Collapse
Affiliation(s)
- Yash Melkani
- Department of Pathology, Division of Molecular and Cellular Pathology, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA
- Engineering Physics Department, College of Engineering, University of California, Berkeley, CA, USA
| | - Aniket Pant
- Department of Pathology, Division of Molecular and Cellular Pathology, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA
- Department of Materials Science and Engineering, Georgia Institute of Technology, Atlanta, GA, USA
| | - Yiming Guo
- Department of Pathology, Division of Molecular and Cellular Pathology, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Girish C Melkani
- Department of Pathology, Division of Molecular and Cellular Pathology, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA.
| |
Collapse
|
24
|
Mehdi RR, Kadivar N, Mukherjee T, Mendiola EA, Shah DJ, Karniadakis G, Avazmohammadi R. Multi-Modality Deep Infarct: Non-invasive identification of infarcted myocardium using composite in-silico-human data learning. RESEARCH SQUARE 2024:rs.3.rs-4468678. [PMID: 38883756 PMCID: PMC11177985 DOI: 10.21203/rs.3.rs-4468678/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2024]
Abstract
Myocardial infarction (MI) continues to be a leading cause of death worldwide. The precise quantification of infarcted tissue is crucial to diagnosis, therapeutic management, and post-MI care. Late gadolinium enhancement-cardiac magnetic resonance (LGE-CMR) is regarded as the gold standard for precise infarct tissue localization in MI patients. A fundamental limitation of LGE-CMR is the invasive intravenous introduction of gadolinium-based contrast agents that present potential high-risk toxicity, particularly for individuals with underlying chronic kidney diseases. Herein, we develop a completely non-invasive methodology that identifies the location and extent of an infarct region in the left ventricle via a machine learning (ML) model using only cardiac strains as inputs. In this transformative approach, we demonstrate the remarkable performance of a multi-fidelity ML model that combines rodent-based in-silico-generated training data (low-fidelity) with very limited patient-specific human data (high-fidelity) in predicting LGE ground truth. Our results offer a new paradigm for developing feasible prognostic tools by augmenting synthetic simulation-based data with very small amounts of in-vivo human data. More broadly, the proposed approach can significantly assist with addressing biomedical challenges in healthcare where human data are limited.
Collapse
Affiliation(s)
- Rana Raza Mehdi
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Nikhil Kadivar
- School of Engineering, Brown University, Providence, RI 02912, USA
| | - Tanmay Mukherjee
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Emilio A. Mendiola
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
| | - Dipan J. Shah
- Houston Methodist DeBakey Heart & Vascular Center, Houston, TX 77030, USA
| | - George Karniadakis
- School of Engineering, Brown University, Providence, RI 02912, USA
- Division of Applied Mathematics, Brown University, Providence, RI 02912, USA
| | - Reza Avazmohammadi
- Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA
- School of Engineering Medicine, Texas A&M University, Houston, TX 77030, USA
- J. Mike Walker ‘66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA
- Department of Cardiovascular Sciences, Houston Methodist Research Institute, Houston, TX 77030, USA
| |
Collapse
|
25
|
Jian M, Tao C, Wu R, Zhang H, Li X, Wang R, Wang Y, Peng L, Zhu J. HRU-Net: A high-resolution convolutional neural network for esophageal cancer radiotherapy target segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108177. [PMID: 38648704 DOI: 10.1016/j.cmpb.2024.108177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Revised: 04/10/2024] [Accepted: 04/13/2024] [Indexed: 04/25/2024]
Abstract
BACKGROUND AND OBJECTIVE The effective segmentation of esophageal squamous carcinoma lesions in CT scans is significant for auxiliary diagnosis and treatment. However, accurate lesion segmentation is still a challenging task due to the irregular form of the esophagus and small size, the inconsistency of spatio-temporal structure, and low contrast of esophagus and its peripheral tissues in medical images. The objective of this study is to improve the segmentation effect of esophageal squamous cell carcinoma lesions. METHODS It is critical for a segmentation network to effectively extract 3D discriminative features to distinguish esophageal cancers from some visually closed adjacent esophageal tissues and organs. In this work, an efficient HRU-Net architecture (High-Resolution U-Net) was exploited for esophageal cancer and esophageal carcinoma segmentation in CT slices. Based on the idea of localization first and segmentation later, the HRU-Net locates the esophageal region before segmentation. In addition, an Resolution Fusion Module (RFM) was designed to integrate the information of adjacent resolution feature maps to obtain strong semantic information, as well as preserve the high-resolution features. RESULTS Compared with the other five typical methods, the devised HRU-Net is capable of generating superior segmentation results. CONCLUSIONS Our proposed HRU-NET improves the accuracy of segmentation for squamous esophageal cancer. Compared to other models, our model performs the best. The designed method may improve the efficiency of clinical diagnosis of esophageal squamous cell carcinoma lesions.
Collapse
Affiliation(s)
- Muwei Jian
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China; School of Information Science and Technology, Linyi University, Linyi, China.
| | - Chen Tao
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Ronghua Wu
- School of Information Science and Technology, Linyi University, Linyi, China
| | - Haoran Zhang
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China
| | - Xiaoguang Li
- Faculty of Information Technology, Beijing University of Technology, Beijing, China
| | - Rui Wang
- School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China
| | - Yanlei Wang
- Youth League Committee, Shandong University of Political Science and Law, Jinan, China
| | - Lizhi Peng
- Shandong Provincial Key Laboratory of Network based Intelligent Computing, University of Jinan, Jinan, China
| | - Jian Zhu
- Department of Radiation Oncology Physics and Technology, Shandong Cancer Hospital affiliated to Shandong First Medical University, Jinan, China
| |
Collapse
|
26
|
Wang Z, Liu Z, Yu J, Gao Y, Liu M. Multi-scale nested UNet with transformer for colorectal polyp segmentation. J Appl Clin Med Phys 2024; 25:e14351. [PMID: 38551396 PMCID: PMC11163511 DOI: 10.1002/acm2.14351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 02/13/2024] [Accepted: 02/19/2024] [Indexed: 06/11/2024] Open
Abstract
BACKGROUND Polyp detection and localization are essential tasks for colonoscopy. U-shape network based convolutional neural networks have achieved remarkable segmentation performance for biomedical images, but lack of long-range dependencies modeling limits their receptive fields. PURPOSE Our goal was to develop and test a novel architecture for polyp segmentation, which takes advantage of learning local information with long-range dependencies modeling. METHODS A novel architecture combining with multi-scale nested UNet structure integrated transformer for polyp segmentation was developed. The proposed network takes advantage of both CNN and transformer to extract distinct feature information. The transformer layer is embedded between the encoder and decoder of a U-shape net to learn explicit global context and long-range semantic information. To address the challenging of variant polyp sizes, a MSFF unit was proposed to fuse features with multiple resolution. RESULTS Four public datasets and one in-house dataset were used to train and test the model performance. Ablation study was also conducted to verify each component of the model. For dataset Kvasir-SEG and CVC-ClinicDB, the proposed model achieved mean dice score of 0.942 and 0.950 respectively, which were more accurate than the other methods. To show the generalization of different methods, we processed two cross dataset validations, the proposed model achieved the highest mean dice score. The results demonstrate that the proposed network has powerful learning and generalization capability, significantly improving segmentation accuracy and outperforming state-of-the-art methods. CONCLUSIONS The proposed model produced more accurate polyp segmentation than current methods on four different public and one in-house datasets. Its capability of polyps segmentation in different sizes shows the potential clinical application.
Collapse
Affiliation(s)
- Zenan Wang
- Department of Gastroenterology, Beijing Chaoyang Hospitalthe Third Clinical Medical College of Capital Medical UniversityBeijingChina
| | - Zhen Liu
- Department of Gastroenterology, Beijing Chaoyang Hospitalthe Third Clinical Medical College of Capital Medical UniversityBeijingChina
| | - Jianfeng Yu
- Department of Gastroenterology, Beijing Chaoyang Hospitalthe Third Clinical Medical College of Capital Medical UniversityBeijingChina
| | - Yingxin Gao
- Department of Gastroenterology, Beijing Chaoyang Hospitalthe Third Clinical Medical College of Capital Medical UniversityBeijingChina
| | - Ming Liu
- Hunan Key Laboratory of Nonferrous Resources and Geological Hazard ExplorationChangshaChina
| |
Collapse
|
27
|
Zhang G, Gu W, Wang S, Li Y, Zhao D, Liang T, Gong Z, Ju R. MOTC: Abdominal Multi-objective Segmentation Model with Parallel Fusion of Global and Local Information. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1-16. [PMID: 38347391 PMCID: PMC11169149 DOI: 10.1007/s10278-024-00978-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 11/19/2023] [Accepted: 11/21/2023] [Indexed: 06/13/2024]
Abstract
Convolutional Neural Networks have been widely applied in medical image segmentation. However, the existence of local inductive bias in convolutional operations restricts the modeling of long-term dependencies. The introduction of Transformer enables the modeling of long-term dependencies and partially eliminates the local inductive bias in convolutional operations, thereby improving the accuracy of tasks such as segmentation and classification. Researchers have proposed various hybrid structures combining Transformer and Convolutional Neural Networks. One strategy is to stack Transformer blocks and convolutional blocks to concentrate on eliminating the accumulated local bias of convolutional operations. Another strategy is to nest convolutional blocks and Transformer blocks to eliminate bias within each nested block. However, due to the granularity of bias elimination operations, these two strategies cannot fully exploit the potential of Transformer. In this paper, a parallel hybrid model is proposed for segmentation, which includes a Transformer branch and a Convolutional Neural Network branch in encoder. After parallel feature extraction, inter-layer information fusion and exchange of complementary information are performed between the two branches, simultaneously extracting local and global features while eliminating the local bias generated by convolutional operations within the current layer. A pure convolutional operation is used in decoder to obtain final segmentation results. To validate the impact of the granularity of bias elimination operations on the effectiveness of local bias elimination, the experiments in this paper were conducted on Flare21 dataset and Amos22 dataset. The average Dice coefficient reached 92.65% on Flare21 dataset, and 91.61% on Amos22 dataset, surpassing comparative methods. The experimental results demonstrate that smaller granularity of bias elimination operations leads to better performance.
Collapse
Affiliation(s)
- GuoDong Zhang
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China
| | - WenWen Gu
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China
| | - SuRan Wang
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China
| | - YanLin Li
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China
| | - DaZhe Zhao
- Key Laboratory of Intelligent Computing in Medical Image, Northeastern University, Wenhua Street, Shenyang, 110819, Liaoning Province, China
| | - TingYu Liang
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China
| | - ZhaoXuan Gong
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China
| | - RongHui Ju
- School of Computer, Shenyang Aerospace University, Daoyi South Street, Shenyang, 110135, Liaoning Province, China.
- Department of Radiology, The Peoples Hospital of Liaoning Province, Wenyi Street, Shenyang, 110016, Liaoning Province, China.
| |
Collapse
|
28
|
Lin H, Falahkheirkhah K, Kindratenko V, Bhargava R. INSTRAS: INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation. MACHINE LEARNING WITH APPLICATIONS 2024; 16:100549. [PMID: 39036499 PMCID: PMC11258863 DOI: 10.1016/j.mlwa.2024.100549] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/23/2024] Open
Abstract
Infrared (IR) spectroscopic imaging is of potentially wide use in medical imaging applications due to its ability to capture both chemical and spatial information. This complexity of the data both necessitates using machine intelligence as well as presents an opportunity to harness a high-dimensionality data set that offers far more information than today's manually-interpreted images. While convolutional neural networks (CNNs), including the well-known U-Net model, have demonstrated impressive performance in image segmentation, the inherent locality of convolution limits the effectiveness of these models for encoding IR data, resulting in suboptimal performance. In this work, we propose an INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation (INSTRAS). This novel model leverages the strength of the transformer encoders to segment IR breast images effectively. Incorporating skip-connection and transformer encoders, INSTRAS overcomes the issue of pure convolution models, such as the difficulty of capturing long-range dependencies. To evaluate the performance of our model and existing convolutional models, we conducted training on various encoder-decoder models using a breast dataset of IR images. INSTRAS, utilizing 9 spectral bands for segmentation, achieved a remarkable AUC score of 0.9788, underscoring its superior capabilities compared to purely convolutional models. These experimental results attest to INSTRAS's advanced and improved segmentation abilities for IR imaging.
Collapse
Affiliation(s)
- Hangzheng Lin
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, IL, United States
| | | | - Volodymyr Kindratenko
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, IL, United States
- Center for Artificial Intelligence Innovation, National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, IL, United States
| | - Rohit Bhargava
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, IL, United States
- Beckman Institute, University of Illinois at Urbana-Champaign, IL, United States
- Departments of Bioengineering, Mechanical Science and Engineering and Cancer Center at Illinois, University of Illinois at Urbana-Champaign, IL, United States
| |
Collapse
|
29
|
Yang Y, Yue S, Quan H. CS-UNet: Cross-scale U-Net with Semantic-position dependencies for retinal vessel segmentation. NETWORK (BRISTOL, ENGLAND) 2024; 35:134-153. [PMID: 38050997 DOI: 10.1080/0954898x.2023.2288858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 11/23/2023] [Indexed: 12/07/2023]
Abstract
Accurate retinal vessel segmentation is the prerequisite for early recognition and treatment of retina-related diseases. However, segmenting retinal vessels is still challenging due to the intricate vessel tree in fundus images, which has a significant number of tiny vessels, low contrast, and lesion interference. For this task, the u-shaped architecture (U-Net) has become the de-facto standard and has achieved considerable success. However, U-Net is a pure convolutional network, which usually shows limitations in global modelling. In this paper, we propose a novel Cross-scale U-Net with Semantic-position Dependencies (CS-UNet) for retinal vessel segmentation. In particular, we first designed a Semantic-position Dependencies Aggregator (SPDA) and incorporate it into each layer of the encoder to better focus on global contextual information by integrating the relationship of semantic and position. To endow the model with the capability of cross-scale interaction, the Cross-scale Relation Refine Module (CSRR) is designed to dynamically select the information associated with the vessels, which helps guide the up-sampling operation. Finally, we have evaluated CS-UNet on three public datasets: DRIVE, CHASE_DB1, and STARE. Compared to most existing state-of-the-art methods, CS-UNet demonstrated better performance.
Collapse
Affiliation(s)
- Ying Yang
- College of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Shengbin Yue
- College of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan, China
- Yunnan Provincial Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming, Yunnan, China
| | - Haiyan Quan
- College of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan, China
| |
Collapse
|
30
|
Yang X, Zheng Y, Mei C, Jiang G, Tian B, Wang L. UGLS: an uncertainty guided deep learning strategy for accurate image segmentation. Front Physiol 2024; 15:1362386. [PMID: 38651048 PMCID: PMC11033460 DOI: 10.3389/fphys.2024.1362386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 03/26/2024] [Indexed: 04/25/2024] Open
Abstract
Accurate image segmentation plays a crucial role in computer vision and medical image analysis. In this study, we developed a novel uncertainty guided deep learning strategy (UGLS) to enhance the performance of an existing neural network (i.e., U-Net) in segmenting multiple objects of interest from images with varying modalities. In the developed UGLS, a boundary uncertainty map was introduced for each object based on its coarse segmentation (obtained by the U-Net) and then combined with input images for the fine segmentation of the objects. We validated the developed method by segmenting optic cup (OC) regions from color fundus images and left and right lung regions from Xray images. Experiments on public fundus and Xray image datasets showed that the developed method achieved a average Dice Score (DS) of 0.8791 and a sensitivity (SEN) of 0.8858 for the OC segmentation, and 0.9605, 0.9607, 0.9621, and 0.9668 for the left and right lung segmentation, respectively. Our method significantly improved the segmentation performance of the U-Net, making it comparable or superior to five sophisticated networks (i.e., AU-Net, BiO-Net, AS-Net, Swin-Unet, and TransUNet).
Collapse
Affiliation(s)
- Xiaoguo Yang
- Wenzhou People’s Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou, China
| | - Yanyan Zheng
- Wenzhou People’s Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou, China
| | - Chenyang Mei
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Gaoqiang Jiang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Bihan Tian
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Lei Wang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
31
|
Li G, Xie J, Zhang L, Sun M, Li Z, Sun Y. MCAFNet: multiscale cross-layer attention fusion network for honeycomb lung lesion segmentation. Med Biol Eng Comput 2024; 62:1121-1137. [PMID: 38150110 DOI: 10.1007/s11517-023-02995-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/07/2023] [Indexed: 12/28/2023]
Abstract
Accurate segmentation of honeycomb lung lesions from lung CT images plays a crucial role in the diagnosis and treatment of various lung diseases. However, the availability of algorithms for automatic segmentation of honeycomb lung lesions remains limited. In this study, we propose a novel multi-scale cross-layer attention fusion network (MCAFNet) specifically designed for the segmentation of honeycomb lung lesions, taking into account their shape specificity and similarity to surrounding vascular shadows. The MCAFNet incorporates several key modules to enhance the segmentation performance. Firstly, a multiscale aggregation (MIA) module is introduced in the input part to preserve spatial information during downsampling. Secondly, a cross-layer attention fusion (CAF) module is proposed to capture multiscale features by integrating channel information and spatial information from different layers of the feature maps. Lastly, a bidirectional attention gate (BAG) module is constructed within the skip connection to enhance the model's ability to filter out background information and focus on the segmentation target. Experimental results demonstrate the effectiveness of the proposed MCAFNet. On the honeycomb lung segmentation dataset, the network achieves an Intersection over Union (IoU) of 0.895, mean IoU (mIoU) of 0.921, and mean Dice coefficient (mDice) of 0.949, outperforming existing medical image segmentation algorithms. Furthermore, experiments conducted on additional datasets confirm the generalizability and robustness of the proposed model. The contribution of this study lies in the development of the MCAFNet, which addresses the lack of automated segmentation algorithms for honeycomb lung lesions. The proposed network demonstrates superior performance in accurately segmenting honeycomb lung lesions, thereby facilitating the diagnosis and treatment of lung diseases. This work contributes to the existing literature by presenting a novel approach that effectively combines multi-scale features and attention mechanisms for lung lesion segmentation. The code is available at https://github.com/Oran9er/MCAFNet .
Collapse
Affiliation(s)
- Gang Li
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Jinjie Xie
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Ling Zhang
- Taiyuan University of Technology Software College, Taiyuan, China.
| | - Mengxia Sun
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Zhichao Li
- Taiyuan University of Technology Software College, Taiyuan, China
| | - Yuanjin Sun
- Taiyuan University of Technology Software College, Taiyuan, China
| |
Collapse
|
32
|
Li Z, Zhang N, Gong H, Qiu R, Zhang W. SG-MIAN: Self-guided multiple information aggregation network for image-level weakly supervised skin lesion segmentation. Comput Biol Med 2024; 170:107988. [PMID: 38232452 DOI: 10.1016/j.compbiomed.2024.107988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 12/11/2023] [Accepted: 01/13/2024] [Indexed: 01/19/2024]
Abstract
Nowadays, skin disease is becoming one of the most malignant diseases that threaten people's health. Computer aided diagnosis based on deep learning has become a widely used technology to assist medical professionals in diagnosis, and segmentation of lesion areas is one of the most important steps in it. However, traditional medical image segmentation methods rely on numerous pixel-level labels for fully supervised training, and such labeling process is time-consuming and requires professional competence. In order to reduce the costs of pixel-level labeling, we proposed a method only using image-level label to segment skin lesion areas. Due to the lack of lesion's spatial and intensity information in image-level labels, and the wide distribution range of irregular shape and different texture on skin lesions, the algorithm must pay great attention to the automatic lesion localization and perception of lesion boundary. In this paper, we proposed a Self-Guided Multiple Information Aggregation Network (SG-MIAN). Our backbone network MIAN utilizes the Multiple Spatial Perceptron (MSP) solely using classification information as guidance to discriminate the key classification features of lesion areas, and thereby performing more accurate localization and activation of lesion areas. Additionally, adjunct to MSP, we also proposed an Auxiliary Activation Structure (AAS) and two auxiliary loss functions to further self-guided boundary correction, achieving the goal of accurate boundary activation. To verify the effectiveness of the proposed method, we conducted extensive experiments using the HAM10000 dataset and the PH2dataset, which demonstrated superior performance compared to most existing weakly supervised segmentation methods.
Collapse
Affiliation(s)
- Zhixun Li
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| | - Nan Zhang
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| | - Huiling Gong
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| | - Ruiyun Qiu
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China.
| | - Wei Zhang
- School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China
| |
Collapse
|
33
|
Chen J, Yang G, Liu A, Chen X, Liu J. SFE-Net: Spatial-Frequency Enhancement Network for robust nuclei segmentation in histopathology images. Comput Biol Med 2024; 171:108131. [PMID: 38447498 DOI: 10.1016/j.compbiomed.2024.108131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 01/19/2024] [Accepted: 02/04/2024] [Indexed: 03/08/2024]
Abstract
Morphological features of individual nuclei serve as a dependable foundation for pathologists in making accurate diagnoses. Existing methods that rely on spatial information for feature extraction have achieved commendable results in nuclei segmentation tasks. However, these approaches are not sufficient to extract edge information of nuclei with small sizes and blurred outlines. Moreover, the lack of attention to the interior of the nuclei leads to significant internal inconsistencies. To address these challenges, we introduce a novel Spatial-Frequency Enhancement Network (SFE-Net) to incorporate spatial-frequency features and promote intra-nuclei consistency for robust nuclei segmentation. Specifically, SFE-Net incorporates a distinctive Spatial-Frequency Feature Extraction module and a Spatial-Guided Feature Enhancement module, which are designed to preserve spatial-frequency information and enhance feature representation respectively, to achieve comprehensive extraction of edge information. Furthermore, we introduce the Label-Guided Distillation method, which utilizes semantic features to guide the segmentation network in strengthening boundary constraints and learning the intra-nuclei consistency of individual nuclei, to improve the robustness of nuclei segmentation. Extensive experiments on three publicly available histopathology image datasets (MoNuSeg, TNBC and CryoNuSeg) demonstrate the superiority of our proposed method, which achieves 79.23%, 81.96% and 73.26% Aggregated Jaccard Index, respectively. The proposed model is available at https://github.com/jinshachen/SFE-Net.
Collapse
Affiliation(s)
- Jinsha Chen
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China
| | - Gang Yang
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China
| | - Aiping Liu
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China
| | - Xun Chen
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China
| | - Ji Liu
- School of Biomedical Engineering, Division of Life Sciences and Medicine, School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China; Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China.
| |
Collapse
|
34
|
Prethija G, Katiravan J. EAMR-Net: A multiscale effective spatial and cross-channel attention network for retinal vessel segmentation. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:4742-4761. [PMID: 38549347 DOI: 10.3934/mbe.2024208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/02/2024]
Abstract
Delineation of retinal vessels in fundus images is essential for detecting a range of eye disorders. An automated technique for vessel segmentation can assist clinicians and enhance the efficiency of the diagnostic process. Traditional methods fail to extract multiscale information, discard unnecessary information, and delineate thin vessels. In this paper, a novel residual U-Net architecture that incorporates multi-scale feature learning and effective attention is proposed to delineate the retinal vessels precisely. Since drop block regularization performs better than drop out in preventing overfitting, drop block was used in this study. A multi-scale feature learning module was added instead of a skip connection to learn multi-scale features. A novel effective attention block was proposed and integrated with the decoder block to obtain precise spatial and channel information. Experimental findings indicated that the proposed model exhibited outstanding performance in retinal vessel delineation. The sensitivities achieved for DRIVE, STARE, and CHASE_DB datasets were 0.8293, 0.8151 and 0.8084, respectively.
Collapse
Affiliation(s)
- G Prethija
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai 600127, India
| | - Jeevaa Katiravan
- Department of Information Technology, Velammal Engineering College, Chennai 600066, India
| |
Collapse
|
35
|
Jiao R, Zhang Y, Ding L, Xue B, Zhang J, Cai R, Jin C. Learning with limited annotations: A survey on deep semi-supervised learning for medical image segmentation. Comput Biol Med 2024; 169:107840. [PMID: 38157773 DOI: 10.1016/j.compbiomed.2023.107840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 10/30/2023] [Accepted: 12/07/2023] [Indexed: 01/03/2024]
Abstract
Medical image segmentation is a fundamental and critical step in many image-guided clinical approaches. Recent success of deep learning-based segmentation methods usually relies on a large amount of labeled data, which is particularly difficult and costly to obtain, especially in the medical imaging domain where only experts can provide reliable and accurate annotations. Semi-supervised learning has emerged as an appealing strategy and been widely applied to medical image segmentation tasks to train deep models with limited annotations. In this paper, we present a comprehensive review of recently proposed semi-supervised learning methods for medical image segmentation and summarize both the technical novelties and empirical results. Furthermore, we analyze and discuss the limitations and several unsolved problems of existing approaches. We hope this review can inspire the research community to explore solutions to this challenge and further advance the field of medical image segmentation.
Collapse
Affiliation(s)
- Rushi Jiao
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; School of Engineering Medicine, Beihang University, Beijing, 100191, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
| | - Yichi Zhang
- School of Data Science, Fudan University, Shanghai, 200433, China; Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, 200433, China.
| | - Le Ding
- School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China.
| | - Bingsen Xue
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
| | - Jicong Zhang
- School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China; Hefei Innovation Research Institute, Beihang University, Hefei, 230012, China.
| | - Rong Cai
- School of Engineering Medicine, Beihang University, Beijing, 100191, China; Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, Beihang University, Beijing, 100191, China.
| | - Cheng Jin
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China; Beijing Anding Hospital, Capital Medical University, Beijing, 100088, China.
| |
Collapse
|
36
|
Mahootiha M, Qadir HA, Bergsland J, Balasingham I. Multimodal deep learning for personalized renal cell carcinoma prognosis: Integrating CT imaging and clinical data. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107978. [PMID: 38113804 DOI: 10.1016/j.cmpb.2023.107978] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 12/05/2023] [Accepted: 12/12/2023] [Indexed: 12/21/2023]
Abstract
BACKGROUND AND OBJECTIVE Renal cell carcinoma represents a significant global health challenge with a low survival rate. The aim of this research was to devise a comprehensive deep-learning model capable of predicting survival probabilities in patients with renal cell carcinoma by integrating CT imaging and clinical data and addressing the limitations observed in prior studies. The aim is to facilitate the identification of patients requiring urgent treatment. METHODS The proposed framework comprises three modules: a 3D image feature extractor, clinical variable selection, and survival prediction. Based on the 3D CNN architecture, the feature extractor module predicts the ISUP grade of renal cell carcinoma tumors linked to mortality rates from CT images. Clinical variables are systematically selected using the Spearman score and random forest importance score as criteria. A deep learning-based network, trained with discrete LogisticHazard-based loss, performs the survival prediction. Nine distinct experiments are performed, with varying numbers of clinical variables determined by different thresholds of the Spearman and importance scores. RESULTS Our findings demonstrate that the proposed strategy surpasses the current literature on renal cancer prognosis based on CT scans and clinical factors. The best-performing experiment yielded a concordance index of 0.84 and an area under the curve value of 0.8 on the test cohort, which suggests strong predictive power. CONCLUSIONS The multimodal deep-learning approach developed in this study shows promising results in estimating survival probabilities for renal cell carcinoma patients using CT imaging and clinical data. This may have potential implications in identifying patients who require urgent treatment, potentially improving patient outcomes. The code created for this project is available for the public on: GitHub.
Collapse
Affiliation(s)
- Maryamalsadat Mahootiha
- The Intervention Centre, Oslo University Hospital, Oslo, 0372, Norway; Faculty of Medicine, University of Oslo, Oslo, 0372, Norway.
| | - Hemin Ali Qadir
- The Intervention Centre, Oslo University Hospital, Oslo, 0372, Norway
| | - Jacob Bergsland
- The Intervention Centre, Oslo University Hospital, Oslo, 0372, Norway
| | - Ilangko Balasingham
- The Intervention Centre, Oslo University Hospital, Oslo, 0372, Norway; Department of Electronic Systems, Norwegian University of Science and Technology, Trondheim, Norway
| |
Collapse
|
37
|
Zhang D, Fan X, Kang X, Tian S, Xiao G, Yu L, Wu W. Class key feature extraction and fusion for 2D medical image segmentation. Med Phys 2024; 51:1263-1276. [PMID: 37552522 DOI: 10.1002/mp.16636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 06/28/2023] [Accepted: 07/07/2023] [Indexed: 08/09/2023] Open
Abstract
BACKGROUND The size variation, complex semantic environment and high similarity in medical images often prevent deep learning models from achieving good performance. PURPOSE To overcome these problems and improve the model segmentation performance and generalizability. METHODS We propose the key class feature reconstruction module (KCRM), which ranks channel weights and selects key features (KFs) that contribute more to the segmentation results for each class. Meanwhile, KCRM reconstructs all local features to establish the dependence relationship from local features to KFs. In addition, we propose the spatial gating module (SGM), which employs KFs to generate two spatial maps to suppress irrelevant regions, strengthening the ability to locate semantic objects. Finally, we enable the model to adapt to size variations by diversifying the receptive field. RESULTS We integrate these modules into class key feature extraction and fusion network (CKFFNet) and validate its performance on three public medical datasets: CHAOS, UW-Madison, and ISIC2017. The experimental results show that our method achieves better segmentation results and generalizability than those of mainstream methods. CONCLUSION Through quantitative and qualitative research, the proposed module improves the segmentation results and enhances the model generalizability, making it suitable for application and expansion.
Collapse
Affiliation(s)
- Dezhi Zhang
- Department of Dermatology and Venereology, People's Hospital of Xinjiang Uygur Autonomous Region, Xinjiang Clinical Research Center For Dermatologic Diseases, Xinjiang Key Laboratory of Dermatology Research (XJYS1707), Urmuqi, China
| | - Xin Fan
- College of Software, Xinjiang University, Urmuqi, Xinjiang, China
| | - Xiaojing Kang
- Department of Dermatology and Venereology, People's Hospital of Xinjiang Uygur Autonomous Region, Xinjiang Clinical Research Center For Dermatologic Diseases, Xinjiang Key Laboratory of Dermatology Research (XJYS1707), Urmuqi, China
| | - Shengwei Tian
- College of Software, Xinjiang University, Urmuqi, Xinjiang, China
- Key Laboratory of Software Engineering Technology, College of Software, Xin Jiang University, Urumqi, China
| | - Guangli Xiao
- College of Software, Xinjiang University, Urmuqi, Xinjiang, China
| | - Long Yu
- College of Network Center, Xinjiang University, Urumqi, China
- Signal and Signal Processing Laboratory, College of Information Science and Engineering, Xinjiang University, Urumqi, China
| | - Weidong Wu
- Department of Dermatology and Venereology, People's Hospital of Xinjiang Uygur Autonomous Region, Xinjiang Clinical Research Center For Dermatologic Diseases, Xinjiang Key Laboratory of Dermatology Research (XJYS1707), Urmuqi, China
| |
Collapse
|
38
|
Sun K, Chen Y, Dong F, Wu Q, Geng J, Chen Y. Retinal vessel segmentation method based on RSP-SA Unet network. Med Biol Eng Comput 2024; 62:605-620. [PMID: 37964177 DOI: 10.1007/s11517-023-02960-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 10/28/2023] [Indexed: 11/16/2023]
Abstract
Segmenting retinal vessels plays a significant role in the diagnosis of fundus disorders. However, there are two problems in the retinal vessel segmentation methods. First, fine-grained features of fine blood vessels are difficult to be extracted. Second, it is easy to lose track of the details of blood vessel edges. To solve the problems above, the Residual SimAM Pyramid-Spatial Attention Unet (RSP-SA Unet) is proposed, in which the encoding, decoding, and upsampling layers of the Unet are mainly improved. Firstly, the RSP structure proposed in this paper approximates a residual structure combined with SimAM and Pyramid Segmentation Attention (PSA), which is applied to the encoding and decoding parts to extract multi-scale spatial information and important features across dimensions at a finer level. Secondly, the spatial attention (SA) is used in the upsampling layer to perform multi-attention mapping on the input feature map, which could enhance the segmentation effect of small blood vessels with low contrast. Finally, the RSP-SA Unet is verified on the CHASE_DB1, DRIVE, and STARE datasets, and the segmentation accuracy (ACC) of the RSP-SA Unet could reach 0.9763, 0.9704, and 0.9724, respectively. Area under the ROC curve (AUC) could reach 0.9896, 0.9858, and 0.9906, respectively. The RSP-SA Unet overall performance is better than the comparison methods.
Collapse
Affiliation(s)
- Kun Sun
- The Higher Educational Key Laboratory for Measuring & Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, Harbin, China
- Teaching Demonstration Center for Measurement and Control Technology and Instrumentation, National Experimental, Harbin University of Science and Technology, Harbin, China
| | - Yang Chen
- The Higher Educational Key Laboratory for Measuring & Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, Harbin, China
- Teaching Demonstration Center for Measurement and Control Technology and Instrumentation, National Experimental, Harbin University of Science and Technology, Harbin, China
| | - Fuxuan Dong
- The Higher Educational Key Laboratory for Measuring & Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, Harbin, China
- Teaching Demonstration Center for Measurement and Control Technology and Instrumentation, National Experimental, Harbin University of Science and Technology, Harbin, China
| | - Qing Wu
- The Higher Educational Key Laboratory for Measuring & Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, Harbin, China.
- Teaching Demonstration Center for Measurement and Control Technology and Instrumentation, National Experimental, Harbin University of Science and Technology, Harbin, China.
- Heilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application, Harbin University of Science and Technology, Harbin, China.
| | - Jiameng Geng
- The Higher Educational Key Laboratory for Measuring & Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, Harbin, China
- Teaching Demonstration Center for Measurement and Control Technology and Instrumentation, National Experimental, Harbin University of Science and Technology, Harbin, China
| | - Yinsheng Chen
- The Higher Educational Key Laboratory for Measuring & Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, Harbin, China
- Teaching Demonstration Center for Measurement and Control Technology and Instrumentation, National Experimental, Harbin University of Science and Technology, Harbin, China
| |
Collapse
|
39
|
Li Z, Huang G, Zou B, Chen W, Zhang T, Xu Z, Cai K, Wang T, Sun Y, Wang Y, Jin K, Huang X. Segmentation of Low-Light Optical Coherence Tomography Angiography Images under the Constraints of Vascular Network Topology. SENSORS (BASEL, SWITZERLAND) 2024; 24:774. [PMID: 38339491 PMCID: PMC10856982 DOI: 10.3390/s24030774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 12/27/2023] [Accepted: 01/03/2024] [Indexed: 02/12/2024]
Abstract
Optical coherence tomography angiography (OCTA) offers critical insights into the retinal vascular system, yet its full potential is hindered by challenges in precise image segmentation. Current methodologies struggle with imaging artifacts and clarity issues, particularly under low-light conditions and when using various high-speed CMOS sensors. These challenges are particularly pronounced when diagnosing and classifying diseases such as branch vein occlusion (BVO). To address these issues, we have developed a novel network based on topological structure generation, which transitions from superficial to deep retinal layers to enhance OCTA segmentation accuracy. Our approach not only demonstrates improved performance through qualitative visual comparisons and quantitative metric analyses but also effectively mitigates artifacts caused by low-light OCTA, resulting in reduced noise and enhanced clarity of the images. Furthermore, our system introduces a structured methodology for classifying BVO diseases, bridging a critical gap in this field. The primary aim of these advancements is to elevate the quality of OCTA images and bolster the reliability of their segmentation. Initial evaluations suggest that our method holds promise for establishing robust, fine-grained standards in OCTA vascular segmentation and analysis.
Collapse
Affiliation(s)
- Zhi Li
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
| | - Gaopeng Huang
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
| | - Binfeng Zou
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
| | - Wenhao Chen
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
| | - Tianyun Zhang
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
| | - Zhaoyang Xu
- Department of Paediatrics, University of Cambridge, Cambridge CB2 1TN, UK;
| | - Kunyan Cai
- Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR 999078, China;
| | - Tingyu Wang
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
| | - Yaoqi Sun
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
- Lishui Institute, Hangzhou Dianzi University, Lishui 323000, China
| | - Yaqi Wang
- College of Media Engineering, Communication University of Zhejiang, Hangzhou 310018, China;
| | - Kai Jin
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou 310027, China;
| | - Xingru Huang
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China; (Z.L.); (G.H.); (B.Z.); (W.C.); (T.Z.); (T.W.); (Y.S.)
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London E3 4BL, UK
| |
Collapse
|
40
|
Jiang Y, Chen J, Yan W, Zhang Z, Qiao H, Wang M. MAG-Net : Multi-fusion network with grouped attention for retinal vessel segmentation. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:1938-1958. [PMID: 38454669 DOI: 10.3934/mbe.2024086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/09/2024]
Abstract
Retinal vessel segmentation plays a vital role in the clinical diagnosis of ophthalmic diseases. Despite convolutional neural networks (CNNs) excelling in this task, challenges persist, such as restricted receptive fields and information loss from downsampling. To address these issues, we propose a new multi-fusion network with grouped attention (MAG-Net). First, we introduce a hybrid convolutional fusion module instead of the original encoding block to learn more feature information by expanding the receptive field. Additionally, the grouped attention enhancement module uses high-level features to guide low-level features and facilitates detailed information transmission through skip connections. Finally, the multi-scale feature fusion module aggregates features at different scales, effectively reducing information loss during decoder upsampling. To evaluate the performance of the MAG-Net, we conducted experiments on three widely used retinal datasets: DRIVE, CHASE and STARE. The results demonstrate remarkable segmentation accuracy, specificity and Dice coefficients. Specifically, the MAG-Net achieved segmentation accuracy values of 0.9708, 0.9773 and 0.9743, specificity values of 0.9836, 0.9875 and 0.9906 and Dice coefficients of 0.8576, 0.8069 and 0.8228, respectively. The experimental results demonstrate that our method outperforms existing segmentation methods exhibiting superior performance and segmentation outcomes.
Collapse
Affiliation(s)
- Yun Jiang
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
| | - Jie Chen
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
| | - Wei Yan
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
| | - Zequn Zhang
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
| | - Hao Qiao
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
| | - Meiqi Wang
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
| |
Collapse
|
41
|
Qian L, Wen C, Li Y, Hu Z, Zhou X, Xia X, Kim SH. Multi-scale context UNet-like network with redesigned skip connections for medical image segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 243:107885. [PMID: 37897988 DOI: 10.1016/j.cmpb.2023.107885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Revised: 10/22/2023] [Accepted: 10/24/2023] [Indexed: 10/30/2023]
Abstract
BACKGROUND AND OBJECTIVE Medical image segmentation has garnered significant research attention in the neural network community as a fundamental requirement for developing intelligent medical assistant systems. A series of UNet-like networks with an encoder-decoder architecture have achieved remarkable success in medical image segmentation. Among these networks, UNet2+ (UNet++) and UNet3+ (UNet+++) have introduced redesigned skip connections, dense skip connections, and full-scale skip connections, respectively, surpassing the performance of the original UNet. However, UNet2+ lacks comprehensive information obtained from the entire scale, which hampers its ability to learn organ placement and boundaries. Similarly, due to the limited number of neurons in its structure, UNet3+ fails to effectively segment small objects when trained with a small number of samples. METHOD In this study, we propose UNet_sharp (UNet#), a novel network topology named after the "#" symbol, which combines dense skip connections and full-scale skip connections. In the decoder sub-network, UNet# can effectively integrate feature maps of different scales and capture fine-grained features and coarse-grained semantics from the entire scale. This approach enhances the understanding of organ and lesion positions and enables accurate boundary segmentation. We employ deep supervision for model pruning to accelerate testing and enable mobile device deployment. Additionally, we construct two classification-guided modules to reduce false positives and improve segmentation accuracy. RESULTS Compared to current UNet-like networks, our proposed method achieves the highest Intersection over Union (IoU) values ((92.67±0.96)%, (92.38±1.29)%, (95.36±1.22)%, (74.01±2.03)%) and F1 scores ((91.64±1.86)%, (95.70±2.16)%, (97.34±2.76)%, (84.77±2.65)%) on the semantic segmentation tasks of nuclei, brain tumors, liver, and lung nodules, respectively. CONCLUSIONS The experimental results demonstrate that the reconstructed skip connections in UNet successfully incorporate multi-scale contextual semantic information. Compared to most state-of-the-art medical image segmentation models, our proposed method more accurately locates organs and lesions and precisely segments boundaries.
Collapse
Affiliation(s)
- Ledan Qian
- College of Mathematics and Physics, Wenzhou University, Wenzhou, 325035, Zhejiang, China
| | - Caiyun Wen
- The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, 325035, Zhejiang, China
| | - Yi Li
- College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, 325035, Zhejiang, China
| | - Zhongyi Hu
- Key Laboratory of Intelligent Image Processing and Analysis, Wenzhou, 325035, Zhejiang, China
| | - Xiao Zhou
- Information Technology Center, Wenzhou University, Wenzhou, 325035, Zhejiang, China.
| | - Xiaonyu Xia
- College of Mathematics and Physics, Wenzhou University, Wenzhou, 325035, Zhejiang, China
| | - Soo-Hyung Kim
- College of AI Convergence, Chonnam National University, Gwangju, 61186, Korea
| |
Collapse
|
42
|
Ma Z, Li X. An improved supervised and attention mechanism-based U-Net algorithm for retinal vessel segmentation. Comput Biol Med 2024; 168:107770. [PMID: 38056215 DOI: 10.1016/j.compbiomed.2023.107770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Revised: 11/08/2023] [Accepted: 11/26/2023] [Indexed: 12/08/2023]
Abstract
The segmentation results of retinal blood vessels are crucial for automatically diagnosing ophthalmic diseases such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular diseases. To improve the accuracy of vessel segmentation and better extract information about small vessels and edges, we introduce the U-Net algorithm with a supervised attention mechanism for retinal vessel segmentation. We achieve this by introducing a decoder fusion module (DFM) in the encoding part, effectively combining different convolutional blocks to extract features comprehensively. Additionally, in the decoding part of U-Net, we propose the context squeeze and excitation (CSE) decoding module to enhance important contextual feature information and the detection of tiny blood vessels. For the final output, we introduce the supervised fusion mechanism (SFM), which combines multiple branches from shallow to deep layers, effectively fusing multi-scale features and capturing information from different levels, fully integrating low-level and high-level features to improve segmentation performance. Our experimental results on the public datasets of DRIVE, STARE, and CHASED_B1 demonstrate the excellent performance of our proposed network.
Collapse
Affiliation(s)
- Zhendi Ma
- School of Computer Science and Technology, Zhejiang Normal University, Jinhua 321004, China
| | - Xiaobo Li
- School of Computer Science and Technology, Zhejiang Normal University, Jinhua 321004, China.
| |
Collapse
|
43
|
Kato S, Hotta K. Expanded tube attention for tubular structure segmentation. Int J Comput Assist Radiol Surg 2023:10.1007/s11548-023-03038-2. [PMID: 38112883 DOI: 10.1007/s11548-023-03038-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Accepted: 11/13/2023] [Indexed: 12/21/2023]
Abstract
PURPOSE Semantic segmentation of tubular structures, such as blood vessels and cell membranes, is a very difficult task, and it tends to break many predicted regions in the middle. This problem is due to the fact that tubular ground truth is very thin, and the number of pixels is extremely unbalanced compared to the background. METHODS We present a novel training method using pseudo-labels generated by morphological transformation. Furthermore, we present an attention module using thickened pseudo-labels, called the expanded tube attention (ETA) module. By using the ETA module, the network learns thickened regions based on pseudo-labels at first and then gradually learns thinned original regions while transferring information in the thickened regions as an attention map. RESULTS Through experiments conducted on retina vessel image datasets using various evaluation measures, we confirmed that the proposed method using ETA modules improved the clDice metric accuracy in comparison with the conventional methods. CONCLUSIONS We demonstrated that the proposed novel expanded tube attention module using thickened pseudo-labels can achieve easy-to-hard learning.
Collapse
Affiliation(s)
- Sota Kato
- Department of Electrical, Information, Materials and Materials Engineering, Meijo University, Tempaku-ku, Nagoya, Aichi, 468-8502, Japan.
| | - Kazuhiro Hotta
- Department of Electrical and Electronic Engineering, Meijo University, Tempaku-ku, Nagoya, Aichi, Japan
| |
Collapse
|
44
|
Liu R, Pu W, Nan H, Zou Y. Retina image segmentation using the three-path Unet model. Sci Rep 2023; 13:22579. [PMID: 38114637 PMCID: PMC10730848 DOI: 10.1038/s41598-023-50141-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 12/15/2023] [Indexed: 12/21/2023] Open
Abstract
Unsupervised image segmentation is a technique that divides an image into distinct regions or objects without prior labeling. This approach offers flexibility and adaptability to various types of image data. Particularly for large datasets, it eliminates the need for manual labeling, thereby it presents advantages in terms of time and labor costs. However, when applied to retinal image segmentation, challenges arise due to variations in data, presence of noise, and manual threshold adjustments, which can lead to over-segmentation or under-segmentation of small blood vessel boundaries and endpoints. In order to enhance the precision and accuracy of retinal image segmentation, we propose a novel image supervised segmentation network based on three-path Unet model.Firstly, the Haar wavelet transform is employed to extract high-frequency image information, which forms the foundation for the proposed HaarNet, a Unet-inspired architecture. Next, the HaarNet is integrated with the Unet and SegNet frameworks to develop a three-path Unet model, referred to as TP-Unet. Finally, the model is further refined into TP-Unet+AE+DSL by incorporating the advantages of auto-encoding (AE) and deep supervised learning (DSL) techniques, thereby enhancing the overall performance of the system. To evaluate the effectiveness of our proposed model, we conduct experiments using the DRIVE and CHASE public datasets. On the DRIVE dataset, our recommended model achieves a Dice coefficient of 0.8291 and a sensitivity index of 0.8184. These results significantly outperform the Unet model by [Formula: see text] and [Formula: see text], respectively. Furthermore, our model demonstrates excellent performance on the CHASE dataset, with a Dice coefficient of 0.8162, a sensitivity of 0.8242, and an accuracy of 0.9664. These metrics surpass the Unet model by [Formula: see text], [Formula: see text], and [Formula: see text], respectively. Our proposed model provides more accurate and reliable results for retinal vessel segmentation, which holds significant potential for assisting doctors in their diagnosis.
Collapse
Affiliation(s)
- Ruihua Liu
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China.
| | - Wei Pu
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China
- Chongqing Vocational College of Transportation, Chongqing, China
| | - Haoyu Nan
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China.
- OPT Machine Vision Tech Co., Ltd., Guangdong, China.
| | - Yangyang Zou
- School of Artificial Intelligence, Chongqing University of Technology, Chongqing, China
| |
Collapse
|
45
|
Zhou J, Hou Z, Lu H, Wang W, Zhao W, Wang Z, Zheng D, Wang S, Tang W, Qu X. A deep supervised transformer U-shaped full-resolution residual network for the segmentation of breast ultrasound image. Med Phys 2023; 50:7513-7524. [PMID: 37816131 DOI: 10.1002/mp.16765] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 09/05/2023] [Accepted: 09/07/2023] [Indexed: 10/12/2023] Open
Abstract
PURPOSE Breast ultrasound (BUS) is an important breast imaging tool. Automatic BUS image segmentation can measure the breast tumor size objectively and reduce doctors' workload. In this article, we proposed a deep supervised transformer U-shaped full-resolution residual network (DSTransUFRRN) to segment BUS images. METHODS In the proposed method, a full-resolution residual stream and a deep supervision mechanism were introduced into TransU-Net. The residual stream can keep full resolution features from different levels and enhance features fusion. Then, the deep supervision can suppress gradient dispersion. Moreover, the transformer module can suppress irrelevant features and improve feature extraction process. Two datasets (dataset A and B) were used for training and evaluation. The dataset A included 980 BUS image samples and the dataset B had 163 BUS image samples. RESULTS Cross-validation was conducted. For the dataset A, the proposed DSTransUFRRN achieved significantly higher Dice (91.04 ± 0.86%) than all compared methods (p < 0.05). For the dataset B, the Dice was lower than that for the dataset A due to the small number of samples, but the Dice of DSTransUFRRN (88.15% ± 2.11%) was significantly higher than that of other compared methods (p < 0.05). CONCLUSIONS In this study, we proposed DSTransUFRRN for BUS image segmentation. The proposed methods achieved significantly higher accuracy than the compared previous methods.
Collapse
Affiliation(s)
- Jiale Zhou
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Zuoxun Hou
- Beijing Institute of Mechanics & Electricity, Beijing, China
| | - Hongyan Lu
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Wenhan Wang
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Wanchen Zhao
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Zenan Wang
- Department of Gastroenterology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Dezhi Zheng
- Research Institute for Frontier Science, Beihang University, Beijing, China
| | - Shuai Wang
- School of Computer Science and Engineering, Beihang University, Beijing, China
| | - Wenzhong Tang
- School of Computer Science and Engineering, Beihang University, Beijing, China
| | - Xiaolei Qu
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| |
Collapse
|
46
|
Joskowicz L, Szeskin A, Rochman S, Dodi A, Lederman R, Fruchtman-Brot H, Azraq Y, Sosna J. Follow-up of liver metastases: a comparison of deep learning and RECIST 1.1. Eur Radiol 2023; 33:9320-9327. [PMID: 37480549 DOI: 10.1007/s00330-023-09926-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2023] [Revised: 04/25/2023] [Accepted: 05/14/2023] [Indexed: 07/24/2023]
Abstract
OBJECTIVES To compare liver metastases changes in CT assessed by radiologists using RECIST 1.1 and with aided simultaneous deep learning-based volumetric lesion changes analysis. METHODS A total of 86 abdominal CT studies from 43 patients (prior and current scans) of abdominal CT scans of patients with 1041 liver metastases (mean = 12.1, std = 11.9, range 1-49) were analyzed. Two radiologists performed readings of all pairs; conventional with RECIST 1.1 and with computer-aided assessment. For computer-aided reading, we used a novel simultaneous multi-channel 3D R2U-Net classifier trained and validated on other scans. The reference was established by having an expert radiologist validate the computed lesion detection and segmentation. The results were then verified and modified as needed by another independent radiologist. The primary outcome measure was the disease status assessment with the conventional and the computer-aided readings with respect to the reference. RESULTS For conventional and computer-aided reading, there was a difference in disease status classification in 40 out of 86 (46.51%) and 10 out of 86 (27.9%) CT studies with respect to the reference, respectively. Computer-aided reading improved conventional reading in 30 CT studies by 34.5% for two readers (23.2% and 46.51%) with respect to the reference standard. The main reason for the difference between the two readings was lesion volume differences (p = 0.01). CONCLUSIONS AI-based computer-aided analysis of liver metastases may improve the accuracy of the evaluation of neoplastic liver disease status. CLINICAL RELEVANCE STATEMENT AI may aid radiologists to improve the accuracy of evaluating changes over time in metastasis of the liver. KEY POINTS • Classification of liver metastasis changes improved significantly in one-third of the cases with an automatically generated comprehensive lesion and lesion changes report. • Simultaneous deep learning changes detection and volumetric assessment may improve the evaluation of liver metastases temporal changes potentially improving disease management.
Collapse
Affiliation(s)
- Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Adi Szeskin
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Shalom Rochman
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Aviv Dodi
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Richard Lederman
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel
| | - Hila Fruchtman-Brot
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel
| | - Yusef Azraq
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel
| | - Jacob Sosna
- Dept of Radiology, Hadassah Medical Center, Faculty of Medicine, The Hebrew University of Jerusalem, POB 12000, 91120, Jerusalem, Israel.
| |
Collapse
|
47
|
Soleimani P, Farezi N. Utilizing deep learning via the 3D U-net neural network for the delineation of brain stroke lesions in MRI image. Sci Rep 2023; 13:19808. [PMID: 37957203 PMCID: PMC10643611 DOI: 10.1038/s41598-023-47107-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2023] [Accepted: 11/09/2023] [Indexed: 11/15/2023] Open
Abstract
The segmentation of acute stroke lesions plays a vital role in healthcare by assisting doctors in making prompt and well-informed treatment choices. Although Magnetic Resonance Imaging (MRI) is a time-intensive procedure, it produces high-fidelity images widely regarded as the most reliable diagnostic tool available. Employing deep learning techniques for automated stroke lesion segmentation can offer valuable insights into the precise location and extent of affected tissue, enabling medical professionals to effectively evaluate treatment risks and make informed assessments. In this research, a deep learning approach is introduced for segmenting acute and sub-acute stroke lesions from MRI images. To enhance feature learning through brain hemisphere symmetry, pre-processing techniques are applied to the data. To tackle the class imbalance challenge, we employed a strategy of using small patches with balanced sampling during training, along with a dynamically weighted loss function that incorporates f1-score and IOU-score (Intersection over Union). Furthermore, the 3D U-Net architecture is used to generate predictions for complete patches, employing a high degree of overlap between patches to minimize the requirement for subsequent post-processing steps. The 3D U-Net model, utilizing ResnetV2 as the pre-trained encoder for IOU-score and Seresnext101 for f1-score, stands as the leading state-of-the-art (SOTA) model for segmentation tasks. However, recent research has introduced a novel model that surpasses these metrics and demonstrates superior performance compared to other backbone architectures. The f1-score and IOU-score were computed for various backbones, with Seresnext101 achieving the highest f1-score and ResnetV2 performing the highest IOU-score. These calculations were conducted using a threshold value of 0.5. This research proposes a valuable model based on transfer learning for the classification of brain diseases in MRI scans. The achieved f1-score using the recommended classifiers demonstrates the effectiveness of the approach employed in this study. The findings indicate that Seresnext101 attains the highest f1-score of 0.94226, while ResnetV2 achieves the best IOU-score of 0.88342, making it the preferred architecture for segmentation methods. Furthermore, the study presents experimental results of the 3D U-Net model applied to brain stroke lesion segmentation, suggesting prospects for researchers interested in segmenting brain strokes and enhancing 3D U-Net models.
Collapse
Affiliation(s)
- Parisa Soleimani
- Faculty of Physics, University of Tabriz, Tabriz, Iran.
- Department of Engineering Sciences, Faculty of Advanced Technologies, University of Mohaghegh Ardabili, Namin, Iran.
| | - Navid Farezi
- Faculty of Physics, University of Tabriz, Tabriz, Iran
| |
Collapse
|
48
|
Mu N, Lyu Z, Rezaeitaleshmahalleh M, Bonifas C, Gosnell J, Haw M, Vettukattil J, Jiang J. S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications. Front Physiol 2023; 14:1209659. [PMID: 38028762 PMCID: PMC10653444 DOI: 10.3389/fphys.2023.1209659] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 09/25/2023] [Indexed: 12/01/2023] Open
Abstract
With the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-level features are learned, and the receptive field size of the network increases, which results in unsatisfactory performance for detecting low-level small/thin structures such as atrial walls and small arteries. To address this issue, we propose to keep the different encoding layer features at their original sizes to constrain the receptive field from increasing as the network goes deeper. Accordingly, we develop a novel S-shaped multiple cross-aggregation segmentation architecture named S-Net, which has two branches in the encoding stage, i.e., a resampling branch to capture low-level fine-grained details and thin/small structures and a downsampling branch to learn high-level discriminative knowledge. In particular, these two branches learn complementary features by residual cross-aggregation; the fusion of the complementary features from different decoding layers can be effectively accomplished through lateral connections. Meanwhile, we perform supervised prediction at all decoding layers to incorporate coarse-level features with high semantic meaning and fine-level features with high localization capability to detect multi-scale structures, especially for small/thin volumes fully. To validate the effectiveness of our S-Net, we conducted extensive experiments on the segmentation of cardiac wall and intracranial aneurysm (IA) vasculature, and quantitative and qualitative evaluations demonstrated the superior performance of our method for predicting small/thin structures in medical images.
Collapse
Affiliation(s)
- Nan Mu
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United States
- Center for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United States
| | - Zonghan Lyu
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United States
- Center for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United States
| | - Mostafa Rezaeitaleshmahalleh
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United States
- Center for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United States
| | - Cassie Bonifas
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United States
- Center for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United States
| | - Jordan Gosnell
- Betz Congenital Health Center, Helen DeVos Children’s Hospital, Grand Rapids, MI, United States
| | - Marcus Haw
- Betz Congenital Health Center, Helen DeVos Children’s Hospital, Grand Rapids, MI, United States
| | - Joseph Vettukattil
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United States
- Betz Congenital Health Center, Helen DeVos Children’s Hospital, Grand Rapids, MI, United States
| | - Jingfeng Jiang
- Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United States
- Center for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United States
| |
Collapse
|
49
|
Pang S, Du A, Orgun MA, Wang Y, Sheng QZ, Wang S, Huang X, Yu Z. Beyond CNNs: Exploiting Further Inherent Symmetries in Medical Image Segmentation. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:6776-6787. [PMID: 36044511 DOI: 10.1109/tcyb.2022.3195447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Automatic tumor or lesion segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because, although the human visual system can detect symmetries in 2-D images effectively, regular CNNs can only exploit translation invariance, overlooking further inherent symmetries existing in medical images, such as rotations and reflections. To solve this problem, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning more precise representations. First, kernel-based equivariant operations are devised on each orientation, which allows it to effectively address the gaps of learning symmetries in existing approaches. Then, to keep segmentation networks globally equivariant, we design distinctive group layers with layer-wise symmetry constraints. Finally, based on our novel framework, extensive experiments conducted on real-world clinical data demonstrate that a group equivariant Res-UNet (called GER-UNet) outperforms its regular CNN-based counterpart and the state-of-the-art segmentation methods in the tasks of hepatic tumor segmentation, COVID-19 lung infection segmentation, and retinal vessel detection. More importantly, the newly built GER-UNet also shows potential in reducing the sample complexity and the redundancy of filters, upgrading current segmentation CNNs, and delineating organs on other medical imaging modalities.
Collapse
|
50
|
Gudhe NR, Kosma VM, Behravan H, Mannermaa A. Nuclei instance segmentation from histopathology images using Bayesian dropout based deep learning. BMC Med Imaging 2023; 23:162. [PMID: 37858043 PMCID: PMC10585914 DOI: 10.1186/s12880-023-01121-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 10/05/2023] [Indexed: 10/21/2023] Open
Abstract
BACKGROUND The deterministic deep learning models have achieved state-of-the-art performance in various medical image analysis tasks, including nuclei segmentation from histopathology images. The deterministic models focus on improving the model prediction accuracy without assessing the confidence in the predictions. METHODS We propose a semantic segmentation model using Bayesian representation to segment nuclei from the histopathology images and to further quantify the epistemic uncertainty. We employ Bayesian approximation with Monte-Carlo (MC) dropout during the inference time to estimate the model's prediction uncertainty. RESULTS We evaluate the performance of the proposed approach on the PanNuke dataset, which consists of 312 visual fields from 19 organ types. We compare the nuclei segmentation accuracy of our approach with that of a fully convolutional neural network, U-Net, SegNet, and the state-of-the-art Hover-net. We use F1-score and intersection over union (IoU) as the evaluation metrics. The proposed approach achieves a mean F1-score of 0.893 ± 0.008 and an IoU value of 0.868 ± 0.003 on the test set of the PanNuke dataset. These results outperform the Hover-net, which has a mean F1-score of 0.871 ± 0.010 and an IoU value of 0.840 ± 0.032. CONCLUSIONS The proposed approach, which incorporates Bayesian representation and Monte-Carlo dropout, demonstrates superior performance in segmenting nuclei from histopathology images compared to existing models such as U-Net, SegNet, and Hover-net. By considering the epistemic uncertainty, our model provides a more reliable estimation of the prediction confidence. These findings highlight the potential of Bayesian deep learning for improving medical image analysis tasks and can contribute to the development of more accurate and reliable computer-aided diagnostic systems.
Collapse
Affiliation(s)
- Naga Raju Gudhe
- Institute of Clinical Medicine, Pathology and Forensic Medicine, Multidisciplinary Cancer research community RC Cancer, University of Eastern Finland, P.O. Box 1627, Kuopio, 70211, Finland.
| | - Veli-Matti Kosma
- Institute of Clinical Medicine, Pathology and Forensic Medicine, Multidisciplinary Cancer research community RC Cancer, University of Eastern Finland, P.O. Box 1627, Kuopio, 70211, Finland
- Biobank of Eastern Finland, Kuopio University Hospital, Kuopio, Finland
| | - Hamid Behravan
- Institute of Clinical Medicine, Pathology and Forensic Medicine, Multidisciplinary Cancer research community RC Cancer, University of Eastern Finland, P.O. Box 1627, Kuopio, 70211, Finland
| | - Arto Mannermaa
- Institute of Clinical Medicine, Pathology and Forensic Medicine, Multidisciplinary Cancer research community RC Cancer, University of Eastern Finland, P.O. Box 1627, Kuopio, 70211, Finland
- Biobank of Eastern Finland, Kuopio University Hospital, Kuopio, Finland
| |
Collapse
|