1
|
Qian Z, Wang Z, Zhang X, Wei B, Lai M, Shou J, Fan Y, Xu Y. MSNSegNet: attention-based multi-shape nuclei instance segmentation in histopathology images. Med Biol Eng Comput 2024; 62:1821-1836. [PMID: 38401007 DOI: 10.1007/s11517-024-03050-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 02/13/2024] [Indexed: 02/26/2024]
Abstract
In clinical research, the segmentation of irregularly shaped nuclei, particularly in mesenchymal areas like fibroblasts, is crucial yet often neglected. These irregular nuclei are significant for assessing tissue repair in immunotherapy, a process involving neovascularization and fibroblast proliferation. Proper segmentation of these nuclei is vital for evaluating immunotherapy's efficacy, as it provides insights into pathological features. However, the challenge lies in the pronounced curvature variations of these non-convex nuclei, making their segmentation more difficult than that of regular nuclei. In this work, we introduce an undefined task to segment nuclei with both regular and irregular morphology, namely multi-shape nuclei segmentation. We propose a proposal-based method to perform multi-shape nuclei segmentation. By leveraging the two-stage structure of the proposal-based method, a powerful refinement module with high computational costs can be selectively deployed only in local regions, improving segmentation accuracy without compromising computational efficiency. We introduce a novel self-attention module to refine features in proposals for the sake of effectiveness and efficiency in the second stage. The self-attention module improves segmentation performance by capturing long-range dependencies to assist in distinguishing the foreground from the background. In this process, similar features get high attention weights while dissimilar ones get low attention weights. In the first stage, we introduce a residual attention module and a semantic-aware module to accurately predict candidate proposals. The two modules capture more interpretable features and introduce additional supervision through semantic-aware loss. In addition, we construct a dataset with a proportion of non-convex nuclei compared with existing nuclei datasets, namely the multi-shape nuclei (MsN) dataset. Our MSNSegNet method demonstrates notable improvements across various metrics compared to the second-highest-scoring methods. For all nuclei, the D i c e score improved by approximately 1.66 % , A J I by about 2.15 % , and D i c e obj by roughly 0.65 % . For non-convex nuclei, which are crucial in clinical applications, our method's A J I improved significantly by approximately 3.86 % and D i c e obj by around 2.54 % . These enhancements underscore the effectiveness of our approach on multi-shape nuclei segmentation, particularly in challenging scenarios involving irregularly shaped nuclei.
Collapse
Affiliation(s)
- Ziniu Qian
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Zihua Wang
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Xin Zhang
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Bingzheng Wei
- Xiaomi Corporation, Haidian District, Beijing, 100085, Beijing, China
| | - Maode Lai
- Department of Pathology, School of Medicine, Zhejiang Provincial Key Laboratory of Disease Proteomics and Alibaba-Zhejiang University Joint Research Center of Future Digital Healthcare, Zhejiang University, Hangzhou, 310027, Zhejiang, China
| | - Jianzhong Shou
- Department of Urology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Changyang District, Beijing, 100021, Beijing, China
| | - Yubo Fan
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Yan Xu
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China.
| |
Collapse
|
2
|
A scale and region-enhanced decoding network for nuclei classification in histology image. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104626] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
|
3
|
Ehsaeyan E. An efficient image segmentation method based on expectation maximization and Salp swarm algorithm. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-31. [PMID: 37362643 PMCID: PMC10061417 DOI: 10.1007/s11042-023-15149-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/06/2022] [Revised: 11/14/2022] [Accepted: 03/13/2023] [Indexed: 06/28/2023]
Abstract
Multilevel image thresholding using Expectation Maximization (EM) is an efficient method for image segmentation. However, it has two weaknesses: 1) EM is a greedy algorithm and cannot jump out of local optima. 2) it cannot guarantee the number of required classes while estimating the histogram by Gaussian Mixture Models (GMM). in this paper, to overcome these shortages, a novel thresholding approach by combining EM and Salp Swarm Algorithm (SSA) is developed. SSA suggests potential points to the EM algorithm to fly to a better position. Moreover, a new mechanism is considered to maintain the number of desired clusters. Twenty-four medical test images are selected and examined by standard metrics such as PSNR and FSIM. The proposed method is compared with the traditional EM algorithm, and an average improvement of 5.27% in PSNR values and 2.01% in FSIM values were recorded. Also, the proposed approach is compared with four existing segmentation techniques by using CT scan images that Qatar University has collected. Experimental results depict that the proposed method obtains the first rank in terms of PSNR and the second rank in terms of FSIM. It has been observed that the proposed technique performs better performance in the segmentation result compared to other considered state-of-the-art methods.
Collapse
Affiliation(s)
- Ehsan Ehsaeyan
- Electrical Engineering Department, Sirjan University of Technology, Sirjan, Iran
| |
Collapse
|
4
|
Guo R, Xie K, Pagnucco M, Song Y. SAC-Net: Learning with weak and noisy labels in histopathology image segmentation. Med Image Anal 2023; 86:102790. [PMID: 36878159 DOI: 10.1016/j.media.2023.102790] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2022] [Revised: 11/24/2022] [Accepted: 02/23/2023] [Indexed: 03/06/2023]
Abstract
Deep convolutional neural networks have been highly effective in segmentation tasks. However, segmentation becomes more difficult when training images include many complex instances to segment, such as the task of nuclei segmentation in histopathology images. Weakly supervised learning can reduce the need for large-scale, high-quality ground truth annotations by involving non-expert annotators or algorithms to generate supervision information for segmentation. However, there is still a significant performance gap between weakly supervised learning and fully supervised learning approaches. In this work, we propose a weakly-supervised nuclei segmentation method in a two-stage training manner that only requires annotation of the nuclear centroids. First, we generate boundary and superpixel-based masks as pseudo ground truth labels to train our SAC-Net, which is a segmentation network enhanced by a constraint network and an attention network to effectively address the problems caused by noisy labels. Then, we refine the pseudo labels at the pixel level based on Confident Learning to train the network again. Our method shows highly competitive performance of cell nuclei segmentation in histopathology images on three public datasets. Code will be available at: https://github.com/RuoyuGuo/MaskGA_Net.
Collapse
Affiliation(s)
- Ruoyu Guo
- School of Computer Science and Engineering, University of New South Wales, Australia
| | - Kunzi Xie
- School of Computer Science and Engineering, University of New South Wales, Australia
| | - Maurice Pagnucco
- School of Computer Science and Engineering, University of New South Wales, Australia
| | - Yang Song
- School of Computer Science and Engineering, University of New South Wales, Australia.
| |
Collapse
|
5
|
Nasir ES, Parvaiz A, Fraz MM. Nuclei and glands instance segmentation in histology images: a narrative review. Artif Intell Rev 2022. [DOI: 10.1007/s10462-022-10372-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
6
|
Yu Y, Tao Y, Guan H, Xiao S, Li F, Yu C, Liu Z, Li J. A multi-branch hierarchical attention network for medical target segmentation. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.104021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|