1
|
Li W, Ye X, Chen X, Jiang X, Yang Y. A deep learning-based method for the detection and segmentation of breast masses in ultrasound images. Phys Med Biol 2024; 69:155027. [PMID: 38986480 DOI: 10.1088/1361-6560/ad61b6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 07/10/2024] [Indexed: 07/12/2024]
Abstract
Objective.Automated detection and segmentation of breast masses in ultrasound images are critical for breast cancer diagnosis, but remain challenging due to limited image quality and complex breast tissues. This study aims to develop a deep learning-based method that enables accurate breast mass detection and segmentation in ultrasound images.Approach.A novel convolutional neural network-based framework that combines the You Only Look Once (YOLO) v5 network and the Global-Local (GOLO) strategy was developed. First, YOLOv5 was applied to locate the mass regions of interest (ROIs). Second, a Global Local-Connected Multi-Scale Selection (GOLO-CMSS) network was developed to segment the masses. The GOLO-CMSS operated on both the entire images globally and mass ROIs locally, and then integrated the two branches for a final segmentation output. Particularly, in global branch, CMSS applied Multi-Scale Selection (MSS) modules to automatically adjust the receptive fields, and Multi-Input (MLI) modules to enable fusion of shallow and deep features at different resolutions. The USTC dataset containing 28 477 breast ultrasound images was collected for training and test. The proposed method was also tested on three public datasets, UDIAT, BUSI and TUH. The segmentation performance of GOLO-CMSS was compared with other networks and three experienced radiologists.Main results.YOLOv5 outperformed other detection models with average precisions of 99.41%, 95.15%, 93.69% and 96.42% on the USTC, UDIAT, BUSI and TUH datasets, respectively. The proposed GOLO-CMSS showed superior segmentation performance over other state-of-the-art networks, with Dice similarity coefficients (DSCs) of 93.19%, 88.56%, 87.58% and 90.37% on the USTC, UDIAT, BUSI and TUH datasets, respectively. The mean DSC between GOLO-CMSS and each radiologist was significantly better than that between radiologists (p< 0.001).Significance.Our proposed method can accurately detect and segment breast masses with a decent performance comparable to radiologists, highlighting its great potential for clinical implementation in breast ultrasound examination.
Collapse
Affiliation(s)
- Wanqing Li
- Department of Engineering and Applied Physics, University of Science and Technology of China, Hefei, Anhui 230026, People's Republic of China
| | - Xianjun Ye
- Department of Ultrasound Medicine, The First Affiliate Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui 230001, People's Republic of China
| | - Xuemin Chen
- Health Management Center, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui 230001, People's Republic of China
| | - Xianxian Jiang
- Graduate School of Bengbu Medical College, Bengbu, Anhui 233030, People's Republic of China
| | - Yidong Yang
- Department of Engineering and Applied Physics, University of Science and Technology of China, Hefei, Anhui 230026, People's Republic of China
- Ion Medical Research Institute, University of Science and Technology of China, Hefei, Anhui 230026, People's Republic of China
- Department of Radiation Oncology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui 230001, People's Republic of China
| |
Collapse
|
2
|
Wu R, Lu X, Yao Z, Ma Y. MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation. Comput Biol Med 2024; 177:108616. [PMID: 38795419 DOI: 10.1016/j.compbiomed.2024.108616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 03/21/2024] [Accepted: 05/11/2024] [Indexed: 05/28/2024]
Abstract
Breast tumor segmentation in ultrasound images is fundamental for quantitative analysis and plays a crucial role in the diagnosis and treatment of breast cancer. Recently, existing methods have mainly focused on spatial domain implementations, with less attention to the frequency domain. In this paper, we propose a Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network (MFMSNet). Specifically, we utilize Octave convolutions instead of conventional convolutions to effectively separate high-frequency and low-frequency components while reducing computational complexity. Introducing the Multi-frequency Transformer block (MF-Trans) enables efficient interaction between high-frequency and low-frequency information, thereby capturing long-range dependencies. Additionally, we incorporate Multi-scale interactive fusion module (MSIF) to merge high-frequency feature maps of different sizes, enhancing the emphasis on tumor edges by integrating local contextual information. Experimental results demonstrate the superiority of our MFMSNet over seven state-of-the-art methods on two publicly available breast ultrasound datasets and one thyroid ultrasound dataset. In the evaluation of MFMSNet, tests were conducted on the BUSI, BUI, and DDTI datasets, comprising 130 images (BUSI), 47 images (BUI), and 128 images (DDTI) in the respective test sets. Employing a five-fold cross-validation approach, the obtained dice coefficients are as follows: 83.42 % (BUSI), 90.79 % (BUI), and 79.96 % (DDTI). The code is available at https://github.com/wrc990616/MFMSNet.
Collapse
Affiliation(s)
- Ruichao Wu
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Xiangyu Lu
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Zihuan Yao
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Yide Ma
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China.
| |
Collapse
|
3
|
Sun S, Fu C, Xu S, Wen Y, Ma T. GLFNet: Global-local fusion network for the segmentation in ultrasound images. Comput Biol Med 2024; 171:108103. [PMID: 38335822 DOI: 10.1016/j.compbiomed.2024.108103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 01/27/2024] [Accepted: 02/04/2024] [Indexed: 02/12/2024]
Abstract
Ultrasound imaging, as a portable and radiation-free modality, presents challenges for accurate segmentation due to the variability of lesions and the similar intensity values of surrounding tissues. Current deep learning approaches leverage convolution for extracting local features and self-attention for handling global dependencies. However, traditional CNNs are spatially local, and Vision Transformers lack image specific bias and are computationally demanding. In response, we propose the Global-Local Fusion Network (GLFNet), a hybrid structure addressing the limitations of both CNNs and Vision Transformers. The GLFNet, featuring Global-Local Fusion Blocks (GLFBlocks), integrates global semantic information with local details to improve segmentation. Each GLFBlock comprises Global and Local Branches for feature extraction in parallel. Within the Global and Local Branches, we introduce the Self-Attention Convolution Fusion Block (SACFBlock), which includes a Spatial-Attention Module and Channel-Attention Module. Experimental results show that our proposed GLFNet surpasses its counterparts in the segmentation tasks, achieving the overall best results with an mIoU of 79.58% and Dice coefficient of 74.62% in the DDTI dataset, an mIoU of 76.61% and Dice coefficient of 71.04% in the BUSI dataset, and an mIoU of 86.77% and Dice coefficient of 87.38% in the BUID dataset. The fusion of local and global features contributes to enhanced performance, making GLFNet a promising approach for ultrasound image segmentation.
Collapse
Affiliation(s)
- Shiyao Sun
- School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
| | - Chong Fu
- School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang 110819, China; Engineering Research Center of Security Technology of Complex Network System, Ministry of Education, China.
| | - Sen Xu
- General Hospital of Northern Theatre Command, Shenyang 110016, China
| | - Yingyou Wen
- School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China; Medical Imaging Research Department, Neusoft Research of Intelligent Healthcare Technology, Co. Ltd., Shenyang, China
| | - Tao Ma
- Dopamine Group Ltd., Auckland, 1542, New Zealand
| |
Collapse
|
4
|
Zhu C, Chai X, Xiao Y, Liu X, Zhang R, Yang Z, Wang Z. Swin-Net: A Swin-Transformer-Based Network Combing with Multi-Scale Features for Segmentation of Breast Tumor Ultrasound Images. Diagnostics (Basel) 2024; 14:269. [PMID: 38337784 PMCID: PMC10854866 DOI: 10.3390/diagnostics14030269] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 01/19/2024] [Accepted: 01/22/2024] [Indexed: 02/12/2024] Open
Abstract
Breast cancer is one of the most common cancers in the world, especially among women. Breast tumor segmentation is a key step in the identification and localization of the breast tumor region, which has important clinical significance. Inspired by the swin-transformer model with powerful global modeling ability, we propose a semantic segmentation framework named Swin-Net for breast ultrasound images, which combines Transformer and Convolutional Neural Networks (CNNs) to effectively improve the accuracy of breast ultrasound segmentation. Firstly, our model utilizes a swin-transformer encoder with stronger learning ability, which can extract features of images more precisely. In addition, two new modules are introduced in our method, including the feature refinement and enhancement module (RLM) and the hierarchical multi-scale feature fusion module (HFM), given that the influence of ultrasonic image acquisition methods and the characteristics of tumor lesions is difficult to capture. Among them, the RLM module is used to further refine and enhance the feature map learned by the transformer encoder. The HFM module is used to process multi-scale high-level semantic features and low-level details, so as to achieve effective cross-layer feature fusion, suppress noise, and improve model segmentation performance. Experimental results show that Swin-Net performs significantly better than the most advanced methods on the two public benchmark datasets. In particular, it achieves an absolute improvement of 1.4-1.8% on Dice. Additionally, we provide a new dataset of breast ultrasound images on which we test the effect of our model, further demonstrating the validity of our method. In summary, the proposed Swin-Net framework makes significant advancements in breast ultrasound image segmentation, providing valuable exploration for research and applications in this domain.
Collapse
Affiliation(s)
- Chengzhang Zhu
- School of Humanities, Central South University, Changsha 410012, China;
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Xian Chai
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Yalong Xiao
- School of Humanities, Central South University, Changsha 410012, China;
| | - Xu Liu
- Department of Medical Ultrasound, Hunan Cancer Hospital/The Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha 410031, China;
| | - Renmao Zhang
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Zhangzheng Yang
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Zhiyuan Wang
- Department of Medical Ultrasound, Hunan Cancer Hospital/The Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha 410031, China;
| |
Collapse
|
5
|
Pramanik P, Pramanik R, Schwenker F, Sarkar R. DBU-Net: Dual branch U-Net for tumor segmentation in breast ultrasound images. PLoS One 2023; 18:e0293615. [PMID: 37930947 PMCID: PMC10627442 DOI: 10.1371/journal.pone.0293615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Accepted: 10/16/2023] [Indexed: 11/08/2023] Open
Abstract
Breast ultrasound medical images often have low imaging quality along with unclear target boundaries. These issues make it challenging for physicians to accurately identify and outline tumors when diagnosing patients. Since precise segmentation is crucial for diagnosis, there is a strong need for an automated method to enhance the segmentation accuracy, which can serve as a technical aid in diagnosis. Recently, the U-Net and its variants have shown great success in medical image segmentation. In this study, drawing inspiration from the U-Net concept, we propose a new variant of the U-Net architecture, called DBU-Net, for tumor segmentation in breast ultrasound images. To enhance the feature extraction capabilities of the encoder, we introduce a novel approach involving the utilization of two distinct encoding paths. In the first path, the original image is employed, while in the second path, we use an image created using the Roberts edge filter, in which edges are highlighted. This dual branch encoding strategy helps to extract the semantic rich information through a mutually informative learning process. At each level of the encoder, both branches independently undergo two convolutional layers followed by a pooling layer. To facilitate cross learning between the branches, a weighted addition scheme is implemented. These weights are dynamically learned by considering the gradient with respect to the loss function. We evaluate the performance of our proposed DBU-Net model on two datasets, namely BUSI and UDIAT, and our experimental results demonstrate superior performance compared to state-of-the-art models.
Collapse
Affiliation(s)
- Payel Pramanik
- Department of Computer Science and Engineering, Jadavpur University, Kolkata, India
| | - Rishav Pramanik
- Department of Computer Science and Engineering, Jadavpur University, Kolkata, India
| | | | - Ram Sarkar
- Department of Computer Science and Engineering, Jadavpur University, Kolkata, India
| |
Collapse
|
6
|
Jiang J, Jiang X, Xu L, Zhang Y, Zheng Y, Kong D. Noise-robustness test for ultrasound breast nodule neural network models as medical devices. Front Oncol 2023; 13:1177225. [PMID: 37427110 PMCID: PMC10325648 DOI: 10.3389/fonc.2023.1177225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 06/05/2023] [Indexed: 07/11/2023] Open
Abstract
Background Deep learning technology has been widely applied to medical image analysis. But due to the limitations of its own imaging principle, ultrasound image has the disadvantages of low resolution and high Speckle Noise density, which not only hinder the diagnosis of patients' conditions but also affect the extraction of ultrasound image features by computer technology. Objective In this study, we investigate the robustness of deep convolutional neural network (CNN) for classification, segmentation, and target detection of breast ultrasound image through random Salt & Pepper Noise and Gaussian Noise. Methods We trained and validated 9 CNN architectures in 8617 breast ultrasound images, but tested the models with noisy test set. Then, we trained and validated 9 CNN architectures with different levels of noise in these breast ultrasound images, and tested the models with noisy test set. Diseases of each breast ultrasound image in our dataset were annotated and voted by three sonographers based on their malignancy suspiciousness. we use evaluation indexes to evaluate the robustness of the neural network algorithm respectively. Results There is a moderate to high impact (The accuracy of the model decreased by about 5%-40%) on model accuracy when Salt and Pepper Noise, Speckle Noise, or Gaussian Noise is introduced to the images respectively. Consequently, DenseNet, UNet++ and Yolov5 were selected as the most robust model based on the selected index. When any two of these three kinds of noise are introduced into the image at the same time, the accuracy of the model will be greatly affected. Conclusions Our experimental results reveal new insights: The variation trend of accuracy with the noise level in Each network used for classification tasks and object detection tasks has some unique characteristics. This finding provides us with a method to reveal the black-box architecture of computer-aided diagnosis (CAD) systems. On the other hand, the purpose of this study is to explore the impact of adding noise directly to the image on the performance of neural networks, which is different from the existing articles on robustness in the field of medical image processing. Consequently, it provides a new way to evaluate the robustness of CAD systems in the future.
Collapse
Affiliation(s)
- Jiaxin Jiang
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Xiaoya Jiang
- Department of Mathematics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR, China
| | - Lei Xu
- Zhejiang Demetics Medical Technology Co., Ltd, Hangzhou, China
| | - Yan Zhang
- Department of Interventional Therapy, The Hwa Mei Hospital of the University of Chinese Academy of Sciences, Ningbo, China
| | - Yuwen Zheng
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Dexing Kong
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| |
Collapse
|
7
|
An imbalance-aware nuclei segmentation methodology for H&E stained histopathology images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2023]
|
8
|
Ahmed MR, Ashrafi AF, Ahmed RU, Shatabda S, Islam AKMM, Islam S. DoubleU-NetPlus: a novel attention and context-guided dual U-Net with multi-scale residual feature fusion network for semantic segmentation of medical images. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08493-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/28/2023]
|
9
|
Ma Z, Qi Y, Xu C, Zhao W, Lou M, Wang Y, Ma Y. ATFE-Net: Axial Transformer and Feature Enhancement-based CNN for ultrasound breast mass segmentation. Comput Biol Med 2023; 153:106533. [PMID: 36638617 DOI: 10.1016/j.compbiomed.2022.106533] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 11/25/2022] [Accepted: 12/31/2022] [Indexed: 01/05/2023]
Abstract
Breast mass is one of the main clinical symptoms of breast cancer. Recently, many CNN-based methods for breast mass segmentation have been proposed. However, these methods have difficulties in capturing long-range dependencies, causing poor segmentation of large-scale breast masses. In this paper, we propose an axial Transformer and feature enhancement-based CNN (ATFE-Net) for ultrasound breast mass segmentation. Specially, an axial Transformer (Axial-Trans) module and a Transformer-based feature enhancement (Trans-FE) module are proposed to capture long-range dependencies. Axial-Trans module only calculates self-attention in width and height directions of input feature maps, which reduces the complexity of self-attention significantly from O(n2) to O(n). In addition, Trans-FE module can enhance feature representation by capturing dependencies between different feature layers, since deeper feature layers have richer semantic information and shallower feature layers have more detailed information. The experimental results show that our ATFE-Net achieved better performance than several state-of-the-art methods on two publicly available breast ultrasound datasets, with Dice coefficient of 82.46% for BUSI and 86.78% for UDIAT, respectively.
Collapse
Affiliation(s)
- Zhou Ma
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Yunliang Qi
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Chunbo Xu
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Wei Zhao
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Meng Lou
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Yiming Wang
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China
| | - Yide Ma
- School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China.
| |
Collapse
|
10
|
Woon Cho S, Rae Baek N, Ryoung Park K. Deep Learning-based Multi-stage Segmentation Method Using Ultrasound Images for Breast Cancer Diagnosis. JOURNAL OF KING SAUD UNIVERSITY - COMPUTER AND INFORMATION SCIENCES 2022. [DOI: 10.1016/j.jksuci.2022.10.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|