1
|
Bobowicz M, Badocha M, Gwozdziewicz K, Rygusik M, Kalinowska P, Szurowska E, Dziubich T. Segmentation-based BI-RADS ensemble classification of breast tumours in ultrasound images. Int J Med Inform 2024; 189:105522. [PMID: 38852288 DOI: 10.1016/j.ijmedinf.2024.105522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Revised: 05/19/2024] [Accepted: 06/05/2024] [Indexed: 06/11/2024]
Abstract
BACKGROUND The development of computer-aided diagnosis systems in breast cancer imaging is exponential. Since 2016, 81 papers have described the automated segmentation of breast lesions in ultrasound images using artificial intelligence. However, only two papers have dealt with complex BI-RADS classifications. PURPOSE This study addresses the automatic classification of breast lesions into binary classes (benign vs. malignant) and multiple BI-RADS classes based on a single ultrasonographic image. Achieving this task should reduce the subjectivity of an individual operator's assessment. MATERIALS AND METHODS Automatic image segmentation methods (PraNet, CaraNet and FCBFormer) adapted to the specific segmentation task were investigated using the U-Net model as a reference. A new classification method was developed using an ensemble of selected segmentation approaches. All experiments were performed on publicly available BUS B, OASBUD, BUSI and private datasets. RESULTS FCBFormer achieved the best outcomes for the segmentation task with intersection over union metric values of 0.81, 0.80 and 0.73 and Dice values of 0.89, 0.87 and 0.82, respectively, for the BUS B, BUSI and OASBUD datasets. Through a series of experiments, we determined that adding an extra 30-pixel margin to the segmentation mask counteracts the potential errors introduced by the segmentation algorithm. An assembly of the full image classifier, bounding box classifier and masked image classifier was the most accurate for binary classification and had the best accuracy (ACC; 0.908), F1 (0.846) and area under the receiver operating characteristics curve (AUROC; 0.871) in the BUS B and ACC (0.982), F1 (0.984) and AUROC (0.998) in the UCC BUS datasets, outperforming each classifier used separately. It was also the most effective for BI-RADS classification, with ACC of 0.953, F1 of 0.920 and AUROC of 0.986 in UCC BUS. Hard voting was the most effective method for dichotomous classification. For the multi-class BI-RADS classification, the soft voting approach was employed. CONCLUSIONS The proposed new classification approach with an ensemble of segmentation and classification approaches proved more accurate than most published results for binary and multi-class BI-RADS classifications.
Collapse
Affiliation(s)
- Maciej Bobowicz
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Mikołaj Badocha
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Katarzyna Gwozdziewicz
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Marlena Rygusik
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Paulina Kalinowska
- Department of Thoracic Radiology, Karolinska University Hospital, Anna Steckséns g 41, Solna 17176, Sweden.
| | - Edyta Szurowska
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Tomasz Dziubich
- Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, 11/12 G. Narutowicza Str., Gdańsk 80-233, Poland.
| |
Collapse
|
2
|
Liu Z, Lv Q, Lee CH, Shen L. Segmenting medical images with limited data. Neural Netw 2024; 177:106367. [PMID: 38754215 DOI: 10.1016/j.neunet.2024.106367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 05/03/2024] [Accepted: 05/03/2024] [Indexed: 05/18/2024]
Abstract
While computer vision has proven valuable for medical image segmentation, its application faces challenges such as limited dataset sizes and the complexity of effectively leveraging unlabeled images. To address these challenges, we present a novel semi-supervised, consistency-based approach termed the data-efficient medical segmenter (DEMS). The DEMS features an encoder-decoder architecture and incorporates the developed online automatic augmenter (OAA) and residual robustness enhancement (RRE) blocks. The OAA augments input data with various image transformations, thereby diversifying the dataset to improve the generalization ability. The RRE enriches feature diversity and introduces perturbations to create varied inputs for different decoders, thereby providing enhanced variability. Moreover, we introduce a sensitive loss to further enhance consistency across different decoders and stabilize the training process. Extensive experimental results on both our own and three public datasets affirm the effectiveness of DEMS. Under extreme data shortage scenarios, our DEMS achieves 16.85% and 10.37% improvement in dice score compared with the U-Net and top-performed state-of-the-art method, respectively. Given its superior data efficiency, DEMS could present significant advancements in medical segmentation under small data regimes. The project homepage can be accessed at https://github.com/NUS-Tim/DEMS.
Collapse
Affiliation(s)
- Zhaoshan Liu
- Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117575, Singapore.
| | - Qiujie Lv
- Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117575, Singapore; School of Intelligent Systems Engineering, Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, 518107, China.
| | - Chau Hung Lee
- Department of Radiology, Tan Tock Seng Hospital, 11 Jalan Tan Tock Seng, Singapore, 308433, Singapore.
| | - Lei Shen
- Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117575, Singapore.
| |
Collapse
|
3
|
Jiménez-Gaona Y, Álvarez MJR, Castillo-Malla D, García-Jaen S, Carrión-Figueroa D, Corral-Domínguez P, Lakshminarayanan V. BraNet: a mobil application for breast image classification based on deep learning algorithms. Med Biol Eng Comput 2024; 62:2737-2756. [PMID: 38693328 PMCID: PMC11330402 DOI: 10.1007/s11517-024-03084-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Accepted: 03/26/2024] [Indexed: 05/03/2024]
Abstract
Mobile health apps are widely used for breast cancer detection using artificial intelligence algorithms, providing radiologists with second opinions and reducing false diagnoses. This study aims to develop an open-source mobile app named "BraNet" for 2D breast imaging segmentation and classification using deep learning algorithms. During the phase off-line, an SNGAN model was previously trained for synthetic image generation, and subsequently, these images were used to pre-trained SAM and ResNet18 segmentation and classification models. During phase online, the BraNet app was developed using the react native framework, offering a modular deep-learning pipeline for mammography (DM) and ultrasound (US) breast imaging classification. This application operates on a client-server architecture and was implemented in Python for iOS and Android devices. Then, two diagnostic radiologists were given a reading test of 290 total original RoI images to assign the perceived breast tissue type. The reader's agreement was assessed using the kappa coefficient. The BraNet App Mobil exhibited the highest accuracy in benign and malignant US images (94.7%/93.6%) classification compared to DM during training I (80.9%/76.9%) and training II (73.7/72.3%). The information contrasts with radiological experts' accuracy, with DM classification being 29%, concerning US 70% for both readers, because they achieved a higher accuracy in US ROI classification than DM images. The kappa value indicates a fair agreement (0.3) for DM images and moderate agreement (0.4) for US images in both readers. It means that not only the amount of data is essential in training deep learning algorithms. Also, it is vital to consider the variety of abnormalities, especially in the mammography data, where several BI-RADS categories are present (microcalcifications, nodules, mass, asymmetry, and dense breasts) and can affect the API accuracy model.
Collapse
Affiliation(s)
- Yuliana Jiménez-Gaona
- Departamento de Química y Ciencias Exactas, Universidad Técnica Particular de Loja, San Cayetano Alto s/n CP1101608, Loja, Ecuador.
- Instituto de Instrumentación para la Imagen Molecular I3M, Universitat Politécnica de Valencia, 46022, Valencia, Spain.
- Theoretical and Experimental Epistemology Lab, School of Opto ΩN2L3G1, Waterloo, Canada.
| | - María José Rodríguez Álvarez
- Instituto de Instrumentación para la Imagen Molecular I3M, Universitat Politécnica de Valencia, 46022, Valencia, Spain
| | - Darwin Castillo-Malla
- Departamento de Química y Ciencias Exactas, Universidad Técnica Particular de Loja, San Cayetano Alto s/n CP1101608, Loja, Ecuador
- Instituto de Instrumentación para la Imagen Molecular I3M, Universitat Politécnica de Valencia, 46022, Valencia, Spain
- Theoretical and Experimental Epistemology Lab, School of Opto ΩN2L3G1, Waterloo, Canada
| | - Santiago García-Jaen
- Departamento de Química y Ciencias Exactas, Universidad Técnica Particular de Loja, San Cayetano Alto s/n CP1101608, Loja, Ecuador
| | | | - Patricio Corral-Domínguez
- Corporación Médica Monte Sinaí-CIPAM (Centro Integral de Patología Mamaria) Cuenca-Ecuador, Facultad de Ciencias Médicas, Universidad de Cuenca, Cuenca, 010203, Ecuador
| | - Vasudevan Lakshminarayanan
- Department of Systems Design Engineering, Physics, and Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, N2L3G1, Canada
| |
Collapse
|
4
|
Magnuska ZA, Roy R, Palmowski M, Kohlen M, Winkler BS, Pfeil T, Boor P, Schulz V, Krauss K, Stickeler E, Kiessling F. Combining Radiomics and Autoencoders to Distinguish Benign and Malignant Breast Tumors on US Images. Radiology 2024; 312:e232554. [PMID: 39254446 DOI: 10.1148/radiol.232554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/11/2024]
Abstract
Background US is clinically established for breast imaging, but its diagnostic performance depends on operator experience. Computer-assisted (real-time) image analysis may help in overcoming this limitation. Purpose To develop precise real-time-capable US-based breast tumor categorization by combining classic radiomics and autoencoder-based features from automatically localized lesions. Materials and Methods A total of 1619 B-mode US images of breast tumors were retrospectively analyzed between April 2018 and January 2024. nnU-Net was trained for lesion segmentation. Features were extracted from tumor segments, bounding boxes, and whole images using either classic radiomics, autoencoder, or both. Feature selection was performed to generate radiomics signatures, which were used to train machine learning algorithms for tumor categorization. Models were evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity and were statistically compared with histopathologically or follow-up-confirmed diagnosis. Results The model was developed on 1191 (mean age, 61 years ± 14 [SD]) female patients and externally validated on 50 (mean age, 55 years ± 15]). The development data set was divided into two parts: testing and training lesion segmentation (419 and 179 examinations) and lesion categorization (503 and 90 examinations). nnU-Net demonstrated precision and reproducibility in lesion segmentation in test set of data set 1 (median Dice score [DS]: 0.90 [IQR, 0.84-0.93]; P = .01) and data set 2 (median DS: 0.89 [IQR, 0.80-0.92]; P = .001). The best model, trained with 23 mixed features from tumor bounding boxes, achieved an AUC of 0.90 (95% CI: 0.83, 0.97), sensitivity of 81% (46 of 57; 95% CI: 70, 91), and specificity of 87% (39 of 45; 95% CI: 77, 87). No evidence of difference was found between model and human readers (AUC = 0.90 [95% CI: 0.83, 0.97] vs 0.83 [95% CI: 0.76, 0.90]; P = .55 and 0.90 vs 0.82 [95% CI: 0.75, 0.90]; P = .45) in tumor classification or between model and histopathologically or follow-up-confirmed diagnosis (AUC = 0.90 [95% CI: 0.83, 0.97] vs 1.00 [95% CI: 1.00,1.00]; P = .10). Conclusion Precise real-time US-based breast tumor categorization was developed by mixing classic radiomics and autoencoder-based features from tumor bounding boxes. ClinicalTrials.gov identifier: NCT04976257 Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Bahl in this issue.
Collapse
Affiliation(s)
- Zuzanna Anna Magnuska
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Rijo Roy
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Moritz Palmowski
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Matthias Kohlen
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Brigitte Sophia Winkler
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Tatjana Pfeil
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Peter Boor
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Volkmar Schulz
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Katja Krauss
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Elmar Stickeler
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| | - Fabian Kiessling
- From the Institute for Experimental Molecular Imaging (Z.A.M., R.R., M.P., V.S., F.K.), Institute of Pathology (P.B.), and Department of Obstetrics and Gynecology (M.K., B.S.W., T.P., K.K., E.S.), University Clinic Aachen, RWTH Aachen University, Forckenbeckstrasse 55, 52074 Aachen, Germany; Physics Institute III B, RWTH Aachen University, Aachen, Germany (V.S.); Comprehensive Diagnostic Center Aachen, Uniklinik RWTH Aachen, Aachen, Germany (P.B., V.S., E.S., F.K.); and Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany (P.B., V.S., F.K.)
| |
Collapse
|
5
|
Zhang L, Wu Z, Yu X, Lyu Y, Wu Z, Dai H, Zhao L, Wang L, Li G, Wang X, Liu T, Zhu D. Learning lifespan brain anatomical correspondence via cortical developmental continuity transfer. Med Image Anal 2024; 99:103328. [PMID: 39243599 DOI: 10.1016/j.media.2024.103328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 07/04/2024] [Accepted: 08/26/2024] [Indexed: 09/09/2024]
Abstract
Identifying anatomical correspondences in the human brain throughout the lifespan is an essential prerequisite for studying brain development and aging. But given the tremendous individual variability in cortical folding patterns, the heterogeneity of different neurodevelopmental stages, and the scarce of neuroimaging data, it is difficult to infer reliable lifespan anatomical correspondence at finer scales. To solve this problem, in this work, we take the advantage of the developmental continuity of the cerebral cortex and propose a novel transfer learning strategy: the model is trained from scratch using the age group with the largest sample size, and then is transferred and adapted to the other groups following the cortical developmental trajectory. A novel loss function is designed to ensure that during the transfer process the common patterns will be extracted and preserved, while the group-specific new patterns will be captured. The proposed framework was evaluated using multiple datasets covering four lifespan age groups with 1,000+ brains (from 34 gestational weeks to young adult). Our experimental results show that: 1) the proposed transfer strategy can dramatically improve the model performance on populations (e.g., early neurodevelopment) with very limited number of training samples; and 2) with the transfer learning we are able to robustly infer the complicated many-to-many anatomical correspondences among different brains at different neurodevelopmental stages. (Code will be released soon: https://github.com/qidianzl/CDC-transfer).
Collapse
Affiliation(s)
- Lu Zhang
- Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, 76019, USA
| | - Zhengwang Wu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Xiaowei Yu
- Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, 76019, USA
| | - Yanjun Lyu
- Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, 76019, USA
| | - Zihao Wu
- Department of Computer Science, University of Georgia, Athens, GA, 30602, USA
| | - Haixing Dai
- Department of Computer Science, University of Georgia, Athens, GA, 30602, USA
| | - Lin Zhao
- Department of Computer Science, University of Georgia, Athens, GA, 30602, USA
| | - Li Wang
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Gang Li
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Xianqiao Wang
- College of Engineering, University of Georgia, Athens, GA, 30602, USA
| | - Tianming Liu
- Department of Computer Science, University of Georgia, Athens, GA, 30602, USA
| | - Dajiang Zhu
- Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX, 76019, USA.
| |
Collapse
|
6
|
Cao W, Guo J, You X, Liu Y, Li L, Cui W, Cao Y, Chen X, Zheng J. NeighborNet: Learning Intra- and Inter-Image Pixel Neighbor Representation for Breast Lesion Segmentation. IEEE J Biomed Health Inform 2024; 28:4761-4771. [PMID: 38743530 DOI: 10.1109/jbhi.2024.3400802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
Breast lesion segmentation from ultrasound images is essential in computer-aided breast cancer diagnosis. To alleviate the problems of blurry lesion boundaries and irregular morphologies, common practices combine CNN and attention to integrate global and local information. However, previous methods use two independent modules to extract global and local features separately, such feature-wise inflexible integration ignores the semantic gap between them, resulting in representation redundancy/insufficiency and undesirable restrictions in clinic practices. Moreover, medical images are highly similar to each other due to the imaging methods and human tissues, but the captured global information by transformer-based methods in the medical domain is limited within images, the semantic relations and common knowledge across images are largely ignored. To alleviate the above problems, in the neighbor view, this paper develops a pixel neighbor representation learning method (NeighborNet) to flexibly integrate global and local context within and across images for lesion morphology and boundary modeling. Concretely, we design two neighbor layers to investigate two properties (i.e., number and distribution) of neighbors. The neighbor number for each pixel is not fixed but determined by itself. The neighbor distribution is extended from one image to all images in the datasets. With the two properties, for each pixel at each feature level, the proposed NeighborNet can evolve into the transformer or degenerate into the CNN for adaptive context representation learning to cope with the irregular lesion morphologies and blurry boundaries. The state-of-the-art performances on three ultrasound datasets prove the effectiveness of the proposed NeighborNet.
Collapse
|
7
|
Barekatrezaei S, Kozegar E, Salamati M, Soryani M. Mass detection in automated three dimensional breast ultrasound using cascaded convolutional neural networks. Phys Med 2024; 124:103433. [PMID: 39002423 DOI: 10.1016/j.ejmp.2024.103433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 07/03/2024] [Accepted: 07/08/2024] [Indexed: 07/15/2024] Open
Abstract
PURPOSE Early detection of breast cancer has a significant effect on reducing its mortality rate. For this purpose, automated three-dimensional breast ultrasound (3-D ABUS) has been recently used alongside mammography. The 3-D volume produced in this imaging system includes many slices. The radiologist must review all the slices to find the mass, a time-consuming task with a high probability of mistakes. Therefore, many computer-aided detection (CADe) systems have been developed to assist radiologists in this task. In this paper, we propose a novel CADe system for mass detection in 3-D ABUS images. METHODS The proposed system includes two cascaded convolutional neural networks. The goal of the first network is to achieve the highest possible sensitivity, and the second network's goal is to reduce false positives while maintaining high sensitivity. In both networks, an improved version of 3-D U-Net architecture is utilized in which two types of modified Inception modules are used in the encoder section. In the second network, new attention units are also added to the skip connections that receive the results of the first network as saliency maps. RESULTS The system was evaluated on a dataset containing 60 3-D ABUS volumes from 43 patients and 55 masses. A sensitivity of 91.48% and a mean false positive of 8.85 per patient were achieved. CONCLUSIONS The suggested mass detection system is fully automatic without any user interaction. The results indicate that the sensitivity and the mean FP per patient of the CADe system outperform competing techniques.
Collapse
Affiliation(s)
- Sepideh Barekatrezaei
- School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran.
| | - Ehsan Kozegar
- Department of Computer Engineering and Engineering Sciences, Faculty of Technology and Engineering, University of Guilan, Rudsar-Vajargah, Guilan, Iran.
| | - Masoumeh Salamati
- Department of Reproductive Imaging, Reproductive Biomedicine Research Center, Royan Institute for Reproductive Biomedicine, ACECR, Tehran, Iran.
| | - Mohsen Soryani
- School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran.
| |
Collapse
|
8
|
Li W, Ye X, Chen X, Jiang X, Yang Y. A deep learning-based method for the detection and segmentation of breast masses in ultrasound images. Phys Med Biol 2024; 69:155027. [PMID: 38986480 DOI: 10.1088/1361-6560/ad61b6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 07/10/2024] [Indexed: 07/12/2024]
Abstract
Objective.Automated detection and segmentation of breast masses in ultrasound images are critical for breast cancer diagnosis, but remain challenging due to limited image quality and complex breast tissues. This study aims to develop a deep learning-based method that enables accurate breast mass detection and segmentation in ultrasound images.Approach.A novel convolutional neural network-based framework that combines the You Only Look Once (YOLO) v5 network and the Global-Local (GOLO) strategy was developed. First, YOLOv5 was applied to locate the mass regions of interest (ROIs). Second, a Global Local-Connected Multi-Scale Selection (GOLO-CMSS) network was developed to segment the masses. The GOLO-CMSS operated on both the entire images globally and mass ROIs locally, and then integrated the two branches for a final segmentation output. Particularly, in global branch, CMSS applied Multi-Scale Selection (MSS) modules to automatically adjust the receptive fields, and Multi-Input (MLI) modules to enable fusion of shallow and deep features at different resolutions. The USTC dataset containing 28 477 breast ultrasound images was collected for training and test. The proposed method was also tested on three public datasets, UDIAT, BUSI and TUH. The segmentation performance of GOLO-CMSS was compared with other networks and three experienced radiologists.Main results.YOLOv5 outperformed other detection models with average precisions of 99.41%, 95.15%, 93.69% and 96.42% on the USTC, UDIAT, BUSI and TUH datasets, respectively. The proposed GOLO-CMSS showed superior segmentation performance over other state-of-the-art networks, with Dice similarity coefficients (DSCs) of 93.19%, 88.56%, 87.58% and 90.37% on the USTC, UDIAT, BUSI and TUH datasets, respectively. The mean DSC between GOLO-CMSS and each radiologist was significantly better than that between radiologists (p< 0.001).Significance.Our proposed method can accurately detect and segment breast masses with a decent performance comparable to radiologists, highlighting its great potential for clinical implementation in breast ultrasound examination.
Collapse
Affiliation(s)
- Wanqing Li
- Department of Engineering and Applied Physics, University of Science and Technology of China, Hefei, Anhui 230026, People's Republic of China
| | - Xianjun Ye
- Department of Ultrasound Medicine, The First Affiliate Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui 230001, People's Republic of China
| | - Xuemin Chen
- Health Management Center, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui 230001, People's Republic of China
| | - Xianxian Jiang
- Graduate School of Bengbu Medical College, Bengbu, Anhui 233030, People's Republic of China
| | - Yidong Yang
- Department of Engineering and Applied Physics, University of Science and Technology of China, Hefei, Anhui 230026, People's Republic of China
- Ion Medical Research Institute, University of Science and Technology of China, Hefei, Anhui 230026, People's Republic of China
- Department of Radiation Oncology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui 230001, People's Republic of China
| |
Collapse
|
9
|
Ahmad J, Akram S, Jaffar A, Ali Z, Bhatti SM, Ahmad A, Rehman SU. Deep learning empowered breast cancer diagnosis: Advancements in detection and classification. PLoS One 2024; 19:e0304757. [PMID: 38990817 PMCID: PMC11239011 DOI: 10.1371/journal.pone.0304757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2024] [Accepted: 05/18/2024] [Indexed: 07/13/2024] Open
Abstract
Recent advancements in AI, driven by big data technologies, have reshaped various industries, with a strong focus on data-driven approaches. This has resulted in remarkable progress in fields like computer vision, e-commerce, cybersecurity, and healthcare, primarily fueled by the integration of machine learning and deep learning models. Notably, the intersection of oncology and computer science has given rise to Computer-Aided Diagnosis (CAD) systems, offering vital tools to aid medical professionals in tumor detection, classification, recurrence tracking, and prognosis prediction. Breast cancer, a significant global health concern, is particularly prevalent in Asia due to diverse factors like lifestyle, genetics, environmental exposures, and healthcare accessibility. Early detection through mammography screening is critical, but the accuracy of mammograms can vary due to factors like breast composition and tumor characteristics, leading to potential misdiagnoses. To address this, an innovative CAD system leveraging deep learning and computer vision techniques was introduced. This system enhances breast cancer diagnosis by independently identifying and categorizing breast lesions, segmenting mass lesions, and classifying them based on pathology. Thorough validation using the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) demonstrated the CAD system's exceptional performance, with a 99% success rate in detecting and classifying breast masses. While the accuracy of detection is 98.5%, when segmenting breast masses into separate groups for examination, the method's performance was approximately 95.39%. Upon completing all the analysis, the system's classification phase yielded an overall accuracy of 99.16% for classification. The potential for this integrated framework to outperform current deep learning techniques is proposed, despite potential challenges related to the high number of trainable parameters. Ultimately, this recommended framework offers valuable support to researchers and physicians in breast cancer diagnosis by harnessing cutting-edge AI and image processing technologies, extending recent advances in deep learning to the medical domain.
Collapse
Affiliation(s)
- Jawad Ahmad
- Faculty of Computer Science & Information Technology, The Superior University, Lahore, Pakistan
- Intelligent Data Visual Computing Research (IDVCR), Lahore, Pakistan
| | - Sheeraz Akram
- Information Systems Department, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia
| | - Arfan Jaffar
- Faculty of Computer Science & Information Technology, The Superior University, Lahore, Pakistan
- Intelligent Data Visual Computing Research (IDVCR), Lahore, Pakistan
| | - Zulfiqar Ali
- School of Computer Science and Electronic Engineering (CSEE), University of Essex, Wivenhoe Park, Colchester, United Kingdom
| | - Sohail Masood Bhatti
- Faculty of Computer Science & Information Technology, The Superior University, Lahore, Pakistan
- Intelligent Data Visual Computing Research (IDVCR), Lahore, Pakistan
| | - Awais Ahmad
- Information Systems Department, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia
| | - Shafiq Ur Rehman
- Information Systems Department, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia
| |
Collapse
|
10
|
Guan H, Yap PT, Bozoki A, Liu M. Federated learning for medical image analysis: A survey. PATTERN RECOGNITION 2024; 151:110424. [PMID: 38559674 PMCID: PMC10976951 DOI: 10.1016/j.patcog.2024.110424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. We have systematically gathered research papers on federated learning and its applications in medical image analysis published between 2017 and 2023. Our search and compilation were conducted using databases from IEEE Xplore, ACM Digital Library, Science Direct, Springer Link, Web of Science, Google Scholar, and PubMed. In this survey, we first introduce the background of federated learning for dealing with privacy protection and collaborative learning issues. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field.
Collapse
Affiliation(s)
- Hao Guan
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Pew-Thian Yap
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Andrea Bozoki
- Department of Neurology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
11
|
Wang J, Qiao L, Zhou S, Zhou J, Wang J, Li J, Ying S, Chang C, Shi J. Weakly Supervised Lesion Detection and Diagnosis for Breast Cancers With Partially Annotated Ultrasound Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2509-2521. [PMID: 38373131 DOI: 10.1109/tmi.2024.3366940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/21/2024]
Abstract
Deep learning (DL) has proven highly effective for ultrasound-based computer-aided diagnosis (CAD) of breast cancers. In an automatic CAD system, lesion detection is critical for the following diagnosis. However, existing DL-based methods generally require voluminous manually-annotated region of interest (ROI) labels and class labels to train both the lesion detection and diagnosis models. In clinical practice, the ROI labels, i.e. ground truths, may not always be optimal for the classification task due to individual experience of sonologists, resulting in the issue of coarse annotation to limit the diagnosis performance of a CAD model. To address this issue, a novel Two-Stage Detection and Diagnosis Network (TSDDNet) is proposed based on weakly supervised learning to improve diagnostic accuracy of the ultrasound-based CAD for breast cancers. In particular, all the initial ROI-level labels are considered as coarse annotations before model training. In the first training stage, a candidate selection mechanism is then designed to refine manual ROIs in the fully annotated images and generate accurate pseudo-ROIs for the partially annotated images under the guidance of class labels. The training set is updated with more accurate ROI labels for the second training stage. A fusion network is developed to integrate detection network and classification network into a unified end-to-end framework as the final CAD model in the second training stage. A self-distillation strategy is designed on this model for joint optimization to further improves its diagnosis performance. The proposed TSDDNet is evaluated on three B-mode ultrasound datasets, and the experimental results indicate that it achieves the best performance on both lesion detection and diagnosis tasks, suggesting promising application potential.
Collapse
|
12
|
Carrilero-Mardones M, Parras-Jurado M, Nogales A, Pérez-Martín J, Díez FJ. Deep Learning for Describing Breast Ultrasound Images with BI-RADS Terms. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01155-1. [PMID: 38926264 DOI: 10.1007/s10278-024-01155-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Revised: 05/12/2024] [Accepted: 05/13/2024] [Indexed: 06/28/2024]
Abstract
Breast cancer is the most common cancer in women. Ultrasound is one of the most used techniques for diagnosis, but an expert in the field is necessary to interpret the test. Computer-aided diagnosis (CAD) systems aim to help physicians during this process. Experts use the Breast Imaging-Reporting and Data System (BI-RADS) to describe tumors according to several features (shape, margin, orientation...) and estimate their malignancy, with a common language. To aid in tumor diagnosis with BI-RADS explanations, this paper presents a deep neural network for tumor detection, description, and classification. An expert radiologist described with BI-RADS terms 749 nodules taken from public datasets. The YOLO detection algorithm is used to obtain Regions of Interest (ROIs), and then a model, based on a multi-class classification architecture, receives as input each ROI and outputs the BI-RADS descriptors, the BI-RADS classification (with 6 categories), and a Boolean classification of malignancy. Six hundred of the nodules were used for 10-fold cross-validation (CV) and 149 for testing. The accuracy of this model was compared with state-of-the-art CNNs for the same task. This model outperforms plain classifiers in the agreement with the expert (Cohen's kappa), with a mean over the descriptors of 0.58 in CV and 0.64 in testing, while the second best model yielded kappas of 0.55 and 0.59, respectively. Adding YOLO to the model significantly enhances the performance (0.16 in CV and 0.09 in testing). More importantly, training the model with BI-RADS descriptors enables the explainability of the Boolean malignancy classification without reducing accuracy.
Collapse
Affiliation(s)
- Mikel Carrilero-Mardones
- Department of Artificial Intelligence, Universidad Nacional de Educacion a Distancia (UNED), Madrid, Spain.
| | | | - Alberto Nogales
- CEIEC Research Institute, Universidad Francisco de Vitoria, Madrid, Spain
| | - Jorge Pérez-Martín
- Department of Artificial Intelligence, Universidad Nacional de Educacion a Distancia (UNED), Madrid, Spain
| | - Francisco Javier Díez
- Department of Artificial Intelligence, Universidad Nacional de Educacion a Distancia (UNED), Madrid, Spain
| |
Collapse
|
13
|
Ru J, Zhu Z, Shi J. Spatial and geometric learning for classification of breast tumors from multi-center ultrasound images: a hybrid learning approach. BMC Med Imaging 2024; 24:133. [PMID: 38840240 PMCID: PMC11155188 DOI: 10.1186/s12880-024-01307-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2024] [Accepted: 05/27/2024] [Indexed: 06/07/2024] Open
Abstract
BACKGROUND Breast cancer is the most common cancer among women, and ultrasound is a usual tool for early screening. Nowadays, deep learning technique is applied as an auxiliary tool to provide the predictive results for doctors to decide whether to make further examinations or treatments. This study aimed to develop a hybrid learning approach for breast ultrasound classification by extracting more potential features from local and multi-center ultrasound data. METHODS We proposed a hybrid learning approach to classify the breast tumors into benign and malignant. Three multi-center datasets (BUSI, BUS, OASBUD) were used to pretrain a model by federated learning, then every dataset was fine-tuned at local. The proposed model consisted of a convolutional neural network (CNN) and a graph neural network (GNN), aiming to extract features from images at a spatial level and from graphs at a geometric level. The input images are small-sized and free from pixel-level labels, and the input graphs are generated automatically in an unsupervised manner, which saves the costs of labor and memory space. RESULTS The classification AUCROC of our proposed method is 0.911, 0.871 and 0.767 for BUSI, BUS and OASBUD. The balanced accuracy is 87.6%, 85.2% and 61.4% respectively. The results show that our method outperforms conventional methods. CONCLUSIONS Our hybrid approach can learn the inter-feature among multi-center data and the intra-feature of local data. It shows potential in aiding doctors for breast tumor classification in ultrasound at an early stage.
Collapse
Affiliation(s)
- Jintao Ru
- Department of Medical Engineering, Shaoxing Hospital of Traditional Chinese Medicine, Shaoxing, Zhejiang, People's Republic of China.
| | - Zili Zhu
- Department of Radiology, The First Affiliated Hospital of Ningbo University, Ningbo, Zhejiang, People's Republic of China
| | - Jialin Shi
- Rehabilitation Medicine Institute, Zhejiang Rehabilitation Medical Center, Hangzhou, Zhejiang, People's Republic of China
| |
Collapse
|
14
|
S S, Rufus NHA. Investigation on ultrasound images for detection of fetal congenital heart defects. Biomed Phys Eng Express 2024; 10:042001. [PMID: 38781934 DOI: 10.1088/2057-1976/ad4f91] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2023] [Accepted: 05/23/2024] [Indexed: 05/25/2024]
Abstract
Congenital heart defects (CHD) are one of the serious problems that arise during pregnancy. Early CHD detection reduces death rates and morbidity but is hampered by the relatively low detection rates (i.e., 60%) of current screening technology. The detection rate could be increased by supplementing ultrasound imaging with fetal ultrasound image evaluation (FUSI) using deep learning techniques. As a result, the non-invasive foetal ultrasound image has clear potential in the diagnosis of CHD and should be considered in addition to foetal echocardiography. This review paper highlights cutting-edge technologies for detecting CHD using ultrasound images, which involve pre-processing, localization, segmentation, and classification. Existing technique of preprocessing includes spatial domain filter, non-linear mean filter, transform domain filter, and denoising methods based on Convolutional Neural Network (CNN); segmentation includes thresholding-based techniques, region growing-based techniques, edge detection techniques, Artificial Neural Network (ANN) based segmentation methods, non-deep learning approaches and deep learning approaches. The paper also suggests future research directions for improving current methodologies.
Collapse
Affiliation(s)
- Satish S
- Department of Electronics and Communication Engineering, Vel Tech Rangarajan Dr Sagunthala R&D Institute of Science and Technology, Chennai-600062, Tamil Nadu, India
| | - N Herald Anantha Rufus
- Department of Electronics and Communication Engineering, Vel Tech Rangarajan Dr Sagunthala R&D Institute of Science and Technology, Chennai-600062, Tamil Nadu, India
| |
Collapse
|
15
|
Pramanik P, Roy A, Cuevas E, Perez-Cisneros M, Sarkar R. DAU-Net: Dual attention-aided U-Net for segmenting tumor in breast ultrasound images. PLoS One 2024; 19:e0303670. [PMID: 38820462 PMCID: PMC11142567 DOI: 10.1371/journal.pone.0303670] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Accepted: 04/29/2024] [Indexed: 06/02/2024] Open
Abstract
Breast cancer remains a critical global concern, underscoring the urgent need for early detection and accurate diagnosis to improve survival rates among women. Recent developments in deep learning have shown promising potential for computer-aided detection (CAD) systems to address this challenge. In this study, a novel segmentation method based on deep learning is designed to detect tumors in breast ultrasound images. Our proposed approach combines two powerful attention mechanisms: the novel Positional Convolutional Block Attention Module (PCBAM) and Shifted Window Attention (SWA), integrated into a Residual U-Net model. The PCBAM enhances the Convolutional Block Attention Module (CBAM) by incorporating the Positional Attention Module (PAM), thereby improving the contextual information captured by CBAM and enhancing the model's ability to capture spatial relationships within local features. Additionally, we employ SWA within the bottleneck layer of the Residual U-Net to further enhance the model's performance. To evaluate our approach, we perform experiments using two widely used datasets of breast ultrasound images and the obtained results demonstrate its capability in accurately detecting tumors. Our approach achieves state-of-the-art performance with dice score of 74.23% and 78.58% on BUSI and UDIAT datasets, respectively in segmenting the breast tumor region, showcasing its potential to help with precise tumor detection. By leveraging the power of deep learning and integrating innovative attention mechanisms, our study contributes to the ongoing efforts to improve breast cancer detection and ultimately enhance women's survival rates. The source code of our work can be found here: https://github.com/AyushRoy2001/DAUNet.
Collapse
Affiliation(s)
- Payel Pramanik
- Department of Computer Science and Engineering, Jadavpur University, Kolkata, India
| | - Ayush Roy
- Department of Electrical Engineering, Jadavpur University, Kolkata, India
| | - Erik Cuevas
- Departamento de Electrónica, Universidad de Guadalajara, Guadalajara, Mexico
| | - Marco Perez-Cisneros
- División de Tecnologías Para La Integración Ciber-Humana, Universidad de Guadalajara, Guadalajara, Mexico
| | - Ram Sarkar
- Department of Computer Science and Engineering, Jadavpur University, Kolkata, India
| |
Collapse
|
16
|
Jiang Z, Gandomkar Z, Trieu PDY, Taba ST, Barron ML, Lewis SJ. AI for interpreting screening mammograms: implications for missed cancer in double reading practices and challenging-to-locate lesions. Sci Rep 2024; 14:11893. [PMID: 38789575 PMCID: PMC11126609 DOI: 10.1038/s41598-024-62324-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Accepted: 05/15/2024] [Indexed: 05/26/2024] Open
Abstract
Although the value of adding AI as a surrogate second reader in various scenarios has been investigated, it is unknown whether implementing an AI tool within double reading practice would capture additional subtle cancers missed by both radiologists who independently assessed the mammograms. This paper assesses the effectiveness of two state-of-the-art Artificial Intelligence (AI) models in detecting retrospectively-identified missed cancers within a screening program employing double reading practices. The study also explores the agreement between AI and radiologists in locating the lesions, considering various levels of concordance among the radiologists in locating the lesions. The Globally-aware Multiple Instance Classifier (GMIC) and Global-Local Activation Maps (GLAM) models were fine-tuned for our dataset. We evaluated the sensitivity of both models on missed cancers retrospectively identified by a panel of three radiologists who reviewed prior examinations of 729 cancer cases detected in a screening program with double reading practice. Two of these experts annotated the lesions, and based on their concordance levels, cases were categorized as 'almost perfect,' 'substantial,' 'moderate,' and 'poor.' We employed Similarity or Histogram Intersection (SIM) and Kullback-Leibler Divergence (KLD) metrics to compare saliency maps of malignant cases from the AI model with annotations from radiologists in each category. In total, 24.82% of cancers were labeled as "missed." The performance of GMIC and GLAM on the missed cancer cases was 82.98% and 79.79%, respectively, while for the true screen-detected cancers, the performances were 89.54% and 87.25%, respectively (p-values for the difference in sensitivity < 0.05). As anticipated, SIM and KLD from saliency maps were best in 'almost perfect,' followed by 'substantial,' 'moderate,' and 'poor.' Both GMIC and GLAM (p-values < 0.05) exhibited greater sensitivity at higher concordance. Even in a screening program with independent double reading, adding AI could potentially identify missed cancers. However, the challenging-to-locate lesions for radiologists impose a similar challenge for AI.
Collapse
Affiliation(s)
- Zhengqiang Jiang
- Discipline of Medical Imaging Sciences, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia.
| | - Ziba Gandomkar
- Discipline of Medical Imaging Sciences, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Phuong Dung Yun Trieu
- Discipline of Medical Imaging Sciences, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Seyedamir Tavakoli Taba
- Discipline of Medical Imaging Sciences, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Melissa L Barron
- Discipline of Medical Imaging Sciences, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Sarah J Lewis
- Discipline of Medical Imaging Sciences, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
- School of Health Sciences, Western Sydney University, Campbelltown, Australia
| |
Collapse
|
17
|
Yan L, Liang Z, Zhang H, Zhang G, Zheng W, Han C, Yu D, Zhang H, Xie X, Liu C, Zhang W, Zheng H, Pei J, Shen D, Qian X. A domain knowledge-based interpretable deep learning system for improving clinical breast ultrasound diagnosis. COMMUNICATIONS MEDICINE 2024; 4:90. [PMID: 38760506 PMCID: PMC11101659 DOI: 10.1038/s43856-024-00518-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Accepted: 05/03/2024] [Indexed: 05/19/2024] Open
Abstract
BACKGROUND Though deep learning has consistently demonstrated advantages in the automatic interpretation of breast ultrasound images, its black-box nature hinders potential interactions with radiologists, posing obstacles for clinical deployment. METHODS We proposed a domain knowledge-based interpretable deep learning system for improving breast cancer risk prediction via paired multimodal ultrasound images. The deep learning system was developed on 4320 multimodal breast ultrasound images of 1440 biopsy-confirmed lesions from 1348 prospectively enrolled patients across two hospitals between August 2019 and December 2022. The lesions were allocated to 70% training cohort, 10% validation cohort, and 20% test cohort based on case recruitment date. RESULTS Here, we show that the interpretable deep learning system can predict breast cancer risk as accurately as experienced radiologists, with an area under the receiver operating characteristic curve of 0.902 (95% confidence interval = 0.882 - 0.921), sensitivity of 75.2%, and specificity of 91.8% on the test cohort. With the aid of the deep learning system, particularly its inherent explainable features, junior radiologists tend to achieve better clinical outcomes, while senior radiologists experience increased confidence levels. Multimodal ultrasound images augmented with domain knowledge-based reasoning cues enable an effective human-machine collaboration at a high level of prediction performance. CONCLUSIONS Such a clinically applicable deep learning system may be incorporated into future breast cancer screening and support assisted or second-read workflows.
Collapse
Affiliation(s)
- Lin Yan
- School of Mathematics, Xi'an University of Finance and Economics, Xi'an, China
| | - Zhiying Liang
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - Hao Zhang
- Department of Neurosurgery, Beijing Friendship Hospital, Capital Medical University, Beijing, China
| | - Gaosong Zhang
- Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Weiwei Zheng
- Department of Ultrasound, Xuancheng People's Hospital, Xuancheng, China
| | - Chunguang Han
- Department of General Surgery, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Dongsheng Yu
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China
| | - Hanqi Zhang
- Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Xinxin Xie
- Department of Ultrasound, Peking University Third Hospital, Beijing, China
| | - Chang Liu
- Department of General Surgery, The First Affiliated Hospital of Anhui Medical University, Hefei, China
- Department of Breast Surgery, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Wenxin Zhang
- Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Hui Zheng
- Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Jing Pei
- Department of General Surgery, The First Affiliated Hospital of Anhui Medical University, Hefei, China.
- Department of Breast Surgery, The First Affiliated Hospital of Anhui Medical University, Hefei, China.
| | - Dinggang Shen
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.
- State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai, China.
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China.
- Shanghai Clinical Research and Trial Center, Shanghai, China.
| | - Xuejun Qian
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.
- State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai, China.
- Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China.
| |
Collapse
|
18
|
Bosco E, Spairani E, Toffali E, Meacci V, Ramalli A, Matrone G. A Deep Learning Approach for Beamforming and Contrast Enhancement of Ultrasound Images in Monostatic Synthetic Aperture Imaging: A Proof-of-Concept. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2024; 5:376-382. [PMID: 38899024 PMCID: PMC11186640 DOI: 10.1109/ojemb.2024.3401098] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 03/29/2024] [Accepted: 05/08/2024] [Indexed: 06/21/2024] Open
Abstract
Goal: In this study, we demonstrate that a deep neural network (DNN) can be trained to reconstruct high-contrast images, resembling those produced by the multistatic Synthetic Aperture (SA) method using a 128-element array, leveraging pre-beamforming radiofrequency (RF) signals acquired through the monostatic SA approach. Methods: A U-net was trained using 27200 pairs of RF signals, simulated considering a monostatic SA architecture, with their corresponding delay-and-sum beamformed target images in a multistatic 128-element SA configuration. The contrast was assessed on 500 simulated test images of anechoic/hyperechoic targets. The DNN's performance in reconstructing experimental images of a phantom and different in vivo scenarios was tested too. Results: The DNN, compared to the simple monostatic SA approach used to acquire pre-beamforming signals, generated better-quality images with higher contrast and reduced noise/artifacts. Conclusions: The obtained results suggest the potential for the development of a single-channel setup, simultaneously providing good-quality images and reducing hardware complexity.
Collapse
Affiliation(s)
- Edoardo Bosco
- Department of Electrical, Computer and Biomedical EngineeringUniversity of Pavia27100PaviaItaly
| | - Edoardo Spairani
- Department of Electrical, Computer and Biomedical EngineeringUniversity of Pavia27100PaviaItaly
| | - Eleonora Toffali
- Department of Electrical, Computer and Biomedical EngineeringUniversity of Pavia27100PaviaItaly
| | - Valentino Meacci
- Department of Information EngineeringUniversity of Florence50134FlorenceItaly
| | - Alessandro Ramalli
- Department of Information EngineeringUniversity of Florence50134FlorenceItaly
| | - Giulia Matrone
- Department of Electrical, Computer and Biomedical EngineeringUniversity of Pavia27100PaviaItaly
| |
Collapse
|
19
|
Al-Karawi D, Al-Zaidi S, Helael KA, Obeidat N, Mouhsen AM, Ajam T, Alshalabi BA, Salman M, Ahmed MH. A Review of Artificial Intelligence in Breast Imaging. Tomography 2024; 10:705-726. [PMID: 38787015 PMCID: PMC11125819 DOI: 10.3390/tomography10050055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 04/14/2024] [Accepted: 05/06/2024] [Indexed: 05/25/2024] Open
Abstract
With the increasing dominance of artificial intelligence (AI) techniques, the important prospects for their application have extended to various medical fields, including domains such as in vitro diagnosis, intelligent rehabilitation, medical imaging, and prognosis. Breast cancer is a common malignancy that critically affects women's physical and mental health. Early breast cancer screening-through mammography, ultrasound, or magnetic resonance imaging (MRI)-can substantially improve the prognosis for breast cancer patients. AI applications have shown excellent performance in various image recognition tasks, and their use in breast cancer screening has been explored in numerous studies. This paper introduces relevant AI techniques and their applications in the field of medical imaging of the breast (mammography and ultrasound), specifically in terms of identifying, segmenting, and classifying lesions; assessing breast cancer risk; and improving image quality. Focusing on medical imaging for breast cancer, this paper also reviews related challenges and prospects for AI.
Collapse
Affiliation(s)
- Dhurgham Al-Karawi
- Medical Analytica Ltd., 26a Castle Park Industrial Park, Flint CH6 5XA, UK;
| | - Shakir Al-Zaidi
- Medical Analytica Ltd., 26a Castle Park Industrial Park, Flint CH6 5XA, UK;
| | - Khaled Ahmad Helael
- Royal Medical Services, King Hussein Medical Hospital, King Abdullah II Ben Al-Hussein Street, Amman 11855, Jordan;
| | - Naser Obeidat
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Abdulmajeed Mounzer Mouhsen
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Tarek Ajam
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Bashar A. Alshalabi
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Mohamed Salman
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Mohammed H. Ahmed
- School of Computing, Coventry University, 3 Gulson Road, Coventry CV1 5FB, UK;
| |
Collapse
|
20
|
He Q, Yang Q, Su H, Wang Y. Multi-task learning for segmentation and classification of breast tumors from ultrasound images. Comput Biol Med 2024; 173:108319. [PMID: 38513394 DOI: 10.1016/j.compbiomed.2024.108319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 03/03/2024] [Accepted: 03/12/2024] [Indexed: 03/23/2024]
Abstract
Segmentation and classification of breast tumors are critical components of breast ultrasound (BUS) computer-aided diagnosis (CAD), which significantly improves the diagnostic accuracy of breast cancer. However, the characteristics of tumor regions in BUS images, such as non-uniform intensity distributions, ambiguous or missing boundaries, and varying tumor shapes and sizes, pose significant challenges to automated segmentation and classification solutions. Many previous studies have proposed multi-task learning methods to jointly tackle tumor segmentation and classification by sharing the features extracted by the encoder. Unfortunately, this often introduces redundant or misleading information, which hinders effective feature exploitation and adversely affects performance. To address this issue, we present ACSNet, a novel multi-task learning network designed to optimize tumor segmentation and classification in BUS images. The segmentation network incorporates a novel gate unit to allow optimal transfer of valuable contextual information from the encoder to the decoder. In addition, we develop the Deformable Spatial Attention Module (DSAModule) to improve segmentation accuracy by overcoming the limitations of conventional convolution in dealing with morphological variations of tumors. In the classification branch, multi-scale feature extraction and channel attention mechanisms are integrated to discriminate between benign and malignant breast tumors. Experiments on two publicly available BUS datasets demonstrate that ACSNet not only outperforms mainstream multi-task learning methods for both breast tumor segmentation and classification tasks, but also achieves state-of-the-art results for BUS tumor segmentation. Code and models are available at https://github.com/qqhe-frank/BUS-segmentation-and-classification.git.
Collapse
Affiliation(s)
- Qiqi He
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China; School of Life Science and Technology, Xidian University, Xi'an, China
| | - Qiuju Yang
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China.
| | - Hang Su
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| | - Yixuan Wang
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| |
Collapse
|
21
|
Jiang Q, Ye H, Yang B, Cao F. Label-Decoupled Medical Image Segmentation With Spatial-Channel Graph Convolution and Dual Attention Enhancement. IEEE J Biomed Health Inform 2024; 28:2830-2841. [PMID: 38376972 DOI: 10.1109/jbhi.2024.3367756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
Deep learning-based methods have been widely used in medical image segmentation recently. However, existing works are usually difficult to simultaneously capture global long-range information from images and topological correlations among feature maps. Further, medical images often suffer from blurred target edges. Accordingly, this paper proposes a novel medical image segmentation framework named a label-decoupled network with spatial-channel graph convolution and dual attention enhancement mechanism (LADENet for short). It constructs learnable adjacency matrices and utilizes graph convolutions to effectively capture global long-range information on spatial locations and topological dependencies between different channels in an image. Then a label-decoupled strategy based on distance transformation is introduced to decouple an original segmentation label into a body label and an edge label for supervising the body branch and edge branch. Again, a dual attention enhancement mechanism, designing a body attention block in the body branch and an edge attention block in the edge branch, is built to promote the learning ability of spatial region and boundary features. Besides, a feature interactor is devised to fully consider the information interaction between the body and edge branches to improve segmentation performance. Experiments on benchmark datasets reveal the superiority of LADENet compared to state-of-the-art approaches.
Collapse
|
22
|
Subhashini R, Velswamy R, Sree Rathna Lakshmi NVS, Sivanandam C. An innovative breast cancer detection framework using multiscale dilated densenet with attention mechanism. NETWORK (BRISTOL, ENGLAND) 2024:1-37. [PMID: 38648017 DOI: 10.1080/0954898x.2024.2343348] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2024] [Accepted: 04/05/2024] [Indexed: 04/25/2024]
Abstract
Cancer-related deadly diseases affect both developed and underdeveloped nations worldwide. Effective network learning is crucial to more reliably identify and categorize breast carcinoma in vast and unbalanced image datasets. The absence of early cancer symptoms makes the early identification process challenging. Therefore, from the perspectives of diagnosis, prevention, and therapy, cancer continues to be among the healthcare concerns that numerous researchers work to advance. It is highly essential to design an innovative breast cancer detection model by considering the complications presented in the classical techniques. Initially, breast cancer images are gathered from online sources and it is further subjected to the segmentation region. Here, it is segmented using Adaptive Trans-Dense-Unet (A-TDUNet), and their parameters are tuned using the developed Modified Sheep Flock Optimization Algorithm (MSFOA). The segmented images are further subjected to the breast cancer detection stage and effective breast cancer detection is performed by Multiscale Dilated Densenet with Attention Mechanism (MDD-AM). Throughout the result validation, the Net Present Value (NPV) and accuracy rate of the designed approach are 96.719% and 93.494%. Hence, the implemented breast cancer detection model secured a better efficacy rate than the baseline detection methods in diverse experimental conditions.
Collapse
Affiliation(s)
- R Subhashini
- Department of Information Technology, Sona College of Technology, Salem, Tamil Nadu, India
| | - Rajasekar Velswamy
- Department of Computer Science and Engineering, SRM Institute of Science and Technology, Kattankulathur, Tamil Nadu, India
| | - N V S Sree Rathna Lakshmi
- Department of Electronics and Communication Engineering, Agni College of Technology, Thazhambur, Tamil Nadu, India
| | - Chakaravarthi Sivanandam
- Department of Computer Science and Engineering, Panimalar Engineering College, Poonamallee, Chennai, Tamil Nadu, India
| |
Collapse
|
23
|
VanBerlo B, Hoey J, Wong A. A survey of the impact of self-supervised pretraining for diagnostic tasks in medical X-ray, CT, MRI, and ultrasound. BMC Med Imaging 2024; 24:79. [PMID: 38580932 PMCID: PMC10998380 DOI: 10.1186/s12880-024-01253-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 03/18/2024] [Indexed: 04/07/2024] Open
Abstract
Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning, leveraging large amounts of unlabelled data. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging, concentrating on studies that compare self-supervised pretraining to fully supervised learning for diagnostic tasks such as classification and segmentation. The most pertinent finding is that self-supervised pretraining generally improves downstream task performance compared to full supervision, most prominently when unlabelled examples greatly outnumber labelled examples. Based on the aggregate evidence, recommendations are provided for practitioners considering using self-supervised learning. Motivated by limitations identified in current research, directions and practices for future study are suggested, such as integrating clinical knowledge with theoretically justified self-supervised learning methods, evaluating on public datasets, growing the modest body of evidence for ultrasound, and characterizing the impact of self-supervised pretraining on generalization.
Collapse
Affiliation(s)
- Blake VanBerlo
- Cheriton School of Computer Science, 200 University Ave W, N2L 3G1, Waterloo, Canada.
| | - Jesse Hoey
- Cheriton School of Computer Science, 200 University Ave W, N2L 3G1, Waterloo, Canada
| | - Alexander Wong
- Department of Systems Design Engineering, 200 University Ave W, N2L 3G1, Waterloo, Canada
| |
Collapse
|
24
|
Gómez-Flores W, Gregorio-Calas MJ, Coelho de Albuquerque Pereira W. BUS-BRA: A breast ultrasound dataset for assessing computer-aided diagnosis systems. Med Phys 2024; 51:3110-3123. [PMID: 37937827 DOI: 10.1002/mp.16812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 10/10/2023] [Accepted: 10/12/2023] [Indexed: 11/09/2023] Open
Abstract
PURPOSE Computer-aided diagnosis (CAD) systems on breast ultrasound (BUS) aim to increase the efficiency and effectiveness of breast screening, helping specialists to detect and classify breast lesions. CAD system development requires a set of annotated images, including lesion segmentation, biopsy results to specify benign and malignant cases, and BI-RADS categories to indicate the likelihood of malignancy. Besides, standardized partitions of training, validation, and test sets promote reproducibility and fair comparisons between different approaches. Thus, we present a publicly available BUS dataset whose novelty is the substantial increment of cases with the above-mentioned annotations and the inclusion of standardized partitions to objectively assess and compare CAD systems. ACQUISITION AND VALIDATION METHODS The BUS dataset comprises 1875 anonymized images from 1064 female patients acquired via four ultrasound scanners during systematic studies at the National Institute of Cancer (Rio de Janeiro, Brazil). The dataset includes biopsy-proven tumors divided into 722 benign and 342 malignant cases. Besides, a senior ultrasonographer performed a BI-RADS assessment in categories 2 to 5. Additionally, the ultrasonographer manually outlined the breast lesions to obtain ground truth segmentations. Furthermore, 5- and 10-fold cross-validation partitions are provided to standardize the training and test sets to evaluate and reproduce CAD systems. Finally, to validate the utility of the BUS dataset, an evaluation framework is implemented to assess the performance of deep neural networks for segmenting and classifying breast lesions. DATA FORMAT AND USAGE NOTES The BUS dataset is publicly available for academic and research purposes through an open-access repository under the name BUS-BRA: A Breast Ultrasound Dataset for Assessing CAD Systems. BUS images and reference segmentations are saved in Portable Network Graphic (PNG) format files, and the dataset information is stored in separate Comma-Separated Value (CSV) files. POTENTIAL APPLICATIONS The BUS-BRA dataset can be used to develop and assess artificial intelligence-based lesion detection and segmentation methods, and the classification of BUS images into pathological classes and BI-RADS categories. Other potential applications include developing image processing methods like despeckle filtering and contrast enhancement methods to improve image quality and feature engineering for image description.
Collapse
Affiliation(s)
- Wilfrido Gómez-Flores
- Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, Tamaulipas, Mexico
| | | | | |
Collapse
|
25
|
Tang Z, Mahmoodi S, Meng D, Darekar A, Vollmer B. Rule-based deep learning method for prognosis of neonatal hypoxic-ischemic encephalopathy by using susceptibility weighted image analysis. MAGMA (NEW YORK, N.Y.) 2024; 37:227-239. [PMID: 38252196 DOI: 10.1007/s10334-023-01139-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Revised: 12/06/2023] [Accepted: 12/11/2023] [Indexed: 01/23/2024]
Abstract
OBJECTIVE Susceptibility weighted imaging (SWI) of neonatal hypoxic-ischemic brain injury can provide assistance in the prognosis of neonatal hypoxic-ischemic encephalopathy (HIE). We propose a convolutional neural network model to classify SWI images with HIE. MATERIALS AND METHODS Due to the lack of a large dataset, transfer learning method with fine-tuning a pre-trained ResNet 50 is introduced. We randomly select 11 datasets from patients with normal neurology outcomes (n = 31) and patients with abnormal neurology outcomes (n = 11) at 24 months of age to avoid bias in classification due to any imbalance in the data. RESULTS We develop a rule-based system to improve the classification performance, with an accuracy of 0.93 ± 0.09. We also compute heatmaps produced by the Grad-CAM technique to analyze which areas of SWI images contributed more to the classification patients with abnormal neurology outcome. CONCLUSION Such regions that are important in the classification accuracy can interpret the relationship between the brain regions affected by hypoxic-ischemic and neurodevelopmental outcomes of infants with HIE at the age of 2 years.
Collapse
Affiliation(s)
- Zhen Tang
- School of Computer Science and Technology, AnHui University of Technology, Maxiang Street, Maanshan, 243032, Anhui, China.
| | - Sasan Mahmoodi
- School of Electronics and Computer Science, University of Southampton, Southampton, SO17 1BJ, UK
| | - Di Meng
- School of Computer Science and Technology, AnHui University of Technology, Maxiang Street, Maanshan, 243032, Anhui, China
| | - Angela Darekar
- Department of Medical Physics, University Hospital Southampton NHS Foundation Trust, Southampton, SO16 6YD, UK
| | - Brigitte Vollmer
- Clinical Neurosciences and Clinical and Experimental Sciences, Faculty of Medicine, University of Southampton, Southampton, SO17 1BJ, UK
| |
Collapse
|
26
|
Hoang QT, Pham XH, Trinh XT, Le AV, Bui MV, Bui TT. An Efficient CNN-Based Method for Intracranial Hemorrhage Segmentation from Computerized Tomography Imaging. J Imaging 2024; 10:77. [PMID: 38667975 PMCID: PMC11051045 DOI: 10.3390/jimaging10040077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Revised: 03/21/2024] [Accepted: 03/22/2024] [Indexed: 04/28/2024] Open
Abstract
Intracranial hemorrhage (ICH) resulting from traumatic brain injury is a serious issue, often leading to death or long-term disability if not promptly diagnosed. Currently, doctors primarily use Computerized Tomography (CT) scans to detect and precisely locate a hemorrhage, typically interpreted by radiologists. However, this diagnostic process heavily relies on the expertise of medical professionals. To address potential errors, computer-aided diagnosis systems have been developed. In this study, we propose a new method that enhances the localization and segmentation of ICH lesions in CT scans by using multiple images created through different data augmentation techniques. We integrate residual connections into a U-Net-based segmentation network to improve the training efficiency. Our experiments, based on 82 CT scans from traumatic brain injury patients, validate the effectiveness of our approach, achieving an IOU score of 0.807 ± 0.03 for ICH segmentation using 10-fold cross-validation.
Collapse
Affiliation(s)
- Quoc Tuan Hoang
- Faculty of Mechanical Engineering, Hung Yen University of Technology and Education, 39Rd., Hung Yen 160000, Vietnam; (Q.T.H.); (X.T.T.)
| | - Xuan Hien Pham
- Faculty of Mechanical Engineering, University of Transport and Communications, Hanoi 100000, Vietnam;
| | - Xuan Thang Trinh
- Faculty of Mechanical Engineering, Hung Yen University of Technology and Education, 39Rd., Hung Yen 160000, Vietnam; (Q.T.H.); (X.T.T.)
| | - Anh Vu Le
- Communication and Signal Processing Research Group, Faculty of Electrical and Electronics Engineering, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam
| | - Minh V. Bui
- Faculty of Engineering and Technology, Nguyen Tat Thanh University, 300A, Nguyen Tat Thanh, Ward 13, District 4, Ho Chi Minh City 700000, Vietnam;
| | - Trung Thanh Bui
- Faculty of Mechanical Engineering, Hung Yen University of Technology and Education, 39Rd., Hung Yen 160000, Vietnam; (Q.T.H.); (X.T.T.)
| |
Collapse
|
27
|
Karunanayake N, Makhanov SS. When deep learning is not enough: artificial life as a supplementary tool for segmentation of ultrasound images of breast cancer. Med Biol Eng Comput 2024:10.1007/s11517-024-03026-x. [PMID: 38498125 DOI: 10.1007/s11517-024-03026-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Accepted: 01/16/2024] [Indexed: 03/20/2024]
Abstract
Segmentation of tumors in ultrasound (US) images of the breast is a critical issue in medical imaging. Due to the poor quality of US images and the varying specifications of US machines, segmentation and classification of abnormalities present difficulties even for trained radiologists. The paper aims to introduce a novel AI-based hybrid model for US segmentation that offers high accuracy, requires relatively smaller datasets, and is capable of handling previously unseen data. The software can be used for diagnostics and the US-guided biopsies. A unique and robust hybrid approach that combines deep learning (DL) and multi-agent artificial life (AL) has been introduced. The algorithms are verified on three US datasets. The method outperforms 14 selected state-of-the-art algorithms applied to US images characterized by complex geometry and high level of noise. The paper offers an original classification of the images and tests to analyze the limits of the DL. The model has been trained and verified on 1264 ultrasound images. The images are in the JPEG and PNG formats. The age of the patients ranges from 22 to 73 years. The 14 benchmark algorithms include deformable shapes, edge linking, superpixels, machine learning, and DL methods. The tests use eight-region shape- and contour-based evaluation metrics. The proposed method (DL-AL) produces excellent results in terms of the dice coefficient (region) and the relative Hausdorff distance H3 (contour-based) as follows: the easiest image complexity level, Dice = 0.96 and H3 = 0.26; the medium complexity level, Dice = 0.91 and H3 = 0.82; and the hardest complexity level, Dice = 0.90 and H3 = 0.84. All other metrics follow the same pattern. The DL-AL outperforms the second best (Unet-based) method by 10-20%. The method has been also tested by a series of unconventional tests. The model was trained on low complexity images and applied to the entire set of images. These results are summarized below. (1) Only the low complexity images have been used for training (68% unknown images): Dice = 0.80 and H3 = 2.01. (2) The low and the medium complexity images have been used for training (51% unknown images): Dice = 0.86 and H3 = 1.32. (3) The low, medium, and hard complexity images have been used for training (35% unknown images): Dice = 0.92 and H3 = 0.76. These tests show a significant advantage of DL-AL over 30%. A video demo illustrating the algorithm is at http://tinyurl.com/mr4ah687 .
Collapse
Affiliation(s)
- Nalan Karunanayake
- Sirindhorn International Institute of Technology, Thammasat University, Pathum Thani, Thailand
| | - Stanislav S Makhanov
- Sirindhorn International Institute of Technology, Thammasat University, Pathum Thani, Thailand.
| |
Collapse
|
28
|
Xu K, You K, Zhu B, Feng M, Feng D, Yang C. Masked Modeling-Based Ultrasound Image Classification via Self-Supervised Learning. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2024; 5:226-237. [PMID: 38606402 PMCID: PMC11008806 DOI: 10.1109/ojemb.2024.3374966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 02/21/2024] [Accepted: 03/05/2024] [Indexed: 04/13/2024] Open
Abstract
Recently, deep learning-based methods have emerged as the preferred approach for ultrasound data analysis. However, these methods often require large-scale annotated datasets for training deep models, which are not readily available in practical scenarios. Additionally, the presence of speckle noise and other imaging artifacts can introduce numerous hard examples for ultrasound data classification. In this paper, drawing inspiration from self-supervised learning techniques, we present a pre-training method based on mask modeling specifically designed for ultrasound data. Our study investigates three different mask modeling strategies: random masking, vertical masking, and horizontal masking. By employing these strategies, our pre-training approach aims to predict the masked portion of the ultrasound images. Notably, our method does not rely on externally labeled data, allowing us to extract representative features without the need for human annotation. Consequently, we can leverage unlabeled datasets for pre-training. Furthermore, to address the challenges posed by hard samples in ultrasound data, we propose a novel hard sample mining strategy. To evaluate the effectiveness of our proposed method, we conduct experiments on two datasets. The experimental results demonstrate that our approach outperforms other state-of-the-art methods in ultrasound image classification. This indicates the superiority of our pre-training method and its ability to extract discriminative features from ultrasound data, even in the presence of hard examples.
Collapse
Affiliation(s)
- Kele Xu
- National University of Defense TechnologyChangsha410073China
| | - Kang You
- Shanghai Jiao Tong UniversityShanghai200240China
| | - Boqing Zhu
- National University of Defense TechnologyChangsha410073China
| | - Ming Feng
- TongJi UniversityShanghai200070China
| | - Dawei Feng
- National University of Defense TechnologyChangsha410073China
| | - Cheng Yang
- National University of Defense TechnologyChangsha410073China
| |
Collapse
|
29
|
Wang Y, Zhu B, Kong L, Wang J, Gao B, Wang J, Tian D, Yao Y. BPSegSys: A Brachial Plexus Nerve Trunk Segmentation System Using Deep Learning. ULTRASOUND IN MEDICINE & BIOLOGY 2024; 50:374-383. [PMID: 38176984 DOI: 10.1016/j.ultrasmedbio.2023.11.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Revised: 11/13/2023] [Accepted: 11/16/2023] [Indexed: 01/06/2024]
Abstract
OBJECTIVE Ultrasound-guided nerve block anesthesia (UGNB) is a high-tech visual nerve block anesthesia method that can be used to observe the target nerve and its surrounding structures, the puncture needle's advancement and local anesthetics spread in real time. The key in UGNB is nerve identification. With the help of deep learning methods, the automatic identification or segmentation of nerves can be realized, assisting doctors in completing nerve block anesthesia accurately and efficiently. METHODS We established a public data set containing 320 ultrasound images of brachial plexus (BP). Three experienced doctors jointly produced the BP segmentation ground truth and labeled brachial plexus trunks. We designed a brachial plexus segmentation system (BPSegSys) based on deep learning. RESULTS BPSegSys achieves experienced-doctor-level nerve identification performance in various experiments. We evaluated BPSegSys performance in terms of intersection-over-union (IoU). Considering three data set groups in our established public data set, the IoUs of BPSegSys were 0.5350, 0.4763 and 0.5043, respectively, which exceed the IoUs 0.5205, 0.4704 and 0.4979 of experienced doctors. In addition, we determined that BPSegSys can help doctors identify brachial plexus trunks more accurately, with IoU improvement up to 27%, which has significant clinical application value. CONCLUSION We establish a data set for brachial plexus trunk identification and designed a BPSegSys to identify the brachial plexus trunks. BPSegSys achieves the doctor-level identification of the brachial plexus trunks and improves the accuracy and efficiency of doctors' identification of the brachial plexus trunks.
Collapse
Affiliation(s)
- Yu Wang
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | - Binbin Zhu
- Department of Anesthesiology, Affiliated Hospital of Medical School of Ningbo University, Ningbo University, China
| | - Lingsi Kong
- Department of Anesthesiology, Ningbo No. 6 Hospital, Ningbo, China
| | - Jianlin Wang
- Department of Anesthesiology, Ningbo No. 6 Hospital, Ningbo, China
| | - Bin Gao
- Department of Anesthesiology, Affiliated Hospital of Medical School of Ningbo University, Ningbo University, China
| | - Jianhua Wang
- Department of Anesthesiology, Affiliated Hospital of Medical School of Ningbo University, Ningbo University, China
| | - Dingcheng Tian
- Research Institute for Medical and Biological Engineering, Ningbo University, Ningbo, China
| | - Yudong Yao
- Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, USA.
| |
Collapse
|
30
|
Manh V, Jia X, Xue W, Xu W, Mei Z, Dong Y, Zhou J, Huang R, Ni D. An efficient framework for lesion segmentation in ultrasound images using global adversarial learning and region-invariant loss. Comput Biol Med 2024; 171:108137. [PMID: 38447499 DOI: 10.1016/j.compbiomed.2024.108137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Revised: 01/16/2024] [Accepted: 02/12/2024] [Indexed: 03/08/2024]
Abstract
Lesion segmentation in ultrasound images is an essential yet challenging step for early evaluation and diagnosis of cancers. In recent years, many automatic CNN-based methods have been proposed to assist this task. However, most modern approaches often lack capturing long-range dependencies and prior information making it difficult to identify the lesions with unfixed shapes, sizes, locations, and textures. To address this, we present a novel lesion segmentation framework that guides the model to learn the global information about lesion characteristics and invariant features (e.g., morphological features) of lesions to improve the segmentation in ultrasound images. Specifically, the segmentation model is guided to learn the characteristics of lesions from the global maps using an adversarial learning scheme with a self-attention-based discriminator. We argue that under such a lesion characteristics-based guidance mechanism, the segmentation model gets more clues about the boundaries, shapes, sizes, and positions of lesions and can produce reliable predictions. In addition, as ultrasound lesions have different textures, we embed this prior knowledge into a novel region-invariant loss to constrain the model to focus on invariant features for robust segmentation. We demonstrate our method on one in-house breast ultrasound (BUS) dataset and two public datasets (i.e., breast lesion (BUS B) and thyroid nodule from TNSCUI2020). Experimental results show that our method is specifically suitable for lesion segmentation in ultrasound images and can outperform the state-of-the-art approaches with Dice of 0.931, 0.906, and 0.876, respectively. The proposed method demonstrates that it can provide more important information about the characteristics of lesions for lesion segmentation in ultrasound images, especially for lesions with irregular shapes and small sizes. It can assist the current lesion segmentation models to better suit clinical needs.
Collapse
Affiliation(s)
- Van Manh
- Medical Ultrasound Image Computing (MUSIC) lab, School of Biomedical Engineering, Shenzhen University, Shenzhen, 518060, China
| | - Xiaohong Jia
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, Shanghai, 200240, China
| | - Wufeng Xue
- Medical Ultrasound Image Computing (MUSIC) lab, School of Biomedical Engineering, Shenzhen University, Shenzhen, 518060, China
| | - Wenwen Xu
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, Shanghai, 200240, China
| | - Zihan Mei
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, Shanghai, 200240, China
| | - Yijie Dong
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, Shanghai, 200240, China
| | - Jianqiao Zhou
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, Shanghai, 200240, China.
| | - Ruobing Huang
- Medical Ultrasound Image Computing (MUSIC) lab, School of Biomedical Engineering, Shenzhen University, Shenzhen, 518060, China.
| | - Dong Ni
- Medical Ultrasound Image Computing (MUSIC) lab, School of Biomedical Engineering, Shenzhen University, Shenzhen, 518060, China.
| |
Collapse
|
31
|
Chen L, Zeng B, Shen J, Xu J, Cai Z, Su S, Chen J, Cai X, Ying T, Hu B, Wu M, Chen X, Zheng Y. Bone age assessment based on three-dimensional ultrasound and artificial intelligence compared with paediatrician-read radiographic bone age: protocol for a prospective, diagnostic accuracy study. BMJ Open 2024; 14:e079969. [PMID: 38401893 PMCID: PMC10895244 DOI: 10.1136/bmjopen-2023-079969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 01/31/2024] [Indexed: 02/26/2024] Open
Abstract
INTRODUCTION Radiographic bone age (BA) assessment is widely used to evaluate children's growth disorders and predict their future height. Moreover, children are more sensitive and vulnerable to X-ray radiation exposure than adults. The purpose of this study is to develop a new, safer, radiation-free BA assessment method for children by using three-dimensional ultrasound (3D-US) and artificial intelligence (AI), and to test the diagnostic accuracy and reliability of this method. METHODS AND ANALYSIS This is a prospective, observational study. All participants will be recruited through Paediatric Growth and Development Clinic. All participants will receive left hand 3D-US and X-ray examination at the Shanghai Sixth People's Hospital on the same day, all images will be recorded. These image related data will be collected and randomly divided into training set (80% of all) and test set (20% of all). The training set will be used to establish a cascade network of 3D-US skeletal image segmentation and BA prediction model to achieve end-to-end prediction of image to BA. The test set will be used to evaluate the accuracy of AI BA model of 3D-US. We have developed a new ultrasonic scanning device, which can be proposed to automatic 3D-US scanning of hands. AI algorithms, such as convolutional neural network, will be used to identify and segment the skeletal structures in the hand 3D-US images. We will achieve automatic segmentation of hand skeletal 3D-US images, establish BA prediction model of 3D-US, and test the accuracy of the prediction model. ETHICS AND DISSEMINATION The Ethics Committee of Shanghai Sixth People's Hospital approved this study. The approval number is 2022-019. A written informed consent will be obtained from their parent or guardian of each participant. Final results will be published in peer-reviewed journals and presented at national and international conferences. TRIAL REGISTRATION NUMBER ChiCTR2200057236.
Collapse
Affiliation(s)
- Li Chen
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Bolun Zeng
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Jian Shen
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Jiangchang Xu
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Zehang Cai
- Shantou Institute of Ultrasonic Instruments Co., Ltd, Shantou, China
| | - Shudian Su
- Shantou Institute of Ultrasonic Instruments Co., Ltd, Shantou, China
| | - Jie Chen
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xiaojun Cai
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Tao Ying
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Bing Hu
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Min Wu
- Department of Pediatrics, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xiaojun Chen
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
- Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
| | - Yuanyi Zheng
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai Institute of Ultrasound in Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| |
Collapse
|
32
|
Azam S, Montaha S, Raiaan MAK, Rafid AKMRH, Mukta SH, Jonkman M. An Automated Decision Support System to Analyze Malignancy Patterns of Breast Masses Employing Medically Relevant Features of Ultrasound Images. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:45-59. [PMID: 38343240 DOI: 10.1007/s10278-023-00925-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 09/22/2023] [Accepted: 10/23/2023] [Indexed: 03/02/2024]
Abstract
An automated computer-aided approach might aid radiologists in diagnosing breast cancer at a primary stage. This study proposes a novel decision support system to classify breast tumors into benign and malignant based on clinically important features, using ultrasound images. Nine handcrafted features, which align with the clinical markers used by radiologists, are extracted from the region of interest (ROI) of ultrasound images. To validate that these elected clinical markers have a significant impact on predicting the benign and malignant classes, ten machine learning (ML) models are experimented with resulting in test accuracies in the range of 96 to 99%. In addition, four feature selection techniques are explored where two features are eliminated according to the feature ranking score of each feature selection method. The Random Forest classifier is trained with the resultant four feature sets. Results indicate that even when eliminating only two features, the performance of the model is reduced for each feature selection technique. These experiments validate the efficiency and effectiveness of the clinically important features. To develop the decision support system, a probability density function (PDF) graph is generated for each feature in order to find a threshold range to distinguish benign and malignant tumors. Based on the threshold range of particular features, a decision support system is developed in such a way that if at least eight out of nine features are within the threshold range, the image will be denoted as true predicted. With this algorithm, a test accuracy of 99.38% and an F1 Score of 99.05% is achieved, which means that our decision support system outperforms all the previously trained ML models. Moreover, after calculating individual class-based test accuracies, for the benign class, a test accuracy of 99.31% has been attained where only three benign instances are misclassified out of 437 instances, and for the malignant class, a test accuracy of 99.52% has been attained where only one malignant instance is misclassified out of 210 instances. This system is robust, time-effective, and reliable as the radiologists' criteria are followed and may aid specialists in making a diagnosis.
Collapse
Affiliation(s)
- Sami Azam
- Faculty of Science and Technology, Charles Darwin University, Casuarina, NT, 0909, Australia.
| | - Sidratul Montaha
- Faculty of Science and Technology, Charles Darwin University, Casuarina, NT, 0909, Australia
| | | | | | | | - Mirjam Jonkman
- Faculty of Science and Technology, Charles Darwin University, Casuarina, NT, 0909, Australia
| |
Collapse
|
33
|
Pawłowska A, Ćwierz-Pieńkowska A, Domalik A, Jaguś D, Kasprzak P, Matkowski R, Fura Ł, Nowicki A, Żołek N. Curated benchmark dataset for ultrasound based breast lesion analysis. Sci Data 2024; 11:148. [PMID: 38297002 PMCID: PMC10830496 DOI: 10.1038/s41597-024-02984-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 01/17/2024] [Indexed: 02/02/2024] Open
Abstract
A new detailed dataset of breast ultrasound scans (BrEaST) containing images of benign and malignant lesions as well as normal tissue examples, is presented. The dataset consists of 256 breast scans collected from 256 patients. Each scan was manually annotated and labeled by a radiologist experienced in breast ultrasound examination. In particular, each tumor was identified in the image using a freehand annotation and labeled according to BIRADS features and lexicon. The histopathological classification of the tumor was also provided for patients who underwent a biopsy. The BrEaST dataset is the first breast ultrasound dataset containing patient-level labels, image-level annotations, and tumor-level labels with all cases confirmed by follow-up care or core needle biopsy result. To enable research into breast disease detection, tumor segmentation and classification, the BrEaST dataset is made publicly available with the CC-BY 4.0 license.
Collapse
Affiliation(s)
- Anna Pawłowska
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Pawinskiego 5B, 02-106, Warsaw, Poland
| | - Anna Ćwierz-Pieńkowska
- Maria Sklodowska-Curie National Institute of Oncology - National Research Institute Branch in Krakow ul, Garncarska 11, 31-115, Kraków, Poland
| | - Agnieszka Domalik
- Maria Sklodowska-Curie National Institute of Oncology - National Research Institute Branch in Krakow ul, Garncarska 11, 31-115, Kraków, Poland
| | - Dominika Jaguś
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Pawinskiego 5B, 02-106, Warsaw, Poland
| | - Piotr Kasprzak
- Breast Unit, Lower Silesian Oncology, Pulmonology and Hematology Center, pl. Ludwika Hirszfelda 12, 53-413, Wrocław, Poland
| | - Rafał Matkowski
- Breast Unit, Lower Silesian Oncology, Pulmonology and Hematology Center, pl. Ludwika Hirszfelda 12, 53-413, Wrocław, Poland
- Department of Oncology, Wrocław Medical University, Wrocław, Poland
| | - Łukasz Fura
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Pawinskiego 5B, 02-106, Warsaw, Poland
| | - Andrzej Nowicki
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Pawinskiego 5B, 02-106, Warsaw, Poland
| | - Norbert Żołek
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Pawinskiego 5B, 02-106, Warsaw, Poland.
| |
Collapse
|
34
|
Tagnamas J, Ramadan H, Yahyaouy A, Tairi H. Multi-task approach based on combined CNN-transformer for efficient segmentation and classification of breast tumors in ultrasound images. Vis Comput Ind Biomed Art 2024; 7:2. [PMID: 38273164 PMCID: PMC10811315 DOI: 10.1186/s42492-024-00155-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 01/11/2024] [Indexed: 01/27/2024] Open
Abstract
Accurate segmentation of breast ultrasound (BUS) images is crucial for early diagnosis and treatment of breast cancer. Further, the task of segmenting lesions in BUS images continues to pose significant challenges due to the limitations of convolutional neural networks (CNNs) in capturing long-range dependencies and obtaining global context information. Existing methods relying solely on CNNs have struggled to address these issues. Recently, ConvNeXts have emerged as a promising architecture for CNNs, while transformers have demonstrated outstanding performance in diverse computer vision tasks, including the analysis of medical images. In this paper, we propose a novel breast lesion segmentation network CS-Net that combines the strengths of ConvNeXt and Swin Transformer models to enhance the performance of the U-Net architecture. Our network operates on BUS images and adopts an end-to-end approach to perform segmentation. To address the limitations of CNNs, we design a hybrid encoder that incorporates modified ConvNeXt convolutions and Swin Transformer. Furthermore, to enhance capturing the spatial and channel attention in feature maps we incorporate the Coordinate Attention Module. Second, we design an Encoder-Decoder Features Fusion Module that facilitates the fusion of low-level features from the encoder with high-level semantic features from the decoder during the image reconstruction. Experimental results demonstrate the superiority of our network over state-of-the-art image segmentation methods for BUS lesions segmentation.
Collapse
Affiliation(s)
- Jaouad Tagnamas
- Department of Informatics, Faculty of Sciences Dhar El Mahraz, University of Sidi Mohamed Ben Abdellah, 30000, Fez, Morocco.
| | - Hiba Ramadan
- Department of Informatics, Faculty of Sciences Dhar El Mahraz, University of Sidi Mohamed Ben Abdellah, 30000, Fez, Morocco
| | - Ali Yahyaouy
- Department of Informatics, Faculty of Sciences Dhar El Mahraz, University of Sidi Mohamed Ben Abdellah, 30000, Fez, Morocco
| | - Hamid Tairi
- Department of Informatics, Faculty of Sciences Dhar El Mahraz, University of Sidi Mohamed Ben Abdellah, 30000, Fez, Morocco
| |
Collapse
|
35
|
Zhu C, Chai X, Xiao Y, Liu X, Zhang R, Yang Z, Wang Z. Swin-Net: A Swin-Transformer-Based Network Combing with Multi-Scale Features for Segmentation of Breast Tumor Ultrasound Images. Diagnostics (Basel) 2024; 14:269. [PMID: 38337784 PMCID: PMC10854866 DOI: 10.3390/diagnostics14030269] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 01/19/2024] [Accepted: 01/22/2024] [Indexed: 02/12/2024] Open
Abstract
Breast cancer is one of the most common cancers in the world, especially among women. Breast tumor segmentation is a key step in the identification and localization of the breast tumor region, which has important clinical significance. Inspired by the swin-transformer model with powerful global modeling ability, we propose a semantic segmentation framework named Swin-Net for breast ultrasound images, which combines Transformer and Convolutional Neural Networks (CNNs) to effectively improve the accuracy of breast ultrasound segmentation. Firstly, our model utilizes a swin-transformer encoder with stronger learning ability, which can extract features of images more precisely. In addition, two new modules are introduced in our method, including the feature refinement and enhancement module (RLM) and the hierarchical multi-scale feature fusion module (HFM), given that the influence of ultrasonic image acquisition methods and the characteristics of tumor lesions is difficult to capture. Among them, the RLM module is used to further refine and enhance the feature map learned by the transformer encoder. The HFM module is used to process multi-scale high-level semantic features and low-level details, so as to achieve effective cross-layer feature fusion, suppress noise, and improve model segmentation performance. Experimental results show that Swin-Net performs significantly better than the most advanced methods on the two public benchmark datasets. In particular, it achieves an absolute improvement of 1.4-1.8% on Dice. Additionally, we provide a new dataset of breast ultrasound images on which we test the effect of our model, further demonstrating the validity of our method. In summary, the proposed Swin-Net framework makes significant advancements in breast ultrasound image segmentation, providing valuable exploration for research and applications in this domain.
Collapse
Affiliation(s)
- Chengzhang Zhu
- School of Humanities, Central South University, Changsha 410012, China;
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Xian Chai
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Yalong Xiao
- School of Humanities, Central South University, Changsha 410012, China;
| | - Xu Liu
- Department of Medical Ultrasound, Hunan Cancer Hospital/The Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha 410031, China;
| | - Renmao Zhang
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Zhangzheng Yang
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; (X.C.); (R.Z.); (Z.Y.)
| | - Zhiyuan Wang
- Department of Medical Ultrasound, Hunan Cancer Hospital/The Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha 410031, China;
| |
Collapse
|
36
|
Jiang Z, Gandomkar Z, Trieu PD(Y, Tavakoli Taba S, Barron ML, Obeidy P, Lewis SJ. Evaluating Recalibrating AI Models for Breast Cancer Diagnosis in a New Context: Insights from Transfer Learning, Image Enhancement and High-Quality Training Data Integration. Cancers (Basel) 2024; 16:322. [PMID: 38254813 PMCID: PMC10814142 DOI: 10.3390/cancers16020322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 01/07/2024] [Accepted: 01/08/2024] [Indexed: 01/24/2024] Open
Abstract
This paper investigates the adaptability of four state-of-the-art artificial intelligence (AI) models to the Australian mammographic context through transfer learning, explores the impact of image enhancement on model performance and analyses the relationship between AI outputs and histopathological features for clinical relevance and accuracy assessment. A total of 1712 screening mammograms (n = 856 cancer cases and n = 856 matched normal cases) were used in this study. The 856 cases with cancer lesions were annotated by two expert radiologists and the level of concordance between their annotations was used to establish two sets: a 'high-concordances subset' with 99% agreement of cancer location and an 'entire dataset' with all cases included. The area under the receiver operating characteristic curve (AUC) was used to evaluate the performance of Globally aware Multiple Instance Classifier (GMIC), Global-Local Activation Maps (GLAM), I&H and End2End AI models, both in the pretrained and transfer learning modes, with and without applying the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm. The four AI models with and without transfer learning in the high-concordance subset outperformed those in the entire dataset. Applying the CLAHE algorithm to mammograms improved the performance of the AI models. In the high-concordance subset with the transfer learning and CLAHE algorithm applied, the AUC of the GMIC model was highest (0.912), followed by the GLAM model (0.909), I&H (0.893) and End2End (0.875). There were significant differences (p < 0.05) in the performances of the four AI models between the high-concordance subset and the entire dataset. The AI models demonstrated significant differences in malignancy probability concerning different tumour size categories in mammograms. The performance of AI models was affected by several factors such as concordance classification, image enhancement and transfer learning. Mammograms with a strong concordance with radiologists' annotations, applying image enhancement and transfer learning could enhance the accuracy of AI models.
Collapse
Affiliation(s)
- Zhengqiang Jiang
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
| | - Ziba Gandomkar
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
| | - Phuong Dung (Yun) Trieu
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
| | - Seyedamir Tavakoli Taba
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
| | - Melissa L. Barron
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
| | - Peyman Obeidy
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
| | - Sarah J. Lewis
- Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney 2006, Australia; (Z.G.); (P.D.T.); (S.T.T.); (M.L.B.); (P.O.)
- School of Health Sciences, Western Sydney University, Campbelltown 2560, Australia
| |
Collapse
|
37
|
Mori R, Okawa M, Tokumaru Y, Niwa Y, Matsuhashi N, Futamura M. Application of an artificial intelligence-based system in the diagnosis of breast ultrasound images obtained using a smartphone. World J Surg Oncol 2024; 22:2. [PMID: 38167161 PMCID: PMC10759682 DOI: 10.1186/s12957-023-03286-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 12/26/2023] [Indexed: 01/05/2024] Open
Abstract
BACKGROUND Breast ultrasound (US) is useful for dense breasts, and the introduction of artificial intelligence (AI)-assisted diagnoses of breast US images should be considered. However, the implementation of AI-based technologies in clinical practice is problematic because of the costs of introducing such approaches to hospital information systems (HISs) and the security risk of connecting HIS to the Internet to access AI services. To solve these problems, we developed a system that applies AI to the analysis of breast US images captured using a smartphone. METHODS Training data were prepared using 115 images of benign lesions and 201 images of malignant lesions acquired at the Division of Breast Surgery, Gifu University Hospital. YOLOv3 (object detection models) was used to detect lesions on US images. A graphical user interface (GUI) was developed to predict an AI server. A smartphone application was also developed for capturing US images displayed on the HIS monitor with its camera and displaying the prediction results received from the AI server. The sensitivity and specificity of the prediction performed on the AI server and via the smartphone were calculated using 60 images spared from the training. RESULTS The established AI showed 100% sensitivity and 75% specificity for malignant lesions and took 0.2 s per prediction with the AI sever. Prediction using a smartphone required 2 s per prediction and showed 100% sensitivity and 97.5% specificity for malignant lesions. CONCLUSIONS Good-quality predictions were obtained using the AI server. Moreover, the quality of the prediction via the smartphone was slightly better than that on the AI server, which can be safely and inexpensively introduced into HISs.
Collapse
Affiliation(s)
- Ryutaro Mori
- Department of Breast Surgery, Gifu University Hospital, 1-1 Yanagido, Gifu, 501-1194, Japan.
| | - Mai Okawa
- Department of Breast Surgery, Gifu University Hospital, 1-1 Yanagido, Gifu, 501-1194, Japan
| | - Yoshihisa Tokumaru
- Department of Breast Surgery, Gifu University Hospital, 1-1 Yanagido, Gifu, 501-1194, Japan
| | - Yoshimi Niwa
- Department of Breast Surgery, Gifu University Hospital, 1-1 Yanagido, Gifu, 501-1194, Japan
| | - Nobuhisa Matsuhashi
- Department of Gastroenterological Surgery Pediatric Surgery, Gifu University, Graduate School of Medicine, 1-1 Yanagido, Gifu, 501-1194, Japan
| | - Manabu Futamura
- Department of Breast Surgery, Gifu University Hospital, 1-1 Yanagido, Gifu, 501-1194, Japan
| |
Collapse
|
38
|
Shankari N, Kudva V, Hegde RB. Breast Mass Detection and Classification Using Machine Learning Approaches on Two-Dimensional Mammogram: A Review. Crit Rev Biomed Eng 2024; 52:41-60. [PMID: 38780105 DOI: 10.1615/critrevbiomedeng.2024051166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/25/2024]
Abstract
Breast cancer is a leading cause of mortality among women, both in India and globally. The prevalence of breast masses is notably common in women aged 20 to 60. These breast masses are classified, according to the breast imaging-reporting and data systems (BI-RADS) standard, into categories such as fibroadenoma, breast cysts, benign, and malignant masses. To aid in the diagnosis of breast disorders, imaging plays a vital role, with mammography being the most widely used modality for detecting breast abnormalities over the years. However, the process of identifying breast diseases through mammograms can be time-consuming, requiring experienced radiologists to review a significant volume of images. Early detection of breast masses is crucial for effective disease management, ultimately reducing mortality rates. To address this challenge, advancements in image processing techniques, specifically utilizing artificial intelligence (AI) and machine learning (ML), have tiled the way for the development of decision support systems. These systems assist radiologists in the accurate identification and classification of breast disorders. This paper presents a review of various studies where diverse machine learning approaches have been applied to digital mammograms. These approaches aim to identify breast masses and classify them into distinct subclasses such as normal, benign and malignant. Additionally, the paper highlights both the advantages and limitations of existing techniques, offering valuable insights for the benefit of future research endeavors in this critical area of medical imaging and breast health.
Collapse
Affiliation(s)
- N Shankari
- NITTE (Deemed to be University), Department of Electronics and Communication Engineering, NMAM Institute of Technology, Nitte 574110, Karnataka, India
| | - Vidya Kudva
- School of Information Sciences, Manipal Academy of Higher Education, Manipal, India -576104; Nitte Mahalinga Adyanthaya Memorial Institute of Technology, Nitte, India - 574110
| | - Roopa B Hegde
- NITTE (Deemed to be University), Department of Electronics and Communication Engineering, NMAM Institute of Technology, Nitte - 574110, Karnataka, India
| |
Collapse
|
39
|
Zhang L, Xu R, Zhao J. Learning technology for detection and grading of cancer tissue using tumour ultrasound images1. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024; 32:157-171. [PMID: 37424493 DOI: 10.3233/xst-230085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
BACKGROUND Early diagnosis of breast cancer is crucial to perform effective therapy. Many medical imaging modalities including MRI, CT, and ultrasound are used to diagnose cancer. OBJECTIVE This study aims to investigate feasibility of applying transfer learning techniques to train convoluted neural networks (CNNs) to automatically diagnose breast cancer via ultrasound images. METHODS Transfer learning techniques helped CNNs recognise breast cancer in ultrasound images. Each model's training and validation accuracies were assessed using the ultrasound image dataset. Ultrasound images educated and tested the models. RESULTS MobileNet had the greatest accuracy during training and DenseNet121 during validation. Transfer learning algorithms can detect breast cancer in ultrasound images. CONCLUSIONS Based on the results, transfer learning models may be useful for automated breast cancer diagnosis in ultrasound images. However, only a trained medical professional should diagnose cancer, and computational approaches should only be used to help make quick decisions.
Collapse
Affiliation(s)
- Liyan Zhang
- Department of Ultrasound, Sunshine Union Hospital, Weifang, China
| | - Ruiyan Xu
- College of Health, Binzhou Polytechnical College, Binzhou, China
| | - Jingde Zhao
- Department of Imaging, Qingdao Hospital of Traditional Chinese Medicine (Qingdao HaiCi Hospital), Qingdao, China
| |
Collapse
|
40
|
Sharma P, Nayak DR, Balabantaray BK, Tanveer M, Nayak R. A survey on cancer detection via convolutional neural networks: Current challenges and future directions. Neural Netw 2024; 169:637-659. [PMID: 37972509 DOI: 10.1016/j.neunet.2023.11.006] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 10/21/2023] [Accepted: 11/04/2023] [Indexed: 11/19/2023]
Abstract
Cancer is a condition in which abnormal cells uncontrollably split and damage the body tissues. Hence, detecting cancer at an early stage is highly essential. Currently, medical images play an indispensable role in detecting various cancers; however, manual interpretation of these images by radiologists is observer-dependent, time-consuming, and tedious. An automatic decision-making process is thus an essential need for cancer detection and diagnosis. This paper presents a comprehensive survey on automated cancer detection in various human body organs, namely, the breast, lung, liver, prostate, brain, skin, and colon, using convolutional neural networks (CNN) and medical imaging techniques. It also includes a brief discussion about deep learning based on state-of-the-art cancer detection methods, their outcomes, and the possible medical imaging data used. Eventually, the description of the dataset used for cancer detection, the limitations of the existing solutions, future trends, and challenges in this domain are discussed. The utmost goal of this paper is to provide a piece of comprehensive and insightful information to researchers who have a keen interest in developing CNN-based models for cancer detection.
Collapse
Affiliation(s)
- Pallabi Sharma
- School of Computer Science, UPES, Dehradun, 248007, Uttarakhand, India.
| | - Deepak Ranjan Nayak
- Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, 302017, Rajasthan, India.
| | - Bunil Kumar Balabantaray
- Computer Science and Engineering, National Institute of Technology Meghalaya, Shillong, 793003, Meghalaya, India.
| | - M Tanveer
- Department of Mathematics, Indian Institute of Technology Indore, Simrol, 453552, Indore, India.
| | - Rajashree Nayak
- School of Applied Sciences, Birla Global University, Bhubaneswar, 751029, Odisha, India.
| |
Collapse
|
41
|
Azad R, Kazerouni A, Heidari M, Aghdam EK, Molaei A, Jia Y, Jose A, Roy R, Merhof D. Advances in medical image analysis with vision Transformers: A comprehensive review. Med Image Anal 2024; 91:103000. [PMID: 37883822 DOI: 10.1016/j.media.2023.103000] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 09/30/2023] [Accepted: 10/11/2023] [Indexed: 10/28/2023]
Abstract
The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.
Collapse
Affiliation(s)
- Reza Azad
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Amirhossein Kazerouni
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Moein Heidari
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | | | - Amirali Molaei
- School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Yiwei Jia
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Abin Jose
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Rijo Roy
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Dorit Merhof
- Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany; Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
| |
Collapse
|
42
|
Irmici G, Cè M, Pepa GD, D'Ascoli E, De Berardinis C, Giambersio E, Rabiolo L, La Rocca L, Carriero S, Depretto C, Scaperrotta G, Cellina M. Exploring the Potential of Artificial Intelligence in Breast Ultrasound. Crit Rev Oncog 2024; 29:15-28. [PMID: 38505878 DOI: 10.1615/critrevoncog.2023048873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/21/2024]
Abstract
Breast ultrasound has emerged as a valuable imaging modality in the detection and characterization of breast lesions, particularly in women with dense breast tissue or contraindications for mammography. Within this framework, artificial intelligence (AI) has garnered significant attention for its potential to improve diagnostic accuracy in breast ultrasound and revolutionize the workflow. This review article aims to comprehensively explore the current state of research and development in harnessing AI's capabilities for breast ultrasound. We delve into various AI techniques, including machine learning, deep learning, as well as their applications in automating lesion detection, segmentation, and classification tasks. Furthermore, the review addresses the challenges and hurdles faced in implementing AI systems in breast ultrasound diagnostics, such as data privacy, interpretability, and regulatory approval. Ethical considerations pertaining to the integration of AI into clinical practice are also discussed, emphasizing the importance of maintaining a patient-centered approach. The integration of AI into breast ultrasound holds great promise for improving diagnostic accuracy, enhancing efficiency, and ultimately advancing patient's care. By examining the current state of research and identifying future opportunities, this review aims to contribute to the understanding and utilization of AI in breast ultrasound and encourage further interdisciplinary collaboration to maximize its potential in clinical practice.
Collapse
Affiliation(s)
- Giovanni Irmici
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Maurizio Cè
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Gianmarco Della Pepa
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Elisa D'Ascoli
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Claudia De Berardinis
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Emilia Giambersio
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Lidia Rabiolo
- Dipartimento di Biomedicina, Neuroscienze e Diagnostica Avanzata, Policlinico Università di Palermo, Palermo, Italy
| | - Ludovica La Rocca
- Postgraduation School in Radiodiagnostics, Università degli Studi di Napoli
| | - Serena Carriero
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Catherine Depretto
- Breast Radiology Unit, Fondazione IRCCS, Istituto Nazionale Tumori, Milano, Italy
| | | | - Michaela Cellina
- Radiology Department, Fatebenefratelli Hospital, ASST Fatebenefratelli Sacco, Milano, Piazza Principessa Clotilde 3, 20121, Milan, Italy
| |
Collapse
|
43
|
Zhou G, Mosadegh B. Distilling Knowledge From an Ensemble of Vision Transformers for Improved Classification of Breast Ultrasound. Acad Radiol 2024; 31:104-120. [PMID: 37666747 DOI: 10.1016/j.acra.2023.08.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 07/20/2023] [Accepted: 08/05/2023] [Indexed: 09/06/2023]
Abstract
RATIONALE AND OBJECTIVES To develop a deep learning model for the automated classification of breast ultrasound images as benign or malignant. More specifically, the application of vision transformers, ensemble learning, and knowledge distillation is explored for breast ultrasound classification. MATERIALS AND METHODS Single view, B-mode ultrasound images were curated from the publicly available Breast Ultrasound Image (BUSI) dataset, which has categorical ground truth labels (benign vs malignant) assigned by radiologists and malignant cases confirmed by biopsy. The performance of vision transformers (ViT) is compared to convolutional neural networks (CNN), followed by a comparison between supervised, self-supervised, and randomly initialized ViT. Subsequently, the ensemble of 10 independently trained ViT, where the ensemble model is the unweighted average of the output of each individual model is compared to the performance of each ViT alone. Finally, we train a single ViT to emulate the ensembled ViT using knowledge distillation. RESULTS On this dataset that was trained using five-fold cross validation, ViT outperforms CNN, while self-supervised ViT outperform supervised and randomly initialized ViT. The ensemble model achieves an area under the receiver operating characteristics curve (AuROC) and area under the precision recall curve (AuPRC) of 0.977 and 0.965 on the test set, outperforming the average AuROC and AuPRC of the independently trained ViTs (0.958 ± 0.05 and 0.931 ± 0.016). The distilled ViT achieves an AuROC and AuPRC of 0.972 and 0.960. CONCLUSION Both transfer learning and ensemble learning can each offer increased performance independently and can be sequentially combined to collectively improve the performance of the final model. Furthermore, a single vision transformer can be trained to match the performance of an ensemble of a set of vision transformers using knowledge distillation.
Collapse
Affiliation(s)
| | - Bobak Mosadegh
- Dalio Institute of Cardiovascular Imaging, Department of Radiology, Weill Cornell Medicine, New York, New York
| |
Collapse
|
44
|
Tasnim J, Hasan MK. CAM-QUS guided self-tuning modular CNNs with multi-loss functions for fully automated breast lesion classification in ultrasound images. Phys Med Biol 2023; 69:015018. [PMID: 38056017 DOI: 10.1088/1361-6560/ad1319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Accepted: 12/06/2023] [Indexed: 12/08/2023]
Abstract
Objective.Breast cancer is the major cause of cancer death among women worldwide. Deep learning-based computer-aided diagnosis (CAD) systems for classifying lesions in breast ultrasound images can help materialise the early detection of breast cancer and enhance survival chances.Approach.This paper presents a completely automated BUS diagnosis system with modular convolutional neural networks tuned with novel loss functions. The proposed network comprises a dynamic channel input enhancement network, an attention-guided InceptionV3-based feature extraction network, a classification network, and a parallel feature transformation network to map deep features into quantitative ultrasound (QUS) feature space. These networks function together to improve classification accuracy by increasing the separation of benign and malignant class-specific features and enriching them simultaneously. Unlike the categorical crossentropy (CCE) loss-based traditional approaches, our method uses two additional novel losses: class activation mapping (CAM)-based and QUS feature-based losses, to capacitate the overall network learn the extraction of clinically valued lesion shape and texture-related properties focusing primarily the lesion area for explainable AI (XAI).Main results.Experiments on four public, one private, and a combined breast ultrasound dataset are used to validate our strategy. The suggested technique obtains an accuracy of 97.28%, sensitivity of 93.87%, F1-score of 95.42% on dataset 1 (BUSI), and an accuracy of 91.50%, sensitivity of 89.38%, and F1-score of 89.31% on the combined dataset, consisting of 1494 images collected from hospitals in five demographic locations using four ultrasound systems of different manufacturers. These results outperform techniques reported in the literature by a considerable margin.Significance.The proposed CAD system provides diagnosis from the auto-focused lesion area of B-mode BUS images, avoiding the explicit requirement of any segmentation or region of interest extraction, and thus can be a handy tool for making accurate and reliable diagnoses even in unspecialized healthcare centers.
Collapse
Affiliation(s)
- Jarin Tasnim
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1205, Bangladesh
| | - Md Kamrul Hasan
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1205, Bangladesh
| |
Collapse
|
45
|
Oh K, Lee SE, Kim EK. 3-D breast nodule detection on automated breast ultrasound using faster region-based convolutional neural networks and U-Net. Sci Rep 2023; 13:22625. [PMID: 38114666 PMCID: PMC10730541 DOI: 10.1038/s41598-023-49794-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 12/12/2023] [Indexed: 12/21/2023] Open
Abstract
Mammography is currently the most commonly used modality for breast cancer screening. However, its sensitivity is relatively low in women with dense breasts. Dense breast tissues show a relatively high rate of interval cancers and are at high risk for developing breast cancer. As a supplemental screening tool, ultrasonography is a widely adopted imaging modality to standard mammography, especially for dense breasts. Lately, automated breast ultrasound imaging has gained attention due to its advantages over hand-held ultrasound imaging. However, automated breast ultrasound imaging requires considerable time and effort for reading because of the lengthy data. Hence, developing a computer-aided nodule detection system for automated breast ultrasound is invaluable and impactful practically. This study proposes a three-dimensional breast nodule detection system based on a simple two-dimensional deep-learning model exploiting automated breast ultrasound. Additionally, we provide several postprocessing steps to reduce false positives. In our experiments using the in-house automated breast ultrasound datasets, a sensitivity of [Formula: see text] with 8.6 false positives is achieved on unseen test data at best.
Collapse
Affiliation(s)
- Kangrok Oh
- Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, 03722, Republic of Korea
| | - Si Eun Lee
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, 363, Dongbaekjukjeon-daero, Giheung-gu, Yongin, Gyeonggi-do, 16995, Republic of Korea
| | - Eun-Kyung Kim
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, 363, Dongbaekjukjeon-daero, Giheung-gu, Yongin, Gyeonggi-do, 16995, Republic of Korea.
| |
Collapse
|
46
|
Ciobotaru A, Bota MA, Goța DI, Miclea LC. Multi-Instance Classification of Breast Tumor Ultrasound Images Using Convolutional Neural Networks and Transfer Learning. Bioengineering (Basel) 2023; 10:1419. [PMID: 38136010 PMCID: PMC10740646 DOI: 10.3390/bioengineering10121419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Revised: 12/07/2023] [Accepted: 12/12/2023] [Indexed: 12/24/2023] Open
Abstract
BACKGROUND Breast cancer is arguably one of the leading causes of death among women around the world. The automation of the early detection process and classification of breast masses has been a prominent focus for researchers in the past decade. The utilization of ultrasound imaging is prevalent in the diagnostic evaluation of breast cancer, with its predictive accuracy being dependent on the expertise of the specialist. Therefore, there is an urgent need to create fast and reliable ultrasound image detection algorithms to address this issue. METHODS This paper aims to compare the efficiency of six state-of-the-art, fine-tuned deep learning models that can classify breast tissue from ultrasound images into three classes: benign, malignant, and normal, using transfer learning. Additionally, the architecture of a custom model is introduced and trained from the ground up on a public dataset containing 780 images, which was further augmented to 3900 and 7800 images, respectively. What is more, the custom model is further validated on another private dataset containing 163 ultrasound images divided into two classes: benign and malignant. The pre-trained architectures used in this work are ResNet-50, Inception-V3, Inception-ResNet-V2, MobileNet-V2, VGG-16, and DenseNet-121. The performance evaluation metrics that are used in this study are as follows: Precision, Recall, F1-Score and Specificity. RESULTS The experimental results show that the models trained on the augmented dataset with 7800 images obtained the best performance on the test set, having 94.95 ± 0.64%, 97.69 ± 0.52%, 97.69 ± 0.13%, 97.77 ± 0.29%, 95.07 ± 0.41%, 98.11 ± 0.10%, and 96.75 ± 0.26% accuracy for the ResNet-50, MobileNet-V2, InceptionResNet-V2, VGG-16, Inception-V3, DenseNet-121, and our model, respectively. CONCLUSION Our proposed model obtains competitive results, outperforming some state-of-the-art models in terms of accuracy and training time.
Collapse
Affiliation(s)
- Alexandru Ciobotaru
- Department of Automation, Faculty of Automation and Computer Science, Technical University of Cluj-Napoca, 400114 Cluj-Napoca, Romania; (A.C.); (D.I.G.)
| | - Maria Aurora Bota
- Department of Advanced Computing Sciences, Faculty of Sciences and Engineering, Maastricht University, 6229 EN Maastricht, The Netherlands;
| | - Dan Ioan Goța
- Department of Automation, Faculty of Automation and Computer Science, Technical University of Cluj-Napoca, 400114 Cluj-Napoca, Romania; (A.C.); (D.I.G.)
| | - Liviu Cristian Miclea
- Department of Automation, Faculty of Automation and Computer Science, Technical University of Cluj-Napoca, 400114 Cluj-Napoca, Romania; (A.C.); (D.I.G.)
| |
Collapse
|
47
|
Wang Q, Jia X, Luo T, Yu J, Xia S. Deep learning algorithm using bispectrum analysis energy feature maps based on ultrasound radiofrequency signals to detect breast cancer. Front Oncol 2023; 13:1272427. [PMID: 38179175 PMCID: PMC10766103 DOI: 10.3389/fonc.2023.1272427] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Accepted: 11/15/2023] [Indexed: 01/06/2024] Open
Abstract
Background Ultrasonography is an important imaging method for clinical breast cancer screening. As the original echo signals of ultrasonography, ultrasound radiofrequency (RF) signals provide abundant tissue macroscopic and microscopic information and have important development and utilization value in breast cancer detection. Methods In this study, we proposed a deep learning method based on bispectrum analysis feature maps to process RF signals and realize breast cancer detection. The bispectrum analysis energy feature maps with frequency subdivision were first proposed and applied to breast cancer detection in this study. Our deep learning network was based on a weight sharing network framework for the input of multiple feature maps. A feature map attention module was designed for multiple feature maps input of the network to adaptively learn both feature maps and features that were conducive to classification. We also designed a similarity constraint factor, learning the similarity and difference between feature maps by cosine distance. Results The experiment results showed that the areas under the receiver operating characteristic curves of our proposed method in the validation set and two independent test sets for benign and malignant breast tumor classification were 0.913, 0.900, and 0.885, respectively. The performance of the model combining four ultrasound bispectrum analysis energy feature maps in breast cancer detection was superior to that of the model using an ultrasound grayscale image and the model using a single bispectrum analysis energy feature map in this study. Conclusion The combination of deep learning technology and our proposed ultrasound bispectrum analysis energy feature maps effectively realized breast cancer detection and was an efficient method of feature extraction and utilization of ultrasound RF signals.
Collapse
Affiliation(s)
- Qingmin Wang
- School of Information Science and Engineering, Fudan University, Shanghai, China
| | - Xiaohong Jia
- Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Ting Luo
- Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Jinhua Yu
- School of Information Science and Engineering, Fudan University, Shanghai, China
| | - Shujun Xia
- Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
48
|
Harrison P, Hasan R, Park K. State-of-the-Art of Breast Cancer Diagnosis in Medical Images via Convolutional Neural Networks (CNNs). JOURNAL OF HEALTHCARE INFORMATICS RESEARCH 2023; 7:387-432. [PMID: 37927373 PMCID: PMC10620373 DOI: 10.1007/s41666-023-00144-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2022] [Revised: 08/14/2023] [Accepted: 08/22/2023] [Indexed: 11/07/2023]
Abstract
Early detection of breast cancer is crucial for a better prognosis. Various studies have been conducted where tumor lesions are detected and localized on images. This is a narrative review where the studies reviewed are related to five different image modalities: histopathological, mammogram, magnetic resonance imaging (MRI), ultrasound, and computed tomography (CT) images, making it different from other review studies where fewer image modalities are reviewed. The goal is to have the necessary information, such as pre-processing techniques and CNN-based diagnosis techniques for the five modalities, readily available in one place for future studies. Each modality has pros and cons, such as mammograms might give a high false positive rate for radiographically dense breasts, while ultrasounds with low soft tissue contrast result in early-stage false detection, and MRI provides a three-dimensional volumetric image, but it is expensive and cannot be used as a routine test. Various studies were manually reviewed using particular inclusion and exclusion criteria; as a result, 91 recent studies that classify and detect tumor lesions on breast cancer images from 2017 to 2022 related to the five image modalities were included. For histopathological images, the maximum accuracy achieved was around 99 % , and the maximum sensitivity achieved was 97.29 % by using DenseNet, ResNet34, and ResNet50 architecture. For mammogram images, the maximum accuracy achieved was 96.52 % using a customized CNN architecture. For MRI, the maximum accuracy achieved was 98.33 % using customized CNN architecture. For ultrasound, the maximum accuracy achieved was around 99 % by using DarkNet-53, ResNet-50, G-CNN, and VGG. For CT, the maximum sensitivity achieved was 96 % by using Xception architecture. Histopathological and ultrasound images achieved higher accuracy of around 99 % by using ResNet34, ResNet50, DarkNet-53, G-CNN, and VGG compared to other modalities for either of the following reasons: use of pre-trained architectures with pre-processing techniques, use of modified architectures with pre-processing techniques, use of two-stage CNN, and higher number of studies available for Artificial Intelligence (AI)/machine learning (ML) researchers to reference. One of the gaps we found is that only a single image modality is used for CNN-based diagnosis; in the future, a multiple image modality approach can be used to design a CNN architecture with higher accuracy.
Collapse
Affiliation(s)
- Pratibha Harrison
- Department of Computer and Information Science, University of Massachusetts Dartmouth, 285 Old Westport Rd, North Dartmouth, 02747 MA USA
| | - Rakib Hasan
- Department of Mechanical Engineering, Khulna University of Engineering & Technology, PhulBari Gate, Khulna, 9203 Bangladesh
| | - Kihan Park
- Department of Mechanical Engineering, University of Massachusetts Dartmouth, 285 Old Westport Rd, North Dartmouth, 02747 MA USA
| |
Collapse
|
49
|
Zhou J, Hou Z, Lu H, Wang W, Zhao W, Wang Z, Zheng D, Wang S, Tang W, Qu X. A deep supervised transformer U-shaped full-resolution residual network for the segmentation of breast ultrasound image. Med Phys 2023; 50:7513-7524. [PMID: 37816131 DOI: 10.1002/mp.16765] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 09/05/2023] [Accepted: 09/07/2023] [Indexed: 10/12/2023] Open
Abstract
PURPOSE Breast ultrasound (BUS) is an important breast imaging tool. Automatic BUS image segmentation can measure the breast tumor size objectively and reduce doctors' workload. In this article, we proposed a deep supervised transformer U-shaped full-resolution residual network (DSTransUFRRN) to segment BUS images. METHODS In the proposed method, a full-resolution residual stream and a deep supervision mechanism were introduced into TransU-Net. The residual stream can keep full resolution features from different levels and enhance features fusion. Then, the deep supervision can suppress gradient dispersion. Moreover, the transformer module can suppress irrelevant features and improve feature extraction process. Two datasets (dataset A and B) were used for training and evaluation. The dataset A included 980 BUS image samples and the dataset B had 163 BUS image samples. RESULTS Cross-validation was conducted. For the dataset A, the proposed DSTransUFRRN achieved significantly higher Dice (91.04 ± 0.86%) than all compared methods (p < 0.05). For the dataset B, the Dice was lower than that for the dataset A due to the small number of samples, but the Dice of DSTransUFRRN (88.15% ± 2.11%) was significantly higher than that of other compared methods (p < 0.05). CONCLUSIONS In this study, we proposed DSTransUFRRN for BUS image segmentation. The proposed methods achieved significantly higher accuracy than the compared previous methods.
Collapse
Affiliation(s)
- Jiale Zhou
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Zuoxun Hou
- Beijing Institute of Mechanics & Electricity, Beijing, China
| | - Hongyan Lu
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Wenhan Wang
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Wanchen Zhao
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| | - Zenan Wang
- Department of Gastroenterology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Dezhi Zheng
- Research Institute for Frontier Science, Beihang University, Beijing, China
| | - Shuai Wang
- School of Computer Science and Engineering, Beihang University, Beijing, China
| | - Wenzhong Tang
- School of Computer Science and Engineering, Beihang University, Beijing, China
| | - Xiaolei Qu
- School of Instrumentation and Optoelectronics Engineering, Beihang University, Beijing, China
| |
Collapse
|
50
|
Gómez-Flores W, Pereira WCDA. Gray-to-color image conversion in the classification of breast lesions on ultrasound using pre-trained deep neural networks. Med Biol Eng Comput 2023; 61:3193-3207. [PMID: 37713158 DOI: 10.1007/s11517-023-02928-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 08/29/2023] [Indexed: 09/16/2023]
Abstract
Breast ultrasound (BUS) image classification in benign and malignant classes is often based on pre-trained convolutional neural networks (CNNs) to cope with small-sized training data. Nevertheless, BUS images are single-channel gray-level images, whereas pre-trained CNNs learned from color images with red, green, and blue (RGB) components. Thus, a gray-to-color conversion method is applied to fit the BUS image to the CNN's input layer size. This paper evaluates 13 gray-to-color conversion methods proposed in the literature that follow three strategies: replicating the gray-level image to all RGB channels, decomposing the image to enhance inherent information like the lesion's texture and morphology, and learning a matching layer. Besides, we introduce an image decomposition method based on the lesion's structural information to describe its inner and outer complexity. These gray-to-color conversion methods are evaluated under the same experimental framework using a pre-trained CNN architecture named ResNet-18 and a BUS dataset with more than 3000 images. In addition, the Matthews correlation coefficient (MCC), sensitivity (SEN), and specificity (SPE) measure the classification performance. The experimental results show that decomposition methods outperform replication and learning-based methods when using information from the lesion's binary mask (obtained from a segmentation method), reaching an MCC value greater than 0.70 and specificity up to 0.92, although the sensitivity is about 0.80. On the other hand, regarding the proposed method, the trade-off between sensitivity and specificity is better balanced, obtaining about 0.88 for both indices and an MCC of 0.73. This study contributes to the objective assessment of different gray-to-color conversion approaches in classifying breast lesions, revealing that mask-based decomposition methods improve classification performance. Besides, the proposed method based on structural information improves the sensitivity, obtaining more reliable classification results on malignant cases and potentially benefiting clinical practice.
Collapse
Affiliation(s)
- Wilfrido Gómez-Flores
- Centro de Investigación y de Estudios Avanzados del IPN, Unidad Tamaulipas, Ciudad Victoria, 87138, Tamaulipas, Mexico.
| | | |
Collapse
|