1
|
Bobowicz M, Badocha M, Gwozdziewicz K, Rygusik M, Kalinowska P, Szurowska E, Dziubich T. Segmentation-based BI-RADS ensemble classification of breast tumours in ultrasound images. Int J Med Inform 2024; 189:105522. [PMID: 38852288 DOI: 10.1016/j.ijmedinf.2024.105522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Revised: 05/19/2024] [Accepted: 06/05/2024] [Indexed: 06/11/2024]
Abstract
BACKGROUND The development of computer-aided diagnosis systems in breast cancer imaging is exponential. Since 2016, 81 papers have described the automated segmentation of breast lesions in ultrasound images using artificial intelligence. However, only two papers have dealt with complex BI-RADS classifications. PURPOSE This study addresses the automatic classification of breast lesions into binary classes (benign vs. malignant) and multiple BI-RADS classes based on a single ultrasonographic image. Achieving this task should reduce the subjectivity of an individual operator's assessment. MATERIALS AND METHODS Automatic image segmentation methods (PraNet, CaraNet and FCBFormer) adapted to the specific segmentation task were investigated using the U-Net model as a reference. A new classification method was developed using an ensemble of selected segmentation approaches. All experiments were performed on publicly available BUS B, OASBUD, BUSI and private datasets. RESULTS FCBFormer achieved the best outcomes for the segmentation task with intersection over union metric values of 0.81, 0.80 and 0.73 and Dice values of 0.89, 0.87 and 0.82, respectively, for the BUS B, BUSI and OASBUD datasets. Through a series of experiments, we determined that adding an extra 30-pixel margin to the segmentation mask counteracts the potential errors introduced by the segmentation algorithm. An assembly of the full image classifier, bounding box classifier and masked image classifier was the most accurate for binary classification and had the best accuracy (ACC; 0.908), F1 (0.846) and area under the receiver operating characteristics curve (AUROC; 0.871) in the BUS B and ACC (0.982), F1 (0.984) and AUROC (0.998) in the UCC BUS datasets, outperforming each classifier used separately. It was also the most effective for BI-RADS classification, with ACC of 0.953, F1 of 0.920 and AUROC of 0.986 in UCC BUS. Hard voting was the most effective method for dichotomous classification. For the multi-class BI-RADS classification, the soft voting approach was employed. CONCLUSIONS The proposed new classification approach with an ensemble of segmentation and classification approaches proved more accurate than most published results for binary and multi-class BI-RADS classifications.
Collapse
Affiliation(s)
- Maciej Bobowicz
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Mikołaj Badocha
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Katarzyna Gwozdziewicz
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Marlena Rygusik
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Paulina Kalinowska
- Department of Thoracic Radiology, Karolinska University Hospital, Anna Steckséns g 41, Solna 17176, Sweden.
| | - Edyta Szurowska
- 2(nd) Department of Radiology, Medical University of Gdansk, 17 Smoluchowskiego Str., Gdansk 80-214, Poland.
| | - Tomasz Dziubich
- Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, 11/12 G. Narutowicza Str., Gdańsk 80-233, Poland.
| |
Collapse
|
2
|
Bartolotta TV, Militello C, Prinzi F, Ferraro F, Rundo L, Zarcaro C, Dimarco M, Orlando AAM, Matranga D, Vitabile S. Artificial intelligence-based, semi-automated segmentation for the extraction of ultrasound-derived radiomics features in breast cancer: a prospective multicenter study. LA RADIOLOGIA MEDICA 2024; 129:977-988. [PMID: 38724697 PMCID: PMC11252191 DOI: 10.1007/s11547-024-01826-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 04/29/2024] [Indexed: 07/17/2024]
Abstract
PURPOSE To investigate the feasibility of an artificial intelligence (AI)-based semi-automated segmentation for the extraction of ultrasound (US)-derived radiomics features in the characterization of focal breast lesions (FBLs). MATERIAL AND METHODS Two expert radiologists classified according to US BI-RADS criteria 352 FBLs detected in 352 patients (237 at Center A and 115 at Center B). An AI-based semi-automated segmentation was used to build a machine learning (ML) model on the basis of B-mode US of 237 images (center A) and then validated on an external cohort of B-mode US images of 115 patients (Center B). RESULTS A total of 202 of 352 (57.4%) FBLs were benign, and 150 of 352 (42.6%) were malignant. The AI-based semi-automated segmentation achieved a success rate of 95.7% for one reviewer and 96% for the other, without significant difference (p = 0.839). A total of 15 (4.3%) and 14 (4%) of 352 semi-automated segmentations were not accepted due to posterior acoustic shadowing at B-Mode US and 13 and 10 of them corresponded to malignant lesions, respectively. In the validation cohort, the characterization made by the expert radiologist yielded values of sensitivity, specificity, PPV and NPV of 0.933, 0.9, 0.857, 0.955, respectively. The ML model obtained values of sensitivity, specificity, PPV and NPV of 0.544, 0.6, 0.416, 0.628, respectively. The combined assessment of radiologists and ML model yielded values of sensitivity, specificity, PPV and NPV of 0.756, 0.928, 0.872, 0.855, respectively. CONCLUSION AI-based semi-automated segmentation is feasible, allowing an instantaneous and reproducible extraction of US-derived radiomics features of FBLs. The combination of radiomics and US BI-RADS classification led to a potential decrease of unnecessary biopsy but at the expense of a not negligible increase of potentially missed cancers.
Collapse
Affiliation(s)
- Tommaso Vincenzo Bartolotta
- Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University of Palermo, Palermo, Italy.
| | - Carmelo Militello
- Institute for High-Performance Computing and Networking (ICAR-CNR), Italian National Research Council, Palermo, Italy
| | - Francesco Prinzi
- Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University of Palermo, Palermo, Italy
- Department of Computer Science and Technology, University of Cambridge, Cambridge, UK
| | - Fabiola Ferraro
- Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University of Palermo, Palermo, Italy
| | - Leonardo Rundo
- Department of Information and Electrical Engineering and Applied Mathematics (DIEM), University of Salerno, Fisciano, SA, Italy
| | - Calogero Zarcaro
- Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University of Palermo, Palermo, Italy
| | | | | | - Domenica Matranga
- Department of Health Promotion, Mother and Child Care, Internal Medicine and Medical Specialties (ProMISE), University of Palermo, Palermo, Italy
| | - Salvatore Vitabile
- Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University of Palermo, Palermo, Italy
| |
Collapse
|
3
|
Schnitzler T, Ruppert C, Hejduk P, Borkowski K, Kajüter J, Rossi C, Ciritsis A, Landsmann A, Zaytoun H, Boss A, Schindera S, Burn F. Automatic Detection of Post-Operative Clips in Mammography Using a U-Net Convolutional Neural Network. J Imaging 2024; 10:147. [PMID: 38921624 PMCID: PMC11205252 DOI: 10.3390/jimaging10060147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2024] [Revised: 06/04/2024] [Accepted: 06/13/2024] [Indexed: 06/27/2024] Open
Abstract
BACKGROUND After breast conserving surgery (BCS), surgical clips indicate the tumor bed and, thereby, the most probable area for tumor relapse. The aim of this study was to investigate whether a U-Net-based deep convolutional neural network (dCNN) may be used to detect surgical clips in follow-up mammograms after BCS. METHODS 884 mammograms and 517 tomosynthetic images depicting surgical clips and calcifications were manually segmented and classified. A U-Net-based segmentation network was trained with 922 images and validated with 394 images. An external test dataset consisting of 39 images was annotated by two radiologists with up to 7 years of experience in breast imaging. The network's performance was compared to that of human readers using accuracy and interrater agreement (Cohen's Kappa). RESULTS The overall classification accuracy on the validation set after 45 epochs ranged between 88.2% and 92.6%, indicating that the model's performance is comparable to the decisions of a human reader. In 17.4% of cases, calcifications have been misclassified as post-operative clips. The interrater reliability of the model compared to the radiologists showed substantial agreement (κreader1 = 0.72, κreader2 = 0.78) while the readers compared to each other revealed a Cohen's Kappa of 0.84, thus showing near-perfect agreement. CONCLUSIONS With this study, we show that surgery clips can adequately be identified by an AI technique. A potential application of the proposed technique is patient triage as well as the automatic exclusion of post-operative cases from PGMI (Perfect, Good, Moderate, Inadequate) evaluation, thus improving the quality management workflow.
Collapse
Affiliation(s)
- Tician Schnitzler
- Institute of Radiology, Cantonal Hospital Aarau, 5001 Aarau, Switzerland; (H.Z.); (F.B.)
| | - Carlotta Ruppert
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Patryk Hejduk
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Karol Borkowski
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Jonas Kajüter
- Institute of Diagnostic and Interventional Radiology, University Hospital Basel, 4031 Basel, Switzerland
| | - Cristina Rossi
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Alexander Ciritsis
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Anna Landsmann
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Hasan Zaytoun
- Institute of Radiology, Cantonal Hospital Aarau, 5001 Aarau, Switzerland; (H.Z.); (F.B.)
| | - Andreas Boss
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, 8091 Zurich, Switzerland (K.B.)
| | - Sebastian Schindera
- Institute of Radiology, Cantonal Hospital Aarau, 5001 Aarau, Switzerland; (H.Z.); (F.B.)
| | - Felice Burn
- Institute of Radiology, Cantonal Hospital Aarau, 5001 Aarau, Switzerland; (H.Z.); (F.B.)
| |
Collapse
|
4
|
Guldogan N, Taskin F, Icten GE, Yilmaz E, Turk EB, Erdemli S, Parlakkilic UT, Turkoglu O, Aribal E. Artificial Intelligence in BI-RADS Categorization of Breast Lesions on Ultrasound: Can We Omit Excessive Follow-ups and Biopsies? Acad Radiol 2024; 31:2194-2202. [PMID: 38087719 DOI: 10.1016/j.acra.2023.11.031] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 11/18/2023] [Accepted: 11/20/2023] [Indexed: 07/01/2024]
Abstract
RATIONALE AND OBJECTIVES Artificial intelligence (AI) systems have been increasingly applied to breast ultrasonography. They are expected to decrease the workload of radiologists and to improve diagnostic accuracy. The aim of this study is to evaluate the performance of an AI system for the BI-RADS category assessment in breast masses detected on breast ultrasound. MATERIALS AND METHODS: A total of 715 masses detected in 530 patients were analyzed. Three breast imaging centers of the same institution and nine breast radiologists participated in this study. Ultrasound was performed by one radiologist who obtained two orthogonal views of each detected lesion. These images were retrospectively reviewed by a second radiologist blinded to the patient's clinical data. A commercial AI system evaluated images. The level of agreement between the AI system and the two radiologists and their diagnostic performance were calculated according to dichotomic BI-RADS category assessment. RESULTS This study included 715 breast masses. Of these, 134 (18.75%) were malignant, and 581 (81.25%) were benign. In discriminating benign and probably benign from suspicious lesions, the agreement between AI and the first and second radiologists was moderate statistically. The sensitivity and specificity of radiologist 1, radiologist 2, and AI were calculated as 98.51% and 80.72%, 97.76% and 75.56%, and 98.51% and 65.40%, respectively. For radiologist 1, the positive predictive value (PPV) was 54.10%, the negative predictive value (NPV) was 99.58%, and the accuracy was 84.06%. Radiologist 2 achieved a PPV of 47.99%, NPV of 99.32%, and accuracy of 79.72%. The AI system exhibited a PPV of 39.64%, NPV of 99.48%, and accuracy of 71.61%. Notably, none of the lesions categorized as BI-RADS 2 by AI were malignant, while 2 of the lesions classified as BI-RADS 3 by AI were subsequently confirmed as malignant. By considering AI-assigned BI-RADS 2 as safe, we could potentially avoid 11% (18 out of 163) of benign lesion biopsies and 46.2% (110 out of 238) of follow-ups. CONCLUSION AI proves effective in predicting malignancy. Integrating it into the clinical workflow has the potential to reduce unnecessary biopsies and short-term follow-ups, which, in turn, can contribute to sustainability in healthcare practices.
Collapse
Affiliation(s)
- Nilgun Guldogan
- Breast Clinic, Acibadem Altunizade Hospital, 34662, Istanbul, Turkey (N.G., E.Y., E.B.T., E.A.).
| | - Fusun Taskin
- Department of Radiology, Acibadem M.A.A. University School of Medicine, Atakent University Hospital, 34755, Istanbul, Turkey (F.T., S.E.); Acibadem M.A.A. University Senology Research Institute, 34457, Sarıyer, Istanbul, Turkey (F.T., G.E.I., U.T.P.)
| | - Gul Esen Icten
- Acibadem M.A.A. University Senology Research Institute, 34457, Sarıyer, Istanbul, Turkey (F.T., G.E.I., U.T.P.); Department of Radiology, Acibadem M.A.A. University School of Medicine, Acıbadem Maslak Hospital, Büyükdere St. 40, 34457, Maslak, Istanbul, Turkey (G.E.I.)
| | - Ebru Yilmaz
- Breast Clinic, Acibadem Altunizade Hospital, 34662, Istanbul, Turkey (N.G., E.Y., E.B.T., E.A.)
| | - Ebru Banu Turk
- Breast Clinic, Acibadem Altunizade Hospital, 34662, Istanbul, Turkey (N.G., E.Y., E.B.T., E.A.)
| | - Servet Erdemli
- Department of Radiology, Acibadem M.A.A. University School of Medicine, Atakent University Hospital, 34755, Istanbul, Turkey (F.T., S.E.)
| | - Ulku Tuba Parlakkilic
- Acibadem M.A.A. University Senology Research Institute, 34457, Sarıyer, Istanbul, Turkey (F.T., G.E.I., U.T.P.)
| | - Ozlem Turkoglu
- Department of Radiology, Taksim Training and Research Hospital, Istanbul, Turkey (O.T.)
| | - Erkin Aribal
- Breast Clinic, Acibadem Altunizade Hospital, 34662, Istanbul, Turkey (N.G., E.Y., E.B.T., E.A.); Department of Radiology, Acibadem M.A.A. University School of Medicine, Istanbul, Turkey (E.A.)
| |
Collapse
|
5
|
Al-Karawi D, Al-Zaidi S, Helael KA, Obeidat N, Mouhsen AM, Ajam T, Alshalabi BA, Salman M, Ahmed MH. A Review of Artificial Intelligence in Breast Imaging. Tomography 2024; 10:705-726. [PMID: 38787015 PMCID: PMC11125819 DOI: 10.3390/tomography10050055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 04/14/2024] [Accepted: 05/06/2024] [Indexed: 05/25/2024] Open
Abstract
With the increasing dominance of artificial intelligence (AI) techniques, the important prospects for their application have extended to various medical fields, including domains such as in vitro diagnosis, intelligent rehabilitation, medical imaging, and prognosis. Breast cancer is a common malignancy that critically affects women's physical and mental health. Early breast cancer screening-through mammography, ultrasound, or magnetic resonance imaging (MRI)-can substantially improve the prognosis for breast cancer patients. AI applications have shown excellent performance in various image recognition tasks, and their use in breast cancer screening has been explored in numerous studies. This paper introduces relevant AI techniques and their applications in the field of medical imaging of the breast (mammography and ultrasound), specifically in terms of identifying, segmenting, and classifying lesions; assessing breast cancer risk; and improving image quality. Focusing on medical imaging for breast cancer, this paper also reviews related challenges and prospects for AI.
Collapse
Affiliation(s)
- Dhurgham Al-Karawi
- Medical Analytica Ltd., 26a Castle Park Industrial Park, Flint CH6 5XA, UK;
| | - Shakir Al-Zaidi
- Medical Analytica Ltd., 26a Castle Park Industrial Park, Flint CH6 5XA, UK;
| | - Khaled Ahmad Helael
- Royal Medical Services, King Hussein Medical Hospital, King Abdullah II Ben Al-Hussein Street, Amman 11855, Jordan;
| | - Naser Obeidat
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Abdulmajeed Mounzer Mouhsen
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Tarek Ajam
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Bashar A. Alshalabi
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Mohamed Salman
- Department of Diagnostic Radiology and Nuclear Medicine, Faculty of Medicine, Jordan University of Science and Technology, Irbid 22110, Jordan; (N.O.); (A.M.M.); (T.A.); (B.A.A.); (M.S.)
| | - Mohammed H. Ahmed
- School of Computing, Coventry University, 3 Gulson Road, Coventry CV1 5FB, UK;
| |
Collapse
|
6
|
Nagao A, Inagaki Y, Nogami K, Yamasaki N, Iwasaki F, Liu Y, Murakami Y, Ito T, Takedani H. Artificial intelligence-assisted ultrasound imaging in hemophilia: research, development, and evaluation of hemarthrosis and synovitis detection. Res Pract Thromb Haemost 2024; 8:102439. [PMID: 38993620 PMCID: PMC11238186 DOI: 10.1016/j.rpth.2024.102439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Accepted: 05/03/2024] [Indexed: 07/13/2024] Open
Abstract
Background Joint bleeding can lead to synovitis and arthropathy in people with hemophilia, reducing quality of life. Although early diagnosis is associated with improved therapeutic outcomes, diagnostic ultrasonography requires specialist experience. Artificial intelligence (AI) algorithms may support ultrasonography diagnoses. Objectives This study will research, develop, and evaluate the diagnostic precision of an AI algorithm for detecting the presence or absence of hemarthrosis and synovitis in people with hemophilia. Methods Elbow, knee, and ankle ultrasound images were obtained from people with hemophilia from January 2010 to March 2022. The images were used to train and test the AI models to estimate the presence/absence of hemarthrosis and synovitis. The primary endpoint was the area under the curve for the diagnostic precision to diagnose hemarthrosis and synovitis. Other endpoints were the rate of accuracy, precision, sensitivity, and specificity. Results Out of 5649 images collected, 3435 were used for analysis. The area under the curve for hemarthrosis detection for the elbow, knee, and ankle joints was ≥0.87 and for synovitis, it was ≥0.90. The accuracy and precision for hemarthrosis detection were ≥0.74 and ≥0.67, respectively, and those for synovitis were ≥0.83 and ≥0.74, respectively. Analysis across people with hemophilia aged 10 to 60 years showed consistent results. Conclusion AI models have the potential to aid diagnosis and enable earlier therapeutic interventions, helping people with hemophilia achieve healthy and active lives. Although AI models show potential in diagnosis, evidence is unclear on required control for abnormal findings. Long-term observation is crucial for assessing impact on joint health.
Collapse
Affiliation(s)
- Azusa Nagao
- Department of Blood Coagulation, Ogikubo Hospital, Tokyo, Japan
| | - Yusuke Inagaki
- Department of Rehabilitation Medicine, Nara Medical University, Nara, Japan
| | - Keiji Nogami
- Department of Pediatrics, Nara Medical University, Nara, Japan
| | - Naoya Yamasaki
- Department of Transfusion Medicine, Hiroshima University, Hiroshima, Japan
| | - Fuminori Iwasaki
- Division of Hematology and Oncology, Kanagawa Children’s Medical Center, Kanagawa, Japan
| | - Yang Liu
- Clinical Development Division, Chugai Pharmaceutical Co, Ltd, Tokyo, Japan
| | - Yoichi Murakami
- Medical Affairs Division, Chugai Pharmaceutical Co, Ltd, Tokyo, Japan
| | - Takahiro Ito
- Medical Affairs Division, Chugai Pharmaceutical Co, Ltd, Tokyo, Japan
| | - Hideyuki Takedani
- Department of Rehabilitation, National Hospital Organization Tsuruga Medical Center, Fukui, Japan
| |
Collapse
|
7
|
Kim HJ, Kim HH, Kim KH, Lee JS, Choi WJ, Chae EY, Shin HJ, Cha JH, Shim WH. Use of a commercial artificial intelligence-based mammography analysis software for improving breast ultrasound interpretations. Eur Radiol 2024:10.1007/s00330-024-10718-3. [PMID: 38570382 DOI: 10.1007/s00330-024-10718-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 02/22/2024] [Accepted: 03/13/2024] [Indexed: 04/05/2024]
Abstract
OBJECTIVES To evaluate the use of a commercial artificial intelligence (AI)-based mammography analysis software for improving the interpretations of breast ultrasound (US)-detected lesions. METHODS A retrospective analysis was performed on 1109 breasts that underwent both mammography and US-guided breast biopsy. The AI software processed mammograms and provided an AI score ranging from 0 to 100 for each breast, indicating the likelihood of malignancy. The performance of the AI score in differentiating mammograms with benign outcomes from those revealing cancers following US-guided breast biopsy was evaluated. In addition, prediction models for benign outcomes were constructed based on clinical and imaging characteristics with and without AI scores, using logistic regression analysis. RESULTS The AI software had an area under the receiver operating characteristics curve (AUROC) of 0.79 (95% CI, 0.79-0.82) in differentiating between benign and cancer cases. The prediction models that did not include AI scores (non-AI model), only used AI scores (AI-only model), and included AI scores (integrated model) had AUROCs of 0.79 (95% CI, 0.75-0.83), 0.78 (95% CI, 0.74-0.82), and 0.85 (95% CI, 0.81-0.88) in the development cohort, and 0.75 (95% CI, 0.68-0.81), 0.82 (95% CI, 0.76-0.88), and 0.84 (95% CI, 0.79-0.90) in the validation cohort, respectively. The integrated model outperformed the non-AI model in the development and validation cohorts (p < 0.001 for both). CONCLUSION The commercial AI-based mammography analysis software could be a valuable adjunct to clinical decision-making for managing US-detected breast lesions. CLINICAL RELEVANCE STATEMENT The commercial AI-based mammography analysis software could potentially reduce unnecessary biopsies and improve patient outcomes. KEY POINTS • Breast US has high rates of false-positive interpretations. • A commercial AI-based mammography analysis software could distinguish mammograms having benign outcomes from those revealing cancers after US-guided breast biopsy. • A commercial AI-based mammography analysis software may improve interpretations for breast US-detected lesions.
Collapse
Affiliation(s)
- Hee Jeong Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea
| | - Hak Hee Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea.
| | - Ki Hwan Kim
- Lunit Inc., 15F, 27, Teheran-Ro 2-Gil, Gangnam-Gu, Seoul, 06241, South Korea
| | - Ji Sung Lee
- Department of Clinical Epidemiology and Biostatistics, Asan Medical Center, University of Ulsan College, Ulsan, South Korea
| | - Woo Jung Choi
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea
| | - Eun Young Chae
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea
| | - Hee Jung Shin
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea
| | - Joo Hee Cha
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea
| | - Woo Hyun Shim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea
| |
Collapse
|
8
|
Du Y, Wang D, Liu M, Zhang X, Ren W, Sun J, Yin C, Yang S, Zhang L. Study on the differential diagnosis of benign and malignant breast lesions using a deep learning model based on multimodal images. J Cancer Res Ther 2024; 20:625-632. [PMID: 38687933 DOI: 10.4103/jcrt.jcrt_1796_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 12/01/2023] [Indexed: 05/02/2024]
Abstract
OBJECTIVE To establish a multimodal model for distinguishing benign and malignant breast lesions. MATERIALS AND METHODS Clinical data, mammography, and MRI images (including T2WI, diffusion-weighted images (DWI), apparent diffusion coefficient (ADC), and DCE-MRI images) of 132 benign and breast cancer patients were analyzed retrospectively. The region of interest (ROI) in each image was marked and segmented using MATLAB software. The mammography, T2WI, DWI, ADC, and DCE-MRI models based on the ResNet34 network were trained. Using an integrated learning method, the five models were used as a basic model, and voting methods were used to construct a multimodal model. The dataset was divided into a training set and a prediction set. The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the model were calculated. The diagnostic efficacy of each model was analyzed using a receiver operating characteristic curve (ROC) and an area under the curve (AUC). The diagnostic value was determined by the DeLong test with statistically significant differences set at P < 0.05. RESULTS We evaluated the ability of the model to classify benign and malignant tumors using the test set. The AUC values of the multimodal model, mammography model, T2WI model, DWI model, ADC model and DCE-MRI model were 0.943, 0.645, 0.595, 0.905, 0.900, and 0.865, respectively. The diagnostic ability of the multimodal model was significantly higher compared with that of the mammography and T2WI models. However, compared with the DWI, ADC, and DCE-MRI models, there was no significant difference in the diagnostic ability of these models. CONCLUSION Our deep learning model based on multimodal image training has practical value for the diagnosis of benign and malignant breast lesions.
Collapse
Affiliation(s)
- Yanan Du
- Department of Health Management, The First Affiliated Hospital of Shandong First Medical University and Qianfoshan Hospital, Jinan City, Shandong Province, China
| | - Dawei Wang
- Department of Health Management Shandong University of Traditional Chinese Medicine, Jinan City, Shandong Province, China
| | - Menghan Liu
- Department of Health Management, The First Affiliated Hospital of Shandong First Medical University and Qianfoshan Hospital, Jinan City, Shandong Province, China
| | - Xiaodong Zhang
- Postgraduate Department, Shandong First Medical University (Shandong Academy of Medical Sciences), Jinan City, Shandong Province, China
| | - Wanqing Ren
- Postgraduate Department, Shandong First Medical University (Shandong Academy of Medical Sciences), Jinan City, Shandong Province, China
- Department of Radiology, The First Affiliated Hospital of Shandong First Medical University and Qianfoshan Hospital, Jinan City, Shandong Province, China
| | - Jingxiang Sun
- Postgraduate Department, Shandong First Medical University (Shandong Academy of Medical Sciences), Jinan City, Shandong Province, China
- Department of Radiology, The First Affiliated Hospital of Shandong First Medical University and Qianfoshan Hospital, Jinan City, Shandong Province, China
| | - Chao Yin
- Department of Radiology, Yantai Taocun Central Hospital, Yantai City, Shandong Province, China
| | - Shiwei Yang
- Department of Anorectal Surgery, The First Affiliated Hospital of Shandong First Medical University and Qianfoshan Hospital, Jinan City, Shandong Province, China
| | - Li Zhang
- Department of Pharmacology, Jinan Central Hospital Affiliated to Shandong First Medical University, Jinan City, Shandong Province, China
| |
Collapse
|
9
|
Lokaj B, Pugliese MT, Kinkel K, Lovis C, Schmid J. Barriers and facilitators of artificial intelligence conception and implementation for breast imaging diagnosis in clinical practice: a scoping review. Eur Radiol 2024; 34:2096-2109. [PMID: 37658895 PMCID: PMC10873444 DOI: 10.1007/s00330-023-10181-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 06/07/2023] [Accepted: 07/10/2023] [Indexed: 09/05/2023]
Abstract
OBJECTIVE Although artificial intelligence (AI) has demonstrated promise in enhancing breast cancer diagnosis, the implementation of AI algorithms in clinical practice encounters various barriers. This scoping review aims to identify these barriers and facilitators to highlight key considerations for developing and implementing AI solutions in breast cancer imaging. METHOD A literature search was conducted from 2012 to 2022 in six databases (PubMed, Web of Science, CINHAL, Embase, IEEE, and ArXiv). The articles were included if some barriers and/or facilitators in the conception or implementation of AI in breast clinical imaging were described. We excluded research only focusing on performance, or with data not acquired in a clinical radiology setup and not involving real patients. RESULTS A total of 107 articles were included. We identified six major barriers related to data (B1), black box and trust (B2), algorithms and conception (B3), evaluation and validation (B4), legal, ethical, and economic issues (B5), and education (B6), and five major facilitators covering data (F1), clinical impact (F2), algorithms and conception (F3), evaluation and validation (F4), and education (F5). CONCLUSION This scoping review highlighted the need to carefully design, deploy, and evaluate AI solutions in clinical practice, involving all stakeholders to yield improvement in healthcare. CLINICAL RELEVANCE STATEMENT The identification of barriers and facilitators with suggested solutions can guide and inform future research, and stakeholders to improve the design and implementation of AI for breast cancer detection in clinical practice. KEY POINTS • Six major identified barriers were related to data; black-box and trust; algorithms and conception; evaluation and validation; legal, ethical, and economic issues; and education. • Five major identified facilitators were related to data, clinical impact, algorithms and conception, evaluation and validation, and education. • Coordinated implication of all stakeholders is required to improve breast cancer diagnosis with AI.
Collapse
Affiliation(s)
- Belinda Lokaj
- Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Delémont, Switzerland.
- Faculty of Medicine, University of Geneva, Geneva, Switzerland.
- Division of Medical Information Sciences, Geneva University Hospitals, Geneva, Switzerland.
| | - Marie-Thérèse Pugliese
- Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Delémont, Switzerland
| | - Karen Kinkel
- Réseau Hospitalier Neuchâtelois, Neuchâtel, Switzerland
| | - Christian Lovis
- Faculty of Medicine, University of Geneva, Geneva, Switzerland
- Division of Medical Information Sciences, Geneva University Hospitals, Geneva, Switzerland
| | - Jérôme Schmid
- Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Delémont, Switzerland
| |
Collapse
|
10
|
Dan Q, Xu Z, Burrows H, Bissram J, Stringer JSA, Li Y. Diagnostic performance of deep learning in ultrasound diagnosis of breast cancer: a systematic review. NPJ Precis Oncol 2024; 8:21. [PMID: 38280946 PMCID: PMC10821881 DOI: 10.1038/s41698-024-00514-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 12/08/2023] [Indexed: 01/29/2024] Open
Abstract
Deep learning (DL) has been widely investigated in breast ultrasound (US) for distinguishing between benign and malignant breast masses. This systematic review of test diagnosis aims to examine the accuracy of DL, compared to human readers, for the diagnosis of breast cancer in the US under clinical settings. Our literature search included records from databases including PubMed, Embase, Scopus, and Cochrane Library. Test accuracy outcomes were synthesized to compare the diagnostic performance of DL and human readers as well as to evaluate the assistive role of DL to human readers. A total of 16 studies involving 9238 female participants were included. There were no prospective studies comparing the test accuracy of DL versus human readers in clinical workflows. Diagnostic test results varied across the included studies. In 14 studies employing standalone DL systems, DL showed significantly lower sensitivities in 5 studies with comparable specificities and outperformed human readers at higher specificities in another 4 studies; in the remaining studies, DL models and human readers showed equivalent test outcomes. In 12 studies that assessed assistive DL systems, no studies proved the assistive role of DL in the overall diagnostic performance of human readers. Current evidence is insufficient to conclude that DL outperforms human readers or enhances the accuracy of diagnostic breast US in a clinical setting. Standardization of study methodologies is required to improve the reproducibility and generalizability of DL research, which will aid in clinical translation and application.
Collapse
Affiliation(s)
- Qing Dan
- Department of Ultrasound, Nanfang Hospital, Southern Medical University, 510515, Guangzhou, China
- Global Women's Health, The University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Ziting Xu
- Department of Ultrasound, Nanfang Hospital, Southern Medical University, 510515, Guangzhou, China
| | - Hannah Burrows
- Health Sciences Library, The University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Jennifer Bissram
- Health Sciences Library, The University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Jeffrey S A Stringer
- Global Women's Health, The University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA.
| | - Yingjia Li
- Department of Ultrasound, Nanfang Hospital, Southern Medical University, 510515, Guangzhou, China.
| |
Collapse
|
11
|
Irmici G, Cè M, Pepa GD, D'Ascoli E, De Berardinis C, Giambersio E, Rabiolo L, La Rocca L, Carriero S, Depretto C, Scaperrotta G, Cellina M. Exploring the Potential of Artificial Intelligence in Breast Ultrasound. Crit Rev Oncog 2024; 29:15-28. [PMID: 38505878 DOI: 10.1615/critrevoncog.2023048873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/21/2024]
Abstract
Breast ultrasound has emerged as a valuable imaging modality in the detection and characterization of breast lesions, particularly in women with dense breast tissue or contraindications for mammography. Within this framework, artificial intelligence (AI) has garnered significant attention for its potential to improve diagnostic accuracy in breast ultrasound and revolutionize the workflow. This review article aims to comprehensively explore the current state of research and development in harnessing AI's capabilities for breast ultrasound. We delve into various AI techniques, including machine learning, deep learning, as well as their applications in automating lesion detection, segmentation, and classification tasks. Furthermore, the review addresses the challenges and hurdles faced in implementing AI systems in breast ultrasound diagnostics, such as data privacy, interpretability, and regulatory approval. Ethical considerations pertaining to the integration of AI into clinical practice are also discussed, emphasizing the importance of maintaining a patient-centered approach. The integration of AI into breast ultrasound holds great promise for improving diagnostic accuracy, enhancing efficiency, and ultimately advancing patient's care. By examining the current state of research and identifying future opportunities, this review aims to contribute to the understanding and utilization of AI in breast ultrasound and encourage further interdisciplinary collaboration to maximize its potential in clinical practice.
Collapse
Affiliation(s)
- Giovanni Irmici
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Maurizio Cè
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Gianmarco Della Pepa
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Elisa D'Ascoli
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Claudia De Berardinis
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Emilia Giambersio
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Lidia Rabiolo
- Dipartimento di Biomedicina, Neuroscienze e Diagnostica Avanzata, Policlinico Università di Palermo, Palermo, Italy
| | - Ludovica La Rocca
- Postgraduation School in Radiodiagnostics, Università degli Studi di Napoli
| | - Serena Carriero
- Postgraduation School in Radiodiagnostics, Università degli Studi di Milano, Via Festa del Perdono, 7, 20122 Milan, Italy
| | - Catherine Depretto
- Breast Radiology Unit, Fondazione IRCCS, Istituto Nazionale Tumori, Milano, Italy
| | | | - Michaela Cellina
- Radiology Department, Fatebenefratelli Hospital, ASST Fatebenefratelli Sacco, Milano, Piazza Principessa Clotilde 3, 20121, Milan, Italy
| |
Collapse
|
12
|
Zhou G, Mosadegh B. Distilling Knowledge From an Ensemble of Vision Transformers for Improved Classification of Breast Ultrasound. Acad Radiol 2024; 31:104-120. [PMID: 37666747 DOI: 10.1016/j.acra.2023.08.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 07/20/2023] [Accepted: 08/05/2023] [Indexed: 09/06/2023]
Abstract
RATIONALE AND OBJECTIVES To develop a deep learning model for the automated classification of breast ultrasound images as benign or malignant. More specifically, the application of vision transformers, ensemble learning, and knowledge distillation is explored for breast ultrasound classification. MATERIALS AND METHODS Single view, B-mode ultrasound images were curated from the publicly available Breast Ultrasound Image (BUSI) dataset, which has categorical ground truth labels (benign vs malignant) assigned by radiologists and malignant cases confirmed by biopsy. The performance of vision transformers (ViT) is compared to convolutional neural networks (CNN), followed by a comparison between supervised, self-supervised, and randomly initialized ViT. Subsequently, the ensemble of 10 independently trained ViT, where the ensemble model is the unweighted average of the output of each individual model is compared to the performance of each ViT alone. Finally, we train a single ViT to emulate the ensembled ViT using knowledge distillation. RESULTS On this dataset that was trained using five-fold cross validation, ViT outperforms CNN, while self-supervised ViT outperform supervised and randomly initialized ViT. The ensemble model achieves an area under the receiver operating characteristics curve (AuROC) and area under the precision recall curve (AuPRC) of 0.977 and 0.965 on the test set, outperforming the average AuROC and AuPRC of the independently trained ViTs (0.958 ± 0.05 and 0.931 ± 0.016). The distilled ViT achieves an AuROC and AuPRC of 0.972 and 0.960. CONCLUSION Both transfer learning and ensemble learning can each offer increased performance independently and can be sequentially combined to collectively improve the performance of the final model. Furthermore, a single vision transformer can be trained to match the performance of an ensemble of a set of vision transformers using knowledge distillation.
Collapse
Affiliation(s)
| | - Bobak Mosadegh
- Dalio Institute of Cardiovascular Imaging, Department of Radiology, Weill Cornell Medicine, New York, New York
| |
Collapse
|
13
|
Li JW, Sheng DL, Chen JG, You C, Liu S, Xu HX, Chang C. Artificial intelligence in breast imaging: potentials and challenges. Phys Med Biol 2023; 68:23TR01. [PMID: 37722385 DOI: 10.1088/1361-6560/acfade] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2023] [Accepted: 09/18/2023] [Indexed: 09/20/2023]
Abstract
Breast cancer, which is the most common type of malignant tumor among humans, is a leading cause of death in females. Standard treatment strategies, including neoadjuvant chemotherapy, surgery, postoperative chemotherapy, targeted therapy, endocrine therapy, and radiotherapy, are tailored for individual patients. Such personalized therapies have tremendously reduced the threat of breast cancer in females. Furthermore, early imaging screening plays an important role in reducing the treatment cycle and improving breast cancer prognosis. The recent innovative revolution in artificial intelligence (AI) has aided radiologists in the early and accurate diagnosis of breast cancer. In this review, we introduce the necessity of incorporating AI into breast imaging and the applications of AI in mammography, ultrasonography, magnetic resonance imaging, and positron emission tomography/computed tomography based on published articles since 1994. Moreover, the challenges of AI in breast imaging are discussed.
Collapse
Affiliation(s)
- Jia-Wei Li
- Department of Medical Ultrasound, Fudan University Shanghai Cancer Center; Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, 200032, People's Republic of China
| | - Dan-Li Sheng
- Department of Medical Ultrasound, Fudan University Shanghai Cancer Center; Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, 200032, People's Republic of China
| | - Jian-Gang Chen
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication & Electronic Engineering, East China Normal University, People's Republic of China
| | - Chao You
- Department of Radiology, Fudan University Shanghai Cancer Center; Department of Oncology, Shanghai Medical College, Fudan University, Shanghai 200032, People's Republic of China
| | - Shuai Liu
- Department of Nuclear Medicine, Fudan University Shanghai Cancer Center; Department of Oncology, Shanghai Medical College, Fudan University, Shanghai 200032, People's Republic of China
| | - Hui-Xiong Xu
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, 200032, People's Republic of China
| | - Cai Chang
- Department of Medical Ultrasound, Fudan University Shanghai Cancer Center; Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, 200032, People's Republic of China
| |
Collapse
|
14
|
Qiu S, Zhuang S, Li B, Wang J, Zhuang Z. Prospective assessment of breast lesions AI classification model based on ultrasound dynamic videos and ACR BI-RADS characteristics. Front Oncol 2023; 13:1274557. [PMID: 38023255 PMCID: PMC10656688 DOI: 10.3389/fonc.2023.1274557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 10/16/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction AI-assisted ultrasound diagnosis is considered a fast and accurate new method that can reduce the subjective and experience-dependent nature of handheld ultrasound. In order to meet clinical diagnostic needs better, we first proposed a breast lesions AI classification model based on ultrasound dynamic videos and ACR BI-RADS characteristics (hereafter, Auto BI-RADS). In this study, we prospectively verify its performance. Methods In this study, the model development was based on retrospective data including 480 ultrasound dynamic videos equivalent to 18122 static images of pathologically proven breast lesions from 420 patients. A total of 292 breast lesions ultrasound dynamic videos from the internal and external hospital were prospectively tested by Auto BI-RADS. The performance of Auto BI-RADS was compared with both experienced and junior radiologists using the DeLong method, Kappa test, and McNemar test. Results The Auto BI-RADS achieved an accuracy, sensitivity, and specificity of 0.87, 0.93, and 0.81, respectively. The consistency of the BI-RADS category between Auto BI-RADS and the experienced group (Kappa:0.82) was higher than that of the juniors (Kappa:0.60). The consistency rates between Auto BI-RADS and the experienced group were higher than those between Auto BI-RADS and the junior group for shape (93% vs. 80%; P = .01), orientation (90% vs. 84%; P = .02), margin (84% vs. 71%; P = .01), echo pattern (69% vs. 56%; P = .001) and posterior features (76% vs. 71%; P = .0046), While the difference of calcification was not significantly different. Discussion In this study, we aimed to prospectively verify a novel AI tool based on ultrasound dynamic videos and ACR BI-RADS characteristics. The prospective assessment suggested that the AI tool not only meets the clinical needs better but also reaches the diagnostic efficiency of experienced radiologists.
Collapse
Affiliation(s)
- Shunmin Qiu
- Department of Ultrasound, First Affiliated Hospital of Shantou University Medical College, Shantou, Guangdong, China
| | - Shuxin Zhuang
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Bin Li
- Product Development Department, Shantou Institute of Ultrasonic Instruments, Shantou, Guangdong, China
| | - Jinhong Wang
- Department of Ultrasound, Shantou Chaonan Minsheng Hospital, Shantou, Guangdong, China
| | - Zhemin Zhuang
- Engineering College, Shantou University, Shantou, Guangdong, China
| |
Collapse
|
15
|
Kumar A, Kempski Leadingham KM, Kerensky MJ, Sankar S, Thakor NV, Manbachi A. Visualizing tactile feedback: an overview of current technologies with a focus on ultrasound elastography. FRONTIERS IN MEDICAL TECHNOLOGY 2023; 5:1238129. [PMID: 37854637 PMCID: PMC10579802 DOI: 10.3389/fmedt.2023.1238129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Accepted: 09/14/2023] [Indexed: 10/20/2023] Open
Abstract
Tissue elasticity remains an essential biomarker of health and is indicative of irregularities such as tumors or infection. The timely detection of such abnormalities is crucial for the prevention of disease progression and complications that arise from late-stage illnesses. However, at both the bedside and the operating table, there is a distinct lack of tactile feedback for deep-seated tissue. As surgical techniques advance toward remote or minimally invasive options to reduce infection risk and hasten healing time, surgeons lose the ability to manually palpate tissue. Furthermore, palpation of deep structures results in decreased accuracy, with the additional barrier of needing years of experience for adequate confidence of diagnoses. This review delves into the current modalities used to fulfill the clinical need of quantifying physical touch. It covers research efforts involving tactile sensing for remote or minimally invasive surgeries, as well as the potential of ultrasound elastography to further this field with non-invasive real-time imaging of the organ's biomechanical properties. Elastography monitors tissue response to acoustic or mechanical energy and reconstructs an image representative of the elastic profile in the region of interest. This intuitive visualization of tissue elasticity surpasses the tactile information provided by sensors currently used to augment or supplement manual palpation. Focusing on common ultrasound elastography modalities, we evaluate various sensing mechanisms used for measuring tactile information and describe their emerging use in clinical settings where palpation is insufficient or restricted. With the ongoing advancements in ultrasound technology, particularly the emergence of micromachined ultrasound transducers, these devices hold great potential in facilitating early detection of tissue abnormalities and providing an objective measure of patient health.
Collapse
Affiliation(s)
- Avisha Kumar
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
- HEPIUS Innovation Lab, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Kelley M. Kempski Leadingham
- HEPIUS Innovation Lab, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Max J. Kerensky
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
- HEPIUS Innovation Lab, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Sriramana Sankar
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Nitish V. Thakor
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Amir Manbachi
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, United States
- HEPIUS Innovation Lab, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD, United States
- Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| |
Collapse
|
16
|
You C, Shen Y, Sun S, Zhou J, Li J, Su G, Michalopoulou E, Peng W, Gu Y, Guo W, Cao H. Artificial intelligence in breast imaging: Current situation and clinical challenges. EXPLORATION (BEIJING, CHINA) 2023; 3:20230007. [PMID: 37933287 PMCID: PMC10582610 DOI: 10.1002/exp.20230007] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 04/30/2023] [Indexed: 11/08/2023]
Abstract
Breast cancer ranks among the most prevalent malignant tumours and is the primary contributor to cancer-related deaths in women. Breast imaging is essential for screening, diagnosis, and therapeutic surveillance. With the increasing demand for precision medicine, the heterogeneous nature of breast cancer makes it necessary to deeply mine and rationally utilize the tremendous amount of breast imaging information. With the rapid advancement of computer science, artificial intelligence (AI) has been noted to have great advantages in processing and mining of image information. Therefore, a growing number of scholars have started to focus on and research the utility of AI in breast imaging. Here, an overview of breast imaging databases and recent advances in AI research are provided, the challenges and problems in this field are discussed, and then constructive advice is further provided for ongoing scientific developments from the perspective of the National Natural Science Foundation of China.
Collapse
Affiliation(s)
- Chao You
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Yiyuan Shen
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Shiyun Sun
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Jiayin Zhou
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Jiawei Li
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Guanhua Su
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
- Department of Breast SurgeryKey Laboratory of Breast Cancer in ShanghaiFudan University Shanghai Cancer CenterShanghaiChina
| | | | - Weijun Peng
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Yajia Gu
- Department of RadiologyFudan University Shanghai Cancer CenterShanghaiChina
- Department of OncologyShanghai Medical CollegeFudan UniversityShanghaiChina
| | - Weisheng Guo
- Department of Minimally Invasive Interventional RadiologyKey Laboratory of Molecular Target and Clinical PharmacologySchool of Pharmaceutical Sciences and The Second Affiliated HospitalGuangzhou Medical UniversityGuangzhouChina
| | - Heqi Cao
- Department of Health SciencesNational Natural Science Foundation of ChinaBeijingChina
| |
Collapse
|
17
|
Wanderley MC, Soares CMA, Morais MMM, Cruz RM, Lima IRM, Chojniak R, Bitencourt AGV. Application of artificial intelligence in predicting malignancy risk in breast masses on ultrasound. Radiol Bras 2023; 56:229-234. [PMID: 38204896 PMCID: PMC10775818 DOI: 10.1590/0100-3984.2023.0034] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 05/16/2023] [Accepted: 07/05/2023] [Indexed: 01/12/2024] Open
Abstract
Objective To evaluate the results obtained with an artificial intelligence-based software for predicting the risk of malignancy in breast masses from ultrasound images. Materials and Methods This was a retrospective, single-center study evaluating 555 breast masses submitted to percutaneous biopsy at a cancer referral center. Ultrasonographic findings were classified in accordance with the BI-RADS lexicon. The images were analyzed by using Koios DS Breast software and classified as benign, probably benign, low to intermediate suspicion, high suspicion, or probably malignant. The histological classification was considered the reference standard. Results The mean age of the patients was 51 years, and the mean mass size was 16 mm. The radiologist evaluation had a sensitivity and specificity of 99.1% and 34.0%, respectively, compared with 98.2% and 39.0%, respectively, for the software evaluation. The positive predictive value for malignancy for the BI-RADS categories was similar between the radiologist and software evaluations. Two false-negative results were identified in the radiologist evaluation, the masses in question being classified as suspicious by the software, whereas four false-negative results were identified in the software evaluation, the masses in question being classified as suspicious by the radiologist. Conclusion In our sample, the performance of artificial intelligence-based software was comparable to that of a radiologist.
Collapse
Affiliation(s)
| | | | | | | | | | - Rubens Chojniak
- Department of Imaging, A.C.Camargo Cancer Center, São Paulo,
SP, Brazil
| | | |
Collapse
|
18
|
Kondo S, Satoh M, Nishida M, Sakano R, Takagi K. Ceusia-Breast: computer-aided diagnosis with contrast enhanced ultrasound image analysis for breast lesions. BMC Med Imaging 2023; 23:114. [PMID: 37644398 PMCID: PMC10466705 DOI: 10.1186/s12880-023-01072-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 08/02/2023] [Indexed: 08/31/2023] Open
Abstract
BACKGROUND In recent years, contrast-enhanced ultrasonography (CEUS) has been used for various applications in breast diagnosis. The superiority of CEUS over conventional B-mode imaging in the ultrasound diagnosis of the breast lesions in clinical practice has been widely confirmed. On the other hand, there have been many proposals for computer-aided diagnosis of breast lesions on B-mode ultrasound images, but few for CEUS. We propose a semi-automatic classification method based on machine learning in CEUS of breast lesions. METHODS The proposed method extracts spatial and temporal features from CEUS videos and breast tumors are classified as benign or malignant using linear support vector machines (SVM) with combination of selected optimal features. In the proposed method, tumor regions are extracted using the guidance information specified by the examiners, then morphological and texture features of tumor regions obtained from B-mode and CEUS images and TIC features obtained from CEUS video are extracted. Then, our method uses SVM classifiers to classify breast tumors as benign or malignant. During SVM training, many features are prepared, and useful features are selected. We name our proposed method "Ceucia-Breast" (Contrast Enhanced UltraSound Image Analysis for BREAST lesions). RESULTS The experimental results on 119 subjects show that the area under the receiver operating curve, accuracy, precision, and recall are 0.893, 0.816, 0.841 and 0.920, respectively. The classification performance is improved by our method over conventional methods using only B-mode images. In addition, we confirm that the selected features are consistent with the CEUS guidelines for breast tumor diagnosis. Furthermore, we conduct an experiment on the operator dependency of specifying guidance information and find that the intra-operator and inter-operator kappa coefficients are 1.0 and 0.798, respectively. CONCLUSION The experimental results show a significant improvement in classification performance compared to conventional classification methods using only B-mode images. We also confirm that the selected features are related to the findings that are considered important in clinical practice. Furthermore, we verify the intra- and inter-examiner correlation in the guidance input for region extraction and confirm that both correlations are in strong agreement.
Collapse
|
19
|
Gao Y, Soh NYT, Liu N, Lim G, Ting D, Cheng LTE, Wong KM, Liew C, Oh HC, Tan JR, Venkataraman N, Goh SH, Yan YY. Application of a deep learning algorithm in the detection of hip fractures. iScience 2023; 26:107350. [PMID: 37554447 PMCID: PMC10404720 DOI: 10.1016/j.isci.2023.107350] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 05/30/2023] [Accepted: 07/06/2023] [Indexed: 08/10/2023] Open
Abstract
This paper describes the development of a deep learning model for prediction of hip fractures on pelvic radiographs (X-rays). Developed using over 40,000 pelvic radiographs from a single institution, the model demonstrated high sensitivity and specificity when applied to a test set of emergency department radiographs. This study approximates the real-world application of a deep learning fracture detection model by including radiographs with sub-optimal image quality, other non-hip fractures, and metallic implants, which were excluded from prior published work. The study also explores the effect of ethnicity on model performance, as well as the accuracy of visualization algorithm for fracture localization.
Collapse
Affiliation(s)
- Yan Gao
- Health Services Research, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
| | - Nicholas Yock Teck Soh
- Department of Diagnostic Radiology, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Gilbert Lim
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Daniel Ting
- Singapore Health Services (SingHealth), Duke-NUS Medical School, Singapore, Singapore
| | - Lionel Tim-Ee Cheng
- Department of Diagnostic Radiology, Singapore General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
- Radiological Sciences ACP, Duke-NUS Medical School, Singapore, Singapore
| | - Kang Min Wong
- Department of Diagnostic Radiology, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
- Radiological Sciences ACP, Duke-NUS Medical School, Singapore, Singapore
| | - Charlene Liew
- Department of Diagnostic Radiology, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
- Radiological Sciences ACP, Duke-NUS Medical School, Singapore, Singapore
| | - Hong Choon Oh
- Health Services Research, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
| | - Jin Rong Tan
- Department of Diagnostic Radiology, Singapore General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
| | - Narayan Venkataraman
- Department of Medical Informatics, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
| | - Siang Hiong Goh
- Department of Emergency Medicine, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
| | - Yet Yen Yan
- Department of Diagnostic Radiology, Changi General Hospital, Singapore Health Services (SingHealth), Singapore, Singapore
- Radiological Sciences ACP, Duke-NUS Medical School, Singapore, Singapore
| |
Collapse
|
20
|
GadAllah MT, Mohamed AENA, Hefnawy AA, Zidan HE, El-Banby GM, Mohamed Badawy S. Convolutional Neural Networks Based Classification of Segmented Breast Ultrasound Images – A Comparative Preliminary Study. 2023 INTELLIGENT METHODS, SYSTEMS, AND APPLICATIONS (IMSA) 2023. [DOI: 10.1109/imsa58542.2023.10217585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Affiliation(s)
| | - Abd El-Naser A. Mohamed
- Menoufia University,Faculty of Electronic Engineering,Electronics and Electrical Communications Engineering Department,Menoufia,Egypt
| | - Alaa A. Hefnawy
- Electronics Research Institute (ERI),Computers and Systems Department,Cairo,Egypt
| | - Hassan E. Zidan
- Electronics Research Institute (ERI),Computers and Systems Department,Cairo,Egypt
| | - Ghada M. El-Banby
- Menoufia University,Faculty of Electronic Engineering,Industrial Electronics and Control Engineering Department,Menoufia,Egypt
| | - Samir Mohamed Badawy
- Menoufia University,Faculty of Electronic Engineering,Industrial Electronics and Control Engineering Department,Menoufia,Egypt
| |
Collapse
|
21
|
Sexauer R, Hejduk P, Borkowski K, Ruppert C, Weikert T, Dellas S, Schmidt N. Diagnostic accuracy of automated ACR BI-RADS breast density classification using deep convolutional neural networks. Eur Radiol 2023; 33:4589-4596. [PMID: 36856841 PMCID: PMC10289992 DOI: 10.1007/s00330-023-09474-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 01/17/2023] [Accepted: 01/26/2023] [Indexed: 03/02/2023]
Abstract
OBJECTIVES High breast density is a well-known risk factor for breast cancer. This study aimed to develop and adapt two (MLO, CC) deep convolutional neural networks (DCNN) for automatic breast density classification on synthetic 2D tomosynthesis reconstructions. METHODS In total, 4605 synthetic 2D images (1665 patients, age: 57 ± 37 years) were labeled according to the ACR (American College of Radiology) density (A-D). Two DCNNs with 11 convolutional layers and 3 fully connected layers each, were trained with 70% of the data, whereas 20% was used for validation. The remaining 10% were used as a separate test dataset with 460 images (380 patients). All mammograms in the test dataset were read blinded by two radiologists (reader 1 with two and reader 2 with 11 years of dedicated mammographic experience in breast imaging), and the consensus was formed as the reference standard. The inter- and intra-reader reliabilities were assessed by calculating Cohen's kappa coefficients, and diagnostic accuracy measures of automated classification were evaluated. RESULTS The two models for MLO and CC projections had a mean sensitivity of 80.4% (95%-CI 72.2-86.9), a specificity of 89.3% (95%-CI 85.4-92.3), and an accuracy of 89.6% (95%-CI 88.1-90.9) in the differentiation between ACR A/B and ACR C/D. DCNN versus human and inter-reader agreement were both "substantial" (Cohen's kappa: 0.61 versus 0.63). CONCLUSION The DCNN allows accurate, standardized, and observer-independent classification of breast density based on the ACR BI-RADS system. KEY POINTS • A DCNN performs on par with human experts in breast density assessment for synthetic 2D tomosynthesis reconstructions. • The proposed technique may be useful for accurate, standardized, and observer-independent breast density evaluation of tomosynthesis.
Collapse
Affiliation(s)
- Raphael Sexauer
- Department of Radiology and Nuclear Medicine, University Hospital Basel, Petersgraben 4, CH-4031, Basel, Switzerland.
| | - Patryk Hejduk
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, Rämistrasse 100, CH-8091, Zurich, Switzerland
| | - Karol Borkowski
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, Rämistrasse 100, CH-8091, Zurich, Switzerland
| | - Carlotta Ruppert
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, Rämistrasse 100, CH-8091, Zurich, Switzerland
| | - Thomas Weikert
- Department of Radiology and Nuclear Medicine, University Hospital Basel, Petersgraben 4, CH-4031, Basel, Switzerland
| | - Sophie Dellas
- Department of Radiology and Nuclear Medicine, University Hospital Basel, Petersgraben 4, CH-4031, Basel, Switzerland
| | - Noemi Schmidt
- Department of Radiology and Nuclear Medicine, University Hospital Basel, Petersgraben 4, CH-4031, Basel, Switzerland
| |
Collapse
|
22
|
Afrin H, Larson NB, Fatemi M, Alizad A. Deep Learning in Different Ultrasound Methods for Breast Cancer, from Diagnosis to Prognosis: Current Trends, Challenges, and an Analysis. Cancers (Basel) 2023; 15:3139. [PMID: 37370748 PMCID: PMC10296633 DOI: 10.3390/cancers15123139] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 06/02/2023] [Accepted: 06/08/2023] [Indexed: 06/29/2023] Open
Abstract
Breast cancer is the second-leading cause of mortality among women around the world. Ultrasound (US) is one of the noninvasive imaging modalities used to diagnose breast lesions and monitor the prognosis of cancer patients. It has the highest sensitivity for diagnosing breast masses, but it shows increased false negativity due to its high operator dependency. Underserved areas do not have sufficient US expertise to diagnose breast lesions, resulting in delayed management of breast lesions. Deep learning neural networks may have the potential to facilitate early decision-making by physicians by rapidly yet accurately diagnosing and monitoring their prognosis. This article reviews the recent research trends on neural networks for breast mass ultrasound, including and beyond diagnosis. We discussed original research recently conducted to analyze which modes of ultrasound and which models have been used for which purposes, and where they show the best performance. Our analysis reveals that lesion classification showed the highest performance compared to those used for other purposes. We also found that fewer studies were performed for prognosis than diagnosis. We also discussed the limitations and future directions of ongoing research on neural networks for breast ultrasound.
Collapse
Affiliation(s)
- Humayra Afrin
- Department of Physiology and Biomedical Engineering, Mayo Clinic College of Medicine and Science, Rochester, MN 55905, USA
| | - Nicholas B. Larson
- Department of Quantitative Health Sciences, Mayo Clinic College of Medicine and Science, Rochester, MN 55905, USA
| | - Mostafa Fatemi
- Department of Physiology and Biomedical Engineering, Mayo Clinic College of Medicine and Science, Rochester, MN 55905, USA
| | - Azra Alizad
- Department of Physiology and Biomedical Engineering, Mayo Clinic College of Medicine and Science, Rochester, MN 55905, USA
- Department of Radiology, Mayo Clinic College of Medicine and Science, Rochester, MN 55905, USA
| |
Collapse
|
23
|
Hejduk P, Sexauer R, Ruppert C, Borkowski K, Unkelbach J, Schmidt N. Automatic and standardized quality assurance of digital mammography and tomosynthesis with deep convolutional neural networks. Insights Imaging 2023; 14:90. [PMID: 37199794 DOI: 10.1186/s13244-023-01396-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 03/06/2023] [Indexed: 05/19/2023] Open
Abstract
OBJECTIVES The aim of this study was to develop and validate a commercially available AI platform for the automatic determination of image quality in mammography and tomosynthesis considering a standardized set of features. MATERIALS AND METHODS In this retrospective study, 11,733 mammograms and synthetic 2D reconstructions from tomosynthesis of 4200 patients from two institutions were analyzed by assessing the presence of seven features which impact image quality in regard to breast positioning. Deep learning was applied to train five dCNN models on features detecting the presence of anatomical landmarks and three dCNN models for localization features. The validity of models was assessed by the calculation of the mean squared error in a test dataset and was compared to the reading by experienced radiologists. RESULTS Accuracies of the dCNN models ranged between 93.0% for the nipple visualization and 98.5% for the depiction of the pectoralis muscle in the CC view. Calculations based on regression models allow for precise measurements of distances and angles of breast positioning on mammograms and synthetic 2D reconstructions from tomosynthesis. All models showed almost perfect agreement compared to human reading with Cohen's kappa scores above 0.9. CONCLUSIONS An AI-based quality assessment system using a dCNN allows for precise, consistent and observer-independent rating of digital mammography and synthetic 2D reconstructions from tomosynthesis. Automation and standardization of quality assessment enable real-time feedback to technicians and radiologists that shall reduce a number of inadequate examinations according to PGMI (Perfect, Good, Moderate, Inadequate) criteria, reduce a number of recalls and provide a dependable training platform for inexperienced technicians.
Collapse
Affiliation(s)
- Patryk Hejduk
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, Rämistr. 100, 8091, Zurich, Switzerland.
| | - Raphael Sexauer
- Breast Imaging, Radiology and Nuclear Medicine, University Hospital Basel, Basel, Switzerland
| | - Carlotta Ruppert
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, Rämistr. 100, 8091, Zurich, Switzerland
| | - Karol Borkowski
- Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, Rämistr. 100, 8091, Zurich, Switzerland
| | - Jan Unkelbach
- Department of Radiation Oncology, University Hospital Zurich, Zurich, Switzerland
| | - Noemi Schmidt
- Breast Imaging, Radiology and Nuclear Medicine, University Hospital Basel, Basel, Switzerland
| |
Collapse
|
24
|
Gu Y, Xu W, Liu T, An X, Tian J, Ran H, Ren W, Chang C, Yuan J, Kang C, Deng Y, Wang H, Luo B, Guo S, Zhou Q, Xue E, Zhan W, Zhou Q, Li J, Zhou P, Chen M, Gu Y, Chen W, Zhang Y, Li J, Cong L, Zhu L, Wang H, Jiang Y. Ultrasound-based deep learning in the establishment of a breast lesion risk stratification system: a multicenter study. Eur Radiol 2023; 33:2954-2964. [PMID: 36418619 DOI: 10.1007/s00330-022-09263-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 09/03/2022] [Accepted: 10/22/2022] [Indexed: 11/25/2022]
Abstract
OBJECTIVES To establish a breast lesion risk stratification system using ultrasound images to predict breast malignancy and assess Breast Imaging Reporting and Data System (BI-RADS) categories simultaneously. METHODS This multicenter study prospectively collected a dataset of ultrasound images for 5012 patients at thirty-two hospitals from December 2018 to December 2020. A deep learning (DL) model was developed to conduct binary categorization (benign and malignant) and BI-RADS categories (2, 3, 4a, 4b, 4c, and 5) simultaneously. The training set of 4212 patients and the internal test set of 416 patients were from thirty hospitals. The remaining two hospitals with 384 patients were used as an external test set. Three experienced radiologists performed a reader study on 324 patients randomly selected from the test sets. We compared the performance of the DL model with that of three radiologists and the consensus of the three radiologists. RESULTS In the external test set, the DL model achieved areas under the receiver operating characteristic curve (AUCs) of 0.980 and 0.945 for the binary categorization and six-way categorizations, respectively. In the reader study set, the DL BI-RADS categories achieved a similar AUC (0.901 vs. 0.933, p = 0.0632), sensitivity (90.98% vs. 95.90%, p = 0.1094), and accuracy (83.33% vs. 79.01%, p = 0.0541), but higher specificity (78.71% vs. 68.81%, p = 0.0012) than those of the consensus of the three radiologists. CONCLUSIONS The DL model performed well in distinguishing benign from malignant breast lesions and yielded outcomes similar to experienced radiologists. This indicates the potential applicability of the DL model in clinical diagnosis. KEY POINTS • The DL model can achieve binary categorization for benign and malignant breast lesions and six-way BI-RADS categorizations for categories 2, 3, 4a, 4b, 4c, and 5, simultaneously. • The DL model showed acceptable agreement with radiologists for the classification of breast lesions. • The DL model performed well in distinguishing benign from malignant breast lesions and had promise in helping reduce unnecessary biopsies of BI-RADS 4a lesions.
Collapse
Affiliation(s)
- Yang Gu
- Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing, 100730, China
| | - Wen Xu
- Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing, 100730, China
| | - Ting Liu
- Department of Medical Imaging Advanced Research, Beijing Research Institute, Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Beijing, China
| | - Xing An
- Department of Medical Imaging Advanced Research, Beijing Research Institute, Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Beijing, China
| | - Jiawei Tian
- Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, China
| | - Haitao Ran
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University & Chongqing Key Laboratory of Ultrasound Molecular Imaging, Chongqing, China
| | - Weidong Ren
- Department of Ultrasound, Shengjing Hospital of China Medical University, Shenyang, China
| | - Cai Chang
- Department of Medical Ultrasound, Fudan University Shanghai Cancer Center, Fudan University, Shanghai, China
| | - Jianjun Yuan
- Department of Ultrasonography, Henan Provincial People's Hospital, Zhengzhou, China
| | - Chunsong Kang
- Department of Ultrasound, Shanxi Bethune Hospital, Shanxi Academy of Medical Sciences, Taiyuan, China
| | - Youbin Deng
- Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College of Huazhong University of Science and Technology, Wuhan, China
| | - Hui Wang
- Department of Ultrasound, China-Japan Union Hospital of Jilin University, Changchun, China
| | - Baoming Luo
- Department of Ultrasound, The Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Shenglan Guo
- Department of Ultrasonography, First Affiliated Hospital of Guangxi Medical University, Nanning, China
| | - Qi Zhou
- Department of Medical Ultrasound, The Second Affiliated Hospital, School of Medicine, Xi'an Jiaotong University, Xi'an, China
| | - Ensheng Xue
- Department of Ultrasound, Union Hospital of Fujian Medical University, Fujian Institute of Ultrasound Medicine, Fuzhou, China
| | - Weiwei Zhan
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University, School of Medicine, Shanghai, China
| | - Qing Zhou
- Department of Ultrasonography, Renmin Hospital of Wuhan University, Wuhan, China
| | - Jie Li
- Department of Ultrasound, Qilu Hospital, Shandong University, Jinan, China
| | - Ping Zhou
- Department of Ultrasound, The Third Xiangya Hospital of Central South University, Changsha, China
| | - Man Chen
- Department of Ultrasound Medicine, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Ying Gu
- Department of Ultrasonography, The Affiliated Hospital of Guizhou Medical University, Guiyang, China
| | - Wu Chen
- Department of Ultrasound, The First Hospital of Shanxi Medical University, Taiyuan, China
| | - Yuhong Zhang
- Department of Ultrasound, The Second Hospital of Dalian Medical University, Dalian, China
| | - Jianchu Li
- Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing, 100730, China
| | - Longfei Cong
- Department of Medical Imaging Advanced Research, Beijing Research Institute, Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Beijing, China
| | - Lei Zhu
- Department of Medical Imaging Advanced Research, Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China
| | - Hongyan Wang
- Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing, 100730, China.
| | - Yuxin Jiang
- Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing, 100730, China.
| |
Collapse
|
25
|
Artificial intelligence-aided method to detect uterine fibroids in ultrasound images: a retrospective study. Sci Rep 2023; 13:3714. [PMID: 36878941 PMCID: PMC9988965 DOI: 10.1038/s41598-022-26771-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Accepted: 12/20/2022] [Indexed: 03/08/2023] Open
Abstract
We explored a new artificial intelligence-assisted method to assist junior ultrasonographers in improving the diagnostic performance of uterine fibroids and further compared it with senior ultrasonographers to confirm the effectiveness and feasibility of the artificial intelligence method. In this retrospective study, we collected a total of 3870 ultrasound images from 667 patients with a mean age of 42.45 years ± 6.23 [SD] for those who received a pathologically confirmed diagnosis of uterine fibroids and 570 women with a mean age of 39.24 years ± 5.32 [SD] without uterine lesions from Shunde Hospital of Southern Medical University between 2015 and 2020. The DCNN model was trained and developed on the training dataset (2706 images) and internal validation dataset (676 images). To evaluate the performance of the model on the external validation dataset (488 images), we assessed the diagnostic performance of the DCNN with ultrasonographers possessing different levels of seniority. The DCNN model aided the junior ultrasonographers (Averaged) in diagnosing uterine fibroids with higher accuracy (94.72% vs. 86.63%, P < 0.001), sensitivity (92.82% vs. 83.21%, P = 0.001), specificity (97.05% vs. 90.80%, P = 0.009), positive predictive value (97.45% vs. 91.68%, P = 0.007), and negative predictive value (91.73% vs. 81.61%, P = 0.001) than they achieved alone. Their ability was comparable to that of senior ultrasonographers (Averaged) in terms of accuracy (94.72% vs. 95.24%, P = 0.66), sensitivity (92.82% vs. 93.66%, P = 0.73), specificity (97.05% vs. 97.16%, P = 0.79), positive predictive value (97.45% vs. 97.57%, P = 0.77), and negative predictive value (91.73% vs. 92.63%, P = 0.75). The DCNN-assisted strategy can considerably improve the uterine fibroid diagnosis performance of junior ultrasonographers to make them more comparable to senior ultrasonographers.
Collapse
|
26
|
Catalano O, Fusco R, De Muzio F, Simonetti I, Palumbo P, Bruno F, Borgheresi A, Agostini A, Gabelloni M, Varelli C, Barile A, Giovagnoni A, Gandolfo N, Miele V, Granata V. Recent Advances in Ultrasound Breast Imaging: From Industry to Clinical Practice. Diagnostics (Basel) 2023; 13:diagnostics13050980. [PMID: 36900124 PMCID: PMC10000574 DOI: 10.3390/diagnostics13050980] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Accepted: 03/02/2023] [Indexed: 03/08/2023] Open
Abstract
Breast ultrasound (US) has undergone dramatic technological improvement through recent decades, moving from a low spatial resolution, grayscale-limited technique to a highly performing, multiparametric modality. In this review, we first focus on the spectrum of technical tools that have become commercially available, including new microvasculature imaging modalities, high-frequency transducers, extended field-of-view scanning, elastography, contrast-enhanced US, MicroPure, 3D US, automated US, S-Detect, nomograms, images fusion, and virtual navigation. In the subsequent section, we discuss the broadened current application of US in breast clinical scenarios, distinguishing among primary US, complementary US, and second-look US. Finally, we mention the still ongoing limitations and the challenging aspects of breast US.
Collapse
Affiliation(s)
- Orlando Catalano
- Department of Radiology, Istituto Diagnostico Varelli, 80126 Naples, Italy
| | - Roberta Fusco
- Medical Oncology Division, Igea SpA, 80013 Naples, Italy
- Correspondence:
| | - Federica De Muzio
- Department of Medicine and Health Sciences “V. Tiberio”, University of Molise, 86100 Campobasso, Italy
| | - Igino Simonetti
- Division of Radiology, “Istituto Nazionale Tumori IRCCS Fondazione Pascale-IRCCS di Napoli”, 80131 Naples, Italy
| | - Pierpaolo Palumbo
- Department of Diagnostic Imaging, Area of Cardiovascular and Interventional Imaging, Abruzzo Health Unit 1, 67100 L’Aquila, Italy
- Italian Society of Medical and Interventional Radiology (SIRM), SIRM Foundation, 20122 Milan, Italy
| | - Federico Bruno
- Department of Diagnostic Imaging, Area of Cardiovascular and Interventional Imaging, Abruzzo Health Unit 1, 67100 L’Aquila, Italy
- Italian Society of Medical and Interventional Radiology (SIRM), SIRM Foundation, 20122 Milan, Italy
| | - Alessandra Borgheresi
- Department of Clinical, Special and Dental Sciences, University Politecnica delle Marche, 60126 Ancona, Italy
- Department of Radiology, University Hospital “Azienda Ospedaliera Universitaria delle Marche”, 60126 Ancona, Italy
| | - Andrea Agostini
- Department of Clinical, Special and Dental Sciences, University Politecnica delle Marche, 60126 Ancona, Italy
- Department of Radiology, University Hospital “Azienda Ospedaliera Universitaria delle Marche”, 60126 Ancona, Italy
| | - Michela Gabelloni
- Department of Translational Research, Diagnostic and Interventional Radiology, University of Pisa, 56126 Pisa, Italy
| | - Carlo Varelli
- Department of Radiology, Istituto Diagnostico Varelli, 80126 Naples, Italy
| | - Antonio Barile
- Department of Applied Clinical Sciences and Biotechnology, University of L’Aquila, 67100 L’Aquila, Italy
| | - Andrea Giovagnoni
- Department of Clinical, Special and Dental Sciences, University Politecnica delle Marche, 60126 Ancona, Italy
- Department of Radiology, University Hospital “Azienda Ospedaliera Universitaria delle Marche”, 60126 Ancona, Italy
| | - Nicoletta Gandolfo
- Diagnostic Imaging Department, Villa Scassi Hospital-ASL 3, Corso Scassi 1, 16149 Genoa, Italy
| | - Vittorio Miele
- Italian Society of Medical and Interventional Radiology (SIRM), SIRM Foundation, 20122 Milan, Italy
- Department of Emergency Radiology, Careggi University Hospital, 50134 Florence, Italy
| | - Vincenza Granata
- Division of Radiology, “Istituto Nazionale Tumori IRCCS Fondazione Pascale-IRCCS di Napoli”, 80131 Naples, Italy
| |
Collapse
|
27
|
Kaplan E, Chan WY, Dogan S, Barua PD, Bulut HT, Tuncer T, Cizik M, Tan RS, Acharya UR. Automated BI-RADS classification of lesions using pyramid triple deep feature generator technique on breast ultrasound images. Med Eng Phys 2022; 108:103895. [DOI: 10.1016/j.medengphy.2022.103895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 09/09/2022] [Accepted: 09/13/2022] [Indexed: 10/14/2022]
|
28
|
Syed AH, Khan T. Evolution of research trends in artificial intelligence for breast cancer diagnosis and prognosis over the past two decades: A bibliometric analysis. Front Oncol 2022; 12:854927. [PMID: 36267967 PMCID: PMC9578338 DOI: 10.3389/fonc.2022.854927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Accepted: 08/30/2022] [Indexed: 01/27/2023] Open
Abstract
Objective In recent years, among the available tools, the concurrent application of Artificial Intelligence (AI) has improved the diagnostic performance of breast cancer screening. In this context, the present study intends to provide a comprehensive overview of the evolution of AI for breast cancer diagnosis and prognosis research using bibliometric analysis. Methodology Therefore, in the present study, relevant peer-reviewed research articles published from 2000 to 2021 were downloaded from the Scopus and Web of Science (WOS) databases and later quantitatively analyzed and visualized using Bibliometrix (R package). Finally, open challenges areas were identified for future research work. Results The present study revealed that the number of literature studies published in AI for breast cancer detection and survival prediction has increased from 12 to 546 between the years 2000 to 2021. The United States of America (USA), the Republic of China, and India are the most productive publication-wise in this field. Furthermore, the USA leads in terms of the total citations; however, hungry and Holland take the lead positions in average citations per year. Wang J is the most productive author, and Zhan J is the most relevant author in this field. Stanford University in the USA is the most relevant affiliation by the number of published articles. The top 10 most relevant sources are Q1 journals with PLOS ONE and computer in Biology and Medicine are the leading journals in this field. The most trending topics related to our study, transfer learning and deep learning, were identified. Conclusion The present findings provide insight and research directions for policymakers and academic researchers for future collaboration and research in AI for breast cancer patients.
Collapse
Affiliation(s)
- Asif Hassan Syed
- Department of Computer Science, Faculty of Computing and Information Technology Rabigh (FCITR), King Abdulaziz University, Jeddah, Saudi Arabia
| | - Tabrej Khan
- Department of Information Systems, Faculty of Computing and Information Technology Rabigh (FCITR), King Abdulaziz University, Jeddah, Saudi Arabia
| |
Collapse
|
29
|
Mahant SS, Varma AR. Artificial Intelligence in Breast Ultrasound: The Emerging Future of Modern Medicine. Cureus 2022; 14:e28945. [PMID: 36237807 PMCID: PMC9547651 DOI: 10.7759/cureus.28945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Accepted: 09/08/2022] [Indexed: 11/25/2022] Open
Abstract
In today's world, progressively enormous popularity prevails around artificial intelligence (AI). AI is gaining popularity in the identification of various images. Therefore, it has been widely used in the ultrasound of the breast. Furthermore, AI can perform a quantitative evaluation, which further helps maintain the diagnosis's accuracy. Moreover, breast cancer is the most common cancer in women, posing a severe threat to women's health. Hence, its early detection is usually associated with a patient's prognosis. As a result, using AI in breast cancer screening and detection is highly crucial. The concept of AI in the perspective of breast ultrasound has been highlighted in this brief review article. It tends to focus on early AI, i.e., traditional machine learning and deep learning algorithms. Also, the use of AI in ultrasound and the use of it in mammography, magnetic resonance imaging, nuclear medicine imaging, and classification of breast lesions is broadly explained, along with the challenges faced in bringing AI into daily practice.
Collapse
|
30
|
Wilding R, Sheraton VM, Soto L, Chotai N, Tan EY. Deep learning applied to breast imaging classification and segmentation with human expert intervention. J Ultrasound 2022; 25:659-666. [PMID: 35000127 PMCID: PMC9402837 DOI: 10.1007/s40477-021-00642-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Accepted: 11/22/2021] [Indexed: 11/30/2022] Open
Abstract
PURPOSE Automatic classification and segmentation of tumors in breast ultrasound images enables better diagnosis and planning treatment strategies for breast cancer patients. METHODS We collected 953 breast ultrasound images from two open-source datasets and classified them with help of an expert radiologist according to BI-RADS criteria. The data was split into normal, benign and malignant classes. We then used machine learning to develop classification and segmentation algorithms. RESULTS We found 3.92% of the images across the open-source datasets had erroneous classifications. Post-radiologist intervention, three algorithms were developed based on the classification categories. Classification algorithms distinguished images with healthy breast tissue from those with abnormal tissue with 96% accuracy, and distinguished benign from malignant images with 85% accuracy. Both algorithms generated robust F1 and AUROC metrics. Finally, the masses within images were segmented with an 80.31% DICE score. CONCLUSIONS Our work illustrates the potential of deep learning algorithms to improve the accuracy of breast ultrasound assessments and to facilitate automated assessments.
Collapse
Affiliation(s)
| | | | - Lysabella Soto
- Research Department, RESOMED, Maracaibo, Venezuela
- Postgraduate Division Studies of Radiology, Medicine School of Zulia's University, Maracaibo, Venezuela
| | - Niketa Chotai
- Department of Radiology, RadLink Diagnostic Imaging Center, Singapore, Singapore
| | - Ern Yu Tan
- Department of General Surgery, Tan Tock Seng Hospital, Singapore, Singapore.
| |
Collapse
|
31
|
Gu J, Jiang T. Ultrasound radiomics in personalized breast management: Current status and future prospects. Front Oncol 2022; 12:963612. [PMID: 36059645 PMCID: PMC9428828 DOI: 10.3389/fonc.2022.963612] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 08/01/2022] [Indexed: 11/18/2022] Open
Abstract
Breast cancer is the most common cancer in women worldwide. Providing accurate and efficient diagnosis, risk stratification and timely adjustment of treatment strategies are essential steps in achieving precision medicine before, during and after treatment. Radiomics provides image information that cannot be recognized by the naked eye through deep mining of medical images. Several studies have shown that radiomics, as a second reader of medical images, can assist physicians not only in the detection and diagnosis of breast lesions but also in the assessment of risk stratification and prediction of treatment response. Recently, more and more studies have focused on the application of ultrasound radiomics in breast management. We summarized recent research advances in ultrasound radiomics for the diagnosis of benign and malignant breast lesions, prediction of molecular subtype, assessment of lymph node status, prediction of neoadjuvant chemotherapy response, and prediction of survival. In addition, we discuss the current challenges and future prospects of ultrasound radiomics.
Collapse
Affiliation(s)
- Jionghui Gu
- Department of Ultrasound, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
- Zhejiang University Cancer Center, Hangzhou, China
| | - Tian'an Jiang
- Department of Ultrasound, The First Affiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
- Zhejiang University Cancer Center, Hangzhou, China
- *Correspondence: Tian'an Jiang,
| |
Collapse
|
32
|
Kumar P, Kumar A, Srivastava S, Padma Sai Y. A novel bi-modal extended Huber loss function based refined mask RCNN approach for automatic multi instance detection and localization of breast cancer. Proc Inst Mech Eng H 2022; 236:1036-1053. [DOI: 10.1177/09544119221095416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Breast cancer is an extremely aggressive cancer in women. Its abnormalities can be observed in the form of masses, calcification and lumps. In order to reduce the mortality rate of women its detection is needed at an early stage. The present paper proposes a novel bi-modal extended Huber loss function based refined mask regional convolutional neural network for automatic multi-instance detection and localization of breast cancer. To refine and increase the efficacy of the proposed method three changes are casted. First, a pre-processing step is performed for mammogram and ultrasound breast images. Second, the features of the region proposal network are separately mapped for accurate region of interest. Third, to reduce overfitting and fast convergence, an extended Huber loss function is used at the place of Smooth L1( x) in boundary loss. To extend the functionality of Huber loss, the delta parameter is automated by the aid of median absolute deviation with grid search algorithm. It provides the best optimum value of delta instead of user-based value. The proposed method is compared with pre-existing methods in terms of accuracy, true positive rate, true negative rate, precision, F-score, balanced classification rate, Youden’s index, Jaccard Index and dice coefficient on CBIS-DDSM and ultrasound database. The experimental result shows that the proposed method is a better suited approach for multi-instance detection, localization and classification of breast cancer. It can be used as a diagnostic medium that helps in clinical purposes and leads to a precise diagnosis of breast cancer abnormalities.
Collapse
Affiliation(s)
- Pradeep Kumar
- Department of Electronics and Communication Engineering, National Institute of Technology Patna, Patna, Bihar, India
| | - Abhinav Kumar
- Department of Electronics and Communication Engineering, National Institute of Technology Patna, Patna, Bihar, India
| | - Subodh Srivastava
- Department of Electronics and Communication Engineering, National Institute of Technology Patna, Patna, Bihar, India
| | - Yarlagadda Padma Sai
- Department of Electronics and Communication Engineering, VNR VJIET, Hyderabad, Telangana, India
| |
Collapse
|
33
|
Sabani A, Landsmann A, Hejduk P, Schmidt C, Marcon M, Borkowski K, Rossi C, Ciritsis A, Boss A. BI-RADS-Based Classification of Mammographic Soft Tissue Opacities Using a Deep Convolutional Neural Network. Diagnostics (Basel) 2022; 12:diagnostics12071564. [PMID: 35885470 PMCID: PMC9318280 DOI: 10.3390/diagnostics12071564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 06/16/2022] [Accepted: 06/21/2022] [Indexed: 11/17/2022] Open
Abstract
The aim of this study was to investigate the potential of a machine learning algorithm to classify breast cancer solely by the presence of soft tissue opacities in mammograms, independent of other morphological features, using a deep convolutional neural network (dCNN). Soft tissue opacities were classified based on their radiological appearance using the ACR BI-RADS atlas. We included 1744 mammograms from 438 patients to create 7242 icons by manual labeling. The icons were sorted into three categories: “no opacities” (BI-RADS 1), “probably benign opacities” (BI-RADS 2/3) and “suspicious opacities” (BI-RADS 4/5). A dCNN was trained (70% of data), validated (20%) and finally tested (10%). A sliding window approach was applied to create colored probability maps for visual impression. Diagnostic performance of the dCNN was compared to human readout by experienced radiologists on a “real-world” dataset. The accuracies of the models on the test dataset ranged between 73.8% and 89.8%. Compared to human readout, our dCNN achieved a higher specificity (100%, 95% CI: 85.4–100%; reader 1: 86.2%, 95% CI: 67.4–95.5%; reader 2: 79.3%, 95% CI: 59.7–91.3%), and the sensitivity (84.0%, 95% CI: 63.9–95.5%) was lower than that of human readers (reader 1:88.0%, 95% CI: 67.4–95.4%; reader 2:88.0%, 95% CI: 67.7–96.8%). In conclusion, a dCNN can be used for the automatic detection as well as the standardized and observer-independent classification of soft tissue opacities in mammograms independent of the presence of microcalcifications. Human decision making in accordance with the BI-RADS classification can be mimicked by artificial intelligence.
Collapse
|
34
|
Abstract
Machine learning (ML) methods are pervading an increasing number of fields of application because of their capacity to effectively solve a wide variety of challenging problems. The employment of ML techniques in ultrasound imaging applications started several years ago but the scientific interest in this issue has increased exponentially in the last few years. The present work reviews the most recent (2019 onwards) implementations of machine learning techniques for two of the most popular ultrasound imaging fields, medical diagnostics and non-destructive evaluation. The former, which covers the major part of the review, was analyzed by classifying studies according to the human organ investigated and the methodology (e.g., detection, segmentation, and/or classification) adopted, while for the latter, some solutions to the detection/classification of material defects or particular patterns are reported. Finally, the main merits of machine learning that emerged from the study analysis are summarized and discussed.
Collapse
|
35
|
Zhang L, Dong D, Sun Y, Hu C, Sun C, Wu Q, Tian J. Development and Validation of a Deep Learning Model to Screen for Trisomy 21 During the First Trimester From Nuchal Ultrasonographic Images. JAMA Netw Open 2022; 5:e2217854. [PMID: 35727579 PMCID: PMC9214589 DOI: 10.1001/jamanetworkopen.2022.17854] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
IMPORTANCE Accurate screening of trisomy 21 in the first trimester can provide an early opportunity for decision-making regarding reproductive choices. OBJECTIVE To develop and validate a deep learning model for screening fetuses with trisomy 21 based on ultrasonographic images. DESIGN, SETTING, AND PARTICIPANTS This diagnostic study used data from all available cases and controls enrolled at 2 hospitals in China between January 2009 and September 2020. Two-dimensional images of the midsagittal plane of the fetal face in singleton pregnancies with gestational age more than 11 weeks and less than 14 weeks were examined. Observers were blinded to subjective fetus nuchal translucency (NT) marker measurements. A convolutional neural network was developed to construct a deep learning model. Data augmentation was applied to generate more data. Different groups were randomly selected as training and validation sets to assess the robustness of the deep learning model. The fetal NT was shown and measured. Each detection of trisomy 21 was confirmed by chorionic villus sampling or amniocentesis. Data were analyzed from March 1, 2021, to January 3, 2022. MAIN OUTCOMES AND MEASURES The primary outcome was detection of fetuses with trisomy 21. The receiver operating characteristic curve, metrics of accuracy, area under the curve (AUC), sensitivity, and specificity were used for model performance evaluation. RESULTS A total of 822 case and control participants (mean [SD] age, 31.9 [4.6] years) were enrolled in the study, including 550 participants (mean [SD] age, 31.7 [4.7] years) in the training set and 272 participants (mean [SD] age, 32.3 [4.7] years) in the validation set. The deep learning model showed good performance for trisomy 21 screening in the training (AUC, 0.98; 95% CI, 0.97-0.99) and validation (AUC, 0.95; 95% CI, 0.93-0.98) sets. The deep learning model had better detective performance for fetuses with trisomy 21 than the model with NT marker and maternal age (training: AUC, 0.82; 95% CI, 0.77-0.86; validation: AUC, 0.73; 95% CI, 0.66-0.80). CONCLUSIONS AND RELEVANCE These findings suggest that this deep learning model accurately screened fetuses with trisomy 21, which indicates that the model is a potential tool to facilitate universal primary screening for trisomy 21.
Collapse
Affiliation(s)
- Liwen Zhang
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Di Dong
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Zhuhai Precision Medical Center, Zhuhai People’s Hospital, Jinan University, Zhuhai, China
| | - Yongqing Sun
- Beijing Obstetrics and Gynecology Hospital, Capital Medical University. Beijing Maternal and Child Health Care Hospital, China
| | - Chaoen Hu
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Congxin Sun
- Department of Ultrasound, Shijiazhuang Obstetrics and Gynecology Hospital, Shijiazhuang, China
| | - Qingqing Wu
- Beijing Obstetrics and Gynecology Hospital, Capital Medical University. Beijing Maternal and Child Health Care Hospital, China
| | - Jie Tian
- CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine and Engineering, Beihang University, Beijing, China
| |
Collapse
|
36
|
Yu AC, Mohajer B, Eng J. External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. Radiol Artif Intell 2022; 4:e210064. [PMID: 35652114 DOI: 10.1148/ryai.210064] [Citation(s) in RCA: 94] [Impact Index Per Article: 47.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 03/09/2022] [Accepted: 04/12/2022] [Indexed: 01/17/2023]
Abstract
Purpose To assess generalizability of published deep learning (DL) algorithms for radiologic diagnosis. Materials and Methods In this systematic review, the PubMed database was searched for peer-reviewed studies of DL algorithms for image-based radiologic diagnosis that included external validation, published from January 1, 2015, through April 1, 2021. Studies using nonimaging features or incorporating non-DL methods for feature extraction or classification were excluded. Two reviewers independently evaluated studies for inclusion, and any discrepancies were resolved by consensus. Internal and external performance measures and pertinent study characteristics were extracted, and relationships among these data were examined using nonparametric statistics. Results Eighty-three studies reporting 86 algorithms were included. The vast majority (70 of 86, 81%) reported at least some decrease in external performance compared with internal performance, with nearly half (42 of 86, 49%) reporting at least a modest decrease (≥0.05 on the unit scale) and nearly a quarter (21 of 86, 24%) reporting a substantial decrease (≥0.10 on the unit scale). No study characteristics were found to be associated with the difference between internal and external performance. Conclusion Among published external validation studies of DL algorithms for image-based radiologic diagnosis, the vast majority demonstrated diminished algorithm performance on the external dataset, with some reporting a substantial performance decrease.Keywords: Meta-Analysis, Computer Applications-Detection/Diagnosis, Neural Networks, Computer Applications-General (Informatics), Epidemiology, Technology Assessment, Diagnosis, Informatics Supplemental material is available for this article. © RSNA, 2022.
Collapse
Affiliation(s)
- Alice C Yu
- Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 1800 Orleans St, Baltimore, MD 21287
| | - Bahram Mohajer
- Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 1800 Orleans St, Baltimore, MD 21287
| | - John Eng
- Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 1800 Orleans St, Baltimore, MD 21287
| |
Collapse
|
37
|
Discrimination of Breast Cancer Based on Ultrasound Images and Convolutional Neural Network. JOURNAL OF ONCOLOGY 2022; 2022:7733583. [PMID: 35345516 PMCID: PMC8957444 DOI: 10.1155/2022/7733583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Revised: 02/10/2022] [Accepted: 02/24/2022] [Indexed: 11/17/2022]
Abstract
The aim of our study was to establish an artificial intelligence tool for the diagnosis of breast disease base on ultrasound (US) images. A deep learning algorithm Efficient-Det assisted US diagnosis method was developed to determine breast suspicious lesions as benign, malignant, or normal. Totally 1181 US images from 487 patients of our hospital and 694 publicly accessible images were employed for modeling, including 558 benign images, 370 malignant images, and 253 normal tissue images. The actual diagnosis results for the patients were determined by the biopsy or surgery. Efficient-Det was first retrained using an exclusive public breast cancer US dataset with transfer learning techniques. A blind test set consisting of 50 benign, 50 malignant, and 50 normal tissue images was randomly picked from the patients' images as the independent test set to test its searching ability on suspicious tumor regions. Furthermore, the confusion matrix and classification accuracy were employed as evaluation metrics to select the optimal classification models. Efficient-Det has demonstrated remarkable progress in general image recognition tasks with specific advantages of locating and identifying tumor areas simultaneously. Compared to the manual method (mean accuracy: 95.3% and 60 s per image) and traditional feature engineering method (mean accuracy: 90% and 15 s per image), our Efficient-Det has the capability of providing a competitive (mean accuracy: 92.6%) and fast (0.06 s per image) classification result. The deployment of Efficient-Det in our local breast cancer discrimination task exhibits specific applicability within real clinical workflows.
Collapse
|
38
|
Balkenende L, Teuwen J, Mann RM. Application of Deep Learning in Breast Cancer Imaging. Semin Nucl Med 2022; 52:584-596. [PMID: 35339259 DOI: 10.1053/j.semnuclmed.2022.02.003] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Revised: 02/15/2022] [Accepted: 02/16/2022] [Indexed: 11/11/2022]
Abstract
This review gives an overview of the current state of deep learning research in breast cancer imaging. Breast imaging plays a major role in detecting breast cancer at an earlier stage, as well as monitoring and evaluating breast cancer during treatment. The most commonly used modalities for breast imaging are digital mammography, digital breast tomosynthesis, ultrasound and magnetic resonance imaging. Nuclear medicine imaging techniques are used for detection and classification of axillary lymph nodes and distant staging in breast cancer imaging. All of these techniques are currently digitized, enabling the possibility to implement deep learning (DL), a subset of Artificial intelligence, in breast imaging. DL is nowadays embedded in a plethora of different tasks, such as lesion classification and segmentation, image reconstruction and generation, cancer risk prediction, and prediction and assessment of therapy response. Studies show similar and even better performances of DL algorithms compared to radiologists, although it is clear that large trials are needed, especially for ultrasound and magnetic resonance imaging, to exactly determine the added value of DL in breast cancer imaging. Studies on DL in nuclear medicine techniques are only sparsely available and further research is mandatory. Legal and ethical issues need to be considered before the role of DL can expand to its full potential in clinical breast care practice.
Collapse
Affiliation(s)
- Luuk Balkenende
- Department of Radiology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands; Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jonas Teuwen
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands; Department of Radiation Oncology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Ritse M Mann
- Department of Radiology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands; Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands.
| |
Collapse
|
39
|
Yuan H, Gao Z, He X, Li D, Duan S, Effah CY, Wang W, Wang J, Qu L, Wu Y. Application of logistic regression and convolutional neural network in prediction and diagnosis of high-risk populations of lung cancer. Eur J Cancer Prev 2022; 31:145-151. [PMID: 33859129 DOI: 10.1097/cej.0000000000000684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
OBJECTIVES The early detection, early diagnosis, and early treatment of lung cancer are the best strategies to improve the 5-year survival rate. Logistic regression analysis can be a helpful tool in the early detection of high-risk groups of lung cancer. Convolutional neural network (CNN) could distinguish benign from malignant pulmonary nodules, which is critical for early precise diagnosis and treatment. Here, we developed a risk assessment model of lung cancer and a high-precision classification diagnostic model using these technologies so as to provide a basis for early screening of lung cancer and for intelligent differential diagnosis. METHODS A total of 355 lung cancer patients, 444 patients with benign lung disease and 472 healthy people from The First Affiliated Hospital of Zhengzhou University were included in this study. Moreover, the dataset of 607 lung computed tomography images was collected from the above patients. The logistic regression method was employed to screen the high-risk groups of lung cancer, and the CNN model was designed to classify pulmonary nodules into benign or malignant nodules. RESULTS The area under the curve of the lung cancer risk assessment model in the training set and the testing set were 0.823 and 0.710, respectively. After finely optimizing the settings of the CNN, the area under the curve could reach 0.984. CONCLUSIONS This performance demonstrated that the lung cancer risk assessment model could be used to screen for high-risk individuals with lung cancer and the CNN framework was suitable for the differential diagnosis of pulmonary nodules.
Collapse
Affiliation(s)
| | | | | | - Di Li
- Departments of Toxicology
| | | | | | | | - Jing Wang
- Occupational and Environmental Health
| | - Lingbo Qu
- Nutrition and Food Hygiene, College of Public Health, Zhengzhou University
| | | |
Collapse
|
40
|
Pi Y, Li Q, Qi X, Deng D, Yi Z. Automated assessment of BI-RADS categories for ultrasound images using multi-scale neural networks with an order-constrained loss function. APPL INTELL 2022. [DOI: 10.1007/s10489-021-03140-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
41
|
Landsmann A, Wieler J, Hejduk P, Ciritsis A, Borkowski K, Rossi C, Boss A. Applied Machine Learning in Spiral Breast-CT: Can We Train a Deep Convolutional Neural Network for Automatic, Standardized and Observer Independent Classification of Breast Density? Diagnostics (Basel) 2022; 12:diagnostics12010181. [PMID: 35054348 PMCID: PMC8775263 DOI: 10.3390/diagnostics12010181] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2021] [Revised: 01/05/2022] [Accepted: 01/11/2022] [Indexed: 02/05/2023] Open
Abstract
The aim of this study was to investigate the potential of a machine learning algorithm to accurately classify parenchymal density in spiral breast-CT (BCT), using a deep convolutional neural network (dCNN). In this retrospectively designed study, 634 examinations of 317 patients were included. After image selection and preparation, 5589 images from 634 different BCT examinations were sorted by a four-level density scale, ranging from A to D, using ACR BI-RADS-like criteria. Subsequently four different dCNN models (differences in optimizer and spatial resolution) were trained (70% of data), validated (20%) and tested on a “real-world” dataset (10%). Moreover, dCNN accuracy was compared to a human readout. The overall performance of the model with lowest resolution of input data was highest, reaching an accuracy on the “real-world” dataset of 85.8%. The intra-class correlation of the dCNN and the two readers was almost perfect (0.92) and kappa values between both readers and the dCNN were substantial (0.71–0.76). Moreover, the diagnostic performance between the readers and the dCNN showed very good correspondence with an AUC of 0.89. Artificial Intelligence in the form of a dCNN can be used for standardized, observer-independent and reliable classification of parenchymal density in a BCT examination.
Collapse
|
42
|
Hsieh YH, Hsu FR, Dai ST, Huang HY, Chen DR, Shia WC. Incorporating the Breast Imaging Reporting and Data System Lexicon with a Fully Convolutional Network for Malignancy Detection on Breast Ultrasound. Diagnostics (Basel) 2021; 12:66. [PMID: 35054233 PMCID: PMC8774546 DOI: 10.3390/diagnostics12010066] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Revised: 12/21/2021] [Accepted: 12/25/2021] [Indexed: 11/16/2022] Open
Abstract
In this study, we applied semantic segmentation using a fully convolutional deep learning network to identify characteristics of the Breast Imaging Reporting and Data System (BI-RADS) lexicon from breast ultrasound images to facilitate clinical malignancy tumor classification. Among 378 images (204 benign and 174 malignant images) from 189 patients (102 benign breast tumor patients and 87 malignant patients), we identified seven malignant characteristics related to the BI-RADS lexicon in breast ultrasound. The mean accuracy and mean IU of the semantic segmentation were 32.82% and 28.88, respectively. The weighted intersection over union was 85.35%, and the area under the curve was 89.47%, showing better performance than similar semantic segmentation networks, SegNet and U-Net, in the same dataset. Our results suggest that the utilization of a deep learning network in combination with the BI-RADS lexicon can be an important supplemental tool when using ultrasound to diagnose breast malignancy.
Collapse
Affiliation(s)
- Yung-Hsien Hsieh
- Department of Information Engineering and Computer Science, Feng Chia University, Taichung 40724, Taiwan; (Y.-H.H.); (F.-R.H.); (S.-T.D.)
| | - Fang-Rong Hsu
- Department of Information Engineering and Computer Science, Feng Chia University, Taichung 40724, Taiwan; (Y.-H.H.); (F.-R.H.); (S.-T.D.)
| | - Seng-Tong Dai
- Department of Information Engineering and Computer Science, Feng Chia University, Taichung 40724, Taiwan; (Y.-H.H.); (F.-R.H.); (S.-T.D.)
| | - Hsin-Ya Huang
- Comprehensive Breast Cancer Center, Changhua Christian Hospital, Changhua 500, Taiwan;
| | - Dar-Ren Chen
- Comprehensive Breast Cancer Center, Changhua Christian Hospital, Changhua 500, Taiwan;
- School of Medicine, Chung Shan Medical University, Taichung 40201, Taiwan
| | - Wei-Chung Shia
- Department of Information Engineering and Computer Science, Feng Chia University, Taichung 40724, Taiwan; (Y.-H.H.); (F.-R.H.); (S.-T.D.)
- Molecular Medicine Laboratory, Department of Research, Changhua Christian Hospital, Changhua 500, Taiwan
| |
Collapse
|
43
|
Zhang G, Ren Y, Xi X, Li D, Guo J, Li X, Tian C, Xu Z. LRSCnet: Local Reference Semantic Code learning for breast tumor classification in ultrasound images. Biomed Eng Online 2021; 20:127. [PMID: 34920726 PMCID: PMC8684265 DOI: 10.1186/s12938-021-00968-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Accepted: 12/09/2021] [Indexed: 11/17/2022] Open
Abstract
Purpose This study proposed a novel Local Reference Semantic Code (LRSC) network for automatic breast ultrasound image classification with few labeled data. Methods In the proposed network, the local structure extractor is firstly developed to learn the local reference which describes common local characteristics of tumors. After that, a two-stage hierarchical encoder is developed to encode the local structures of lesion into the high-level semantic code. Based on the learned semantic code, the self-matching layer is proposed for the final classification. Results In the experiment, the proposed method outperformed traditional classification methods and AUC (Area Under Curve), ACC (Accuracy), Sen (Sensitivity), Spec (Specificity), PPV (Positive Predictive Values), and NPV(Negative Predictive Values) are 0.9540, 0.9776, 0.9629, 0.93, 0.9774 and 0.9090, respectively. In addition, the proposed method also improved matching speed. Conclusions LRSC-network is proposed for breast ultrasound images classification with few labeled data. In the proposed network, a two-stage hierarchical encoder is introduced to learn high-level semantic code. The learned code contains more effective high-level classification information and is simpler, leading to better generalization ability.
Collapse
Affiliation(s)
- Guang Zhang
- School of Software, Shandong University, Jinan, China.,Health Management, The First Affiliated Hospital of Shangdong First Medical University & Shandong Provincial Qianfoshan Hospital, Jinan, China
| | - Yanwei Ren
- School of Software, Shandong University, Jinan, China
| | - Xiaoming Xi
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China.
| | - Delin Li
- Health Management, The First Affiliated Hospital of Shangdong First Medical University & Shandong Provincial Qianfoshan Hospital, Jinan, China
| | - Jie Guo
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China
| | - Xiaofeng Li
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China
| | - Cuihuan Tian
- School of Medicine, Shandong Universit, Jinan, China.,Health Management Center, QiLu Hospital of Shandong University, Jinan, China
| | - Zunyi Xu
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China.
| |
Collapse
|
44
|
Decoding the molecular subtypes of breast cancer seen on multimodal ultrasound images using an assembled convolutional neural network model: A prospective and multicentre study. EBioMedicine 2021; 74:103684. [PMID: 34773890 PMCID: PMC8599999 DOI: 10.1016/j.ebiom.2021.103684] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Revised: 10/17/2021] [Accepted: 10/25/2021] [Indexed: 02/06/2023] Open
Abstract
Background Preoperative determination of breast cancer molecular subtypes facilitates individualized treatment plan-making and improves patient prognosis. We aimed to develop an assembled convolutional neural network (ACNN) model for the preoperative prediction of molecular subtypes using multimodal ultrasound (US) images. Methods This multicentre study prospectively evaluated a dataset of greyscale US, colour Doppler flow imaging (CDFI), and shear-wave elastography (SWE) images in 807 patients with 818 breast cancers from November 2016 to February 2021. The St. Gallen molecular subtypes of breast cancer were confirmed by postoperative immunohistochemical examination. The monomodal ACNN model based on greyscale US images, the dual-modal ACNN model based on greyscale US and CDFI images, and the multimodal ACNN model based on greyscale US and CDFI as well as SWE images were constructed in the training cohort. The performances of three ACNN models in predicting four- and five-classification molecular subtypes and identifying triple negative from non-triple negative subtypes were assessed and compared. The performance of the multimodal ACNN was also compared with preoperative core needle biopsy (CNB). Finding The performance of the multimodal ACNN model (macroaverage area under the curve [AUC]: 0.89–0.96) was superior to that of the dual-modal ACNN model (macroaverage AUC: 0.81–0.84) and the monomodal ACNN model (macroaverage AUC: 0.73–0.75) in predicting four-classification breast cancer molecular subtypes, which was also better than that of preoperative CNB (AUC: 0.89–0.99 vs. 0.67–0.82, p < 0.05). In addition, the multimodal ACNN model outperformed the other two ACNN models in predicting five-classification molecular subtypes (AUC: 0.87–0.94 vs. 0.78-0.81 vs. 0.71–0.78) and identifying triple negative from non-triple negative breast cancers (AUC: 0.934–0.970 vs. 0.688–0.830 vs. 0.536–0.650, p < 0.05). Moreover, the multimodal ACNN model obtained satisfactory prediction performance for both T1 and non-T1 lesions (AUC: 0.957–0.958 and 0.932–0.985). Interpretation The multimodal US-based ACNN model is a potential noninvasive decision-making method for the management of patients with breast cancer in clinical practice. Funding This work was supported in part by the National Natural Science Foundation of China (Grants 81725008 and 81927801), Shanghai Municipal Health Commission (Grants 2019LJ21 and SHSLCZDZK03502), and the Science and Technology Commission of Shanghai Municipality (Grants 19441903200, 19DZ2251100, and 21Y11910800).
Collapse
|
45
|
Okawa K, Inoue M, Sakae T. Development of a tracking error prediction system for the CyberKnife Synchrony Respiratory Tracking System with use of support vector regression. Med Biol Eng Comput 2021; 59:2409-2418. [PMID: 34655052 DOI: 10.1007/s11517-021-02445-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2020] [Accepted: 09/17/2021] [Indexed: 12/25/2022]
Abstract
PURPOSE The accuracy of the CyberKnife Synchrony Respiratory Tracking System is dependent on the breathing pattern of a patient. Therefore, the tracking error in each patient must be determined. Support vector regression (SVR) can be used to easily identify the tracking error in each patient. This study aimed to develop a system with SVR that can predict tracking error according to a patient's respiratory waveform. METHODS Datasets of the respiratory waveforms of 93 patients were obtained. The feature variables were variation in respiration amplitude, tumor velocity, and phase shift between tumor and the chest wall, and the target variable was tracking error. A learning model was evaluated with tenfold cross-validation. We documented the difference between the predicted and actual tracking errors and assessed the correlation coefficient and coefficient of determination. RESULTS The average difference and maximum difference between the actual and predicted tracking errors were 0.57 ± 0.63 mm and 2.1 mm, respectively. The correlation coefficient and coefficient of determination were 0.86 and 0.74, respectively. CONCLUSION We developed a system for obtaining tracking error by using SVR. The accuracy of such a system is clinically useful. Moreover, the system can easily evaluate tracking error. We developed a system that can be used to predict the tracking error of SRTS in the CyberKnife Robotic Radiosurgery System using machine learning. The feature variables were the breathing parameters, and the target variable was the tracking error. We used support vector regression algorithm.
Collapse
Affiliation(s)
- Kohei Okawa
- Department Radiotherapy Quality Management, Yokohama CyberKnife Center, Ichizawa-cho 574-1, Asahi-ku, Yokohama, 241-0014, Japan.
- Graduate School of Comprehensive Human Science, University of Tsukuba, Ibaraki, 305-8577, Japan.
| | - Mitsuhiro Inoue
- Department Radiotherapy Quality Management, Yokohama CyberKnife Center, Ichizawa-cho 574-1, Asahi-ku, Yokohama, 241-0014, Japan
| | - Takeji Sakae
- Proton Medical Research Center, University of Tsukuba Hospital, Ibaraki, 305-8576, Japan
- Faculty of Medicine, University of Tsukuba, Ibaraki, 305-8577, Japan
| |
Collapse
|
46
|
Shen Y, Shamout FE, Oliver JR, Witowski J, Kannan K, Park J, Wu N, Huddleston C, Wolfson S, Millet A, Ehrenpreis R, Awal D, Tyma C, Samreen N, Gao Y, Chhor C, Gandhi S, Lee C, Kumari-Subaiya S, Leonard C, Mohammed R, Moczulski C, Altabet J, Babb J, Lewin A, Reig B, Moy L, Heacock L, Geras KJ. Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams. Nat Commun 2021; 12:5645. [PMID: 34561440 PMCID: PMC8463596 DOI: 10.1038/s41467-021-26023-2] [Citation(s) in RCA: 70] [Impact Index Per Article: 23.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Accepted: 09/14/2021] [Indexed: 02/08/2023] Open
Abstract
Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.
Collapse
Affiliation(s)
- Yiqiu Shen
- grid.137628.90000 0004 1936 8753Center for Data Science, New York University, New York, NY USA
| | - Farah E. Shamout
- grid.440573.1Engineering Division, NYU Abu Dhabi, Abu Dhabi, UAE
| | - Jamie R. Oliver
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Jan Witowski
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Kawshik Kannan
- grid.482020.c0000 0001 1089 179XDepartment of Computer Science, Courant Institute, New York University, New York, NY USA
| | - Jungkyu Park
- grid.137628.90000 0004 1936 8753Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY USA
| | - Nan Wu
- grid.137628.90000 0004 1936 8753Center for Data Science, New York University, New York, NY USA
| | - Connor Huddleston
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Stacey Wolfson
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Alexandra Millet
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Robin Ehrenpreis
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Divya Awal
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Cathy Tyma
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Naziya Samreen
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Yiming Gao
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Chloe Chhor
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Stacey Gandhi
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Cindy Lee
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Sheila Kumari-Subaiya
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Cindy Leonard
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Reyhan Mohammed
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Christopher Moczulski
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Jaime Altabet
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - James Babb
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Alana Lewin
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Beatriu Reig
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Linda Moy
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA ,grid.137628.90000 0004 1936 8753Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY USA
| | - Laura Heacock
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Krzysztof J. Geras
- grid.137628.90000 0004 1936 8753Center for Data Science, New York University, New York, NY USA ,grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA ,grid.137628.90000 0004 1936 8753Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY USA
| |
Collapse
|
47
|
Bitencourt A, Daimiel Naranjo I, Lo Gullo R, Rossi Saccarelli C, Pinker K. AI-enhanced breast imaging: Where are we and where are we heading? Eur J Radiol 2021; 142:109882. [PMID: 34392105 PMCID: PMC8387447 DOI: 10.1016/j.ejrad.2021.109882] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 07/15/2021] [Accepted: 07/26/2021] [Indexed: 12/22/2022]
Abstract
Significant advances in imaging analysis and the development of high-throughput methods that can extract and correlate multiple imaging parameters with different clinical outcomes have led to a new direction in medical research. Radiomics and artificial intelligence (AI) studies are rapidly evolving and have many potential applications in breast imaging, such as breast cancer risk prediction, lesion detection and classification, radiogenomics, and prediction of treatment response and clinical outcomes. AI has been applied to different breast imaging modalities, including mammography, ultrasound, and magnetic resonance imaging, in different clinical scenarios. The application of AI tools in breast imaging has an unprecedented opportunity to better derive clinical value from imaging data and reshape the way we care for our patients. The aim of this study is to review the current knowledge and future applications of AI-enhanced breast imaging in clinical practice.
Collapse
Affiliation(s)
- Almir Bitencourt
- Department of Imaging, A.C.Camargo Cancer Center, Sao Paulo, SP, Brazil; Dasa, Sao Paulo, SP, Brazil
| | - Isaac Daimiel Naranjo
- Department of Radiology, Breast Imaging Service, Guy's and St. Thomas' NHS Trust, Great Maze Pond, London, UK
| | - Roberto Lo Gullo
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Katja Pinker
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
| |
Collapse
|
48
|
Lei YM, Yin M, Yu MH, Yu J, Zeng SE, Lv WZ, Li J, Ye HR, Cui XW, Dietrich CF. Artificial Intelligence in Medical Imaging of the Breast. Front Oncol 2021; 11:600557. [PMID: 34367938 PMCID: PMC8339920 DOI: 10.3389/fonc.2021.600557] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2020] [Accepted: 07/07/2021] [Indexed: 12/24/2022] Open
Abstract
Artificial intelligence (AI) has invaded our daily lives, and in the last decade, there have been very promising applications of AI in the field of medicine, including medical imaging, in vitro diagnosis, intelligent rehabilitation, and prognosis. Breast cancer is one of the common malignant tumors in women and seriously threatens women’s physical and mental health. Early screening for breast cancer via mammography, ultrasound and magnetic resonance imaging (MRI) can significantly improve the prognosis of patients. AI has shown excellent performance in image recognition tasks and has been widely studied in breast cancer screening. This paper introduces the background of AI and its application in breast medical imaging (mammography, ultrasound and MRI), such as in the identification, segmentation and classification of lesions; breast density assessment; and breast cancer risk assessment. In addition, we also discuss the challenges and future perspectives of the application of AI in medical imaging of the breast.
Collapse
Affiliation(s)
- Yu-Meng Lei
- Department of Medical Ultrasound, China Resources & Wisco General Hospital, Academic Teaching Hospital of Wuhan University of Science and Technology, Wuhan, China
| | - Miao Yin
- Department of Medical Ultrasound, China Resources & Wisco General Hospital, Academic Teaching Hospital of Wuhan University of Science and Technology, Wuhan, China
| | - Mei-Hui Yu
- Department of Medical Ultrasound, China Resources & Wisco General Hospital, Academic Teaching Hospital of Wuhan University of Science and Technology, Wuhan, China
| | - Jing Yu
- Department of Medical Ultrasound, China Resources & Wisco General Hospital, Academic Teaching Hospital of Wuhan University of Science and Technology, Wuhan, China
| | - Shu-E Zeng
- Department of Medical Ultrasound, Hubei Cancer Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Wen-Zhi Lv
- Department of Artificial Intelligence, Julei Technology, Wuhan, China
| | - Jun Li
- Department of Medical Ultrasound, The First Affiliated Hospital of Medical College, Shihezi University, Xinjiang, China
| | - Hua-Rong Ye
- Department of Medical Ultrasound, China Resources & Wisco General Hospital, Academic Teaching Hospital of Wuhan University of Science and Technology, Wuhan, China
| | - Xin-Wu Cui
- Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Christoph F Dietrich
- Department Allgemeine Innere Medizin (DAIM), Kliniken Beau Site, Salem und Permanence, Bern, Switzerland
| |
Collapse
|
49
|
Badawy SM, Mohamed AENA, Hefnawy AA, Zidan HE, GadAllah MT, El-Banby GM. Classification of Breast Ultrasound Images Based on Convolutional Neural Networks - A Comparative Study. 2021 INTERNATIONAL TELECOMMUNICATIONS CONFERENCE (ITC-EGYPT) 2021. [DOI: 10.1109/itc-egypt52936.2021.9513972] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
|
50
|
Huang R, Lin Z, Dou H, Wang J, Miao J, Zhou G, Jia X, Xu W, Mei Z, Dong Y, Yang X, Zhou J, Ni D. AW3M: An auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound. Med Image Anal 2021; 72:102137. [PMID: 34216958 DOI: 10.1016/j.media.2021.102137] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Revised: 04/23/2021] [Accepted: 06/14/2021] [Indexed: 10/21/2022]
Abstract
Recently, more clinicians have realized the diagnostic value of multi-modal ultrasound in breast cancer identification and began to incorporate Doppler imaging and Elastography in the routine examination. However, accurately recognizing patterns of malignancy in different types of sonography requires expertise. Furthermore, an accurate and robust diagnosis requires proper weights of multi-modal information as well as the ability to process missing data in practice. These two aspects are often overlooked by existing computer-aided diagnosis (CAD) approaches. To overcome these challenges, we propose a novel framework (called AW3M) that utilizes four types of sonography (i.e. B-mode, Doppler, Shear-wave Elastography, and Strain Elastography) jointly to assist breast cancer diagnosis. It can extract both modality-specific and modality-invariant features using a multi-stream CNN model equipped with self-supervised consistency loss. Instead of assigning the weights of different streams empirically, AW3M automatically learns the optimal weights using reinforcement learning techniques. Furthermore, we design a light-weight recovery block that can be inserted to a trained model to handle different modality-missing scenarios. Experimental results on a large multi-modal dataset demonstrate that our method can achieve promising performance compared with state-of-the-art methods. The AW3M framework is also tested on another independent B-mode dataset to prove its efficacy in general settings. Results show that the proposed recovery block can learn from the joint distribution of multi-modal features to further boost the classification accuracy given single modality input during the test.
Collapse
Affiliation(s)
- Ruobing Huang
- Medical UltraSound Image Computing (MUSIC) Lab, School of Biomedical Engineering, Health Science Center, Shenzhen University, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Zehui Lin
- Medical UltraSound Image Computing (MUSIC) Lab, School of Biomedical Engineering, Health Science Center, Shenzhen University, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Haoran Dou
- Medical UltraSound Image Computing (MUSIC) Lab, School of Biomedical Engineering, Health Science Center, Shenzhen University, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Jian Wang
- Medical UltraSound Image Computing (MUSIC) Lab, School of Biomedical Engineering, Health Science Center, Shenzhen University, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Juzheng Miao
- School of Biological Sciences and Medical Engineering, Southeast University, China
| | - Guangquan Zhou
- School of Biological Sciences and Medical Engineering, Southeast University, China
| | - Xiaohong Jia
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, China
| | - Wenwen Xu
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, China
| | - Zihan Mei
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, China
| | - Yijie Dong
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, China
| | - Xin Yang
- Medical UltraSound Image Computing (MUSIC) Lab, School of Biomedical Engineering, Health Science Center, Shenzhen University, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China
| | - Jianqiao Zhou
- Department of Ultrasound Medicine, Ruijin Hospital, School of Medicine, Shanghai Jiaotong University, China.
| | - Dong Ni
- Medical UltraSound Image Computing (MUSIC) Lab, School of Biomedical Engineering, Health Science Center, Shenzhen University, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, China.
| |
Collapse
|