1
|
Pan H, Shi C, Zhang Y, Zhong Z. Artificial intelligence-based classification of breast nodules: a quantitative morphological analysis of ultrasound images. Quant Imaging Med Surg 2024; 14:3381-3392. [PMID: 38720871 PMCID: PMC11074741 DOI: 10.21037/qims-23-1652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Accepted: 03/25/2024] [Indexed: 05/12/2024]
Abstract
Background Accurate classification of breast nodules into benign and malignant types is critical for the successful treatment of breast cancer. Traditional methods rely on subjective interpretation, which can potentially lead to diagnostic errors. Artificial intelligence (AI)-based methods using the quantitative morphological analysis of ultrasound images have been explored for the automated and reliable classification of breast cancer. This study aimed to investigate the effectiveness of AI-based approaches for improving diagnostic accuracy and patient outcomes. Methods In this study, a quantitative analysis approach was adopted, with a focus on five critical features for evaluation: degree of boundary regularity, clarity of boundaries, echo intensity, and uniformity of echoes. Furthermore, the classification results were assessed using five machine learning methods: logistic regression (LR), support vector machine (SVM), decision tree (DT), naive Bayes, and K-nearest neighbor (KNN). Based on these assessments, a multifeature combined prediction model was established. Results We evaluated the performance of our classification model by quantifying various features of the ultrasound images and using the area under the receiver operating characteristic (ROC) curve (AUC). The moment of inertia achieved an AUC value of 0.793, while the variance and mean of breast nodule areas achieved AUC values of 0.725 and 0.772, respectively. The convexity and concavity achieved AUC values of 0.988 and 0.987, respectively. Additionally, we conducted a joint analysis of multiple features after normalization, achieving a recall value of 0.98, which surpasses most medical evaluation indexes on the market. To ensure experimental rigor, we conducted cross-validation experiments, which yielded no significant differences among the classifiers under 5-, 8-, and 10-fold cross-validation (P>0.05). Conclusions The quantitative analysis can accurately differentiate between benign and malignant breast nodules.
Collapse
Affiliation(s)
- Hao Pan
- School of Electronic Information, Xijing University, Xi’an, China
| | - Changbei Shi
- Department of Nuclear Medicine, Shaanxi Provincial Cancer Hospital, Xi’an, China
| | - Yuxing Zhang
- School of Electronic Information, Xijing University, Xi’an, China
- School of Medicine, Xijing University, Xi’an, China
| | - Zijian Zhong
- School of Electronic Information, Xijing University, Xi’an, China
| |
Collapse
|
2
|
Gómez-Flores W, Gregorio-Calas MJ, Coelho de Albuquerque Pereira W. BUS-BRA: A breast ultrasound dataset for assessing computer-aided diagnosis systems. Med Phys 2024; 51:3110-3123. [PMID: 37937827 DOI: 10.1002/mp.16812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 10/10/2023] [Accepted: 10/12/2023] [Indexed: 11/09/2023] Open
Abstract
PURPOSE Computer-aided diagnosis (CAD) systems on breast ultrasound (BUS) aim to increase the efficiency and effectiveness of breast screening, helping specialists to detect and classify breast lesions. CAD system development requires a set of annotated images, including lesion segmentation, biopsy results to specify benign and malignant cases, and BI-RADS categories to indicate the likelihood of malignancy. Besides, standardized partitions of training, validation, and test sets promote reproducibility and fair comparisons between different approaches. Thus, we present a publicly available BUS dataset whose novelty is the substantial increment of cases with the above-mentioned annotations and the inclusion of standardized partitions to objectively assess and compare CAD systems. ACQUISITION AND VALIDATION METHODS The BUS dataset comprises 1875 anonymized images from 1064 female patients acquired via four ultrasound scanners during systematic studies at the National Institute of Cancer (Rio de Janeiro, Brazil). The dataset includes biopsy-proven tumors divided into 722 benign and 342 malignant cases. Besides, a senior ultrasonographer performed a BI-RADS assessment in categories 2 to 5. Additionally, the ultrasonographer manually outlined the breast lesions to obtain ground truth segmentations. Furthermore, 5- and 10-fold cross-validation partitions are provided to standardize the training and test sets to evaluate and reproduce CAD systems. Finally, to validate the utility of the BUS dataset, an evaluation framework is implemented to assess the performance of deep neural networks for segmenting and classifying breast lesions. DATA FORMAT AND USAGE NOTES The BUS dataset is publicly available for academic and research purposes through an open-access repository under the name BUS-BRA: A Breast Ultrasound Dataset for Assessing CAD Systems. BUS images and reference segmentations are saved in Portable Network Graphic (PNG) format files, and the dataset information is stored in separate Comma-Separated Value (CSV) files. POTENTIAL APPLICATIONS The BUS-BRA dataset can be used to develop and assess artificial intelligence-based lesion detection and segmentation methods, and the classification of BUS images into pathological classes and BI-RADS categories. Other potential applications include developing image processing methods like despeckle filtering and contrast enhancement methods to improve image quality and feature engineering for image description.
Collapse
Affiliation(s)
- Wilfrido Gómez-Flores
- Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, Tamaulipas, Mexico
| | | | | |
Collapse
|
3
|
Lee SE, Hong H, Kim EK. Positive Predictive Values of Abnormality Scores From a Commercial Artificial Intelligence-Based Computer-Aided Diagnosis for Mammography. Korean J Radiol 2024; 25:343-350. [PMID: 38528692 PMCID: PMC10973732 DOI: 10.3348/kjr.2023.0907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 11/17/2023] [Accepted: 12/05/2023] [Indexed: 03/27/2024] Open
Abstract
OBJECTIVE Artificial intelligence-based computer-aided diagnosis (AI-CAD) is increasingly used in mammography. While the continuous scores of AI-CAD have been related to malignancy risk, the understanding of how to interpret and apply these scores remains limited. We investigated the positive predictive values (PPVs) of the abnormality scores generated by a deep learning-based commercial AI-CAD system and analyzed them in relation to clinical and radiological findings. MATERIALS AND METHODS From March 2020 to May 2022, 656 breasts from 599 women (mean age 52.6 ± 11.5 years, including 0.6% [4/599] high-risk women) who underwent mammography and received positive AI-CAD results (Lunit Insight MMG, abnormality score ≥ 10) were retrospectively included in this study. Univariable and multivariable analyses were performed to evaluate the associations between the AI-CAD abnormality scores and clinical and radiological factors. The breasts were subdivided according to the abnormality scores into groups 1 (10-49), 2 (50-69), 3 (70-89), and 4 (90-100) using the optimal binning method. The PPVs were calculated for all breasts and subgroups. RESULTS Diagnostic indications and positive imaging findings by radiologists were associated with higher abnormality scores in the multivariable regression analysis. The overall PPV of AI-CAD was 32.5% (213/656) for all breasts, including 213 breast cancers, 129 breasts with benign biopsy results, and 314 breasts with benign outcomes in the follow-up or diagnostic studies. In the screening mammography subgroup, the PPVs were 18.6% (58/312) overall and 5.1% (12/235), 29.0% (9/31), 57.9% (11/19), and 96.3% (26/27) for score groups 1, 2, 3, and 4, respectively. The PPVs were significantly higher in women with diagnostic indications (45.1% [155/344]), palpability (51.9% [149/287]), fatty breasts (61.2% [60/98]), and certain imaging findings (masses with or without calcifications and distortion). CONCLUSION PPV increased with increasing AI-CAD abnormality scores. The PPVs of AI-CAD satisfied the acceptable PPV range according to Breast Imaging-Reporting and Data System for screening mammography and were higher for diagnostic mammography.
Collapse
Affiliation(s)
- Si Eun Lee
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea
| | - Hanpyo Hong
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea
| | - Eun-Kyung Kim
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea.
| |
Collapse
|
4
|
Chafai N, Bonizzi L, Botti S, Badaoui B. Emerging applications of machine learning in genomic medicine and healthcare. Crit Rev Clin Lab Sci 2024; 61:140-163. [PMID: 37815417 DOI: 10.1080/10408363.2023.2259466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 09/12/2023] [Indexed: 10/11/2023]
Abstract
The integration of artificial intelligence technologies has propelled the progress of clinical and genomic medicine in recent years. The significant increase in computing power has facilitated the ability of artificial intelligence models to analyze and extract features from extensive medical data and images, thereby contributing to the advancement of intelligent diagnostic tools. Artificial intelligence (AI) models have been utilized in the field of personalized medicine to integrate clinical data and genomic information of patients. This integration allows for the identification of customized treatment recommendations, ultimately leading to enhanced patient outcomes. Notwithstanding the notable advancements, the application of artificial intelligence (AI) in the field of medicine is impeded by various obstacles such as the limited availability of clinical and genomic data, the diversity of datasets, ethical implications, and the inconclusive interpretation of AI models' results. In this review, a comprehensive evaluation of multiple machine learning algorithms utilized in the fields of clinical and genomic medicine is conducted. Furthermore, we present an overview of the implementation of artificial intelligence (AI) in the fields of clinical medicine, drug discovery, and genomic medicine. Finally, a number of constraints pertaining to the implementation of artificial intelligence within the healthcare industry are examined.
Collapse
Affiliation(s)
- Narjice Chafai
- Laboratory of Biodiversity, Ecology, and Genome, Faculty of Sciences, Department of Biology, Mohammed V University in Rabat, Rabat, Morocco
| | - Luigi Bonizzi
- Department of Biomedical, Surgical and Dental Science, University of Milan, Milan, Italy
| | - Sara Botti
- PTP Science Park, Via Einstein - Loc. Cascina Codazza, Lodi, Italy
| | - Bouabid Badaoui
- Laboratory of Biodiversity, Ecology, and Genome, Faculty of Sciences, Department of Biology, Mohammed V University in Rabat, Rabat, Morocco
- African Sustainable Agriculture Research Institute (ASARI), Mohammed VI Polytechnic University (UM6P), Laâyoune, Morocco
| |
Collapse
|
5
|
Chhillar I, Singh A. A feature engineering-based machine learning technique to detect and classify lung and colon cancer from histopathological images. Med Biol Eng Comput 2024; 62:913-924. [PMID: 38091162 DOI: 10.1007/s11517-023-02984-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 11/29/2023] [Indexed: 02/22/2024]
Abstract
Globally, lung and colon cancers are among the most prevalent and lethal tumors. Early cancer identification is essential to increase the likelihood of survival. Histopathological images are considered an appropriate tool for diagnosing cancer, which is tedious and error-prone if done manually. Recently, machine learning methods based on feature engineering have gained prominence in automatic histopathological image classification. Furthermore, these methods are more interpretable than deep learning, which operates in a "black box" manner. In the medical profession, the interpretability of a technique is critical to gaining the trust of end users to adopt it. In view of the above, this work aims to create an accurate and interpretable machine-learning technique for the automated classification of lung and colon cancers from histopathology images. In the proposed approach, following the preprocessing steps, texture and color features are retrieved by utilizing the Haralick and Color histogram feature extraction algorithms, respectively. The obtained features are concatenated to form a single feature set. The three feature sets (texture, color, and combined features) are passed into the Light Gradient Boosting Machine (LightGBM) classifier for classification. And their performance is evaluated on the LC25000 dataset using hold-out and stratified 10-fold cross-validation (Stratified 10-FCV) techniques. With a test/hold-out set, the LightGBM with texture, color, and combined features classifies the lung and colon cancer images with 97.72%, 99.92%, and 100% accuracy respectively. In addition, a stratified 10-fold cross-validation method also revealed that LightGBM's combined or color features performed well, with an excellent mean auc_mu score and a low mean multi_logloss value. Thus, this proposed technique can help histologists detect and classify lung and colon histopathology images more efficiently, effectively, and economically, resulting in more productivity.
Collapse
Affiliation(s)
- Indu Chhillar
- Department of Computer Science and Engineering, Deenbandhu Chhotu Ram University of Science and Technology, Murthal, Haryana, India.
| | - Ajmer Singh
- Department of Computer Science and Engineering, Deenbandhu Chhotu Ram University of Science and Technology, Murthal, Haryana, India
| |
Collapse
|
6
|
Lokaj B, Pugliese MT, Kinkel K, Lovis C, Schmid J. Barriers and facilitators of artificial intelligence conception and implementation for breast imaging diagnosis in clinical practice: a scoping review. Eur Radiol 2024; 34:2096-2109. [PMID: 37658895 PMCID: PMC10873444 DOI: 10.1007/s00330-023-10181-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 06/07/2023] [Accepted: 07/10/2023] [Indexed: 09/05/2023]
Abstract
OBJECTIVE Although artificial intelligence (AI) has demonstrated promise in enhancing breast cancer diagnosis, the implementation of AI algorithms in clinical practice encounters various barriers. This scoping review aims to identify these barriers and facilitators to highlight key considerations for developing and implementing AI solutions in breast cancer imaging. METHOD A literature search was conducted from 2012 to 2022 in six databases (PubMed, Web of Science, CINHAL, Embase, IEEE, and ArXiv). The articles were included if some barriers and/or facilitators in the conception or implementation of AI in breast clinical imaging were described. We excluded research only focusing on performance, or with data not acquired in a clinical radiology setup and not involving real patients. RESULTS A total of 107 articles were included. We identified six major barriers related to data (B1), black box and trust (B2), algorithms and conception (B3), evaluation and validation (B4), legal, ethical, and economic issues (B5), and education (B6), and five major facilitators covering data (F1), clinical impact (F2), algorithms and conception (F3), evaluation and validation (F4), and education (F5). CONCLUSION This scoping review highlighted the need to carefully design, deploy, and evaluate AI solutions in clinical practice, involving all stakeholders to yield improvement in healthcare. CLINICAL RELEVANCE STATEMENT The identification of barriers and facilitators with suggested solutions can guide and inform future research, and stakeholders to improve the design and implementation of AI for breast cancer detection in clinical practice. KEY POINTS • Six major identified barriers were related to data; black-box and trust; algorithms and conception; evaluation and validation; legal, ethical, and economic issues; and education. • Five major identified facilitators were related to data, clinical impact, algorithms and conception, evaluation and validation, and education. • Coordinated implication of all stakeholders is required to improve breast cancer diagnosis with AI.
Collapse
Affiliation(s)
- Belinda Lokaj
- Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Delémont, Switzerland.
- Faculty of Medicine, University of Geneva, Geneva, Switzerland.
- Division of Medical Information Sciences, Geneva University Hospitals, Geneva, Switzerland.
| | - Marie-Thérèse Pugliese
- Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Delémont, Switzerland
| | - Karen Kinkel
- Réseau Hospitalier Neuchâtelois, Neuchâtel, Switzerland
| | - Christian Lovis
- Faculty of Medicine, University of Geneva, Geneva, Switzerland
- Division of Medical Information Sciences, Geneva University Hospitals, Geneva, Switzerland
| | - Jérôme Schmid
- Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Delémont, Switzerland
| |
Collapse
|
7
|
Shen X, He Z, Shi Y, Liu T, Yang Y, Luo J, Tang X, Chen B, Xu S, Zhou Y, Xiao J, Qin Y. Development and Validation of an Automated Classification System for Osteonecrosis of the Femoral Head Using Deep Learning Approach: A Multicenter Study. J Arthroplasty 2024; 39:379-386.e2. [PMID: 37572719 DOI: 10.1016/j.arth.2023.08.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 08/01/2023] [Accepted: 08/03/2023] [Indexed: 08/14/2023] Open
Abstract
BACKGROUND Accurate classification can facilitate the selection of appropriate interventions to delay the progression of osteonecrosis of the femoral head (ONFH). This study aimed to perform the classification of ONFH through a deep learning approach. METHODS We retrospectively sampled 1,806 midcoronal magnetic resonance images (MRIs) of 1,337 hips from 4 institutions. Of these, 1,472 midcoronal MRIs of 1,155 hips were divided into training, validation, and test datasets with a ratio of 7:1:2 to develop a convolutional neural network model (CNN). An additional 334 midcoronal MRIs of 182 hips were used to perform external validation. The predictive performance of the CNN and the review panel was also compared. RESULTS A multiclass CNN model was successfully developed. In internal validation, the overall accuracy of the CNN for predicting the severity of ONFH based on the Japanese Investigation Committee classification was 87.8%. The macroaverage values of area under the curve (AUC), precision, recall, and F-value were 0.90, 84.8, 84.8, and 84.6%, respectively. In external validation, the overall accuracy of the CNN was 83.8%. The macroaverage values of area under the curve, precision, recall, and F-value were 0.87, 79.5, 80.5, and 79.9%, respectively. In a human-machine comparison study, the CNN outperformed or was comparable to that of the deputy chief orthopaedic surgeons. CONCLUSION The CNN is feasible and robust for classifying ONFH and correctly locating the necrotic area. These findings suggest that classifying ONFH using deep learning with high accuracy and generalizability may aid in predicting femoral head collapse and clinical decision-making.
Collapse
Affiliation(s)
- Xianyue Shen
- Department of Orthopedics, The Second Hospital of Jilin University, Changchun, Jilin province, PR China
| | - Ziling He
- College of Computer Science and Technology, Jilin University, Changchun, Jilin province, PR China
| | - Yi Shi
- Department of Orthopedics, The Second Hospital of Anhui Medical University, Hefei, Anhui province, PR China
| | - Tong Liu
- Department of Orthopedics, China-Japan Union Hospital of Jilin University, Changchun, Jilin province, PR China
| | - Yuhui Yang
- Department of Orthopedics, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, Guangdong province, PR China
| | - Jia Luo
- College of Computer Science and Technology, Jilin University, Changchun, Jilin province, PR China
| | - Xiongfeng Tang
- Department of Orthopedics, The Second Hospital of Jilin University, Changchun, Jilin province, PR China
| | - Bo Chen
- Department of Orthopedics, The Second Hospital of Jilin University, Changchun, Jilin province, PR China
| | - Shenghao Xu
- Department of Orthopedics, China-Japan Union Hospital of Jilin University, Changchun, Jilin province, PR China
| | - You Zhou
- College of Software, Jilin University, Changchun, Jilin province, PR China
| | - Jianlin Xiao
- Department of Orthopedics, China-Japan Union Hospital of Jilin University, Changchun, Jilin province, PR China
| | - Yanguo Qin
- Department of Orthopedics, The Second Hospital of Jilin University, Changchun, Jilin province, PR China
| |
Collapse
|
8
|
Wu D, Ni J, Fan W, Jiang Q, Wang L, Sun L, Cai Z. Opportunities and challenges of computer aided diagnosis in new millennium: A bibliometric analysis from 2000 to 2023. Medicine (Baltimore) 2023; 102:e36703. [PMID: 38134105 PMCID: PMC10735127 DOI: 10.1097/md.0000000000036703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 11/27/2023] [Indexed: 12/24/2023] Open
Abstract
BACKGROUND After entering the new millennium, computer-aided diagnosis (CAD) is rapidly developing as an emerging technology worldwide. Expanding the spectrum of CAD-related diseases is a possible future research trend. Nevertheless, bibliometric studies in this area have not yet been reported. This study aimed to explore the hotspots and frontiers of research on CAD from 2000 to 2023, which may provide a reference for researchers in this field. METHODS In this paper, we use bibliometrics to analyze CAD-related literature in the Web of Science database between 2000 and 2023. The scientometric softwares VOSviewer and CiteSpace were used to visually analyze the countries, institutions, authors, journals, references and keywords involved in the literature. Keywords burst analysis were utilized to further explore the current state and development trends of research on CAD. RESULTS A total of 13,970 publications were included in this study, with a noticeably rising annual publication trend. China and the United States are major contributors to the publication, with the United States being the dominant position in CAD research. The American research institutions, lead by the University of Chicago, are pioneers of CAD. Acharya UR, Zheng B and Chan HP are the most prolific authors. Institute of Electrical and Electronics Engineers Transactions on Medical Imaging focuses on CAD and publishes the most articles. New computer technologies related to CAD are in the forefront of attention. Currently, CAD is used extensively in breast diseases, pulmonary diseases and brain diseases. CONCLUSION Expanding the spectrum of CAD-related diseases is a possible future research trend. How to overcome the lack of large sample datasets and establish a universally accepted standard for the evaluation of CAD system performance are urgent issues for CAD development and validation. In conclusion, this paper provides valuable information on the current state of CAD research and future developments.
Collapse
Affiliation(s)
- Di Wu
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
- Department of Proctology, Bishan Hospital of Traditional Chinese Medicine, Chongqing, China
- Chongqing College of Traditional Chinese Medicine, Chongqing, China
| | - Jiachun Ni
- Department of Coloproctology, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Wenbin Fan
- Department of Proctology, Bishan Hospital of Traditional Chinese Medicine, Chongqing, China
- Chongqing College of Traditional Chinese Medicine, Chongqing, China
| | - Qiong Jiang
- Chongqing College of Traditional Chinese Medicine, Chongqing, China
| | - Ling Wang
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Li Sun
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Zengjin Cai
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| |
Collapse
|
9
|
Gómez-Flores W, Pereira WCDA. Gray-to-color image conversion in the classification of breast lesions on ultrasound using pre-trained deep neural networks. Med Biol Eng Comput 2023; 61:3193-3207. [PMID: 37713158 DOI: 10.1007/s11517-023-02928-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 08/29/2023] [Indexed: 09/16/2023]
Abstract
Breast ultrasound (BUS) image classification in benign and malignant classes is often based on pre-trained convolutional neural networks (CNNs) to cope with small-sized training data. Nevertheless, BUS images are single-channel gray-level images, whereas pre-trained CNNs learned from color images with red, green, and blue (RGB) components. Thus, a gray-to-color conversion method is applied to fit the BUS image to the CNN's input layer size. This paper evaluates 13 gray-to-color conversion methods proposed in the literature that follow three strategies: replicating the gray-level image to all RGB channels, decomposing the image to enhance inherent information like the lesion's texture and morphology, and learning a matching layer. Besides, we introduce an image decomposition method based on the lesion's structural information to describe its inner and outer complexity. These gray-to-color conversion methods are evaluated under the same experimental framework using a pre-trained CNN architecture named ResNet-18 and a BUS dataset with more than 3000 images. In addition, the Matthews correlation coefficient (MCC), sensitivity (SEN), and specificity (SPE) measure the classification performance. The experimental results show that decomposition methods outperform replication and learning-based methods when using information from the lesion's binary mask (obtained from a segmentation method), reaching an MCC value greater than 0.70 and specificity up to 0.92, although the sensitivity is about 0.80. On the other hand, regarding the proposed method, the trade-off between sensitivity and specificity is better balanced, obtaining about 0.88 for both indices and an MCC of 0.73. This study contributes to the objective assessment of different gray-to-color conversion approaches in classifying breast lesions, revealing that mask-based decomposition methods improve classification performance. Besides, the proposed method based on structural information improves the sensitivity, obtaining more reliable classification results on malignant cases and potentially benefiting clinical practice.
Collapse
Affiliation(s)
- Wilfrido Gómez-Flores
- Centro de Investigación y de Estudios Avanzados del IPN, Unidad Tamaulipas, Ciudad Victoria, 87138, Tamaulipas, Mexico.
| | | |
Collapse
|
10
|
Hossain S, Azam S, Montaha S, Karim A, Chowa SS, Mondol C, Zahid Hasan M, Jonkman M. Automated breast tumor ultrasound image segmentation with hybrid UNet and classification using fine-tuned CNN model. Heliyon 2023; 9:e21369. [PMID: 37885728 PMCID: PMC10598544 DOI: 10.1016/j.heliyon.2023.e21369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 10/11/2023] [Accepted: 10/20/2023] [Indexed: 10/28/2023] Open
Abstract
Introduction Breast cancer stands as the second most deadly form of cancer among women worldwide. Early diagnosis and treatment can significantly mitigate mortality rates. Purpose The study aims to classify breast ultrasound images into benign and malignant tumors. This approach involves segmenting the breast's region of interest (ROI) employing an optimized UNet architecture and classifying the ROIs through an optimized shallow CNN model utilizing an ablation study. Method Several image processing techniques are utilized to improve image quality by removing text, artifacts, and speckle noise, and statistical analysis is done to check the enhanced image quality is satisfactory. With the processed dataset, the segmentation of breast tumor ROI is carried out, optimizing the UNet model through an ablation study where the architectural configuration and hyperparameters are altered. After obtaining the tumor ROIs from the fine-tuned UNet model (RKO-UNet), an optimized CNN model is employed to classify the tumor into benign and malignant classes. To enhance the CNN model's performance, an ablation study is conducted, coupled with the integration of an attention unit. The model's performance is further assessed by classifying breast cancer with mammogram images. Result The proposed classification model (RKONet-13) results in an accuracy of 98.41 %. The performance of the proposed model is further compared with five transfer learning models for both pre-segmented and post-segmented datasets. K-fold cross-validation is done to assess the proposed RKONet-13 model's performance stability. Furthermore, the performance of the proposed model is compared with previous literature, where the proposed model outperforms existing methods, demonstrating its effectiveness in breast cancer diagnosis. Lastly, the model demonstrates its robustness for breast cancer classification, delivering an exceptional performance of 96.21 % on a mammogram dataset. Conclusion The efficacy of this study relies on image pre-processing, segmentation with hybrid attention UNet, and classification with fine-tuned robust CNN model. This comprehensive approach aims to determine an effective technique for detecting breast cancer within ultrasound images.
Collapse
Affiliation(s)
- Shahed Hossain
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka, 1341, Bangladesh
| | - Sami Azam
- Faculty of Science and Technology, Charles Darwin University, Casuarina, 0909, NT, Australia
| | - Sidratul Montaha
- Department of Computer Science, University of Calgary, Calgary, AB, T2N 1N4, Canada
| | - Asif Karim
- Faculty of Science and Technology, Charles Darwin University, Casuarina, 0909, NT, Australia
| | - Sadia Sultana Chowa
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka, 1341, Bangladesh
| | - Chaity Mondol
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka, 1341, Bangladesh
| | - Md Zahid Hasan
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka, 1341, Bangladesh
| | - Mirjam Jonkman
- Faculty of Science and Technology, Charles Darwin University, Casuarina, 0909, NT, Australia
| |
Collapse
|
11
|
Wang L, Ni J, Hou C, Wu D, Sun L, Jiang Q, Cai Z, Fan W. Time to change? Present and prospects of hemorrhoidal classification. Front Med (Lausanne) 2023; 10:1252468. [PMID: 37901411 PMCID: PMC10603225 DOI: 10.3389/fmed.2023.1252468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 09/01/2023] [Indexed: 10/31/2023] Open
Abstract
As a common benign anal condition, the high incidence and recurrence of hemorrhoids pose challenges for both patients and doctors. The classification of hemorrhoids plays a crucial role in assessing, diagnosing, and treating the condition. By using appropriate classification and corresponding treatment strategies, we can achieve higher cure rates and lower recurrence rates of hemorrhoids. Since the introduction of the Miles classification in 1919, various classifications have been developed, which include objective classifications based on anatomical or instrumental assessment and subjective classifications based on symptoms and patient sensations. These classifications aim to accurately evaluate the condition. In this study, we discuss the evaluation values of each classification in terms of their advantages, disadvantages, treatment relevance, reproducibility, practicality, and assessment value. We also analyze the significant and essential factors, principles of use, and components of assessment indicators of hemorrhoidal classification. This study proposes several strategies to address the limitations of current hemorrhoidal assessment methods. All these will provide a reference for the development regarding the assessment and classification of hemorrhoids and clinical diagnosis and management of hemorrhoids.
Collapse
Affiliation(s)
- Ling Wang
- Chongqing College of Traditional Chinese Medicine, Chongqing, China
- Chongqing Medical University, Chongqing, China
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Jiachun Ni
- Department of Coloproctology, Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Changcheng Hou
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Di Wu
- Chongqing Medical University, Chongqing, China
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Li Sun
- Chongqing Medical University, Chongqing, China
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Qiong Jiang
- Chongqing Medical University, Chongqing, China
| | - Zengjin Cai
- Department of Proctology, Yongchuan Hospital of Traditional Chinese Medicine, Chongqing Medical University, Chongqing, China
| | - Wenbin Fan
- Chongqing College of Traditional Chinese Medicine, Chongqing, China
- Chongqing Medical University, Chongqing, China
| |
Collapse
|
12
|
Youk JH, Han K, Lee SE, Kim EK. Consistency of Artificial Intelligence (AI)-based Diagnostic Support Software in Short-term Digital Mammography Reimaging After Core Needle Biopsy. J Digit Imaging 2023; 36:1965-1973. [PMID: 37326891 PMCID: PMC10501993 DOI: 10.1007/s10278-023-00863-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 05/08/2023] [Accepted: 05/26/2023] [Indexed: 06/17/2023] Open
Abstract
To evaluate the consistency in the performance of Artificial Intelligence (AI)-based diagnostic support software in short-term digital mammography reimaging after core needle biopsy. Of 276 women who underwent short-term (<3 mo) serial digital mammograms followed by breast cancer surgery from Jan. to Dec. 2017, 550 breasts were included. All core needle biopsies for breast lesions were performed between serial exams. All mammography images were analyzed using a commercially available AI-based software providing an abnormality score (0-100). Demographic data for age, interval between serial exams, biopsy, and final diagnosis were compiled. Mammograms were reviewed for mammographic density and finding. Statistical analysis was performed to evaluate the distribution of variables according to biopsy and to test the interaction effects of variables with the difference in AI-based score according to biopsy. AI-based score of 550 exams (benign or normal in 263 and malignant in 287) showed significant difference between malignant and benign/normal exams (0.48 vs. 91.97 in first exam and 0.62 vs. 87.13 in second exam, P<0.0001). In comparison of serial exams, no significant difference was found in AI-based score. AI-based score difference between serial exams was significantly different according to biopsy performed or not (-0.25 vs. 0.07, P = 0.035). In linear regression analysis, there was no significant interaction effect of all clinical and mammographic characteristics with mammographic examinations performed after biopsy or not. The results from AI-based diagnostic support software for digital mammography was relatively consistent in short-term reimaging even after core needle biopsy.
Collapse
Affiliation(s)
- Ji Hyun Youk
- Department of Radiology, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Kyunghwa Han
- Department of Radiology, Research Institute of Radiological Science, and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Si Eun Lee
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Gyeonggi‑do, Republic of Korea
| | - Eun-Kyung Kim
- Department of Radiology, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Gyeonggi‑do, Republic of Korea.
| |
Collapse
|
13
|
Balaji K. Image Augmentation based on Variational Autoencoder for Breast Tumor Segmentation. Acad Radiol 2023; 30 Suppl 2:S172-S183. [PMID: 36804294 DOI: 10.1016/j.acra.2022.12.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 12/18/2022] [Accepted: 12/21/2022] [Indexed: 02/18/2023]
Abstract
RATIONALE AND OBJECTIVES Breast tumor segmentation based on Dynamic Contrast-Enhanced Magnetic Resonance Imaging is significant step for computable radiomics analysis of breast cancer. Manual tumor annotation is time-consuming process and involves medical acquaintance, biased, inclined to error, and inter-user discrepancy. A number of modern trainings have revealed the capability of deep learning representations in image segmentation. MATERIALS AND METHODS Here, we describe a 3D Connected-UNets for tumor segmentation from 3D Magnetic Resonance Imagings based on encoder-decoder architecture. Due to a restricted training dataset size, a variational auto-encoder outlet is supplementary to renovate the input image itself in order to identify the shared decoder and execute additional controls on its layers. Based on initial segmentation of Connected-UNets, fully connected 3D provisional unsystematic domain is used to enhance segmentation outcomes by discovering 2D neighbor areas and 3D volume statistics. Moreover, 3D connected modules evaluation is used to endure around large modules and decrease segmentation noise. RESULTS The proposed method has been assessed on two widely offered datasets, explicitly INbreast and the curated breast imaging subset of digital database for screening mammography The proposed model has also been estimated using a private dataset. CONCLUSION The experimental results show that the proposed model outperforms the state-of-the-art methods for breast tumor segmentation.
Collapse
Affiliation(s)
- K Balaji
- School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamilnadu, 632014 India.
| |
Collapse
|
14
|
Zakareya S, Izadkhah H, Karimpour J. A New Deep-Learning-Based Model for Breast Cancer Diagnosis from Medical Images. Diagnostics (Basel) 2023; 13:diagnostics13111944. [PMID: 37296796 DOI: 10.3390/diagnostics13111944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 05/15/2023] [Accepted: 05/28/2023] [Indexed: 06/12/2023] Open
Abstract
Breast cancer is one of the most prevalent cancers among women worldwide, and early detection of the disease can be lifesaving. Detecting breast cancer early allows for treatment to begin faster, increasing the chances of a successful outcome. Machine learning helps in the early detection of breast cancer even in places where there is no access to a specialist doctor. The rapid advancement of machine learning, and particularly deep learning, leads to an increase in the medical imaging community's interest in applying these techniques to improve the accuracy of cancer screening. Most of the data related to diseases is scarce. On the other hand, deep-learning models need much data to learn well. For this reason, the existing deep-learning models on medical images cannot work as well as other images. To overcome this limitation and improve breast cancer classification detection, inspired by two state-of-the-art deep networks, GoogLeNet and residual block, and developing several new features, this paper proposes a new deep model to classify breast cancer. Utilizing adopted granular computing, shortcut connection, two learnable activation functions instead of traditional activation functions, and an attention mechanism is expected to improve the accuracy of diagnosis and consequently decrease the load on doctors. Granular computing can improve diagnosis accuracy by capturing more detailed and fine-grained information about cancer images. The proposed model's superiority is demonstrated by comparing it to several state-of-the-art deep models and existing works using two case studies. The proposed model achieved an accuracy of 93% and 95% on ultrasound images and breast histopathology images, respectively.
Collapse
Affiliation(s)
- Salman Zakareya
- Department of Computer Science, University of Tabriz, Tabriz 5166616471, Iran
| | - Habib Izadkhah
- Department of Computer Science, University of Tabriz, Tabriz 5166616471, Iran
- Research Department of Computational Algorithms and Mathematical Models, University of Tabriz, Tabriz 5166616471, Iran
| | - Jaber Karimpour
- Department of Computer Science, University of Tabriz, Tabriz 5166616471, Iran
| |
Collapse
|
15
|
Isosalo A, Inkinen SI, Turunen T, Ipatti PS, Reponen J, Nieminen MT. Independent evaluation of a multi-view multi-task convolutional neural network breast cancer classification model using Finnish mammography screening data. Comput Biol Med 2023; 161:107023. [PMID: 37230016 DOI: 10.1016/j.compbiomed.2023.107023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Revised: 04/30/2023] [Accepted: 05/09/2023] [Indexed: 05/27/2023]
Abstract
BACKGROUND Development of deep convolutional neural networks for breast cancer classification has taken significant steps towards clinical adoption. It is though unclear how the models perform for unseen data, and what is required to adapt them to different demographic populations. In this retrospective study, we adopt an openly available pre-trained mammography breast cancer multi-view classification model and evaluate it by utilizing an independent Finnish dataset. METHODS Transfer learning was used, and the pre-trained model was finetuned with 8,829 examinations from the Finnish dataset (4,321 normal, 362 malignant and 4,146 benign examinations). Holdout dataset with 2,208 examinations from the Finnish dataset (1,082 normal, 70 malignant and 1,056 benign examinations) was used in the evaluation. The performance was also evaluated on a manually annotated malignant suspect subset. Receiver Operating Characteristic (ROC) and Precision-Recall curves were used to performance measures. RESULTS The Area Under ROC [95%CI] values for malignancy classification obtained with the finetuned model for the entire holdout set were 0.82 [0.76, 0.87], 0.84 [0.77, 0.89], 0.85 [0.79, 0.90], and 0.83 [0.76, 0.89] for R-MLO, L-MLO, R-CC and L-CC views respectively. Performance on the malignant suspect subset was slightly better. On the auxiliary benign classification task performance remained low. CONCLUSIONS The results indicate that the model performs well also in an out-of-distribution setting. Finetuning allowed the model to adapt to some of the underlying local demographics. Future research should concentrate to identify breast cancer subgroups adversely affecting performance, as it is a requirement for increasing the model's readiness level for a clinical setting.
Collapse
Affiliation(s)
- A Isosalo
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland.
| | - S I Inkinen
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland; HUS Diagnostic Center, Clinical Physiology and Nuclear Medicine, Helsinki University and Helsinki University Hospital, Helsinki, Finland
| | - T Turunen
- Department of Diagnostic Radiology, Oulu University Hospital, Oulu, Finland
| | - P S Ipatti
- Department of Diagnostic Radiology, Oulu University Hospital, Oulu, Finland
| | - J Reponen
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland; Medical Research Centre Oulu, Oulu University Hospital and University of Oulu, Oulu, Finland
| | - M T Nieminen
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland; Department of Diagnostic Radiology, Oulu University Hospital, Oulu, Finland; Medical Research Centre Oulu, Oulu University Hospital and University of Oulu, Oulu, Finland
| |
Collapse
|
16
|
Trepanier C, Huang A, Liu M, Ha R. Emerging uses of artificial intelligence in breast and axillary ultrasound. Clin Imaging 2023; 100:64-68. [PMID: 37243994 DOI: 10.1016/j.clinimag.2023.05.007] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 05/02/2023] [Indexed: 05/29/2023]
Abstract
Breast ultrasound is a valuable adjunctive tool to mammography in detecting breast cancer, especially in women with dense breasts. Ultrasound also plays an important role in staging breast cancer by assessing axillary lymph nodes. However, its utility is limited by operator dependence, high recall rate, low positive predictive value and low specificity. These limitations present an opportunity for artificial intelligence (AI) to improve diagnostic performance and pioneer novel uses of ultrasound. Research in developing AI for radiology has flourished over the past few years. A subset of AI, deep learning, uses interconnected computational nodes to form a neural network, which extracts complex visual features from image data to train itself into a predictive model. This review summarizes several key studies evaluating AI programs' performance in predicting breast cancer and demonstrates that AI can assist radiologists and address limitations of ultrasound by acting as a decision support tool. This review also touches on how AI programs allow for novel predictive uses of ultrasound, particularly predicting molecular subtypes of breast cancer and response to neoadjuvant chemotherapy, which have the potential to change how breast cancer is managed by providing non-invasive prognostic and treatment data from ultrasound images. Lastly, this review explores how AI programs demonstrate improved diagnostic accuracy in predicting axillary lymph node metastasis. The limitations and future challenges in developing and implementing AI for breast and axillary ultrasound will also be discussed.
Collapse
Affiliation(s)
- Christopher Trepanier
- Columbia University Irving Medical Center, 622 W 168th St, New York, NY 10032, United States of America.
| | - Alice Huang
- Columbia University Irving Medical Center, 622 W 168th St, New York, NY 10032, United States of America.
| | - Michael Liu
- Columbia University Irving Medical Center, 622 W 168th St, New York, NY 10032, United States of America.
| | - Richard Ha
- Columbia University Irving Medical Center, 622 W 168th St, New York, NY 10032, United States of America.
| |
Collapse
|
17
|
Alhussan AA, Eid MM, Towfek SK, Khafaga DS. Breast Cancer Classification Depends on the Dynamic Dipper Throated Optimization Algorithm. Biomimetics (Basel) 2023; 8:163. [PMID: 37092415 PMCID: PMC10123690 DOI: 10.3390/biomimetics8020163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 04/12/2023] [Accepted: 04/14/2023] [Indexed: 04/25/2023] Open
Abstract
According to the American Cancer Society, breast cancer is the second largest cause of mortality among women after lung cancer. Women's death rates can be decreased if breast cancer is diagnosed and treated early. Due to the lengthy duration of manual breast cancer diagnosis, an automated approach is necessary for early cancer identification. This research proposes a novel framework integrating metaheuristic optimization with deep learning and feature selection for robustly classifying breast cancer from ultrasound images. The structure of the proposed methodology consists of five stages, namely, data augmentation to improve the learning of convolutional neural network (CNN) models, transfer learning using GoogleNet deep network for feature extraction, selection of the best set of features using a novel optimization algorithm based on a hybrid of dipper throated and particle swarm optimization algorithms, and classification of the selected features using CNN optimized using the proposed optimization algorithm. To prove the effectiveness of the proposed approach, a set of experiments were conducted on a breast cancer dataset, freely available on Kaggle, to evaluate the performance of the proposed feature selection method and the performance of the optimized CNN. In addition, statistical tests were established to study the stability and difference of the proposed approach compared to state-of-the-art approaches. The achieved results confirmed the superiority of the proposed approach with a classification accuracy of 98.1%, which is better than the other approaches considered in the conducted experiments.
Collapse
Affiliation(s)
- Amel Ali Alhussan
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Marwa M. Eid
- Faculty of Artificial Intelligence, Delta University for Science and Technology, Mansoura 35712, Egypt
| | - S. K. Towfek
- Delta Higher Institute for Engineering and Technology, Mansoura 35111, Egypt
- Computer Science and Intelligent Systems Research Center, Blacksburg, VA 24060, USA
| | - Doaa Sami Khafaga
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| |
Collapse
|
18
|
Alsharif WM. The utilization of artificial intelligence applications to improve breast cancer detection and prognosis. Saudi Med J 2023; 44:119-127. [PMID: 36773967 PMCID: PMC9987701 DOI: 10.15537/smj.2023.44.2.20220611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/13/2023] Open
Abstract
Breast imaging faces challenges with the current increase in medical imaging requests and lesions that breast screening programs can miss. Solutions to improve these challenges are being sought with the recent advancement and adoption of artificial intelligent (AI)-based applications to enhance workflow efficiency as well as patient-healthcare outcomes. rtificial intelligent tools have been proposed and used to analyze different modes of breast imaging, in most of the published studies, mainly for the detection and classification of breast lesions, breast lesion segmentation, breast density evaluation, and breast cancer risk assessment. This article reviews the background of the Conventional Computer-aided Detection system and AI, AI-based applications in breast medical imaging for the identification, segmentation, and categorization of lesions, breast density and cancer risk evaluation. In addition, the challenges, and limitations of AI-based applications in breast imaging are also discussed.
Collapse
Affiliation(s)
- Walaa M. Alsharif
- From the Diagnostic Radiology Technology Department, College of Applied Medical Sciences, Taibah University, Al Madinah Al Munawwarah; and from the Society of Artificial Intelligence in Healthcare, Riyadh, Kingdom of Saudi Arabia.
- Address correspondence and reprint request to: Dr. Walaa M. Alsharif, Diagnostic Radiology Technology Department, College of Applied Medical Sciences, Taibah University, Al Madinah Al Munawwarah, Kingdom of Saudi Arabia. E-mail: ORCID ID: https//:orcid.org/0000-0001-7607-3255
| |
Collapse
|
19
|
Tan T, Rodriguez-Ruiz A, Zhang T, Xu L, Beets-Tan RGH, Shen Y, Karssemeijer N, Xu J, Mann RM, Bao L. Multi-modal artificial intelligence for the combination of automated 3D breast ultrasound and mammograms in a population of women with predominantly dense breasts. Insights Imaging 2023; 14:10. [PMID: 36645507 PMCID: PMC9842825 DOI: 10.1186/s13244-022-01352-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Accepted: 12/09/2022] [Indexed: 01/17/2023] Open
Abstract
OBJECTIVES To assess the stand-alone and combined performance of artificial intelligence (AI) detection systems for digital mammography (DM) and automated 3D breast ultrasound (ABUS) in detecting breast cancer in women with dense breasts. METHODS 430 paired cases of DM and ABUS examinations from a Asian population with dense breasts were retrospectively collected. All cases were analyzed by two AI systems, one for DM exams and one for ABUS exams. A selected subset (n = 152) was read by four radiologists. The performance of AI systems was based on analysis of the area under the receiver operating characteristic curve (AUC). The maximum Youden's index and its associated sensitivity and specificity were also reported for each AI systems. Detection performance of human readers in the subcohort of the reader study was measured in terms of sensitivity and specificity. RESULTS The performance of the AI systems in a multi-modal setting was significantly better when the weights of AI-DM and AI-ABUS were 0.25 and 0.75, respectively, than each system individually in a single-modal setting (AUC-AI-Multimodal = 0.865; AUC-AI-DM = 0.832, p = 0.026; AUC-AI-ABUS = 0.841, p = 0.041). The maximum Youden's index for AI-Multimodal was 0.707 (sensitivity = 79.4%, specificity = 91.2%). In the subcohort that underwent human reading, the panel of four readers achieved a sensitivity of 93.2% and specificity of 32.7%. AI-multimodal achieves superior or equal sensitivity as single human readers at the same specificity operating points on the ROC curve. CONCLUSION Multimodal (ABUS + DM) AI systems for detecting breast cancer in women with dense breasts are a potential solution for breast screening in radiologist-scarce regions.
Collapse
Affiliation(s)
- Tao Tan
- grid.430814.a0000 0001 0674 1393Department of Radiology, Netherlands Cancer Institute (NKI), Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands ,Faculty of Applied Science, Macao Polytechnic University, Macao, 999078 China
| | | | - Tianyu Zhang
- grid.430814.a0000 0001 0674 1393Department of Radiology, Netherlands Cancer Institute (NKI), Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands ,grid.5012.60000 0001 0481 6099GROW School for Oncology and Development Biology, Maastricht University, P. O. Box 616, 6200 MD Maastricht, The Netherlands
| | - Lin Xu
- grid.440637.20000 0004 4657 8879School of Information Science and Technology, ShanghaiTech University, Shanghai, 201210 China
| | - Regina G. H. Beets-Tan
- grid.430814.a0000 0001 0674 1393Department of Radiology, Netherlands Cancer Institute (NKI), Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands ,grid.5012.60000 0001 0481 6099GROW School for Oncology and Development Biology, Maastricht University, P. O. Box 616, 6200 MD Maastricht, The Netherlands
| | - Yingzhao Shen
- grid.13402.340000 0004 1759 700XAffiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Xueshi Road, Hubin Street, Shangcheng District, Hangzhou, 310006 Zhejiang China
| | - Nico Karssemeijer
- grid.10417.330000 0004 0444 9382Department of Diagnostic Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, The Netherlands
| | - Jun Xu
- grid.260478.f0000 0000 9249 2313Institute for AI in Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, 210044 China
| | - Ritse M. Mann
- grid.430814.a0000 0001 0674 1393Department of Radiology, Netherlands Cancer Institute (NKI), Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands ,grid.10417.330000 0004 0444 9382Department of Diagnostic Imaging, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, The Netherlands
| | - Lingyun Bao
- grid.13402.340000 0004 1759 700XAffiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Xueshi Road, Hubin Street, Shangcheng District, Hangzhou, 310006 Zhejiang China
| |
Collapse
|
20
|
Castro E, Costa Pereira J, Cardoso JS. Symmetry-based regularization in deep breast cancer screening. Med Image Anal 2023; 83:102690. [PMID: 36446314 DOI: 10.1016/j.media.2022.102690] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 10/28/2022] [Accepted: 11/09/2022] [Indexed: 11/23/2022]
Abstract
Breast cancer is the most common and lethal form of cancer in women. Recent efforts have focused on developing accurate neural network-based computer-aided diagnosis systems for screening to help anticipate this disease. The ultimate goal is to reduce mortality and improve quality of life after treatment. Due to the difficulty in collecting and annotating data in this domain, data scarcity is - and will continue to be - a limiting factor. In this work, we present a unified view of different regularization methods that incorporate domain-known symmetries in the model. Three general strategies were followed: (i) data augmentation, (ii) invariance promotion in the loss function, and (iii) the use of equivariant architectures. Each of these strategies encodes different priors on the functions learned by the model and can be readily introduced in most settings. Empirically we show that the proposed symmetry-based regularization procedures improve generalization to unseen examples. This advantage is verified in different scenarios, datasets and model architectures. We hope that both the principle of symmetry-based regularization and the concrete methods presented can guide development towards more data-efficient methods for breast cancer screening as well as other medical imaging domains.
Collapse
Affiliation(s)
- Eduardo Castro
- INESC TEC, Campus da Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal; Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal.
| | - Jose Costa Pereira
- INESC TEC, Campus da Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal; Huawei Technologies R&D, Noah's Ark Lab, Gridiron building, 1 Pancras Square, 5th floor, London N1C 4AG, United Kingdom
| | - Jaime S Cardoso
- INESC TEC, Campus da Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal; Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
| |
Collapse
|
21
|
Magni V, Cozzi A, Schiaffino S, Colarieti A, Sardanelli F. Artificial intelligence for digital breast tomosynthesis: Impact on diagnostic performance, reading times, and workload in the era of personalized screening. Eur J Radiol 2023; 158:110631. [PMID: 36481480 DOI: 10.1016/j.ejrad.2022.110631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 11/24/2022] [Indexed: 12/05/2022]
Abstract
The ultimate goals of the application of artificial intelligence (AI) to digital breast tomosynthesis (DBT) are the reduction of reading times, the increase of diagnostic performance, and the reduction of interval cancer rates. In this review, after outlining the journey from computer-aided detection/diagnosis systems to AI applied to digital mammography (DM), we summarize the results of studies where AI was applied to DBT, noting that long-term advantages of DBT screening and its crucial ability to decrease the interval cancer rate are still under scrutiny. AI has shown the capability to overcome some shortcomings of DBT in the screening setting by improving diagnostic performance and by reducing recall rates (from -2 % to -27 %) and reading times (up to -53 %, with an average 20 % reduction), but the ability of AI to reduce interval cancer rates has not yet been clearly investigated. Prospective validation is needed to assess the cost-effectiveness and real-world impact of AI models assisting DBT interpretation, especially in large-scale studies with low breast cancer prevalence. Finally, we focus on the incoming era of personalized and risk-stratified screening that will first see the application of contrast-enhanced breast imaging to screen women with extremely dense breasts. As the diagnostic advantage of DBT over DM was concentrated in this category, we try to understand if the application of AI to DM in the remaining cohorts of women with heterogeneously dense or non-dense breast could close the gap in diagnostic performance between DM and DBT, thus neutralizing the usefulness of AI application to DBT.
Collapse
Affiliation(s)
- Veronica Magni
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Luigi Mangiagalli 31, 20133 Milano, Italy.
| | - Andrea Cozzi
- Unit of Radiology, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 San Donato Milanese, Italy
| | - Simone Schiaffino
- Unit of Radiology, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 San Donato Milanese, Italy
| | - Anna Colarieti
- Unit of Radiology, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 San Donato Milanese, Italy
| | - Francesco Sardanelli
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Luigi Mangiagalli 31, 20133 Milano, Italy; Unit of Radiology, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 San Donato Milanese, Italy.
| |
Collapse
|
22
|
Zhang S, Liao M, Wang J, Zhu Y, Zhang Y, Zhang J, Zheng R, Lv L, Zhu D, Chen H, Wang W. Fully automatic tumor segmentation of breast ultrasound images with deep learning. J Appl Clin Med Phys 2022; 24:e13863. [PMID: 36495018 PMCID: PMC9859996 DOI: 10.1002/acm2.13863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Revised: 09/28/2022] [Accepted: 11/22/2022] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND Breast ultrasound (BUS) imaging is one of the most prevalent approaches for the detection of breast cancers. Tumor segmentation of BUS images can facilitate doctors in localizing tumors and is a necessary step for computer-aided diagnosis systems. While the majority of clinical BUS scans are normal ones without tumors, segmentation approaches such as U-Net often predict mass regions for these images. Such false-positive problem becomes serious if a fully automatic artificial intelligence system is used for routine screening. METHODS In this study, we proposed a novel model which is more suitable for routine BUS screening. The model contains a classification branch that determines whether the image is normal or with tumors, and a segmentation branch that outlines tumors. Two branches share the same encoder network. We also built a new dataset that contains 1600 BUS images from 625 patients for training and a testing dataset with 130 images from 120 patients for testing. The dataset is the largest one with pixel-wise masks manually segmented by experienced radiologists. Our code is available at https://github.com/szhangNJU/BUS_segmentation. RESULTS The area under the receiver operating characteristic curve (AUC) for classifying images into normal/abnormal categories was 0.991. The dice similarity coefficient (DSC) for segmentation of mass regions was 0.898, better than the state-of-the-art models. Testing on an external dataset gave a similar performance, demonstrating a good transferability of our model. Moreover, we simulated the use of the model in actual clinic practice by processing videos recorded during BUS scans; the model gave very low false-positive predictions on normal images without sacrificing sensitivities for images with tumors. CONCLUSIONS Our model achieved better segmentation performance than the state-of-the-art models and showed a good transferability on an external test set. The proposed deep learning architecture holds potential for use in fully automatic BUS health screening.
Collapse
Affiliation(s)
- Shuai Zhang
- Collaborative Innovation Center of Advanced MicrostructuresSchool of PhysicsNanjing UniversityNanjingChina
| | - Mei Liao
- Department of UltrasoundThird Affiliated HospitalSun Yat‐sen UniversityGuangzhouChina
| | - Jing Wang
- Department of Radiation OncologyEmory UniversityAtlantaGeorgiaUSA
| | - Yongyi Zhu
- Department of UltrasoundThird Affiliated HospitalSun Yat‐sen UniversityGuangzhouChina
| | - Yanling Zhang
- Department of UltrasoundThird Affiliated HospitalSun Yat‐sen UniversityGuangzhouChina
| | - Jian Zhang
- Collaborative Innovation Center of Advanced MicrostructuresSchool of PhysicsNanjing UniversityNanjingChina
- Institute for Brain SciencesNanjing UniversityNanjingChina
| | - Rongqin Zheng
- Department of UltrasoundThird Affiliated HospitalSun Yat‐sen UniversityGuangzhouChina
| | | | | | - Hao Chen
- Precision Care technologyHangzhouChina
| | - Wei Wang
- Collaborative Innovation Center of Advanced MicrostructuresSchool of PhysicsNanjing UniversityNanjingChina
- Institute for Brain SciencesNanjing UniversityNanjingChina
| |
Collapse
|
23
|
Li Z, Koban KC, Schenck TL, Giunta RE, Li Q, Sun Y. Artificial Intelligence in Dermatology Image Analysis: Current Developments and Future Trends. J Clin Med 2022; 11:jcm11226826. [PMID: 36431301 PMCID: PMC9693628 DOI: 10.3390/jcm11226826] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 10/24/2022] [Accepted: 10/28/2022] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Thanks to the rapid development of computer-based systems and deep-learning-based algorithms, artificial intelligence (AI) has long been integrated into the healthcare field. AI is also particularly helpful in image recognition, surgical assistance and basic research. Due to the unique nature of dermatology, AI-aided dermatological diagnosis based on image recognition has become a modern focus and future trend. Key scientific concepts of review: The use of 3D imaging systems allows clinicians to screen and label skin pigmented lesions and distributed disorders, which can provide an objective assessment and image documentation of lesion sites. Dermatoscopes combined with intelligent software help the dermatologist to easily correlate each close-up image with the corresponding marked lesion in the 3D body map. In addition, AI in the field of prosthetics can assist in the rehabilitation of patients and help to restore limb function after amputation in patients with skin tumors. THE AIM OF THE STUDY For the benefit of patients, dermatologists have an obligation to explore the opportunities, risks and limitations of AI applications. This study focuses on the application of emerging AI in dermatology to aid clinical diagnosis and treatment, analyzes the current state of the field and summarizes its future trends and prospects so as to help dermatologists realize the impact of new technological innovations on traditional practices so that they can embrace and use AI-based medical approaches more quickly.
Collapse
Affiliation(s)
- Zhouxiao Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | | | - Thilo Ludwig Schenck
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Riccardo Enzo Giunta
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Qingfeng Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| | - Yangbai Sun
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| |
Collapse
|
24
|
Depiction of breast cancers on digital mammograms by artificial intelligence-based computer-assisted diagnosis according to cancer characteristics. Eur Radiol 2022; 32:7400-7408. [PMID: 35499564 DOI: 10.1007/s00330-022-08718-2] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Revised: 02/03/2022] [Accepted: 03/02/2022] [Indexed: 01/03/2023]
Abstract
OBJECTIVE To evaluate how breast cancers are depicted by artificial intelligence-based computer-assisted diagnosis (AI-CAD) according to clinical, radiological, and pathological factors. MATERIALS AND METHODS From January 2017 to December 2017, 896 patients diagnosed with 930 breast cancers were enrolled in this retrospective study. Commercial AI-CAD was applied to digital mammograms and abnormality scores were obtained. We evaluated the abnormality score according to clinical, radiological, and pathological characteristics. False-negative results were defined by abnormality scores less than 10. RESULTS The median abnormality score of 930 breasts was 87.4 (range 0-99). The false-negative rate of AI-CAD was 19.4% (180/930). Cancers with an abnormality score of more than 90 showed a high proportion of palpable lesions, BI-RADS 4c and 5 lesions, cancers presenting as mass with or without microcalcifications and invasive cancers compared with low-scored cancers (all p < 0.001). False-negative cancers were more likely to develop in asymptomatic patients and extremely dense breasts and to be diagnosed as occult breast cancers and DCIS compared to detected cancers. CONCLUSION Breast cancers depicted with high abnormality scores by AI-CAD are associated with higher BI-RADS category, invasive pathology, and higher cancer stage. KEY POINTS • High-scored cancers by AI-CAD included a high proportion of BI-RADS 4c and 5 lesions, masses with or without microcalcifications, and cancers with invasive pathology. • Among invasive cancers, cancers with higher T and N stage and HER2-enriched subtype were depicted with higher abnormality scores by AI-CAD. • Cancers missed by AI-CAD tended to be in asymptomatic patients and extremely dense breasts and to be diagnosed as occult breast cancers by radiologists.
Collapse
|
25
|
Retson TA, Watanabe AT, Vu H, Chim CY. Multicenter, Multivendor Validation of an FDA-approved Algorithm for Mammography Triage. JOURNAL OF BREAST IMAGING 2022; 4:488-495. [PMID: 38416951 DOI: 10.1093/jbi/wbac046] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2021] [Indexed: 03/01/2024]
Abstract
OBJECTIVE Artificial intelligence (AI)-based triage algorithms may improve cancer detection and expedite radiologist workflow. To this end, the performance of a commercial AI-based triage algorithm on screening mammograms was evaluated across breast densities and lesion types. METHODS This retrospective, IRB-exempt, multicenter, multivendor study examined 1255 screening 4-view mammograms (400 positive and 855 negative studies). Images were anonymized by providing institutions and analyzed by a commercially available AI algorithm (cmTriage, CureMetrix, La Jolla, CA) that performed retrospective triage at the study level by flagging exams as "suspicious" or not. Sensitivities and specificities with confidence intervals were derived from area under the curve (AUC) calculations. RESULTS The algorithm demonstrated an AUC of 0.95 (95% CI: 0.94-0.96) for case identification. Area under the curve held across densities (0.95) and lesion types (masses: 0.94 [95% CI: 0.92-0.96] or microcalcifications: 0.97 [95% CI: 0.96-0.99]). The algorithm has a default sensitivity of 93% (95% CI: 95.6%-90.5%) with specificity of 76.3% (95% CI: 79.2%-73.4%). To evaluate real-world performance, a sensitivity of 86.9% (95% CI: 83.6%-90.2%) was tested, as observed for practicing radiologists by the Breast Cancer Surveillance Consortium (BCSC) study. The resulting specificity was 88.5% (95% CI: 86.4%-90.7%), similar to the BCSC specificity of 88.9%, indicating performance comparable to real-world results. CONCLUSION When tested for lesion detection, an AI-based triage software can perform at the level of practicing radiologists. Drawing attention to suspicious exams may improve reader specificity and help streamline radiologist workflow, enabling faster turnaround times and improving care.
Collapse
Affiliation(s)
- Tara A Retson
- University of California School of Medicine, Department of Radiology, La Jolla, CA, USA
| | - Alyssa T Watanabe
- University of Southern California Keck School of Medicine, Department of Radiology, Los Angeles, CA, USA
- CureMetrix, Inc., La Jolla, CA, USA
| | - Hoanh Vu
- CureMetrix, Inc., La Jolla, CA, USA
| | | |
Collapse
|
26
|
Zimmerer D, Full PM, Isensee F, Jager P, Adler T, Petersen J, Kohler G, Ross T, Reinke A, Kascenas A, Jensen BS, O'Neil AQ, Tan J, Hou B, Batten J, Qiu H, Kainz B, Shvetsova N, Fedulova I, Dylov DV, Yu B, Zhai J, Hu J, Si R, Zhou S, Wang S, Li X, Chen X, Zhao Y, Marimont SN, Tarroni G, Saase V, Maier-Hein L, Maier-Hein K. MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:2728-2738. [PMID: 35468060 DOI: 10.1109/tmi.2022.3170077] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate from the distribution of the training data, they often produce incorrect and over-confident predictions. OoD detection algorithms aim to catch erroneous predictions in advance by analysing the data distribution and detecting potential instances of failure. Moreover, flagging OoD cases may support human readers in identifying incidental findings. Due to the increased interest in OoD algorithms, benchmarks for different domains have recently been established. In the medical imaging domain, for which reliable predictions are often essential, an open benchmark has been missing. We introduce the Medical-Out-Of-Distribution-Analysis-Challenge (MOOD) as an open, fair, and unbiased benchmark for OoD methods in the medical imaging domain. The analysis of the submitted algorithms shows that performance has a strong positive correlation with the perceived difficulty, and that all algorithms show a high variance for different anomalies, making it yet hard to recommend them for clinical practice. We also see a strong correlation between challenge ranking and performance on a simple toy test set, indicating that this might be a valuable addition as a proxy dataset during anomaly detection algorithm development.
Collapse
|
27
|
Hendrix N, Lowry KP, Elmore JG, Lotter W, Sorensen G, Hsu W, Liao GJ, Parsian S, Kolb S, Naeim A, Lee CI. Radiologist Preferences for Artificial Intelligence-Based Decision Support During Screening Mammography Interpretation. J Am Coll Radiol 2022; 19:1098-1110. [PMID: 35970474 PMCID: PMC9840464 DOI: 10.1016/j.jacr.2022.06.019] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 06/03/2022] [Accepted: 06/07/2022] [Indexed: 01/17/2023]
Abstract
BACKGROUND Artificial intelligence (AI) may improve cancer detection and risk prediction during mammography screening, but radiologists' preferences regarding its characteristics and implementation are unknown. PURPOSE To quantify how different attributes of AI-based cancer detection and risk prediction tools affect radiologists' intentions to use AI during screening mammography interpretation. MATERIALS AND METHODS Through qualitative interviews with radiologists, we identified five primary attributes for AI-based breast cancer detection and four for breast cancer risk prediction. We developed a discrete choice experiment based on these attributes and invited 150 US-based radiologists to participate. Each respondent made eight choices for each tool between three alternatives: two hypothetical AI-based tools versus screening without AI. We analyzed samplewide preferences using random parameters logit models and identified subgroups with latent class models. RESULTS Respondents (n = 66; 44% response rate) were from six diverse practice settings across eight states. Radiologists were more interested in AI for cancer detection when sensitivity and specificity were balanced (94% sensitivity with <25% of examinations marked) and AI markup appeared at the end of the hanging protocol after radiologists complete their independent review. For AI-based risk prediction, radiologists preferred AI models using both mammography images and clinical data. Overall, 46% to 60% intended to adopt any of the AI tools presented in the study; 26% to 33% approached AI enthusiastically but were deterred if the features did not align with their preferences. CONCLUSION Although most radiologists want to use AI-based decision support, short-term uptake may be maximized by implementing tools that meet the preferences of dissuadable users.
Collapse
Affiliation(s)
- Nathaniel Hendrix
- Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Kathryn P Lowry
- Department of Radiology, University of Washington, Seattle Cancer Care Alliance, Seattle, Washington.
| | - Joann G Elmore
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, California
| | - William Lotter
- Chief Technology Officer, DeepHealth Inc, RadNet AI Solutions, Cambridge, Massachusetts
| | - Gregory Sorensen
- Chief Technology Officer, DeepHealth Inc, RadNet AI Solutions, Cambridge, Massachusetts
| | - William Hsu
- Department of Radiological Sciences, Data Integration, Architecture, and Analytics Group, University of California, Los Angeles, California; American Medical Informatics Association: Member, Governance Committee; RSNA: Deputy Editor, Radiology: Artificial Intelligence
| | - Geraldine J Liao
- Department of Radiology, Virginia Mason Medical Center, Seattle, Washington
| | - Sana Parsian
- Department of Radiology, University of Washington, Seattle Cancer Care Alliance, Seattle, Washington; Department of Radiology, Kaiser Permanente Washington, Seattle, Washington
| | - Suzanne Kolb
- Department of Radiology, University of Washington, Seattle Cancer Care Alliance, Seattle, Washington
| | - Arash Naeim
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, California; Chief Medical Officer for Clinical Research, UCLA Health; Codirector: Clinical and Translational Science Institute and Center for SMART Health; Associate Director: Institute for Precision Health, Jonsson Comprehensive Cancer Center, Garrick Institute for Risk Sciences
| | - Christoph I Lee
- Department of Radiology, University of Washington, Seattle Cancer Care Alliance, Seattle, Washington; Department of Health Services, School of Public Health, University of Washington, Seattle, Washington; and Deputy Editor, JACR
| |
Collapse
|
28
|
Bao C, Shen J, Zhang Y, Zhang Y, Wei W, Wang Z, Ding J, Han L. Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram. Cancer Med 2022; 12:3718-3726. [PMID: 36082949 PMCID: PMC9939225 DOI: 10.1002/cam4.5231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 08/16/2022] [Accepted: 08/30/2022] [Indexed: 11/05/2022] Open
Abstract
BACKGROUND To evaluate the diagnostic performance of radiologists on breast cancer with or without artificial intelligence (AI) support. METHODS A retrospective study was performed. In total, 643 mammograms (average age: 54 years; female: 100%; cancer: 62.05%) were randomly allocated into two groups. Seventy-five percent of mammograms in each group were randomly selected for assessment by two independent radiologists, and the rest were read once. Half of the 71 radiologists could read mammograms with AI support, and the other half could not. Sensitivity, specificity, Youden's index, agreement rate, Kappa value, the area under the receiver operating characteristic curve (AUC) and the reading time of radiologists in each group were analyzed. RESULTS The average AUC was higher if the AI support system was used (unaided: 0.84; with AI support: 0.91; p < 0.01). The average sensitivity increased from 84.77% to 95.07% with AI support (p < 0.01), but the average specificity decreased (p = 0.07). Youden's index, agreement rate and Kappa value were larger in the group with AI support, and the average reading time was shorter (p < 0.01). CONCLUSIONS The AI support system might contribute to enhancing the diagnostic performance (e.g., higher sensitivity and AUC) of radiologists. In the future, the AI algorithm should be improved, and prospective studies should be conducted.
Collapse
Affiliation(s)
- Chengzhen Bao
- Beijing Obstetrics and Gynecology HospitalCapital Medical University. Beijing Maternal and Child Health Care HospitalBeijingChina
| | - Jie Shen
- Beijing Obstetrics and Gynecology HospitalCapital Medical University. Beijing Maternal and Child Health Care HospitalBeijingChina
| | - Yue Zhang
- Beijing Obstetrics and Gynecology HospitalCapital Medical University. Beijing Maternal and Child Health Care HospitalBeijingChina
| | - Yan Zhang
- Beijing Obstetrics and Gynecology HospitalCapital Medical University. Beijing Maternal and Child Health Care HospitalBeijingChina
| | - Wei Wei
- Beijing Obstetrics and Gynecology HospitalCapital Medical University. Beijing Maternal and Child Health Care HospitalBeijingChina
| | | | | | - Lili Han
- Beijing Obstetrics and Gynecology HospitalCapital Medical University. Beijing Maternal and Child Health Care HospitalBeijingChina
| |
Collapse
|
29
|
Chen X, Zhang K, Abdoli N, Gilley PW, Wang X, Liu H, Zheng B, Qiu Y. Transformers Improve Breast Cancer Diagnosis from Unregistered Multi-View Mammograms. Diagnostics (Basel) 2022; 12:diagnostics12071549. [PMID: 35885455 PMCID: PMC9320758 DOI: 10.3390/diagnostics12071549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 06/21/2022] [Accepted: 06/24/2022] [Indexed: 11/16/2022] Open
Abstract
Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operations, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying or mapping corresponding breast lesion features computed from unregistered multiple mammograms. This motivated us to leverage the architecture of Multi-view Vision Transformers to capture long-range relationships of multiple mammograms from the same patient in one examination. For this purpose, we employed local transformer blocks to separately learn patch relationships within four mammograms acquired from two-view (CC/MLO) of two-side (right/left) breasts. The outputs from different views and sides were concatenated and fed into global transformer blocks, to jointly learn patch relationships between four images representing two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of mammograms, which included 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our four-image (two-view-two-side) transformer-based model achieves case classification performance with an area under ROC curve (AUC = 0.818 ± 0.039), which significantly outperforms AUC = 0.784 ± 0.016 achieved by the state-of-the-art multi-view CNNs (p = 0.009). It also outperforms two one-view-two-side models that achieve AUC of 0.724 ± 0.013 (CC view) and 0.769 ± 0.036 (MLO view), respectively. The study demonstrates the potential of using transformers to develop high-performing computer-aided diagnosis schemes that combine four mammograms.
Collapse
Affiliation(s)
- Xuxin Chen
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
- Correspondence: (X.C.); (Y.Q.)
| | - Ke Zhang
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
- Stephenson School of Biomedical Engineering, University of Oklahoma, Norman, OK 73019, USA
| | - Neman Abdoli
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
| | - Patrik W. Gilley
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
| | | | - Hong Liu
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
| | - Bin Zheng
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
| | - Yuchen Qiu
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA; (K.Z.); (N.A.); (P.W.G.); (H.L.); (B.Z.)
- Correspondence: (X.C.); (Y.Q.)
| |
Collapse
|
30
|
Smetherman D, Golding L, Moy L, Rubin E. The Economic Impact of AI on Breast Imaging. JOURNAL OF BREAST IMAGING 2022; 4:302-308. [PMID: 38416968 DOI: 10.1093/jbi/wbac012] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2021] [Indexed: 03/01/2024]
Abstract
This article explores the development of computer-aided detection (CAD) and artificial or augmented intelligence (AI) for breast radiology examinations and describes the current applications of AI in breast imaging. Although radiologists in other subspecialties may be less familiar with the use of these technologies in their practices, CAD has been used in breast imaging for more than two decades, as mammography CAD programs have been commercially available in the United States since the late 1990s. Likewise, breast radiologists have seen payment for CAD in mammography and breast MRI evolve over time. With the rapid expansion of AI products in radiology in recent years, many new applications for these technologies in breast imaging have emerged. This article outlines the current state of reimbursement for breast radiology AI algorithms within the traditional fee-for-service model used by Medicare and commercial insurers as well as potential future payment pathways. In addition, the inherent challenges of employing the existing payment framework in the United States to AI programs in radiology are detailed for the reader. This article aims to give breast radiologists a better understanding of how AI will be reimbursed as they seek to further incorporate these emerging technologies into their practices to advance patient care and improve workflow efficiency.
Collapse
Affiliation(s)
- Dana Smetherman
- Ochsner Health, Department of Radiology, New Orleans, LA, USA
| | - Lauren Golding
- Triad Radiology Associates, PLLC, Winston-Salem, NC, USA
| | - Linda Moy
- NYU Langone Health, New York, NY, USA
| | - Eric Rubin
- Southeast Radiology Limited, Philadelphia, PA,USA
| |
Collapse
|
31
|
Jabeen K, Khan MA, Alhaisoni M, Tariq U, Zhang YD, Hamza A, Mickus A, Damaševičius R. Breast Cancer Classification from Ultrasound Images Using Probability-Based Optimal Deep Learning Feature Fusion. SENSORS 2022; 22:s22030807. [PMID: 35161552 PMCID: PMC8840464 DOI: 10.3390/s22030807] [Citation(s) in RCA: 56] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 01/12/2022] [Accepted: 01/17/2022] [Indexed: 12/11/2022]
Abstract
After lung cancer, breast cancer is the second leading cause of death in women. If breast cancer is detected early, mortality rates in women can be reduced. Because manual breast cancer diagnosis takes a long time, an automated system is required for early cancer detection. This paper proposes a new framework for breast cancer classification from ultrasound images that employs deep learning and the fusion of the best selected features. The proposed framework is divided into five major steps: (i) data augmentation is performed to increase the size of the original dataset for better learning of Convolutional Neural Network (CNN) models; (ii) a pre-trained DarkNet-53 model is considered and the output layer is modified based on the augmented dataset classes; (iii) the modified model is trained using transfer learning and features are extracted from the global average pooling layer; (iv) the best features are selected using two improved optimization algorithms known as reformed differential evaluation (RDE) and reformed gray wolf (RGW); and (v) the best selected features are fused using a new probability-based serial approach and classified using machine learning algorithms. The experiment was conducted on an augmented Breast Ultrasound Images (BUSI) dataset, and the best accuracy was 99.1%. When compared with recent techniques, the proposed framework outperforms them.
Collapse
Affiliation(s)
- Kiran Jabeen
- Department of Computer Science, HITEC University Taxila, Taxila 47080, Pakistan; (K.J.); (M.A.K.); (A.H.)
| | - Muhammad Attique Khan
- Department of Computer Science, HITEC University Taxila, Taxila 47080, Pakistan; (K.J.); (M.A.K.); (A.H.)
| | - Majed Alhaisoni
- College of Computer Science and Engineering, University of Ha’il, Ha’il 55211, Saudi Arabia;
| | - Usman Tariq
- College of Computer Engineering and Science, Prince Sattam Bin Abdulaziz University, Al-Kharaj 11942, Saudi Arabia;
| | - Yu-Dong Zhang
- Department of Informatics, University of Leicester, Leicester LE1 7RH, UK;
| | - Ameer Hamza
- Department of Computer Science, HITEC University Taxila, Taxila 47080, Pakistan; (K.J.); (M.A.K.); (A.H.)
| | - Artūras Mickus
- Department of Applied Informatics, Vytautas Magnus University, LT-44404 Kaunas, Lithuania;
| | - Robertas Damaševičius
- Department of Applied Informatics, Vytautas Magnus University, LT-44404 Kaunas, Lithuania;
- Correspondence:
| |
Collapse
|
32
|
Artificial Intelligence (AI) for Screening Mammography, From the AI Special Series on AI Applications. AJR Am J Roentgenol 2022; 219:369-380. [PMID: 35018795 DOI: 10.2214/ajr.21.27071] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Artificial intelligence (AI) applications for screening mammography are being marketed for clinical use in the interpretative domains of lesion detection and diagnosis, triage, and breast density assessment, and in the noninterpretive domains of breast cancer risk assessment, image quality control, image acquisition, and dose reduction. Evidence in support of these nascent applications, particularly for lesion detection and diagnosis, is largely based on multireader studies with cancer-enriched datasets rather than rigorous clinical evaluation aligned with the application's specific intended clinical use. This article reviews commercial AI algorithms for screening mammography that are currently available for clinical practice, their use, and evidence supporting their performance. Clinical implementation considerations, such as workflow integration, governance, and ethical issues, are also described. In addition, the future of AI for screening mammography is discussed, including the development of interpretive and noninterpretive AI applications and strategic priorities for research and development.
Collapse
|
33
|
Connected-UNets: a deep learning architecture for breast mass segmentation. NPJ Breast Cancer 2021; 7:151. [PMID: 34857755 PMCID: PMC8640011 DOI: 10.1038/s41523-021-00358-x] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 11/01/2021] [Indexed: 12/19/2022] Open
Abstract
Breast cancer analysis implies that radiologists inspect mammograms to detect suspicious breast lesions and identify mass tumors. Artificial intelligence techniques offer automatic systems for breast mass segmentation to assist radiologists in their diagnosis. With the rapid development of deep learning and its application to medical imaging challenges, UNet and its variations is one of the state-of-the-art models for medical image segmentation that showed promising performance on mammography. In this paper, we propose an architecture, called Connected-UNets, which connects two UNets using additional modified skip connections. We integrate Atrous Spatial Pyramid Pooling (ASPP) in the two standard UNets to emphasize the contextual information within the encoder–decoder network architecture. We also apply the proposed architecture on the Attention UNet (AUNet) and the Residual UNet (ResUNet). We evaluated the proposed architectures on two publically available datasets, the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and additionally on a private dataset. Experiments were also conducted using additional synthetic data using the cycle-consistent Generative Adversarial Network (CycleGAN) model between two unpaired datasets to augment and enhance the images. Qualitative and quantitative results show that the proposed architecture can achieve better automatic mass segmentation with a high Dice score of 89.52%, 95.28%, and 95.88% and Intersection over Union (IoU) score of 80.02%, 91.03%, and 92.27%, respectively, on CBIS-DDSM, INbreast, and the private dataset.
Collapse
|
34
|
Siviengphanom S, Gandomkar Z, Lewis SJ, Brennan PC. Mammography-based Radiomics in Breast Cancer: A Scoping Review of Current Knowledge and Future Needs. Acad Radiol 2021; 29:1228-1247. [PMID: 34799256 DOI: 10.1016/j.acra.2021.09.025] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Revised: 09/14/2021] [Accepted: 09/26/2021] [Indexed: 12/19/2022]
Abstract
RATIONALE AND OBJECTIVES Breast cancer is a highly complex heterogeneous disease. Current validated prognostic factors (e.g., histological grade, lymph node involvement, receptor status, and proliferation index), as well as multigene tests (e.g., Oncotype DX and PAM50) are helpful to describe breast cancer characteristics and predict the chance of recurrence risk and survival. Nevertheless, they are invasive and cannot capture a complete heterogeneity of the entire breast tumor resulting in up to 30% of patients being either over- or under-treated for breast cancer. Furthermore, multigene testings are time consuming and expensive. Radiomics is emerging as a reliable, accurate, non-invasive, and cost-effective approach of using quantitative image features to classify breast cancer characteristics and predict patient outcomes. Several recent radiomics reviews have been conducted in breast cancer, however, specific mammography-based radiomics studies have not been well discussed. This scoping review aims to assess and summarize the current evidence on the potential usefulness of mammography-based (i.e., digital mammography, digital breast tomosynthesis, and contrast-enhanced mammography) radiomics in predicting factors that describe breast cancer characteristics, recurrence, and survival. MATERIALS AND METHODS PubMed database and eligible text reference were searched using relevant keywords to identify studies published between 2015 and December 19, 2020. Studies collected were screened and assessed based on the inclusion and exclusion criteria. RESULTS Eighteen eligible studies were included and organized into three main sections: radiomics predicting breast cancer characteristics, radiomics predicting breast cancer recurrence and survival, and radiomics integrating with clinical data. Majority of publications reported retrospective studies while three studies examined prospective cohorts. Encouraging results were reported, suggesting the potential clinical value of mammography-based radiomics. Further efforts are required to standardize radiomics approaches and catalogue reproducible and relevant mammographic radiomic features. The role of integrating radiomics with other information is discussed. CONCLUSION The potential role of mammography-based radiomics appears promising but more efforts are required to further evaluate its reliability as a routine clinical tool.
Collapse
Affiliation(s)
- Somphone Siviengphanom
- Discipline of Medical Imaging Sciences, Sydney School of Health Sciences, Faculty of Medicine and Health, University of Sydney, Level 7, Susan Wakil Health Building D18, Sydney, NSW 2006, Australia..
| | - Ziba Gandomkar
- Discipline of Medical Imaging Sciences, Sydney School of Health Sciences, Faculty of Medicine and Health, University of Sydney, Level 7, Susan Wakil Health Building D18, Sydney, NSW 2006, Australia
| | - Sarah J Lewis
- Discipline of Medical Imaging Sciences, Sydney School of Health Sciences, Faculty of Medicine and Health, University of Sydney, Level 7, Susan Wakil Health Building D18, Sydney, NSW 2006, Australia
| | - Patrick C Brennan
- Discipline of Medical Imaging Sciences, Sydney School of Health Sciences, Faculty of Medicine and Health, University of Sydney, Level 7, Susan Wakil Health Building D18, Sydney, NSW 2006, Australia
| |
Collapse
|
35
|
Keshavamurthy KN, Eickhoff C, Juluru K. Weakly supervised pneumonia localization in chest X-rays using generative adversarial networks. Med Phys 2021; 48:7154-7171. [PMID: 34459001 PMCID: PMC10997001 DOI: 10.1002/mp.15185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Revised: 07/12/2021] [Accepted: 07/27/2021] [Indexed: 11/09/2022] Open
Abstract
PURPOSE Automatic localization of pneumonia on chest X-rays (CXRs) is highly desirable both as an interpretive aid to the radiologist and for timely diagnosis of the disease. However, pneumonia's amorphous appearance on CXRs and complexity of normal anatomy in the chest present key challenges that hinder accurate localization. Existing studies in this area are either not optimized to preserve spatial information of abnormality or depend on expensive expert-annotated bounding boxes. We present a novel generative adversarial network (GAN)-based machine learning approach for this problem, which is weakly supervised (does not require any location annotations), was trained to retain spatial information, and can produce pixel-wise abnormality maps highlighting regions of abnormality (as opposed to bounding boxes around abnormality). METHODS Our method is based on the Wasserstein GAN framework and, to the best of our knowledge, the first application of GANs to this problem. Specifically, from an abnormal CXR as input, we generated the corresponding pseudo normal CXR image as output. The pseudo normal CXR is the "hypothetical" normal, if the same abnormal CXR were not to have any abnormalities. We surmise that the difference between the pseudo normal and the abnormal CXR highlights the pixels suspected to have pneumonia and hence is our output abnormality map. We trained our algorithm on an "unpaired" data set of abnormal and normal CXRs and did not require any location annotations such as bounding boxes/segmentations of abnormal regions. Furthermore, we incorporated additional prior knowledge/constraints into the model and showed that they help improve localization performance. We validated the model on a data set consisting of 14 184 CXRs from the Radiological Society of North America pneumonia detection challenge. RESULTS We evaluated our methods by comparing the generated abnormality maps with radiologist annotated bounding boxes using receiver operating characteristic (ROC) analysis, image similarity metrics such as normalized cross-correlation/mutual information, and abnormality detection rate.We also present visual examples of the abnormality maps, covering various scenarios of abnormality occurrence. Results demonstrate the ability to highlight regions of abnormality with the best method achieving an ROC area under the curve (AUC) of 0.77 and a detection rate of 85%.The GAN tended to perform better as prior knowledge/constraints were incorporated into the model. CONCLUSIONS We presented a novel GAN based approach for localizing pneumonia on CXRs that (1) does not require expensive hand annotated location ground truth; and (2) was trained to produce abnormality maps at the pixel level as opposed to bounding boxes. We demonstrated the efficacy of our methods via quantitative and qualitative results.
Collapse
Affiliation(s)
- Krishna Nand Keshavamurthy
- Brown University, Providence, RI 02912, USA
- Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY 10065, USA
| | | | - Krishna Juluru
- Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY 10065, USA
| |
Collapse
|
36
|
Gandomkar Z, Siviengphanom S, Ekpo EU, Suleiman M, Taba ST, Li T, Xu D, Evans KK, Lewis SJ, Wolfe JM, Brennan PC. Global processing provides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection. Sci Rep 2021; 11:20122. [PMID: 34635726 PMCID: PMC8505651 DOI: 10.1038/s41598-021-99582-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Accepted: 09/13/2021] [Indexed: 12/15/2022] Open
Abstract
The information captured by the gist signal, which refers to radiologists’ first impression arising from an initial global image processing, is poorly understood. We examined whether the gist signal can provide complementary information to data captured by radiologists (experiment 1), or computer algorithms (experiment 2) based on detailed mammogram inspection. In the first experiment, 19 radiologists assessed a case set twice, once based on a half-second image presentation (i.e., gist signal) and once in the usual viewing condition. Their performances in two viewing conditions were compared using repeated measure correlation (rm-corr). The cancer cases (19 cases × 19 readers) exhibited non-significant trend with rm-corr = 0.012 (p = 0.82, CI: −0.09, 0.12). For normal cases (41 cases × 19 readers), a weak correlation of rm-corr = 0.238 (p < 0.001, CI: 0.17, 0.30) was found. In the second experiment, we combined the abnormality score from a state-of-the-art deep learning-based tool (DL) with the radiological gist signal using a support vector machine (SVM). To obtain the gist signal, 53 radiologists assessed images based on half-second image presentation. The SVM performance for each radiologist and an average reader, whose gist responses were the mean abnormality scores given by all 53 readers to each image was assessed using leave-one-out cross-validation. For the average reader, the AUC for gist, DL, and the SVM, were 0.76 (CI: 0.62–0.86), 0.79 (CI: 0.63–0.89), and 0.88 (CI: 0.79–0.94). For all readers with a gist AUC significantly better than chance-level, the SVM outperformed DL. The gist signal provided malignancy evidence with no or weak associations with the information captured by humans in normal radiologic reporting, which involves detailed mammogram inspection. Adding gist signal to a state-of-the-art deep learning-based tool improved its performance for the breast cancer detection.
Collapse
Affiliation(s)
- Ziba Gandomkar
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia.
| | - Somphone Siviengphanom
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| | - Ernest U Ekpo
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| | - Mo'ayyad Suleiman
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| | - Seyedamir Tavakoli Taba
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| | - Tong Li
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| | - Dong Xu
- School of Electrical and Information Engineering, Faculty of Engineering, University of Sydney, Sydney, NSW, 2006, Australia
| | - Karla K Evans
- Department of Psychology, University of York, York, UK
| | - Sarah J Lewis
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| | - Jeremy M Wolfe
- Harvard Medical School, Boston, MA, USA.,Brigham and Women's Hospital, Boston, MA, USA
| | - Patrick C Brennan
- Discipline of Medical Imaging Sciences, Faculty of Medicine and Health, University of Sydney, 512/Block M, Cumberland Campus, Sydney, NSW, 2006, Australia
| |
Collapse
|
37
|
Laishram R, Rabidas R. WDO optimized detection for mammographic masses and its diagnosis: A unified CAD system. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107620] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
|
38
|
Shen Y, Shamout FE, Oliver JR, Witowski J, Kannan K, Park J, Wu N, Huddleston C, Wolfson S, Millet A, Ehrenpreis R, Awal D, Tyma C, Samreen N, Gao Y, Chhor C, Gandhi S, Lee C, Kumari-Subaiya S, Leonard C, Mohammed R, Moczulski C, Altabet J, Babb J, Lewin A, Reig B, Moy L, Heacock L, Geras KJ. Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams. Nat Commun 2021; 12:5645. [PMID: 34561440 PMCID: PMC8463596 DOI: 10.1038/s41467-021-26023-2] [Citation(s) in RCA: 70] [Impact Index Per Article: 23.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Accepted: 09/14/2021] [Indexed: 02/08/2023] Open
Abstract
Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.
Collapse
Affiliation(s)
- Yiqiu Shen
- grid.137628.90000 0004 1936 8753Center for Data Science, New York University, New York, NY USA
| | - Farah E. Shamout
- grid.440573.1Engineering Division, NYU Abu Dhabi, Abu Dhabi, UAE
| | - Jamie R. Oliver
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Jan Witowski
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Kawshik Kannan
- grid.482020.c0000 0001 1089 179XDepartment of Computer Science, Courant Institute, New York University, New York, NY USA
| | - Jungkyu Park
- grid.137628.90000 0004 1936 8753Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY USA
| | - Nan Wu
- grid.137628.90000 0004 1936 8753Center for Data Science, New York University, New York, NY USA
| | - Connor Huddleston
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Stacey Wolfson
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Alexandra Millet
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Robin Ehrenpreis
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Divya Awal
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Cathy Tyma
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Naziya Samreen
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Yiming Gao
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Chloe Chhor
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Stacey Gandhi
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Cindy Lee
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Sheila Kumari-Subaiya
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Cindy Leonard
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Reyhan Mohammed
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Christopher Moczulski
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Jaime Altabet
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - James Babb
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Alana Lewin
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Beatriu Reig
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Linda Moy
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA ,grid.137628.90000 0004 1936 8753Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY USA
| | - Laura Heacock
- grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA
| | - Krzysztof J. Geras
- grid.137628.90000 0004 1936 8753Center for Data Science, New York University, New York, NY USA ,grid.137628.90000 0004 1936 8753Department of Radiology, NYU Grossman School of Medicine, New York, NY USA ,grid.137628.90000 0004 1936 8753Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY USA
| |
Collapse
|
39
|
Sun Y, Ji Y. AAWS-Net: Anatomy-aware weakly-supervised learning network for breast mass segmentation. PLoS One 2021; 16:e0256830. [PMID: 34460852 PMCID: PMC8405027 DOI: 10.1371/journal.pone.0256830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Accepted: 08/16/2021] [Indexed: 11/18/2022] Open
Abstract
Accurate segmentation of breast masses is an essential step in computer aided diagnosis of breast cancer. The scarcity of annotated training data greatly hinders the model’s generalization ability, especially for the deep learning based methods. However, high-quality image-level annotations are time-consuming and cumbersome in medical image analysis scenarios. In addition, a large amount of weak annotations is under-utilized which comprise common anatomy features. To this end, inspired by teacher-student networks, we propose an Anatomy-Aware Weakly-Supervised learning Network (AAWS-Net) for extracting useful information from mammograms with weak annotations for efficient and accurate breast mass segmentation. Specifically, we adopt a weakly-supervised learning strategy in the Teacher to extract anatomy structure from mammograms with weak annotations by reconstructing the original image. Besides, knowledge distillation is used to suggest morphological differences between benign and malignant masses. Moreover, the prior knowledge learned from the Teacher is introduced to the Student in an end-to-end way, which improves the ability of the student network to locate and segment masses. Experiments on CBIS-DDSM have shown that our method yields promising performance compared with state-of-the-art alternative models for breast mass segmentation in terms of segmentation accuracy and IoU.
Collapse
Affiliation(s)
- Yeheng Sun
- School of Business, University of Shanghai for Science and Technology, Shanghai, China
- * E-mail:
| | - Yule Ji
- School of Business, University of Shanghai for Science and Technology, Shanghai, China
| |
Collapse
|
40
|
Kathiravelu P, Sharma P, Sharma A, Banerjee I, Trivedi H, Purkayastha S, Sinha P, Cadrin-Chenevert A, Safdar N, Gichoya JW. A DICOM Framework for Machine Learning and Processing Pipelines Against Real-time Radiology Images. J Digit Imaging 2021; 34:1005-1013. [PMID: 34405297 PMCID: PMC8455728 DOI: 10.1007/s10278-021-00491-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Revised: 04/29/2021] [Accepted: 07/05/2021] [Indexed: 11/30/2022] Open
Abstract
Real-time execution of machine learning (ML) pipelines on radiology images is difficult due to limited computing resources in clinical environments, whereas running them in research clusters requires efficient data transfer capabilities. We developed Niffler, an open-source Digital Imaging and Communications in Medicine (DICOM) framework that enables ML and processing pipelines in research clusters by efficiently retrieving images from the hospitals’ PACS and extracting the metadata from the images. We deployed Niffler at our institution (Emory Healthcare, the largest healthcare network in the state of Georgia) and retrieved data from 715 scanners spanning 12 sites, up to 350 GB/day continuously in real-time as a DICOM data stream over the past 2 years. We also used Niffler to retrieve images bulk on-demand based on user-provided filters to facilitate several research projects. This paper presents the architecture and three such use cases of Niffler. First, we executed an IVC filter detection and segmentation pipeline on abdominal radiographs in real-time, which was able to classify 989 test images with an accuracy of 96.0%. Second, we applied the Niffler Metadata Extractor to understand the operational efficiency of individual MRI systems based on calculated metrics. We benchmarked the accuracy of the calculated exam time windows by comparing Niffler against the Clinical Data Warehouse (CDW). Niffler accurately identified the scanners’ examination timeframes and idling times, whereas CDW falsely depicted several exam overlaps due to human errors. Third, with metadata extracted from the images by Niffler, we identified scanners with misconfigured time and reconfigured five scanners. Our evaluations highlight how Niffler enables real-time ML and processing pipelines in a research cluster.
Collapse
|
41
|
Do YA, Jang M, Yun BL, Shin SU, Kim B, Kim SM. Diagnostic Performance of Artificial Intelligence-Based Computer-Aided Diagnosis for Breast Microcalcification on Mammography. Diagnostics (Basel) 2021; 11:diagnostics11081409. [PMID: 34441343 PMCID: PMC8392744 DOI: 10.3390/diagnostics11081409] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 07/30/2021] [Accepted: 07/31/2021] [Indexed: 11/23/2022] Open
Abstract
The present study evaluated the diagnostic performance of artificial intelligence-based computer-aided diagnosis (AI-CAD) compared to that of dedicated breast radiologists in characterizing suspicious microcalcification on mammography. We retrospectively analyzed 435 unilateral mammographies from 420 patients (286 benign; 149 malignant) undergoing biopsy for suspicious microcalcification from June 2003 to November 2019. Commercial AI-CAD was applied to the mammography images, and malignancy scores were calculated. Diagnostic performance was compared between radiologists and AI-CAD using the area under the receiving operator characteristics curve (AUC). The AUCs of radiologists and AI-CAD were not significantly different (0.722 vs. 0.745, p = 0.393). The AUCs of the adjusted category were 0.726, 0.744, and 0.756 with cutoffs of 2%, 10%, and 38.03% for AI-CAD, respectively, which were all significantly higher than those for radiologists alone (all p < 0.05). None of the 27 cases downgraded to category 3 with a cutoff of 2% were confirmed as malignant on pathological analysis, suggesting that unnecessary biopsies could be avoided. Our findings suggest that the diagnostic performance of AI-CAD in characterizing suspicious microcalcification on mammography was similar to that of the radiologists, indicating that it may aid in making clinical decisions regarding the treatment of breast microcalcification.
Collapse
Affiliation(s)
- Yoon Ah Do
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam 13620, Korea; (Y.A.D.); (M.J.); (B.L.Y.); (S.U.S.)
| | - Mijung Jang
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam 13620, Korea; (Y.A.D.); (M.J.); (B.L.Y.); (S.U.S.)
| | - Bo La Yun
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam 13620, Korea; (Y.A.D.); (M.J.); (B.L.Y.); (S.U.S.)
| | - Sung Ui Shin
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam 13620, Korea; (Y.A.D.); (M.J.); (B.L.Y.); (S.U.S.)
| | - Bohyoung Kim
- Division of Biomedical Engineering, Hankuk University of Foreign Studies, Seoul 17035, Korea;
| | - Sun Mi Kim
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam 13620, Korea; (Y.A.D.); (M.J.); (B.L.Y.); (S.U.S.)
- Correspondence: ; Tel.: +82-31-787-7609
| |
Collapse
|
42
|
Liu QF, Kang BS, Liu HY, Li HA, Chen R, Liu GX. Efficient Medical Image Adaption via Axis-Aligned Mesh Deformation. JOURNAL OF MEDICAL IMAGING AND HEALTH INFORMATICS 2021. [DOI: 10.1166/jmihi.2021.3547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Content-aware medical image adaptation can make medical images be well presented on different display devices. The existing adaption algorithms mainly consider the visual effect of salient regions, such as specific organ areas of the patient body, but either ignore the quality of unimportant
areas or execute more slowly. In order to enhance the effect of adaption and accelerate the speed of adaptation, we propose an efficient medical image adaptation method via axis-aligned mesh deformation. With this method, importance map is firstly produced by combing the weighted edge map
and saliency map. Then, integer programming is used to initialize and deform the axis-aligned mesh based on importance map. Finally, image adaptation is operated rapidly by bi-linear interpolation. With the proposed method, the real-time image adaptation can be realized, and not only the visual
effect of the significant areas but also the contour integrity and continuity of the non significant areas can be maintained. Experiments on open data-sets show that the proposed method has high efficiency, better effect and strong stability, and is suitable for real-time image adaptation.
Collapse
Affiliation(s)
- Qing-Fang Liu
- School of Information Science and Technology, Northwest University, Xi’an 710127, China
| | - Bao-Sheng Kang
- School of Information Science and Technology, Northwest University, Xi’an 710127, China
| | - Hai-Yun Liu
- Network Center, Shijiazhuang Posts and Telecommunications Technical College, Shijiazhuang 050021, China
| | - Hong-An Li
- College of Computer Science and Technology, Xi’an University of Science and Technology, Xi’an 710054, China
| | - Rui Chen
- Software Engineering College, Zhengzhou University of Light Industry, Zhengzhou 450002, China
| | - Gui-Xian Liu
- Sifang College, Shijiazhuang Tiedao University, Shijiazhuang 050043, China
| |
Collapse
|
43
|
Goyal S. An Overview of Current Trends, Techniques, Prospects, and Pitfalls of Artificial Intelligence in Breast Imaging. REPORTS IN MEDICAL IMAGING 2021. [DOI: 10.2147/rmi.s295205] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
|
44
|
Shen Y, Wu N, Phang J, Park J, Liu K, Tyagi S, Heacock L, Kim SG, Moy L, Cho K, Geras KJ. An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization. Med Image Anal 2021; 68:101908. [PMID: 33383334 PMCID: PMC7828643 DOI: 10.1016/j.media.2020.101908] [Citation(s) in RCA: 60] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 11/12/2020] [Accepted: 11/13/2020] [Indexed: 12/12/2022]
Abstract
Medical images differ from natural images in significantly higher resolutions and smaller regions of interest. Because of these differences, neural network architectures that work well for natural images might not be applicable to medical image analysis. In this work, we propose a novel neural network model to address these unique properties of medical images. This model first uses a low-capacity, yet memory-efficient, network on the whole image to identify the most informative regions. It then applies another higher-capacity network to collect details from chosen regions. Finally, it employs a fusion module that aggregates global and local information to make a prediction. While existing methods often require lesion segmentation during training, our model is trained with only image-level labels and can generate pixel-level saliency maps indicating possible malignant findings. We apply the model to screening mammography interpretation: predicting the presence or absence of benign and malignant lesions. On the NYU Breast Cancer Screening Dataset, our model outperforms (AUC = 0.93) ResNet-34 and Faster R-CNN in classifying breasts with malignant findings. On the CBIS-DDSM dataset, our model achieves performance (AUC = 0.858) on par with state-of-the-art approaches. Compared to ResNet-34, our model is 4.1x faster for inference while using 78.4% less GPU memory. Furthermore, we demonstrate, in a reader study, that our model surpasses radiologist-level AUC by a margin of 0.11.
Collapse
Affiliation(s)
- Yiqiu Shen
- Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA
| | - Nan Wu
- Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA
| | - Jason Phang
- Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA
| | - Jungkyu Park
- Department of Radiology, NYU School of Medicine, 530 1st Ave, New York, NY 10016, USA
| | - Kangning Liu
- Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA
| | - Sudarshini Tyagi
- Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, 251 Mercer St, New York, NY 10012, USA
| | - Laura Heacock
- Department of Radiology, NYU School of Medicine, 530 1st Ave, New York, NY 10016, USA; Perlmutter Cancer Center, NYU Langone Health, 160 E 34th St, New York, NY 10016, USA
| | - S Gene Kim
- Department of Radiology, NYU School of Medicine, 530 1st Ave, New York, NY 10016, USA; Center for Advanced Imaging Innovation and Research, NYU Langone Health, 660 1st Ave, New York, NY 10016, USA; Perlmutter Cancer Center, NYU Langone Health, 160 E 34th St, New York, NY 10016, USA
| | - Linda Moy
- Department of Radiology, NYU School of Medicine, 530 1st Ave, New York, NY 10016, USA; Center for Advanced Imaging Innovation and Research, NYU Langone Health, 660 1st Ave, New York, NY 10016, USA; Perlmutter Cancer Center, NYU Langone Health, 160 E 34th St, New York, NY 10016, USA
| | - Kyunghyun Cho
- Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA; Department of Computer Science, Courant Institute of Mathematical Sciences, New York University, 251 Mercer St, New York, NY 10012, USA
| | - Krzysztof J Geras
- Center for Data Science, New York University, 60 5th Ave, New York, NY 10011, USA; Department of Radiology, NYU School of Medicine, 530 1st Ave, New York, NY 10016, USA; Center for Advanced Imaging Innovation and Research, NYU Langone Health, 660 1st Ave, New York, NY 10016, USA.
| |
Collapse
|
45
|
Zerouaoui H, Idri A. Reviewing Machine Learning and Image Processing Based Decision-Making Systems for Breast Cancer Imaging. J Med Syst 2021; 45:8. [PMID: 33404910 DOI: 10.1007/s10916-020-01689-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Accepted: 12/01/2020] [Indexed: 01/11/2023]
Abstract
Breast cancer (BC) is the leading cause of death among women worldwide. It affects in general women older than 40 years old. Medical images analysis is one of the most promising research areas since it provides facilities for diagnosis and decision-making of several diseases such as BC. This paper conducts a Structured Literature Review (SLR) of the use of Machine Learning (ML) and Image Processing (IP) techniques to deal with BC imaging. A set of 530 papers published between 2000 and August 2019 were selected and analyzed according to ten criteria: year and publication channel, empirical type, research type, medical task, machine learning techniques, datasets used, validation methods, performance measures and image processing techniques which include image pre-processing, segmentation, feature extraction and feature selection. Results showed that diagnosis was the most used medical task and that Deep Learning techniques (DL) were largely used to perform classification. Furthermore, we found out that classification was the most ML objective investigated followed by prediction and clustering. Most of the selected studies used Mammograms as imaging modalities rather than Ultrasound or Magnetic Resonance Imaging with the use of public or private datasets with MIAS as the most frequently investigated public dataset. As for image processing techniques, the majority of the selected studies pre-process their input images by reducing the noise and normalizing the colors, and some of them use segmentation to extract the region of interest with the thresholding method. For feature extraction, we note that researchers extracted the relevant features using classical feature extraction techniques (e.g. Texture features, Shape features, etc.) or DL techniques (e. g. VGG16, VGG19, ResNet, etc.), and finally few papers used feature selection techniques in particular the filter methods.
Collapse
Affiliation(s)
- Hasnae Zerouaoui
- Modeling, Simulation and Data Analysis, Mohammed VI Polytechnic University, Benguerir, Morocco
| | - Ali Idri
- Modeling, Simulation and Data Analysis, Mohammed VI Polytechnic University, Benguerir, Morocco. .,Software Project Management Research Team, ENSIAS, Mohammed V University in Rabat, Rabat, Morocco.
| |
Collapse
|
46
|
Artificial intelligence in oncology. Artif Intell Med 2021. [DOI: 10.1016/b978-0-12-821259-2.00018-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
47
|
Deep-Learning-Based Computer-Aided Systems for Breast Cancer Imaging: A Critical Review. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10228298] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
This paper provides a critical review of the literature on deep learning applications in breast tumor diagnosis using ultrasound and mammography images. It also summarizes recent advances in computer-aided diagnosis/detection (CAD) systems, which make use of new deep learning methods to automatically recognize breast images and improve the accuracy of diagnoses made by radiologists. This review is based upon published literature in the past decade (January 2010–January 2020), where we obtained around 250 research articles, and after an eligibility process, 59 articles were presented in more detail. The main findings in the classification process revealed that new DL-CAD methods are useful and effective screening tools for breast cancer, thus reducing the need for manual feature extraction. The breast tumor research community can utilize this survey as a basis for their current and future studies.
Collapse
|
48
|
Gómez-Flores W, Coelho de Albuquerque Pereira W. A comparative study of pre-trained convolutional neural networks for semantic segmentation of breast tumors in ultrasound. Comput Biol Med 2020; 126:104036. [PMID: 33059238 DOI: 10.1016/j.compbiomed.2020.104036] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Revised: 09/23/2020] [Accepted: 10/03/2020] [Indexed: 12/15/2022]
Abstract
The automatic segmentation of breast tumors in ultrasound (BUS) has recently been addressed using convolutional neural networks (CNN). These CNN-based approaches generally modify a previously proposed CNN architecture or they design a new architecture using CNN ensembles. Although these methods have reported satisfactory results, the trained CNN architectures are often unavailable for reproducibility purposes. Moreover, these methods commonly learn from small BUS datasets with particular properties, which limits generalization in new cases. This paper evaluates four public CNN-based semantic segmentation models that were developed by the computer vision community, as follows: (1) Fully Convolutional Network (FCN) with AlexNet network, (2) U-Net network, (3) SegNet using VGG16 and VGG19 networks, and (4) DeepLabV3+ using ResNet18, ResNet50, MobileNet-V2, and Xception networks. By transfer learning, these CNNs are fine-tuned to segment BUS images in normal and tumoral pixels. The goal is to select a potential CNN-based segmentation model to be further used in computer-aided diagnosis (CAD) systems. The main significance of this study is the comparison of eight well-established CNN architectures using a more extensive BUS dataset than those used by approaches that are currently found in the literature. More than 3000 BUS images acquired from seven US machine models are used for training and validation. The F1-score (F1s) and the Intersection over Union (IoU) quantify the segmentation performance. The segmentation models based on SegNet and DeepLabV3+ obtain the best results with F1s>0.90 and IoU>0.81. In the case of U-Net, the segmentation performance is F1s=0.89 and IoU=0.80, whereas FCN-AlexNet attains the lowest results with F1s=0.84 and IoU=0.73. In particular, ResNet18 obtains F1s=0.905 and IoU=0.827 and requires less training time among SegNet and DeepLabV3+ networks. Hence, ResNet18 is a potential candidate for implementing fully automated end-to-end CAD systems. The CNN models generated in this study are available to researchers at https://github.com/wgomezf/CNN-BUS-segment, which attempts to impact the fair comparison with other CNN-based segmentation approaches for BUS images.
Collapse
Affiliation(s)
- Wilfrido Gómez-Flores
- Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, Unidad Tamaulipas, Ciudad Victoria, Tamaulipas, Mexico.
| | | |
Collapse
|
49
|
Tran WT, Sadeghi-Naini A, Lu FI, Gandhi S, Meti N, Brackstone M, Rakovitch E, Curpen B. Computational Radiology in Breast Cancer Screening and Diagnosis Using Artificial Intelligence. Can Assoc Radiol J 2020; 72:98-108. [DOI: 10.1177/0846537120949974] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
Breast cancer screening has been shown to significantly reduce mortality in women. The increased utilization of screening examinations has led to growing demands for rapid and accurate diagnostic reporting. In modern breast imaging centers, full-field digital mammography (FFDM) has replaced traditional analog mammography, and this has opened new opportunities for developing computational frameworks to automate detection and diagnosis. Artificial intelligence (AI), and its subdomain of deep learning, is showing promising results and improvements on diagnostic accuracy, compared to previous computer-based methods, known as computer-aided detection and diagnosis. In this commentary, we review the current status of computational radiology, with a focus on deep neural networks used in breast cancer screening and diagnosis. Recent studies are developing a new generation of computer-aided detection and diagnosis systems, as well as leveraging AI-driven tools to efficiently interpret digital mammograms, and breast tomosynthesis imaging. The use of AI in computational radiology necessitates transparency and rigorous testing. However, the overall impact of AI to radiology workflows will potentially yield more efficient and standardized processes as well as improve the level of care to patients with high diagnostic accuracy.
Collapse
Affiliation(s)
- William T. Tran
- Department of Radiation Oncology, Sunnybrook Health Sciences Centre, Toronto, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, Canada
| | - Ali Sadeghi-Naini
- Department of Radiation Oncology, Sunnybrook Health Sciences Centre, Toronto, Canada
- Department of Electrical Engineering and Computer Science, Lassonde School of Engineering, York University, Toronto, Canada
| | - Fang-I Lu
- Department of Laboratory Medicine and Molecular Diagnostics, Sunnybrook Health Sciences Centre, Toronto, Canada
| | - Sonal Gandhi
- Division of Medical Oncology, Sunnybrook Health Sciences Centre, Toronto, Canada
- Department of Medicine, University of Toronto, Toronto, Canada
| | - Nicholas Meti
- Division of Medical Oncology, Sunnybrook Health Sciences Centre, Toronto, Canada
| | - Muriel Brackstone
- Department of Surgical Oncology, London Health Sciences Centre, London, Ontario
| | - Eileen Rakovitch
- Department of Radiation Oncology, Sunnybrook Health Sciences Centre, Toronto, Canada
- Department of Radiation Oncology, University of Toronto, Toronto, Canada
| | - Belinda Curpen
- Division of Breast Imaging, Sunnybrook Health Sciences Centre, Toronto, Canada
- Department of Medical Imaging, University of Toronto, Toronto, Canada
| |
Collapse
|
50
|
Wang L, Wang H, Xia C, Wang Y, Tang Q, Li J, Zhou XH. Toward standardized premarket evaluation of computer aided diagnosis/detection products: insights from FDA-approved products. Expert Rev Med Devices 2020; 17:899-918. [PMID: 32842797 DOI: 10.1080/17434440.2020.1813566] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
INTRODUCTION Computer aided detection and diagnosis (CADe and CADx) products are an emerging branch of medical device industry. However, limited technical standard has been developed for product verification and validation. It will be helpful to investigate the current practice of preclinical and clinical evaluation of approved products and provide insights for future standardization. AREAS COVERED Document review was conducted on 56 products approved by the United States Food and Drug Administration, including Summary of Safety and Effectiveness Data, 510(k) decision and de novo decision summaries. Key parameters describing product characteristics, preclinical studies and clinical studies were collected. Evaluation strategies for CADe/CADx products were analyzed and assessed. EXPERT OPINION Preclinical studies were widely adopted in the verification of CADe/CADx products. Standalone performance testing was a common procedure, but the selection of testing dataset and performance metrics showed significant variability and flexibility among manufacturers. Clinical studies were reported by all class III products and some class II products, and Multi-Reader Multi-Case design was commonly used. However, statistical analysis and presentation/interpretation of results was oftentimes incomplete. To resolve above issues, systematic development of standards of CADe/CADx is encouraged, which can be implemented at different aspects through the product lifecycle.
Collapse
Affiliation(s)
- Lu Wang
- Beijing International Center for Mathematical Research, Peking University , Beijing, China
| | - Hao Wang
- Institute for Medical Device Control, National Institutes for Food and Drug Control , Beijing, China
| | - Chen Xia
- Institute of Advanced Research, Beijing Infervision Technology Limited Liability Company , Beijing, China
| | - Yao Wang
- Department of Biosciences, University of Chicago , Chicago, Illinois, USA
| | - Qiaohong Tang
- Institute for Medical Device Control, National Institutes for Food and Drug Control , Beijing, China
| | - Jiage Li
- Institute for Medical Device Control, National Institutes for Food and Drug Control , Beijing, China
| | - Xiao-Hua Zhou
- Beijing International Center for Mathematical Research, Peking University , Beijing, China.,Department of Biostatistics, School of Public Health, Peking University , Beijing, China
| |
Collapse
|