Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making

Segmentation-based BI-RADS ensemble classification of breast tumours in ultrasound images

BACKGROUND

The development of computer-aided diagnosis systems in breast cancer imaging is exponential. Since 2016, 81 papers have described the automated segmentation of breast lesions in ultrasound images using artificial intelligence. However, only two papers have dealt with complex BI-RADS classifications.

PURPOSE

This study addresses the automatic classification of breast lesions into binary classes (benign vs. malignant) and multiple BI-RADS classes based on a single ultrasonographic image. Achieving this task should reduce the subjectivity of an individual operator's assessment.

MATERIALS AND METHODS

Automatic image segmentation methods (PraNet, CaraNet and FCBFormer) adapted to the specific segmentation task were investigated using the U-Net model as a reference. A new classification method was developed using an ensemble of selected segmentation approaches. All experiments were performed on publicly available BUS B, OASBUD, BUSI and private datasets.

RESULTS

FCBFormer achieved the best outcomes for the segmentation task with intersection over union metric values of 0.81, 0.80 and 0.73 and Dice values of 0.89, 0.87 and 0.82, respectively, for the BUS B, BUSI and OASBUD datasets. Through a series of experiments, we determined that adding an extra 30-pixel margin to the segmentation mask counteracts the potential errors introduced by the segmentation algorithm. An assembly of the full image classifier, bounding box classifier and masked image classifier was the most accurate for binary classification and had the best accuracy (ACC; 0.908), F1 (0.846) and area under the receiver operating characteristics curve (AUROC; 0.871) in the BUS B and ACC (0.982), F1 (0.984) and AUROC (0.998) in the UCC BUS datasets, outperforming each classifier used separately. It was also the most effective for BI-RADS classification, with ACC of 0.953, F1 of 0.920 and AUROC of 0.986 in UCC BUS. Hard voting was the most effective method for dichotomous classification. For the multi-class BI-RADS classification, the soft voting approach was employed.

CONCLUSIONS

The proposed new classification approach with an ensemble of segmentation and classification approaches proved more accurate than most published results for binary and multi-class BI-RADS classifications.

Artificial intelligence-based, semi-automated segmentation for the extraction of ultrasound-derived radiomics features in breast cancer: a prospective multicenter study

PURPOSE

To investigate the feasibility of an artificial intelligence (AI)-based semi-automated segmentation for the extraction of ultrasound (US)-derived radiomics features in the characterization of focal breast lesions (FBLs).

MATERIAL AND METHODS

Two expert radiologists classified according to US BI-RADS criteria 352 FBLs detected in 352 patients (237 at Center A and 115 at Center B). An AI-based semi-automated segmentation was used to build a machine learning (ML) model on the basis of B-mode US of 237 images (center A) and then validated on an external cohort of B-mode US images of 115 patients (Center B).

RESULTS

A total of 202 of 352 (57.4%) FBLs were benign, and 150 of 352 (42.6%) were malignant. The AI-based semi-automated segmentation achieved a success rate of 95.7% for one reviewer and 96% for the other, without significant difference (p = 0.839). A total of 15 (4.3%) and 14 (4%) of 352 semi-automated segmentations were not accepted due to posterior acoustic shadowing at B-Mode US and 13 and 10 of them corresponded to malignant lesions, respectively. In the validation cohort, the characterization made by the expert radiologist yielded values of sensitivity, specificity, PPV and NPV of 0.933, 0.9, 0.857, 0.955, respectively. The ML model obtained values of sensitivity, specificity, PPV and NPV of 0.544, 0.6, 0.416, 0.628, respectively. The combined assessment of radiologists and ML model yielded values of sensitivity, specificity, PPV and NPV of 0.756, 0.928, 0.872, 0.855, respectively.

CONCLUSION

AI-based semi-automated segmentation is feasible, allowing an instantaneous and reproducible extraction of US-derived radiomics features of FBLs. The combination of radiomics and US BI-RADS classification led to a potential decrease of unnecessary biopsy but at the expense of a not negligible increase of potentially missed cancers.

Automatic Detection of Post-Operative Clips in Mammography Using a U-Net Convolutional Neural Network

BACKGROUND

After breast conserving surgery (BCS), surgical clips indicate the tumor bed and, thereby, the most probable area for tumor relapse. The aim of this study was to investigate whether a U-Net-based deep convolutional neural network (dCNN) may be used to detect surgical clips in follow-up mammograms after BCS.

METHODS

884 mammograms and 517 tomosynthetic images depicting surgical clips and calcifications were manually segmented and classified. A U-Net-based segmentation network was trained with 922 images and validated with 394 images. An external test dataset consisting of 39 images was annotated by two radiologists with up to 7 years of experience in breast imaging. The network's performance was compared to that of human readers using accuracy and interrater agreement (Cohen's Kappa).

RESULTS

The overall classification accuracy on the validation set after 45 epochs ranged between 88.2% and 92.6%, indicating that the model's performance is comparable to the decisions of a human reader. In 17.4% of cases, calcifications have been misclassified as post-operative clips. The interrater reliability of the model compared to the radiologists showed substantial agreement (κreader1 = 0.72, κreader2 = 0.78) while the readers compared to each other revealed a Cohen's Kappa of 0.84, thus showing near-perfect agreement.

CONCLUSIONS

With this study, we show that surgery clips can adequately be identified by an AI technique. A potential application of the proposed technique is patient triage as well as the automatic exclusion of post-operative cases from PGMI (Perfect, Good, Moderate, Inadequate) evaluation, thus improving the quality management workflow.

Artificial Intelligence in BI-RADS Categorization of Breast Lesions on Ultrasound: Can We Omit Excessive Follow-ups and Biopsies?

RATIONALE AND OBJECTIVES

Artificial intelligence (AI) systems have been increasingly applied to breast ultrasonography. They are expected to decrease the workload of radiologists and to improve diagnostic accuracy. The aim of this study is to evaluate the performance of an AI system for the BI-RADS category assessment in breast masses detected on breast ultrasound. MATERIALS AND METHODS: A total of 715 masses detected in 530 patients were analyzed. Three breast imaging centers of the same institution and nine breast radiologists participated in this study. Ultrasound was performed by one radiologist who obtained two orthogonal views of each detected lesion. These images were retrospectively reviewed by a second radiologist blinded to the patient's clinical data. A commercial AI system evaluated images. The level of agreement between the AI system and the two radiologists and their diagnostic performance were calculated according to dichotomic BI-RADS category assessment.

RESULTS

This study included 715 breast masses. Of these, 134 (18.75%) were malignant, and 581 (81.25%) were benign. In discriminating benign and probably benign from suspicious lesions, the agreement between AI and the first and second radiologists was moderate statistically. The sensitivity and specificity of radiologist 1, radiologist 2, and AI were calculated as 98.51% and 80.72%, 97.76% and 75.56%, and 98.51% and 65.40%, respectively. For radiologist 1, the positive predictive value (PPV) was 54.10%, the negative predictive value (NPV) was 99.58%, and the accuracy was 84.06%. Radiologist 2 achieved a PPV of 47.99%, NPV of 99.32%, and accuracy of 79.72%. The AI system exhibited a PPV of 39.64%, NPV of 99.48%, and accuracy of 71.61%. Notably, none of the lesions categorized as BI-RADS 2 by AI were malignant, while 2 of the lesions classified as BI-RADS 3 by AI were subsequently confirmed as malignant. By considering AI-assigned BI-RADS 2 as safe, we could potentially avoid 11% (18 out of 163) of benign lesion biopsies and 46.2% (110 out of 238) of follow-ups.

CONCLUSION

AI proves effective in predicting malignancy. Integrating it into the clinical workflow has the potential to reduce unnecessary biopsies and short-term follow-ups, which, in turn, can contribute to sustainability in healthcare practices.

A Review of Artificial Intelligence in Breast Imaging

With the increasing dominance of artificial intelligence (AI) techniques, the important prospects for their application have extended to various medical fields, including domains such as in vitro diagnosis, intelligent rehabilitation, medical imaging, and prognosis. Breast cancer is a common malignancy that critically affects women's physical and mental health. Early breast cancer screening-through mammography, ultrasound, or magnetic resonance imaging (MRI)-can substantially improve the prognosis for breast cancer patients. AI applications have shown excellent performance in various image recognition tasks, and their use in breast cancer screening has been explored in numerous studies. This paper introduces relevant AI techniques and their applications in the field of medical imaging of the breast (mammography and ultrasound), specifically in terms of identifying, segmenting, and classifying lesions; assessing breast cancer risk; and improving image quality. Focusing on medical imaging for breast cancer, this paper also reviews related challenges and prospects for AI.

Artificial intelligence-assisted ultrasound imaging in hemophilia: research, development, and evaluation of hemarthrosis and synovitis detection

Background

Joint bleeding can lead to synovitis and arthropathy in people with hemophilia, reducing quality of life. Although early diagnosis is associated with improved therapeutic outcomes, diagnostic ultrasonography requires specialist experience. Artificial intelligence (AI) algorithms may support ultrasonography diagnoses.

Objectives

This study will research, develop, and evaluate the diagnostic precision of an AI algorithm for detecting the presence or absence of hemarthrosis and synovitis in people with hemophilia.

Methods

Elbow, knee, and ankle ultrasound images were obtained from people with hemophilia from January 2010 to March 2022. The images were used to train and test the AI models to estimate the presence/absence of hemarthrosis and synovitis. The primary endpoint was the area under the curve for the diagnostic precision to diagnose hemarthrosis and synovitis. Other endpoints were the rate of accuracy, precision, sensitivity, and specificity.

Results

Out of 5649 images collected, 3435 were used for analysis. The area under the curve for hemarthrosis detection for the elbow, knee, and ankle joints was ≥0.87 and for synovitis, it was ≥0.90. The accuracy and precision for hemarthrosis detection were ≥0.74 and ≥0.67, respectively, and those for synovitis were ≥0.83 and ≥0.74, respectively. Analysis across people with hemophilia aged 10 to 60 years showed consistent results.

Conclusion

AI models have the potential to aid diagnosis and enable earlier therapeutic interventions, helping people with hemophilia achieve healthy and active lives. Although AI models show potential in diagnosis, evidence is unclear on required control for abnormal findings. Long-term observation is crucial for assessing impact on joint health.

Use of a commercial artificial intelligence-based mammography analysis software for improving breast ultrasound interpretations

OBJECTIVES

To evaluate the use of a commercial artificial intelligence (AI)-based mammography analysis software for improving the interpretations of breast ultrasound (US)-detected lesions.

METHODS

A retrospective analysis was performed on 1109 breasts that underwent both mammography and US-guided breast biopsy. The AI software processed mammograms and provided an AI score ranging from 0 to 100 for each breast, indicating the likelihood of malignancy. The performance of the AI score in differentiating mammograms with benign outcomes from those revealing cancers following US-guided breast biopsy was evaluated. In addition, prediction models for benign outcomes were constructed based on clinical and imaging characteristics with and without AI scores, using logistic regression analysis.

RESULTS

The AI software had an area under the receiver operating characteristics curve (AUROC) of 0.79 (95% CI, 0.79-0.82) in differentiating between benign and cancer cases. The prediction models that did not include AI scores (non-AI model), only used AI scores (AI-only model), and included AI scores (integrated model) had AUROCs of 0.79 (95% CI, 0.75-0.83), 0.78 (95% CI, 0.74-0.82), and 0.85 (95% CI, 0.81-0.88) in the development cohort, and 0.75 (95% CI, 0.68-0.81), 0.82 (95% CI, 0.76-0.88), and 0.84 (95% CI, 0.79-0.90) in the validation cohort, respectively. The integrated model outperformed the non-AI model in the development and validation cohorts (p < 0.001 for both).

CONCLUSION

The commercial AI-based mammography analysis software could be a valuable adjunct to clinical decision-making for managing US-detected breast lesions.

CLINICAL RELEVANCE STATEMENT

The commercial AI-based mammography analysis software could potentially reduce unnecessary biopsies and improve patient outcomes.

KEY POINTS

• Breast US has high rates of false-positive interpretations. • A commercial AI-based mammography analysis software could distinguish mammograms having benign outcomes from those revealing cancers after US-guided breast biopsy. • A commercial AI-based mammography analysis software may improve interpretations for breast US-detected lesions.

Study on the differential diagnosis of benign and malignant breast lesions using a deep learning model based on multimodal images

OBJECTIVE

To establish a multimodal model for distinguishing benign and malignant breast lesions.

MATERIALS AND METHODS

Clinical data, mammography, and MRI images (including T2WI, diffusion-weighted images (DWI), apparent diffusion coefficient (ADC), and DCE-MRI images) of 132 benign and breast cancer patients were analyzed retrospectively. The region of interest (ROI) in each image was marked and segmented using MATLAB software. The mammography, T2WI, DWI, ADC, and DCE-MRI models based on the ResNet34 network were trained. Using an integrated learning method, the five models were used as a basic model, and voting methods were used to construct a multimodal model. The dataset was divided into a training set and a prediction set. The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the model were calculated. The diagnostic efficacy of each model was analyzed using a receiver operating characteristic curve (ROC) and an area under the curve (AUC). The diagnostic value was determined by the DeLong test with statistically significant differences set at P < 0.05.

RESULTS

We evaluated the ability of the model to classify benign and malignant tumors using the test set. The AUC values of the multimodal model, mammography model, T2WI model, DWI model, ADC model and DCE-MRI model were 0.943, 0.645, 0.595, 0.905, 0.900, and 0.865, respectively. The diagnostic ability of the multimodal model was significantly higher compared with that of the mammography and T2WI models. However, compared with the DWI, ADC, and DCE-MRI models, there was no significant difference in the diagnostic ability of these models.

CONCLUSION

Our deep learning model based on multimodal image training has practical value for the diagnosis of benign and malignant breast lesions.

Barriers and facilitators of artificial intelligence conception and implementation for breast imaging diagnosis in clinical practice: a scoping review

OBJECTIVE

Although artificial intelligence (AI) has demonstrated promise in enhancing breast cancer diagnosis, the implementation of AI algorithms in clinical practice encounters various barriers. This scoping review aims to identify these barriers and facilitators to highlight key considerations for developing and implementing AI solutions in breast cancer imaging.

METHOD

A literature search was conducted from 2012 to 2022 in six databases (PubMed, Web of Science, CINHAL, Embase, IEEE, and ArXiv). The articles were included if some barriers and/or facilitators in the conception or implementation of AI in breast clinical imaging were described. We excluded research only focusing on performance, or with data not acquired in a clinical radiology setup and not involving real patients.

RESULTS

A total of 107 articles were included. We identified six major barriers related to data (B1), black box and trust (B2), algorithms and conception (B3), evaluation and validation (B4), legal, ethical, and economic issues (B5), and education (B6), and five major facilitators covering data (F1), clinical impact (F2), algorithms and conception (F3), evaluation and validation (F4), and education (F5).

CONCLUSION

This scoping review highlighted the need to carefully design, deploy, and evaluate AI solutions in clinical practice, involving all stakeholders to yield improvement in healthcare.

CLINICAL RELEVANCE STATEMENT

The identification of barriers and facilitators with suggested solutions can guide and inform future research, and stakeholders to improve the design and implementation of AI for breast cancer detection in clinical practice.

KEY POINTS

• Six major identified barriers were related to data; black-box and trust; algorithms and conception; evaluation and validation; legal, ethical, and economic issues; and education. • Five major identified facilitators were related to data, clinical impact, algorithms and conception, evaluation and validation, and education. • Coordinated implication of all stakeholders is required to improve breast cancer diagnosis with AI.

Diagnostic performance of deep learning in ultrasound diagnosis of breast cancer: a systematic review

Deep learning (DL) has been widely investigated in breast ultrasound (US) for distinguishing between benign and malignant breast masses. This systematic review of test diagnosis aims to examine the accuracy of DL, compared to human readers, for the diagnosis of breast cancer in the US under clinical settings. Our literature search included records from databases including PubMed, Embase, Scopus, and Cochrane Library. Test accuracy outcomes were synthesized to compare the diagnostic performance of DL and human readers as well as to evaluate the assistive role of DL to human readers. A total of 16 studies involving 9238 female participants were included. There were no prospective studies comparing the test accuracy of DL versus human readers in clinical workflows. Diagnostic test results varied across the included studies. In 14 studies employing standalone DL systems, DL showed significantly lower sensitivities in 5 studies with comparable specificities and outperformed human readers at higher specificities in another 4 studies; in the remaining studies, DL models and human readers showed equivalent test outcomes. In 12 studies that assessed assistive DL systems, no studies proved the assistive role of DL in the overall diagnostic performance of human readers. Current evidence is insufficient to conclude that DL outperforms human readers or enhances the accuracy of diagnostic breast US in a clinical setting. Standardization of study methodologies is required to improve the reproducibility and generalizability of DL research, which will aid in clinical translation and application.

Exploring the Potential of Artificial Intelligence in Breast Ultrasound

Breast ultrasound has emerged as a valuable imaging modality in the detection and characterization of breast lesions, particularly in women with dense breast tissue or contraindications for mammography. Within this framework, artificial intelligence (AI) has garnered significant attention for its potential to improve diagnostic accuracy in breast ultrasound and revolutionize the workflow. This review article aims to comprehensively explore the current state of research and development in harnessing AI's capabilities for breast ultrasound. We delve into various AI techniques, including machine learning, deep learning, as well as their applications in automating lesion detection, segmentation, and classification tasks. Furthermore, the review addresses the challenges and hurdles faced in implementing AI systems in breast ultrasound diagnostics, such as data privacy, interpretability, and regulatory approval. Ethical considerations pertaining to the integration of AI into clinical practice are also discussed, emphasizing the importance of maintaining a patient-centered approach. The integration of AI into breast ultrasound holds great promise for improving diagnostic accuracy, enhancing efficiency, and ultimately advancing patient's care. By examining the current state of research and identifying future opportunities, this review aims to contribute to the understanding and utilization of AI in breast ultrasound and encourage further interdisciplinary collaboration to maximize its potential in clinical practice.

Distilling Knowledge From an Ensemble of Vision Transformers for Improved Classification of Breast Ultrasound

RATIONALE AND OBJECTIVES

To develop a deep learning model for the automated classification of breast ultrasound images as benign or malignant. More specifically, the application of vision transformers, ensemble learning, and knowledge distillation is explored for breast ultrasound classification.

MATERIALS AND METHODS

Single view, B-mode ultrasound images were curated from the publicly available Breast Ultrasound Image (BUSI) dataset, which has categorical ground truth labels (benign vs malignant) assigned by radiologists and malignant cases confirmed by biopsy. The performance of vision transformers (ViT) is compared to convolutional neural networks (CNN), followed by a comparison between supervised, self-supervised, and randomly initialized ViT. Subsequently, the ensemble of 10 independently trained ViT, where the ensemble model is the unweighted average of the output of each individual model is compared to the performance of each ViT alone. Finally, we train a single ViT to emulate the ensembled ViT using knowledge distillation.

RESULTS

On this dataset that was trained using five-fold cross validation, ViT outperforms CNN, while self-supervised ViT outperform supervised and randomly initialized ViT. The ensemble model achieves an area under the receiver operating characteristics curve (AuROC) and area under the precision recall curve (AuPRC) of 0.977 and 0.965 on the test set, outperforming the average AuROC and AuPRC of the independently trained ViTs (0.958 ± 0.05 and 0.931 ± 0.016). The distilled ViT achieves an AuROC and AuPRC of 0.972 and 0.960.

CONCLUSION

Both transfer learning and ensemble learning can each offer increased performance independently and can be sequentially combined to collectively improve the performance of the final model. Furthermore, a single vision transformer can be trained to match the performance of an ensemble of a set of vision transformers using knowledge distillation.

Artificial intelligence in breast imaging: potentials and challenges

Breast cancer, which is the most common type of malignant tumor among humans, is a leading cause of death in females. Standard treatment strategies, including neoadjuvant chemotherapy, surgery, postoperative chemotherapy, targeted therapy, endocrine therapy, and radiotherapy, are tailored for individual patients. Such personalized therapies have tremendously reduced the threat of breast cancer in females. Furthermore, early imaging screening plays an important role in reducing the treatment cycle and improving breast cancer prognosis. The recent innovative revolution in artificial intelligence (AI) has aided radiologists in the early and accurate diagnosis of breast cancer. In this review, we introduce the necessity of incorporating AI into breast imaging and the applications of AI in mammography, ultrasonography, magnetic resonance imaging, and positron emission tomography/computed tomography based on published articles since 1994. Moreover, the challenges of AI in breast imaging are discussed.

Prospective assessment of breast lesions AI classification model based on ultrasound dynamic videos and ACR BI-RADS characteristics

Introduction

AI-assisted ultrasound diagnosis is considered a fast and accurate new method that can reduce the subjective and experience-dependent nature of handheld ultrasound. In order to meet clinical diagnostic needs better, we first proposed a breast lesions AI classification model based on ultrasound dynamic videos and ACR BI-RADS characteristics (hereafter, Auto BI-RADS). In this study, we prospectively verify its performance.

Methods

In this study, the model development was based on retrospective data including 480 ultrasound dynamic videos equivalent to 18122 static images of pathologically proven breast lesions from 420 patients. A total of 292 breast lesions ultrasound dynamic videos from the internal and external hospital were prospectively tested by Auto BI-RADS. The performance of Auto BI-RADS was compared with both experienced and junior radiologists using the DeLong method, Kappa test, and McNemar test.

Results

The Auto BI-RADS achieved an accuracy, sensitivity, and specificity of 0.87, 0.93, and 0.81, respectively. The consistency of the BI-RADS category between Auto BI-RADS and the experienced group (Kappa:0.82) was higher than that of the juniors (Kappa:0.60). The consistency rates between Auto BI-RADS and the experienced group were higher than those between Auto BI-RADS and the junior group for shape (93% vs. 80%; P = .01), orientation (90% vs. 84%; P = .02), margin (84% vs. 71%; P = .01), echo pattern (69% vs. 56%; P = .001) and posterior features (76% vs. 71%; P = .0046), While the difference of calcification was not significantly different.

Discussion

In this study, we aimed to prospectively verify a novel AI tool based on ultrasound dynamic videos and ACR BI-RADS characteristics. The prospective assessment suggested that the AI tool not only meets the clinical needs better but also reaches the diagnostic efficiency of experienced radiologists.

Visualizing tactile feedback: an overview of current technologies with a focus on ultrasound elastography

Tissue elasticity remains an essential biomarker of health and is indicative of irregularities such as tumors or infection. The timely detection of such abnormalities is crucial for the prevention of disease progression and complications that arise from late-stage illnesses. However, at both the bedside and the operating table, there is a distinct lack of tactile feedback for deep-seated tissue. As surgical techniques advance toward remote or minimally invasive options to reduce infection risk and hasten healing time, surgeons lose the ability to manually palpate tissue. Furthermore, palpation of deep structures results in decreased accuracy, with the additional barrier of needing years of experience for adequate confidence of diagnoses. This review delves into the current modalities used to fulfill the clinical need of quantifying physical touch. It covers research efforts involving tactile sensing for remote or minimally invasive surgeries, as well as the potential of ultrasound elastography to further this field with non-invasive real-time imaging of the organ's biomechanical properties. Elastography monitors tissue response to acoustic or mechanical energy and reconstructs an image representative of the elastic profile in the region of interest. This intuitive visualization of tissue elasticity surpasses the tactile information provided by sensors currently used to augment or supplement manual palpation. Focusing on common ultrasound elastography modalities, we evaluate various sensing mechanisms used for measuring tactile information and describe their emerging use in clinical settings where palpation is insufficient or restricted. With the ongoing advancements in ultrasound technology, particularly the emergence of micromachined ultrasound transducers, these devices hold great potential in facilitating early detection of tissue abnormalities and providing an objective measure of patient health.

Artificial intelligence in breast imaging: Current situation and clinical challenges

Breast cancer ranks among the most prevalent malignant tumours and is the primary contributor to cancer-related deaths in women. Breast imaging is essential for screening, diagnosis, and therapeutic surveillance. With the increasing demand for precision medicine, the heterogeneous nature of breast cancer makes it necessary to deeply mine and rationally utilize the tremendous amount of breast imaging information. With the rapid advancement of computer science, artificial intelligence (AI) has been noted to have great advantages in processing and mining of image information. Therefore, a growing number of scholars have started to focus on and research the utility of AI in breast imaging. Here, an overview of breast imaging databases and recent advances in AI research are provided, the challenges and problems in this field are discussed, and then constructive advice is further provided for ongoing scientific developments from the perspective of the National Natural Science Foundation of China.

Application of artificial intelligence in predicting malignancy risk in breast masses on ultrasound

Objective

To evaluate the results obtained with an artificial intelligence-based software for predicting the risk of malignancy in breast masses from ultrasound images.

Materials and Methods

This was a retrospective, single-center study evaluating 555 breast masses submitted to percutaneous biopsy at a cancer referral center. Ultrasonographic findings were classified in accordance with the BI-RADS lexicon. The images were analyzed by using Koios DS Breast software and classified as benign, probably benign, low to intermediate suspicion, high suspicion, or probably malignant. The histological classification was considered the reference standard.

Results

The mean age of the patients was 51 years, and the mean mass size was 16 mm. The radiologist evaluation had a sensitivity and specificity of 99.1% and 34.0%, respectively, compared with 98.2% and 39.0%, respectively, for the software evaluation. The positive predictive value for malignancy for the BI-RADS categories was similar between the radiologist and software evaluations. Two false-negative results were identified in the radiologist evaluation, the masses in question being classified as suspicious by the software, whereas four false-negative results were identified in the software evaluation, the masses in question being classified as suspicious by the radiologist.

Conclusion

In our sample, the performance of artificial intelligence-based software was comparable to that of a radiologist.

Ceusia-Breast: computer-aided diagnosis with contrast enhanced ultrasound image analysis for breast lesions

BACKGROUND

In recent years, contrast-enhanced ultrasonography (CEUS) has been used for various applications in breast diagnosis. The superiority of CEUS over conventional B-mode imaging in the ultrasound diagnosis of the breast lesions in clinical practice has been widely confirmed. On the other hand, there have been many proposals for computer-aided diagnosis of breast lesions on B-mode ultrasound images, but few for CEUS. We propose a semi-automatic classification method based on machine learning in CEUS of breast lesions.

METHODS

The proposed method extracts spatial and temporal features from CEUS videos and breast tumors are classified as benign or malignant using linear support vector machines (SVM) with combination of selected optimal features. In the proposed method, tumor regions are extracted using the guidance information specified by the examiners, then morphological and texture features of tumor regions obtained from B-mode and CEUS images and TIC features obtained from CEUS video are extracted. Then, our method uses SVM classifiers to classify breast tumors as benign or malignant. During SVM training, many features are prepared, and useful features are selected. We name our proposed method "Ceucia-Breast" (Contrast Enhanced UltraSound Image Analysis for BREAST lesions).

RESULTS

The experimental results on 119 subjects show that the area under the receiver operating curve, accuracy, precision, and recall are 0.893, 0.816, 0.841 and 0.920, respectively. The classification performance is improved by our method over conventional methods using only B-mode images. In addition, we confirm that the selected features are consistent with the CEUS guidelines for breast tumor diagnosis. Furthermore, we conduct an experiment on the operator dependency of specifying guidance information and find that the intra-operator and inter-operator kappa coefficients are 1.0 and 0.798, respectively.

CONCLUSION

The experimental results show a significant improvement in classification performance compared to conventional classification methods using only B-mode images. We also confirm that the selected features are related to the findings that are considered important in clinical practice. Furthermore, we verify the intra- and inter-examiner correlation in the guidance input for region extraction and confirm that both correlations are in strong agreement.

Application of a deep learning algorithm in the detection of hip fractures

This paper describes the development of a deep learning model for prediction of hip fractures on pelvic radiographs (X-rays). Developed using over 40,000 pelvic radiographs from a single institution, the model demonstrated high sensitivity and specificity when applied to a test set of emergency department radiographs. This study approximates the real-world application of a deep learning fracture detection model by including radiographs with sub-optimal image quality, other non-hip fractures, and metallic implants, which were excluded from prior published work. The study also explores the effect of ethnicity on model performance, as well as the accuracy of visualization algorithm for fracture localization.

Diagnostic accuracy of automated ACR BI-RADS breast density classification using deep convolutional neural networks

OBJECTIVES

High breast density is a well-known risk factor for breast cancer. This study aimed to develop and adapt two (MLO, CC) deep convolutional neural networks (DCNN) for automatic breast density classification on synthetic 2D tomosynthesis reconstructions.

METHODS

In total, 4605 synthetic 2D images (1665 patients, age: 57 ± 37 years) were labeled according to the ACR (American College of Radiology) density (A-D). Two DCNNs with 11 convolutional layers and 3 fully connected layers each, were trained with 70% of the data, whereas 20% was used for validation. The remaining 10% were used as a separate test dataset with 460 images (380 patients). All mammograms in the test dataset were read blinded by two radiologists (reader 1 with two and reader 2 with 11 years of dedicated mammographic experience in breast imaging), and the consensus was formed as the reference standard. The inter- and intra-reader reliabilities were assessed by calculating Cohen's kappa coefficients, and diagnostic accuracy measures of automated classification were evaluated.

RESULTS

The two models for MLO and CC projections had a mean sensitivity of 80.4% (95%-CI 72.2-86.9), a specificity of 89.3% (95%-CI 85.4-92.3), and an accuracy of 89.6% (95%-CI 88.1-90.9) in the differentiation between ACR A/B and ACR C/D. DCNN versus human and inter-reader agreement were both "substantial" (Cohen's kappa: 0.61 versus 0.63).

CONCLUSION

The DCNN allows accurate, standardized, and observer-independent classification of breast density based on the ACR BI-RADS system.

KEY POINTS

• A DCNN performs on par with human experts in breast density assessment for synthetic 2D tomosynthesis reconstructions. • The proposed technique may be useful for accurate, standardized, and observer-independent breast density evaluation of tomosynthesis.

Deep Learning in Different Ultrasound Methods for Breast Cancer, from Diagnosis to Prognosis: Current Trends, Challenges, and an Analysis

Breast cancer is the second-leading cause of mortality among women around the world. Ultrasound (US) is one of the noninvasive imaging modalities used to diagnose breast lesions and monitor the prognosis of cancer patients. It has the highest sensitivity for diagnosing breast masses, but it shows increased false negativity due to its high operator dependency. Underserved areas do not have sufficient US expertise to diagnose breast lesions, resulting in delayed management of breast lesions. Deep learning neural networks may have the potential to facilitate early decision-making by physicians by rapidly yet accurately diagnosing and monitoring their prognosis. This article reviews the recent research trends on neural networks for breast mass ultrasound, including and beyond diagnosis. We discussed original research recently conducted to analyze which modes of ultrasound and which models have been used for which purposes, and where they show the best performance. Our analysis reveals that lesion classification showed the highest performance compared to those used for other purposes. We also found that fewer studies were performed for prognosis than diagnosis. We also discussed the limitations and future directions of ongoing research on neural networks for breast ultrasound.

Automatic and standardized quality assurance of digital mammography and tomosynthesis with deep convolutional neural networks

OBJECTIVES

The aim of this study was to develop and validate a commercially available AI platform for the automatic determination of image quality in mammography and tomosynthesis considering a standardized set of features.

MATERIALS AND METHODS

In this retrospective study, 11,733 mammograms and synthetic 2D reconstructions from tomosynthesis of 4200 patients from two institutions were analyzed by assessing the presence of seven features which impact image quality in regard to breast positioning. Deep learning was applied to train five dCNN models on features detecting the presence of anatomical landmarks and three dCNN models for localization features. The validity of models was assessed by the calculation of the mean squared error in a test dataset and was compared to the reading by experienced radiologists.

RESULTS

Accuracies of the dCNN models ranged between 93.0% for the nipple visualization and 98.5% for the depiction of the pectoralis muscle in the CC view. Calculations based on regression models allow for precise measurements of distances and angles of breast positioning on mammograms and synthetic 2D reconstructions from tomosynthesis. All models showed almost perfect agreement compared to human reading with Cohen's kappa scores above 0.9.

CONCLUSIONS

An AI-based quality assessment system using a dCNN allows for precise, consistent and observer-independent rating of digital mammography and synthetic 2D reconstructions from tomosynthesis. Automation and standardization of quality assessment enable real-time feedback to technicians and radiologists that shall reduce a number of inadequate examinations according to PGMI (Perfect, Good, Moderate, Inadequate) criteria, reduce a number of recalls and provide a dependable training platform for inexperienced technicians.

Ultrasound-based deep learning in the establishment of a breast lesion risk stratification system: a multicenter study

OBJECTIVES

To establish a breast lesion risk stratification system using ultrasound images to predict breast malignancy and assess Breast Imaging Reporting and Data System (BI-RADS) categories simultaneously.

METHODS

This multicenter study prospectively collected a dataset of ultrasound images for 5012 patients at thirty-two hospitals from December 2018 to December 2020. A deep learning (DL) model was developed to conduct binary categorization (benign and malignant) and BI-RADS categories (2, 3, 4a, 4b, 4c, and 5) simultaneously. The training set of 4212 patients and the internal test set of 416 patients were from thirty hospitals. The remaining two hospitals with 384 patients were used as an external test set. Three experienced radiologists performed a reader study on 324 patients randomly selected from the test sets. We compared the performance of the DL model with that of three radiologists and the consensus of the three radiologists.

RESULTS

In the external test set, the DL model achieved areas under the receiver operating characteristic curve (AUCs) of 0.980 and 0.945 for the binary categorization and six-way categorizations, respectively. In the reader study set, the DL BI-RADS categories achieved a similar AUC (0.901 vs. 0.933, p = 0.0632), sensitivity (90.98% vs. 95.90%, p = 0.1094), and accuracy (83.33% vs. 79.01%, p = 0.0541), but higher specificity (78.71% vs. 68.81%, p = 0.0012) than those of the consensus of the three radiologists.

CONCLUSIONS

The DL model performed well in distinguishing benign from malignant breast lesions and yielded outcomes similar to experienced radiologists. This indicates the potential applicability of the DL model in clinical diagnosis.

KEY POINTS

• The DL model can achieve binary categorization for benign and malignant breast lesions and six-way BI-RADS categorizations for categories 2, 3, 4a, 4b, 4c, and 5, simultaneously. • The DL model showed acceptable agreement with radiologists for the classification of breast lesions. • The DL model performed well in distinguishing benign from malignant breast lesions and had promise in helping reduce unnecessary biopsies of BI-RADS 4a lesions.

Artificial intelligence-aided method to detect uterine fibroids in ultrasound images: a retrospective study

We explored a new artificial intelligence-assisted method to assist junior ultrasonographers in improving the diagnostic performance of uterine fibroids and further compared it with senior ultrasonographers to confirm the effectiveness and feasibility of the artificial intelligence method. In this retrospective study, we collected a total of 3870 ultrasound images from 667 patients with a mean age of 42.45 years ± 6.23 [SD] for those who received a pathologically confirmed diagnosis of uterine fibroids and 570 women with a mean age of 39.24 years ± 5.32 [SD] without uterine lesions from Shunde Hospital of Southern Medical University between 2015 and 2020. The DCNN model was trained and developed on the training dataset (2706 images) and internal validation dataset (676 images). To evaluate the performance of the model on the external validation dataset (488 images), we assessed the diagnostic performance of the DCNN with ultrasonographers possessing different levels of seniority. The DCNN model aided the junior ultrasonographers (Averaged) in diagnosing uterine fibroids with higher accuracy (94.72% vs. 86.63%, P < 0.001), sensitivity (92.82% vs. 83.21%, P = 0.001), specificity (97.05% vs. 90.80%, P = 0.009), positive predictive value (97.45% vs. 91.68%, P = 0.007), and negative predictive value (91.73% vs. 81.61%, P = 0.001) than they achieved alone. Their ability was comparable to that of senior ultrasonographers (Averaged) in terms of accuracy (94.72% vs. 95.24%, P = 0.66), sensitivity (92.82% vs. 93.66%, P = 0.73), specificity (97.05% vs. 97.16%, P = 0.79), positive predictive value (97.45% vs. 97.57%, P = 0.77), and negative predictive value (91.73% vs. 92.63%, P = 0.75). The DCNN-assisted strategy can considerably improve the uterine fibroid diagnosis performance of junior ultrasonographers to make them more comparable to senior ultrasonographers.

Recent Advances in Ultrasound Breast Imaging: From Industry to Clinical Practice

Breast ultrasound (US) has undergone dramatic technological improvement through recent decades, moving from a low spatial resolution, grayscale-limited technique to a highly performing, multiparametric modality. In this review, we first focus on the spectrum of technical tools that have become commercially available, including new microvasculature imaging modalities, high-frequency transducers, extended field-of-view scanning, elastography, contrast-enhanced US, MicroPure, 3D US, automated US, S-Detect, nomograms, images fusion, and virtual navigation. In the subsequent section, we discuss the broadened current application of US in breast clinical scenarios, distinguishing among primary US, complementary US, and second-look US. Finally, we mention the still ongoing limitations and the challenging aspects of breast US.

Evolution of research trends in artificial intelligence for breast cancer diagnosis and prognosis over the past two decades: A bibliometric analysis

Objective

In recent years, among the available tools, the concurrent application of Artificial Intelligence (AI) has improved the diagnostic performance of breast cancer screening. In this context, the present study intends to provide a comprehensive overview of the evolution of AI for breast cancer diagnosis and prognosis research using bibliometric analysis.

Methodology

Therefore, in the present study, relevant peer-reviewed research articles published from 2000 to 2021 were downloaded from the Scopus and Web of Science (WOS) databases and later quantitatively analyzed and visualized using Bibliometrix (R package). Finally, open challenges areas were identified for future research work.

Results

The present study revealed that the number of literature studies published in AI for breast cancer detection and survival prediction has increased from 12 to 546 between the years 2000 to 2021. The United States of America (USA), the Republic of China, and India are the most productive publication-wise in this field. Furthermore, the USA leads in terms of the total citations; however, hungry and Holland take the lead positions in average citations per year. Wang J is the most productive author, and Zhan J is the most relevant author in this field. Stanford University in the USA is the most relevant affiliation by the number of published articles. The top 10 most relevant sources are Q1 journals with PLOS ONE and computer in Biology and Medicine are the leading journals in this field. The most trending topics related to our study, transfer learning and deep learning, were identified.

Conclusion

The present findings provide insight and research directions for policymakers and academic researchers for future collaboration and research in AI for breast cancer patients.

Artificial Intelligence in Breast Ultrasound: The Emerging Future of Modern Medicine

In today's world, progressively enormous popularity prevails around artificial intelligence (AI). AI is gaining popularity in the identification of various images. Therefore, it has been widely used in the ultrasound of the breast. Furthermore, AI can perform a quantitative evaluation, which further helps maintain the diagnosis's accuracy. Moreover, breast cancer is the most common cancer in women, posing a severe threat to women's health. Hence, its early detection is usually associated with a patient's prognosis. As a result, using AI in breast cancer screening and detection is highly crucial. The concept of AI in the perspective of breast ultrasound has been highlighted in this brief review article. It tends to focus on early AI, i.e., traditional machine learning and deep learning algorithms. Also, the use of AI in ultrasound and the use of it in mammography, magnetic resonance imaging, nuclear medicine imaging, and classification of breast lesions is broadly explained, along with the challenges faced in bringing AI into daily practice.

Deep learning applied to breast imaging classification and segmentation with human expert intervention

PURPOSE

Automatic classification and segmentation of tumors in breast ultrasound images enables better diagnosis and planning treatment strategies for breast cancer patients.

METHODS

We collected 953 breast ultrasound images from two open-source datasets and classified them with help of an expert radiologist according to BI-RADS criteria. The data was split into normal, benign and malignant classes. We then used machine learning to develop classification and segmentation algorithms.

RESULTS

We found 3.92% of the images across the open-source datasets had erroneous classifications. Post-radiologist intervention, three algorithms were developed based on the classification categories. Classification algorithms distinguished images with healthy breast tissue from those with abnormal tissue with 96% accuracy, and distinguished benign from malignant images with 85% accuracy. Both algorithms generated robust F1 and AUROC metrics. Finally, the masses within images were segmented with an 80.31% DICE score.

CONCLUSIONS

Our work illustrates the potential of deep learning algorithms to improve the accuracy of breast ultrasound assessments and to facilitate automated assessments.

Ultrasound radiomics in personalized breast management: Current status and future prospects

Breast cancer is the most common cancer in women worldwide. Providing accurate and efficient diagnosis, risk stratification and timely adjustment of treatment strategies are essential steps in achieving precision medicine before, during and after treatment. Radiomics provides image information that cannot be recognized by the naked eye through deep mining of medical images. Several studies have shown that radiomics, as a second reader of medical images, can assist physicians not only in the detection and diagnosis of breast lesions but also in the assessment of risk stratification and prediction of treatment response. Recently, more and more studies have focused on the application of ultrasound radiomics in breast management. We summarized recent research advances in ultrasound radiomics for the diagnosis of benign and malignant breast lesions, prediction of molecular subtype, assessment of lymph node status, prediction of neoadjuvant chemotherapy response, and prediction of survival. In addition, we discuss the current challenges and future prospects of ultrasound radiomics.

A novel bi-modal extended Huber loss function based refined mask RCNN approach for automatic multi instance detection and localization of breast cancer

Breast cancer is an extremely aggressive cancer in women. Its abnormalities can be observed in the form of masses, calcification and lumps. In order to reduce the mortality rate of women its detection is needed at an early stage. The present paper proposes a novel bi-modal extended Huber loss function based refined mask regional convolutional neural network for automatic multi-instance detection and localization of breast cancer. To refine and increase the efficacy of the proposed method three changes are casted. First, a pre-processing step is performed for mammogram and ultrasound breast images. Second, the features of the region proposal network are separately mapped for accurate region of interest. Third, to reduce overfitting and fast convergence, an extended Huber loss function is used at the place of Smooth L1( x) in boundary loss. To extend the functionality of Huber loss, the delta parameter is automated by the aid of median absolute deviation with grid search algorithm. It provides the best optimum value of delta instead of user-based value. The proposed method is compared with pre-existing methods in terms of accuracy, true positive rate, true negative rate, precision, F-score, balanced classification rate, Youden’s index, Jaccard Index and dice coefficient on CBIS-DDSM and ultrasound database. The experimental result shows that the proposed method is a better suited approach for multi-instance detection, localization and classification of breast cancer. It can be used as a diagnostic medium that helps in clinical purposes and leads to a precise diagnosis of breast cancer abnormalities.

BI-RADS-Based Classification of Mammographic Soft Tissue Opacities Using a Deep Convolutional Neural Network

The aim of this study was to investigate the potential of a machine learning algorithm to classify breast cancer solely by the presence of soft tissue opacities in mammograms, independent of other morphological features, using a deep convolutional neural network (dCNN). Soft tissue opacities were classified based on their radiological appearance using the ACR BI-RADS atlas. We included 1744 mammograms from 438 patients to create 7242 icons by manual labeling. The icons were sorted into three categories: “no opacities” (BI-RADS 1), “probably benign opacities” (BI-RADS 2/3) and “suspicious opacities” (BI-RADS 4/5). A dCNN was trained (70% of data), validated (20%) and finally tested (10%). A sliding window approach was applied to create colored probability maps for visual impression. Diagnostic performance of the dCNN was compared to human readout by experienced radiologists on a “real-world” dataset. The accuracies of the models on the test dataset ranged between 73.8% and 89.8%. Compared to human readout, our dCNN achieved a higher specificity (100%, 95% CI: 85.4–100%; reader 1: 86.2%, 95% CI: 67.4–95.5%; reader 2: 79.3%, 95% CI: 59.7–91.3%), and the sensitivity (84.0%, 95% CI: 63.9–95.5%) was lower than that of human readers (reader 1:88.0%, 95% CI: 67.4–95.4%; reader 2:88.0%, 95% CI: 67.7–96.8%). In conclusion, a dCNN can be used for the automatic detection as well as the standardized and observer-independent classification of soft tissue opacities in mammograms independent of the presence of microcalcifications. Human decision making in accordance with the BI-RADS classification can be mimicked by artificial intelligence.

Recent Advances in Machine Learning Applied to Ultrasound Imaging

Machine learning (ML) methods are pervading an increasing number of fields of application because of their capacity to effectively solve a wide variety of challenging problems. The employment of ML techniques in ultrasound imaging applications started several years ago but the scientific interest in this issue has increased exponentially in the last few years. The present work reviews the most recent (2019 onwards) implementations of machine learning techniques for two of the most popular ultrasound imaging fields, medical diagnostics and non-destructive evaluation. The former, which covers the major part of the review, was analyzed by classifying studies according to the human organ investigated and the methodology (e.g., detection, segmentation, and/or classification) adopted, while for the latter, some solutions to the detection/classification of material defects or particular patterns are reported. Finally, the main merits of machine learning that emerged from the study analysis are summarized and discussed.

Development and Validation of a Deep Learning Model to Screen for Trisomy 21 During the First Trimester From Nuchal Ultrasonographic Images

IMPORTANCE

Accurate screening of trisomy 21 in the first trimester can provide an early opportunity for decision-making regarding reproductive choices.

OBJECTIVE

To develop and validate a deep learning model for screening fetuses with trisomy 21 based on ultrasonographic images.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study used data from all available cases and controls enrolled at 2 hospitals in China between January 2009 and September 2020. Two-dimensional images of the midsagittal plane of the fetal face in singleton pregnancies with gestational age more than 11 weeks and less than 14 weeks were examined. Observers were blinded to subjective fetus nuchal translucency (NT) marker measurements. A convolutional neural network was developed to construct a deep learning model. Data augmentation was applied to generate more data. Different groups were randomly selected as training and validation sets to assess the robustness of the deep learning model. The fetal NT was shown and measured. Each detection of trisomy 21 was confirmed by chorionic villus sampling or amniocentesis. Data were analyzed from March 1, 2021, to January 3, 2022.

MAIN OUTCOMES AND MEASURES

The primary outcome was detection of fetuses with trisomy 21. The receiver operating characteristic curve, metrics of accuracy, area under the curve (AUC), sensitivity, and specificity were used for model performance evaluation.

RESULTS

A total of 822 case and control participants (mean [SD] age, 31.9 [4.6] years) were enrolled in the study, including 550 participants (mean [SD] age, 31.7 [4.7] years) in the training set and 272 participants (mean [SD] age, 32.3 [4.7] years) in the validation set. The deep learning model showed good performance for trisomy 21 screening in the training (AUC, 0.98; 95% CI, 0.97-0.99) and validation (AUC, 0.95; 95% CI, 0.93-0.98) sets. The deep learning model had better detective performance for fetuses with trisomy 21 than the model with NT marker and maternal age (training: AUC, 0.82; 95% CI, 0.77-0.86; validation: AUC, 0.73; 95% CI, 0.66-0.80).

CONCLUSIONS AND RELEVANCE

These findings suggest that this deep learning model accurately screened fetuses with trisomy 21, which indicates that the model is a potential tool to facilitate universal primary screening for trisomy 21.

External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review

Purpose

To assess generalizability of published deep learning (DL) algorithms for radiologic diagnosis.

Materials and Methods

In this systematic review, the PubMed database was searched for peer-reviewed studies of DL algorithms for image-based radiologic diagnosis that included external validation, published from January 1, 2015, through April 1, 2021. Studies using nonimaging features or incorporating non-DL methods for feature extraction or classification were excluded. Two reviewers independently evaluated studies for inclusion, and any discrepancies were resolved by consensus. Internal and external performance measures and pertinent study characteristics were extracted, and relationships among these data were examined using nonparametric statistics.

Results

Eighty-three studies reporting 86 algorithms were included. The vast majority (70 of 86, 81%) reported at least some decrease in external performance compared with internal performance, with nearly half (42 of 86, 49%) reporting at least a modest decrease (≥0.05 on the unit scale) and nearly a quarter (21 of 86, 24%) reporting a substantial decrease (≥0.10 on the unit scale). No study characteristics were found to be associated with the difference between internal and external performance.

Conclusion

Among published external validation studies of DL algorithms for image-based radiologic diagnosis, the vast majority demonstrated diminished algorithm performance on the external dataset, with some reporting a substantial performance decrease.Keywords: Meta-Analysis, Computer Applications-Detection/Diagnosis, Neural Networks, Computer Applications-General (Informatics), Epidemiology, Technology Assessment, Diagnosis, Informatics Supplemental material is available for this article. © RSNA, 2022.

Discrimination of Breast Cancer Based on Ultrasound Images and Convolutional Neural Network

The aim of our study was to establish an artificial intelligence tool for the diagnosis of breast disease base on ultrasound (US) images. A deep learning algorithm Efficient-Det assisted US diagnosis method was developed to determine breast suspicious lesions as benign, malignant, or normal. Totally 1181 US images from 487 patients of our hospital and 694 publicly accessible images were employed for modeling, including 558 benign images, 370 malignant images, and 253 normal tissue images. The actual diagnosis results for the patients were determined by the biopsy or surgery. Efficient-Det was first retrained using an exclusive public breast cancer US dataset with transfer learning techniques. A blind test set consisting of 50 benign, 50 malignant, and 50 normal tissue images was randomly picked from the patients' images as the independent test set to test its searching ability on suspicious tumor regions. Furthermore, the confusion matrix and classification accuracy were employed as evaluation metrics to select the optimal classification models. Efficient-Det has demonstrated remarkable progress in general image recognition tasks with specific advantages of locating and identifying tumor areas simultaneously. Compared to the manual method (mean accuracy: 95.3% and 60 s per image) and traditional feature engineering method (mean accuracy: 90% and 15 s per image), our Efficient-Det has the capability of providing a competitive (mean accuracy: 92.6%) and fast (0.06 s per image) classification result. The deployment of Efficient-Det in our local breast cancer discrimination task exhibits specific applicability within real clinical workflows.

Application of Deep Learning in Breast Cancer Imaging

This review gives an overview of the current state of deep learning research in breast cancer imaging. Breast imaging plays a major role in detecting breast cancer at an earlier stage, as well as monitoring and evaluating breast cancer during treatment. The most commonly used modalities for breast imaging are digital mammography, digital breast tomosynthesis, ultrasound and magnetic resonance imaging. Nuclear medicine imaging techniques are used for detection and classification of axillary lymph nodes and distant staging in breast cancer imaging. All of these techniques are currently digitized, enabling the possibility to implement deep learning (DL), a subset of Artificial intelligence, in breast imaging. DL is nowadays embedded in a plethora of different tasks, such as lesion classification and segmentation, image reconstruction and generation, cancer risk prediction, and prediction and assessment of therapy response. Studies show similar and even better performances of DL algorithms compared to radiologists, although it is clear that large trials are needed, especially for ultrasound and magnetic resonance imaging, to exactly determine the added value of DL in breast cancer imaging. Studies on DL in nuclear medicine techniques are only sparsely available and further research is mandatory. Legal and ethical issues need to be considered before the role of DL can expand to its full potential in clinical breast care practice.

Application of logistic regression and convolutional neural network in prediction and diagnosis of high-risk populations of lung cancer

OBJECTIVES

The early detection, early diagnosis, and early treatment of lung cancer are the best strategies to improve the 5-year survival rate. Logistic regression analysis can be a helpful tool in the early detection of high-risk groups of lung cancer. Convolutional neural network (CNN) could distinguish benign from malignant pulmonary nodules, which is critical for early precise diagnosis and treatment. Here, we developed a risk assessment model of lung cancer and a high-precision classification diagnostic model using these technologies so as to provide a basis for early screening of lung cancer and for intelligent differential diagnosis.

METHODS

A total of 355 lung cancer patients, 444 patients with benign lung disease and 472 healthy people from The First Affiliated Hospital of Zhengzhou University were included in this study. Moreover, the dataset of 607 lung computed tomography images was collected from the above patients. The logistic regression method was employed to screen the high-risk groups of lung cancer, and the CNN model was designed to classify pulmonary nodules into benign or malignant nodules.

RESULTS

The area under the curve of the lung cancer risk assessment model in the training set and the testing set were 0.823 and 0.710, respectively. After finely optimizing the settings of the CNN, the area under the curve could reach 0.984.

CONCLUSIONS

This performance demonstrated that the lung cancer risk assessment model could be used to screen for high-risk individuals with lung cancer and the CNN framework was suitable for the differential diagnosis of pulmonary nodules.

Applied Machine Learning in Spiral Breast-CT: Can We Train a Deep Convolutional Neural Network for Automatic, Standardized and Observer Independent Classification of Breast Density?

The aim of this study was to investigate the potential of a machine learning algorithm to accurately classify parenchymal density in spiral breast-CT (BCT), using a deep convolutional neural network (dCNN). In this retrospectively designed study, 634 examinations of 317 patients were included. After image selection and preparation, 5589 images from 634 different BCT examinations were sorted by a four-level density scale, ranging from A to D, using ACR BI-RADS-like criteria. Subsequently four different dCNN models (differences in optimizer and spatial resolution) were trained (70% of data), validated (20%) and tested on a “real-world” dataset (10%). Moreover, dCNN accuracy was compared to a human readout. The overall performance of the model with lowest resolution of input data was highest, reaching an accuracy on the “real-world” dataset of 85.8%. The intra-class correlation of the dCNN and the two readers was almost perfect (0.92) and kappa values between both readers and the dCNN were substantial (0.71–0.76). Moreover, the diagnostic performance between the readers and the dCNN showed very good correspondence with an AUC of 0.89. Artificial Intelligence in the form of a dCNN can be used for standardized, observer-independent and reliable classification of parenchymal density in a BCT examination.

Incorporating the Breast Imaging Reporting and Data System Lexicon with a Fully Convolutional Network for Malignancy Detection on Breast Ultrasound

In this study, we applied semantic segmentation using a fully convolutional deep learning network to identify characteristics of the Breast Imaging Reporting and Data System (BI-RADS) lexicon from breast ultrasound images to facilitate clinical malignancy tumor classification. Among 378 images (204 benign and 174 malignant images) from 189 patients (102 benign breast tumor patients and 87 malignant patients), we identified seven malignant characteristics related to the BI-RADS lexicon in breast ultrasound. The mean accuracy and mean IU of the semantic segmentation were 32.82% and 28.88, respectively. The weighted intersection over union was 85.35%, and the area under the curve was 89.47%, showing better performance than similar semantic segmentation networks, SegNet and U-Net, in the same dataset. Our results suggest that the utilization of a deep learning network in combination with the BI-RADS lexicon can be an important supplemental tool when using ultrasound to diagnose breast malignancy.

LRSCnet: Local Reference Semantic Code learning for breast tumor classification in ultrasound images

Purpose

This study proposed a novel Local Reference Semantic Code (LRSC) network for automatic breast ultrasound image classification with few labeled data.

Methods

In the proposed network, the local structure extractor is firstly developed to learn the local reference which describes common local characteristics of tumors. After that, a two-stage hierarchical encoder is developed to encode the local structures of lesion into the high-level semantic code. Based on the learned semantic code, the self-matching layer is proposed for the final classification.

Results

In the experiment, the proposed method outperformed traditional classification methods and AUC (Area Under Curve), ACC (Accuracy), Sen (Sensitivity), Spec (Specificity), PPV (Positive Predictive Values), and NPV(Negative Predictive Values) are 0.9540, 0.9776, 0.9629, 0.93, 0.9774 and 0.9090, respectively. In addition, the proposed method also improved matching speed.

Conclusions

LRSC-network is proposed for breast ultrasound images classification with few labeled data. In the proposed network, a two-stage hierarchical encoder is introduced to learn high-level semantic code. The learned code contains more effective high-level classification information and is simpler, leading to better generalization ability.

Decoding the molecular subtypes of breast cancer seen on multimodal ultrasound images using an assembled convolutional neural network model: A prospective and multicentre study

Background

Preoperative determination of breast cancer molecular subtypes facilitates individualized treatment plan-making and improves patient prognosis. We aimed to develop an assembled convolutional neural network (ACNN) model for the preoperative prediction of molecular subtypes using multimodal ultrasound (US) images.

Methods

This multicentre study prospectively evaluated a dataset of greyscale US, colour Doppler flow imaging (CDFI), and shear-wave elastography (SWE) images in 807 patients with 818 breast cancers from November 2016 to February 2021. The St. Gallen molecular subtypes of breast cancer were confirmed by postoperative immunohistochemical examination. The monomodal ACNN model based on greyscale US images, the dual-modal ACNN model based on greyscale US and CDFI images, and the multimodal ACNN model based on greyscale US and CDFI as well as SWE images were constructed in the training cohort. The performances of three ACNN models in predicting four- and five-classification molecular subtypes and identifying triple negative from non-triple negative subtypes were assessed and compared. The performance of the multimodal ACNN was also compared with preoperative core needle biopsy (CNB).

Finding

The performance of the multimodal ACNN model (macroaverage area under the curve [AUC]: 0.89–0.96) was superior to that of the dual-modal ACNN model (macroaverage AUC: 0.81–0.84) and the monomodal ACNN model (macroaverage AUC: 0.73–0.75) in predicting four-classification breast cancer molecular subtypes, which was also better than that of preoperative CNB (AUC: 0.89–0.99 vs. 0.67–0.82, p < 0.05). In addition, the multimodal ACNN model outperformed the other two ACNN models in predicting five-classification molecular subtypes (AUC: 0.87–0.94 vs. 0.78-0.81 vs. 0.71–0.78) and identifying triple negative from non-triple negative breast cancers (AUC: 0.934–0.970 vs. 0.688–0.830 vs. 0.536–0.650, p < 0.05). Moreover, the multimodal ACNN model obtained satisfactory prediction performance for both T1 and non-T1 lesions (AUC: 0.957–0.958 and 0.932–0.985).

Interpretation

The multimodal US-based ACNN model is a potential noninvasive decision-making method for the management of patients with breast cancer in clinical practice.

Funding

This work was supported in part by the National Natural Science Foundation of China (Grants 81725008 and 81927801), Shanghai Municipal Health Commission (Grants 2019LJ21 and SHSLCZDZK03502), and the Science and Technology Commission of Shanghai Municipality (Grants 19441903200, 19DZ2251100, and 21Y11910800).

Development of a tracking error prediction system for the CyberKnife Synchrony Respiratory Tracking System with use of support vector regression

PURPOSE

The accuracy of the CyberKnife Synchrony Respiratory Tracking System is dependent on the breathing pattern of a patient. Therefore, the tracking error in each patient must be determined. Support vector regression (SVR) can be used to easily identify the tracking error in each patient. This study aimed to develop a system with SVR that can predict tracking error according to a patient's respiratory waveform.

METHODS

Datasets of the respiratory waveforms of 93 patients were obtained. The feature variables were variation in respiration amplitude, tumor velocity, and phase shift between tumor and the chest wall, and the target variable was tracking error. A learning model was evaluated with tenfold cross-validation. We documented the difference between the predicted and actual tracking errors and assessed the correlation coefficient and coefficient of determination.

RESULTS

The average difference and maximum difference between the actual and predicted tracking errors were 0.57 ± 0.63 mm and 2.1 mm, respectively. The correlation coefficient and coefficient of determination were 0.86 and 0.74, respectively.

CONCLUSION

We developed a system for obtaining tracking error by using SVR. The accuracy of such a system is clinically useful. Moreover, the system can easily evaluate tracking error. We developed a system that can be used to predict the tracking error of SRTS in the CyberKnife Robotic Radiosurgery System using machine learning. The feature variables were the breathing parameters, and the target variable was the tracking error. We used support vector regression algorithm.

Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams

Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.

AI-enhanced breast imaging: Where are we and where are we heading?

Significant advances in imaging analysis and the development of high-throughput methods that can extract and correlate multiple imaging parameters with different clinical outcomes have led to a new direction in medical research. Radiomics and artificial intelligence (AI) studies are rapidly evolving and have many potential applications in breast imaging, such as breast cancer risk prediction, lesion detection and classification, radiogenomics, and prediction of treatment response and clinical outcomes. AI has been applied to different breast imaging modalities, including mammography, ultrasound, and magnetic resonance imaging, in different clinical scenarios. The application of AI tools in breast imaging has an unprecedented opportunity to better derive clinical value from imaging data and reshape the way we care for our patients. The aim of this study is to review the current knowledge and future applications of AI-enhanced breast imaging in clinical practice.

Artificial Intelligence in Medical Imaging of the Breast

Artificial intelligence (AI) has invaded our daily lives, and in the last decade, there have been very promising applications of AI in the field of medicine, including medical imaging, in vitro diagnosis, intelligent rehabilitation, and prognosis. Breast cancer is one of the common malignant tumors in women and seriously threatens women’s physical and mental health. Early screening for breast cancer via mammography, ultrasound and magnetic resonance imaging (MRI) can significantly improve the prognosis of patients. AI has shown excellent performance in image recognition tasks and has been widely studied in breast cancer screening. This paper introduces the background of AI and its application in breast medical imaging (mammography, ultrasound and MRI), such as in the identification, segmentation and classification of lesions; breast density assessment; and breast cancer risk assessment. In addition, we also discuss the challenges and future perspectives of the application of AI in medical imaging of the breast.

AW3M: An auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound

Recently, more clinicians have realized the diagnostic value of multi-modal ultrasound in breast cancer identification and began to incorporate Doppler imaging and Elastography in the routine examination. However, accurately recognizing patterns of malignancy in different types of sonography requires expertise. Furthermore, an accurate and robust diagnosis requires proper weights of multi-modal information as well as the ability to process missing data in practice. These two aspects are often overlooked by existing computer-aided diagnosis (CAD) approaches. To overcome these challenges, we propose a novel framework (called AW3M) that utilizes four types of sonography (i.e. B-mode, Doppler, Shear-wave Elastography, and Strain Elastography) jointly to assist breast cancer diagnosis. It can extract both modality-specific and modality-invariant features using a multi-stream CNN model equipped with self-supervised consistency loss. Instead of assigning the weights of different streams empirically, AW3M automatically learns the optimal weights using reinforcement learning techniques. Furthermore, we design a light-weight recovery block that can be inserted to a trained model to handle different modality-missing scenarios. Experimental results on a large multi-modal dataset demonstrate that our method can achieve promising performance compared with state-of-the-art methods. The AW3M framework is also tested on another independent B-mode dataset to prove its efficacy in general settings. Results show that the proposed recovery block can learn from the joint distribution of multi-modal features to further boost the classification accuracy given single modality input during the test.