Evaluation of computer-aided diagnosis in breast ultrasonography: Improvement in diagnostic performance of inexperienced radiologists

Impact of an artificial intelligence decision support system among radiologists with different levels of experience in breast ultrasound: A prospective study in a tertiary center

PURPOSE

To assess the impact of an artificial intelligence decision support system (Koios DS) on the diagnostic performance of radiologists with different experience in breast ultrasound and to evaluate its potential to reduce unnecessary biopsies.

METHODS

This observational, prospective, single-centre study included consecutive patients scheduled for ultrasound-guided core-needle biopsy of suspicious breast lesions. Three radiologists with different experience in breast ultrasound (senior breast radiologist: 20 years; junior breast radiologist: 3 years; general radiologist: less than 1 year) independently evaluated the lesions, assigning BI-RADS categories before and after Koios DS application. Biopsy reports served as the reference standard. AUCs and the number of unnecessary biopsies before and after implementing Koios DS were compared using DeLong and McNemar's tests.

RESULTS

A total of 222 patients (median age 58 years, interquartile range 46-72 years) with 226 lesions were included, 89/226 (39.4 %) benign and 137/226 (60.6 %) malignant. The application of Koios DS significantly improved (p < 0.001) the AUC of all radiologists, with a 0.078 AUC Δ for the junior breast radiologist (from 0.786 to 0.864), a 0.062 AUC Δ for the general radiologist (from 0.719 to 0.781), and a 0.045 AUC Δ for the senior breast radiologist (from 0.823 to 0.868). Koios DS would have significantly reduced the number of unnecessary biopsies recommended by the senior breast radiologist (from 41/89 [46.1 %] to 30/89 [33.7 %], p < 0.001) and by the junior breast radiologist (from 46/89 [51.7 %] to 29/89 [32.6 %], p = 0.001).

CONCLUSION

The application of Koios DS improved the radiologists' diagnostic performance, particularly for less experienced ones, and could potentially reduce unnecessary biopsies.

Artificial Intelligence for Breast Ultrasound: AJR Expert Panel Narrative Review

Breast ultrasound is used in a wide variety of clinical scenarios, including both diagnostic and screening applications. Limitations of ultrasound, however, include its low specificity and, for automated breast ultrasound screening, the time necessary to review whole-breast ultrasound images. As of this writing, four AI tools that are approved or cleared by the FDA address these limitations. Current tools, which are intended to provide decision support for lesion classification and/or detection, have been shown to increase specificity among nonspecialists and to decrease interpretation times. Potential future applications include triage of patients with palpable masses in low-resource settings, preoperative prediction of axillary lymph node metastasis, and preoperative prediction of neoadjuvant chemotherapy response. Challenges in the development and clinical deployment of AI for ultrasound include the limited availability of curated training datasets compared with mammography, the high variability in ultrasound image acquisition due to equipment- and operator-related factors (which may limit algorithm generalizability), and the lack of postimplementation evaluation studies. Furthermore, current AI tools for lesion classification were developed based on 2D data, but diagnostic accuracy could potentially be improved if multimodal ultrasound data were used, such as color Doppler, elastography, cine clips, and 3D imaging.

Diagnostic performance of deep learning in ultrasound diagnosis of breast cancer: a systematic review

Deep learning (DL) has been widely investigated in breast ultrasound (US) for distinguishing between benign and malignant breast masses. This systematic review of test diagnosis aims to examine the accuracy of DL, compared to human readers, for the diagnosis of breast cancer in the US under clinical settings. Our literature search included records from databases including PubMed, Embase, Scopus, and Cochrane Library. Test accuracy outcomes were synthesized to compare the diagnostic performance of DL and human readers as well as to evaluate the assistive role of DL to human readers. A total of 16 studies involving 9238 female participants were included. There were no prospective studies comparing the test accuracy of DL versus human readers in clinical workflows. Diagnostic test results varied across the included studies. In 14 studies employing standalone DL systems, DL showed significantly lower sensitivities in 5 studies with comparable specificities and outperformed human readers at higher specificities in another 4 studies; in the remaining studies, DL models and human readers showed equivalent test outcomes. In 12 studies that assessed assistive DL systems, no studies proved the assistive role of DL in the overall diagnostic performance of human readers. Current evidence is insufficient to conclude that DL outperforms human readers or enhances the accuracy of diagnostic breast US in a clinical setting. Standardization of study methodologies is required to improve the reproducibility and generalizability of DL research, which will aid in clinical translation and application.

Artificial intelligence in breast ultrasound: application in clinical practice

Ultrasound (US) is a widely accessible and extensively used tool for breast imaging. It is commonly used as an additional screening tool, especially for women with dense breast tissue. Advances in artificial intelligence (AI) have led to the development of various AI systems that assist radiologists in identifying and diagnosing breast lesions using US. This article provides an overview of the background and supporting evidence for the use of AI in hand held breast US. It discusses the impact of AI on clinical workflow, covering breast cancer detection, diagnosis, prediction of molecular subtypes, evaluation of axillary lymph node status, and response to neoadjuvant chemotherapy. Additionally, the article highlights the potential significance of AI in breast US for low and middle income countries.

Toward AI-supported US Triage of Women with Palpable Breast Lumps in a Low-Resource Setting

Background Most low- and middle-income countries lack access to organized breast cancer screening, and women with lumps may wait months for diagnostic assessment. Purpose To demonstrate that artificial intelligence (AI) software applied to breast US images obtained with low-cost portable equipment and by minimally trained observers could accurately classify palpable breast masses for triage in a low-resource setting. Materials and Methods This prospective multicenter study evaluated participants with at least one palpable mass who were enrolled in a hospital in Jalisco, Mexico, from December 2017 through May 2021. Orthogonal US images were obtained first with portable US with and without calipers of any findings at the site of lump and adjacent tissue. Then women were imaged with standard-of-care (SOC) US with Breast Imaging Reporting and Data System assessments by a radiologist. After exclusions, 758 masses in 300 women were analyzable by AI, with outputs of benign, probably benign, suspicious, and malignant. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were determined. Results The mean patient age ± SD was 50.0 years ± 12.5 (range, 18-92 years) and mean largest lesion diameter was 13 mm ± 8 (range, 2-54 mm). Of 758 masses, 360 (47.5%) were palpable and 56 (7.4%) malignant, including six ductal carcinoma in situ. AI correctly identified 47 or 48 of 49 women (96%-98%) with cancer with either portable US or SOC US images, with AUCs of 0.91 and 0.95, respectively. One circumscribed invasive ductal carcinoma was classified as probably benign with SOC US, ipsilateral to a spiculated invasive ductal carcinoma. Of 251 women with benign masses, 168 (67%) imaged with SOC US were classified as benign or probably benign by AI, as were 96 of 251 masses (38%, P < .001) with portable US. AI performance with images obtained by a radiologist was significantly better than with images obtained by a minimally trained observer. Conclusion AI applied to portable US images of breast masses can accurately identify malignancies. Moderate specificity, which could triage 38%-67% of women with benign masses without tertiary referral, should further improve with AI and observer training with portable US. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Slanetz in this issue.

The Role of a Deep Learning-Based Computer-Aided Diagnosis System and Elastography in Reducing Unnecessary Breast Lesion Biopsies

OBJECTIVES

Ultrasound examination has inter-observer and intra-observer variability and a high false-positive rate. The aim of this study was to evaluate the value of the combined use of a deep learning-based computer-aided diagnosis (CAD) system and ultrasound elastography with conventional ultrasound (US) in increasing specificity and reducing unnecessary breast lesions biopsies.

MATERIALS AND METHODS

Conventional US, CAD system, and strain elastography (SE) were retrospectively performed on 216 breast lesions before biopsy or surgery. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and biopsy rate were compared between conventional US and the combination of conventional US, SE, and CAD system.

RESULTS

Of 216 lesions, 54 were malignant and 162 were benign. The addition of CAD system and SE to conventional US increased the AUC from 0.716 to 0.910 and specificity from 46.9% to 85.8% without a loss in sensitivity while 89.2% (66 of 74) of benign lesions in Breast Imaging Reporting and Data System (BI-RADS) category 4A lesions would avoid unnecessary biopsies.

CONCLUSION

The addition of CAD system and SE to conventional US improved specificity and AUC without loss of sensitivity, and reduced unnecessary biopsies.

Artificial Intelligence for Breast US

US is a widely available, commonly used, and indispensable imaging modality for breast evaluation. It is often the primary imaging modality for the detection and diagnosis of breast cancer in low-resource settings. In addition, it is frequently employed as a supplemental screening tool via either whole breast handheld US or automated breast US among women with dense breasts. In recent years, a variety of artificial intelligence systems have been developed to assist radiologists with the detection and diagnosis of breast lesions on US. This article reviews the background and evidence supporting the use of artificial intelligence tools for breast US, describes implementation strategies and impact on clinical workflow, and discusses potential emerging roles and future directions.

Prediction of the Malignancy of a Breast Lesion Detected on Breast Ultrasound: Radiomics Applied to Clinical Practice

The study aimed to evaluate the performance of radiomics features and one ultrasound CAD (computer-aided diagnosis) in the prediction of the malignancy of a breast lesion detected with ultrasound and to develop a nomogram incorporating radiomic score and available information on CAD performance, conventional Breast Imaging Reporting and Data System evaluation (BI-RADS), and clinical information. Data on 365 breast lesions referred for breast US with subsequent histologic analysis between January 2020 and March 2022 were retrospectively collected. Patients were randomly divided into a training group (n = 255) and a validation test group (n = 110). A radiomics score was generated from the US image. The CAD was performed in a subgroup of 209 cases. The radiomics score included seven radiomics features selected with the LASSO logistic regression model. The multivariable logistic model incorporating CAD performance, BI-RADS evaluation, clinical information, and radiomic score as covariates showed promising results in the prediction of the malignancy of breast lesions: Area under the receiver operating characteristic curve, [AUC]: 0.914; 95% Confidence Interval, [CI]: 0.876-0.951. A nomogram was developed based on these results for possible future applications in clinical practice.

Development and External Validation of a Simple-To-Use Dynamic Nomogram for Predicting Breast Malignancy Based on Ultrasound Morphometric Features: A Retrospective Multicenter Study

Background

With advances in high-throughput computational mining techniques, various quantitative predictive models that are based on ultrasound have been developed. However, the lack of reproducibility and interpretability have hampered clinical use. In this study, we aimed at developing and validating an interpretable and simple-to-use US nomogram that is based on quantitative morphometric features for the prediction of breast malignancy.

Methods

Successive 917 patients with histologically confirmed breast lesions were included in this retrospective multicentric study and assigned to one training cohort and two external validation cohorts. Morphometric features were extracted from grayscale US images. After feature selection and validation of regression assumptions, a dynamic nomogram with a web-based calculator was developed. The performance of the nomogram was assessed with respect to calibration, discrimination, and clinical usefulness.

Results

Through feature selection, three morphometric features were identified as being the most optimal for predicting malignancy, and all regression assumptions of the prediction model were met. Combining all these predictors, the nomogram demonstrated a good discriminative performance in the training cohort and in the two external validation cohorts with AUCs of 0.885, 0.907, and 0.927, respectively. In addition, calibration and decision curves analyses showed good calibration and clinical usefulness.

Conclusions

By incorporating US morphometric features, we constructed an interpretable and easy-to-use dynamic nomogram for quantifying the probability of breast malignancy. The developed nomogram has good generalization abilities, which may fit into clinical practice and serve as a potential tool to guide personalized treatment. Our findings show that quantitative morphometric features from different ultrasound machines and systems can be used as imaging surrogate biomarkers for the development of robust and reproducible quantitative ultrasound dynamic models in breast cancer research.