Nonpalpable breast lesions: impact of a second-opinion review at a breast unit on BI-RADS classification

BI-RADS Category Assignments by GPT-3.5, GPT-4, and Google Bard: A Multilanguage Study

Background The performance of publicly available large language models (LLMs) remains unclear for complex clinical tasks. Purpose To evaluate the agreement between human readers and LLMs for Breast Imaging Reporting and Data System (BI-RADS) categories assigned based on breast imaging reports written in three languages and to assess the impact of discordant category assignments on clinical management. Materials and Methods This retrospective study included reports for women who underwent MRI, mammography, and/or US for breast cancer screening or diagnostic purposes at three referral centers. Reports with findings categorized as BI-RADS 1-5 and written in Italian, English, or Dutch were collected between January 2000 and October 2023. Board-certified breast radiologists and the LLMs GPT-3.5 and GPT-4 (OpenAI) and Bard, now called Gemini (Google), assigned BI-RADS categories using only the findings described by the original radiologists. Agreement between human readers and LLMs for BI-RADS categories was assessed using the Gwet agreement coefficient (AC1 value). Frequencies were calculated for changes in BI-RADS category assignments that would affect clinical management (ie, BI-RADS 0 vs BI-RADS 1 or 2 vs BI-RADS 3 vs BI-RADS 4 or 5) and compared using the McNemar test. Results Across 2400 reports, agreement between the original and reviewing radiologists was almost perfect (AC1 = 0.91), while agreement between the original radiologists and GPT-4, GPT-3.5, and Bard was moderate (AC1 = 0.52, 0.48, and 0.42, respectively). Across human readers and LLMs, differences were observed in the frequency of BI-RADS category upgrades or downgrades that would result in changed clinical management (118 of 2400 [4.9%] for human readers, 611 of 2400 [25.5%] for Bard, 573 of 2400 [23.9%] for GPT-3.5, and 435 of 2400 [18.1%] for GPT-4; P < .001) and that would negatively impact clinical management (37 of 2400 [1.5%] for human readers, 435 of 2400 [18.1%] for Bard, 344 of 2400 [14.3%] for GPT-3.5, and 255 of 2400 [10.6%] for GPT-4; P < .001). Conclusion LLMs achieved moderate agreement with human reader-assigned BI-RADS categories across reports written in three languages but also yielded a high percentage of discordant BI-RADS categories that would negatively impact clinical management. © RSNA, 2024 Supplemental material is available for this article.

Diagnostic value of shear wave elastography combined with super microvascular imaging for BI-RADS 3-5 nodules

Background

To investigate the diagnostic value of shear wave elastography (SWE) and super microvascular imaging (SMI) integrated with the traditional ultrasound breast imaging reporting and data system (BI-RADS) classification in differentiating between benign and malignant breast nodules.

Methods

For analysis, 88 patients with 110 breast nodules assessed as BI-RADS 3-5 by conventional ultrasound were selected. SWE and SMI evaluations were conducted separately, and all nodules were verified as benign or malignant ones by pathology. Receiver operating characteristic (ROC) curves were plotted after obtaining quantitative parameters of different shear waves of nodules, including maximum (Emax), mean (Emean), minimum (Emin) Young's modulus, modulus standard deviation (SD), and modulus ratio (Eratio). The best cut-off value, specificity, sensitivity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) for diagnosing malignant nodules employing Emax were obtained, and the diagnostic value of combining Emax and BI-RADS classification was compared. SMI graded nodule based on the Alder blood flow grading standard, whereas the BI-RADS classification was based on microvascular morphology. We assessed the diagnostic value of SMI for breast nodules and investigated the diagnostic efficacy of SWE combined with SMI in differentiating benign and malignant breast nodules with BI-RADS classification 3-5.

Results

The adjusted the BI-RADS classification using SMI and SWE technologies promoted the sensitivity, specificity, and accuracy of discriminating benign and malignant breast nodules (P < 0.05). The combination of traditional ultrasound BI-RADS classification with SWE and SMI technologies offered high sensitivity, specificity, accuracy, PPV, and NPV for identifying benign and malignant breast lesions. Moreover, combining SWE and SMI technologies with the adjusted BI-RADS classificationhad the best diagnostic efficacy for distinguishing benign and malignant breast nodules with BI-RADS 3-5.

Conclusion

The combination of SWE and SMI with the adjusted BI-RADS classification is a promising diagnostic method for differentiating benign and malignant breast nodules.

King Abdulaziz University Breast Cancer Mammogram Dataset (KAU-BCMD)

The current era is characterized by the rapidly increasing use of computer-aided diagnosis (CAD) systems in the medical field. These systems need a variety of datasets to help develop, evaluate, and compare their performances fairly. Physicians indicated that breast anatomy, especially dense ones, and the probability of breast cancer and tumor development, vary highly depending on race. Researchers reported that breast cancer risk factors are related to culture and society. Thus, there is a massive need for a local dataset representing breast cancer in our region to help develop and evaluate automatic breast cancer CAD systems. This paper presents a public mammogram dataset called King Abdulaziz University Breast Cancer Mammogram Dataset (KAU-BCMD) version 1. To our knowledge, KAU-BCMD is the first dataset in Saudi Arabia that deals with a large number of mammogram scans. The dataset was collected from the Sheikh Mohammed Hussein Al-Amoudi Center of Excellence in Breast Cancer at King Abdulaziz University. It contains 1416 cases. Each case has two views for both the right and left breasts, resulting in 5662 images based on the breast imaging reporting and data system. It also contains 205 ultrasound cases corresponding to a part of the mammogram cases, with 405 images as a total. The dataset was annotated and reviewed by three different radiologists. Our dataset is a promising dataset that contains different imaging modalities for breast cancer with different cancer grades for Saudi women.