Interobserver variability in upgraded and non-upgraded BI-RADS 3 lesions

The Impact of Mammographic, Radiologist, and Patient Factors on the Likelihood of Probably Benign (BI-RADS 3) Assessment at Diagnostic Mammography

OBJECTIVE

To evaluate the association of mammographic, radiologist, and patient factors on BI-RADS 3 assessment at diagnostic mammography in patients recalled from screening mammography.

METHODS

This Institutional Review Board-approved retrospective study of consecutive unique diagnostic mammography examinations in asymptomatic patients recalled from screening mammography March 5, 2014, to December 31, 2019, was conducted in a single large United States health care institution. Mammographic features (mass, calcification, distortion, asymmetry), breast density, prior examination, and BI-RADS assessment were extracted from reports by natural language processing. Patient age, race, and ethnicity were extracted from the electronic health record. Radiologist years in practice, recall rate, and number of interpreted diagnostic mammograms were calculated. A mixed effect logistic regression model evaluated factors associated with likelihood of BI-RADS 3 compared with other BI-RADS assessments.

RESULTS

A total of 12 080 diagnostic mammography examinations were performed during the study period, yielding 2010 (16.6%) BI-RADS 3 and 10 070 (83.4%) other BI-RADS assessments. Asymmetry (odds ratio [OR] = 6.49, P <.001) and calcification (OR = 5.59, P <.001) were associated with increased likelihood of BI-RADS 3 assessment; distortion (OR = 0.20, P <.001), dense breast parenchyma (OR = 0.82, P <.001), prior examination (OR = 0.63, P = .01), and increasing patient age (OR = 0.99, P <.001) were associated with decreased likelihood. Mass, patient race or ethnicity, and radiologist factors were not significantly associated with BI-RADS 3 assessment. Malignancy rate for BI-RADS 3 lesions was 1.6%.

CONCLUSION

Asymmetry and calcifications had an increased likelihood of BI-RADS 3 assessment at diagnostic evaluation with low likelihood of malignancy, while radiologist features had no association.

Nomogram for preoperative differentiation of benign and malignant breast tumors using contrast-enhanced cone-beam breast CT (CE CB-BCT) quantitative imaging and assessment features

PURPOSE

Breast cancer's impact necessitates refined diagnostic approaches. This study develops a nomogram using radiology quantitative features from contrast-enhanced cone-beam breast CT for accurate preoperative classification of benign and malignant breast tumors.

MATERIAL AND METHODS

A retrospective study enrolled 234 females with breast tumors, split into training and test sets. Contrast-enhanced cone-beam breast CT-images were acquired using Koning Breast CT-1000. Quantitative assessment features were extracted via 3D-slicer software, identifying independent predictors. The nomogram was constructed to preoperative differentiation benign and malignant breast tumors. Calibration curve was used to assess whether the model showed favorable correspondence with pathological confirmation. Decision curve analysis confirmed the model's superiority.

RESULTS

The study enrolled 234 female patients with a mean age of 50.2 years (SD ± 9.2). The training set had 164 patients (89 benign, 75 malignant), and the test set had 70 patients (29 benign, 41 malignant). The nomogram achieved excellent predictive performance in distinguishing benign and malignant breast lesions with an AUC of 0.940 (95% CI 0.900-0.940) in the training set and 0.970 (95% CI 0.940-0.970) in the test set.

CONCLUSION

This study illustrates the effectiveness of quantitative radiology features derived from contrast-enhanced cone-beam breast CT in distinguishing between benign and malignant breast tumors. Incorporating these features into a nomogram-based diagnostic model allows for breast tumor diagnoses that are objective and possess good accuracy. The application of these insights could substantially increase reliability and efficacy in the management of breast tumors, offering enhanced diagnostic capability.

Application of computer-aided diagnosis to predict malignancy in BI-RADS 3 breast lesions

Purpose

To evaluate the ability of computer-aided diagnosis (CAD) system (S-Detect) to identify malignancy in ultrasound (US) -detected BI-RADS 3 breast lesions.

Materials and methods

148 patients with 148 breast lesions categorized as BI-RADS 3 were included in the study between January 2021 and September 2022. The malignancy rate, accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the curve (AUC) were calculated.

Results

In this study, 143 breast lesions were found to be benign, and 5 breast lesions were malignant (malignancy rate, 3.4 %, 95 % confidence interval (CI): 0.5-6.3). The malignancy rate rose significantly to 18.2 % (4/22, 95 % CI: 2.1-34.3) in the high-risk group with a "possibly malignant" CAD result (p = 0.017). With a "possibly benign" CAD result, the malignancy rate decreased to 0.8 % (1/126, 95 % CI: 0-2.2) in the low-risk group (p = 0.297). The AUC, sensitivity, specificity, accuracy, PPV, and NPV of the CAD system in BI-RADS 3 breast lesions were 0.837 (95 % CI: 77.7-89.6), 80.0 % (95 % CI: 73.6-86.4), 87.4 % (95 % CI: 82.0-92.7), 87.2 % (95 % CI: 81.8-92.6), 18.2 % (95 % CI: 2.1-34.3) and 99.2 % (95 % CI: 97.8-100.0), respectively.

Conclusions

CAD system (S-Detect) enables radiologists to distinguish a high-risk group and a low-risk group among US-detected BI-RADS 3 breast lesions, so that patients in the low-risk group can receive follow-up without anxiety, while those in the high-risk group with a significantly increased malignancy rate should actively receive biopsy to avoid delayed diagnosis of breast cancer.

Outcome of Imaging and Biopsy of BI-RADS Category 3 Lesions: Follow-Up Compliance, Biopsy, and Malignancy Rates in a Large Patient Cohort

OBJECTIVES

To identify biopsy rates and indications for BI-RADS 3 lesions in a large cohort of patients and compare with follow-up compliance and malignancy outcomes.

METHODS

We retrospectively reviewed all BI-RADS category-3 lesions seen on mammography and/or ultrasound between 2013 and 2015. Patient age, lesion size, follow-up rates at 6-, 12-, and 24-months were collected. Biopsy timing, indication, and outcomes (malignant vs benign) were recorded using at least 2-year follow-up or biopsy pathology as endpoint.

RESULTS

Of 2319 BI-RADS 3 lesions in 2075 women analyzed, biopsy was performed in 173 (7.5%). Most biopsies were performed upfront (99, 57.2%), followed by at 6 (44, 25.4%), 12 (21, 12.1%), and 24-month follow-up (9, 5.2%; P < .001). Palpable (P < .001) and larger (median 1.4 vs 1.0 cm, P < .001) lesions in women <40 years (15.2% vs 4.8%, P < .001) were more likely to undergo biopsy. Most biopsies were prompted by patient/physician desire (64.5%, P < .001). Of 783 lesions with available endpoint, 5 (0.6%) were cancer. All cancers were identified either at presentation (in 0-5 months, n = 1) or 6-month follow-up (in 5-9 months, n = 4) with biopsy prompted by either morphology change (n = 3) or lesion growth (n = 2). Of the 1855 lesions which were expected for follow up, only 310 (16.7%) underwent all follow-ups, while 482 (26.1%) had two, 489 (26.5%) one, and 565 (30.6%) had no follow-up.

CONCLUSIONS

In our cohort, BI-RADS category 3 lesions had significantly higher biopsy rates compared with the small malignancy rate, all of which were identified at baseline or first follow-up. Overall patient follow-up compliance low. Imaging follow-up, especially at first 6-month time point, should be encouraged in BI-RADS 3 lesions, instead of upfront biopsies.

A Single-Center Audit of BI-RADS 3 Assessment Category Utilization in Mammography and Breast Ultrasound

Purpose: To evaluate outcomes of breast lesions assessed at our institution as probably benign (Breast Imaging Reporting and Data System [BI-RADS] category 3) with an expected malignancy rate of less than or equal to 2 %. Methods: Average-risk women with a BI-RADS 3 assessment following mammographic and/or ultrasound evaluation at our institution between January 1 and December 31, 2017 were included. Cancer yield was calculated within 90 days and at 6-month intervals up to 36 months. Results: Among 517 women (median age, 52 years; range, 13-89 years) with a BI-RADS 3 assessment, 349 (67.5 %) underwent biopsy or completed follow-up imaging up to 36 months. One hundred and 68 (32.5 %) were lost to follow-up. Thirty of 349 (8.6 %) had their imaging upgraded and underwent biopsy, yielding six cancers (cancer yield, 6 of 349 women [1.7 %]). Among 569 lesions assessed as BI-RADS 3, 92 (16.2 %) were characterized by morphologic features other than those validated as probably benign in prospective clinical studies. Fifty three of 517 women (10.3 %) had follow-up beyond 24 months, and 24 (4.6 %) had follow-up beyond 36 months. Conclusion: Overall utilization of the BI-RADS 3 assessment category at our institution is appropriate with a 1.7 % cancer yield. However, the rate of loss to follow-up, percentage of non-validated findings assessed as probably benign, and redundancy in follow-up protocols are too high, and warrant intervention. A patient handout explaining the BI-RADS 3 assessment category and automatic scheduling of follow-up studies have been implemented at our center to address loss to follow-up.

Impact of artificial intelligence in breast cancer screening with mammography

Objectives

To demonstrate that radiologists, with the help of artificial intelligence (AI), are able to better classify screening mammograms into the correct breast imaging reporting and data system (BI-RADS) category, and as a secondary objective, to explore the impact of AI on cancer detection and mammogram interpretation time.

Methods

A multi-reader, multi-case study with cross-over design, was performed, including 314 mammograms. Twelve radiologists interpreted the examinations in two sessions delayed by a 4 weeks wash-out period with and without AI support. For each breast of each mammogram, they had to mark the most suspicious lesion (if any) and assign it with a forced BI-RADS category and a level of suspicion or “continuous BI-RADS 100”.

Cohen’s kappa correlation coefficient evaluating the inter-observer agreement for BI-RADS category per breast, and the area under the receiver operating characteristic curve (AUC), were used as metrics and analyzed.

Results

On average, the quadratic kappa coefficient increased significantly when using AI for all readers [κ = 0.549, 95% CI (0.528–0.571) without AI and κ = 0.626, 95% CI (0.607–0.6455) with AI]. AUC was significantly improved when using AI (0.74 vs 0.77, p = 0.004). Reading time was not significantly affected for all readers (106 s without AI and vs 102 s with AI; p = 0.754).

Conclusions

When using AI, radiologists were able to better assign mammograms with the correct BI-RADS category without slowing down the interpretation time.

Radiologist Variation in the Rates of Follow-up Imaging Recommendations Made for Pulmonary Nodules

OBJECTIVE

Determine whether differences exist in rates of follow-up recommendations made for pulmonary nodules after accounting for multiple patient and radiologist factors.

METHODS

This Institutional Review Board-approved, retrospective study was performed at an urban academic quaternary care hospital. We analyzed 142,001 chest and abdominal CT reports from January 1, 2016, to December 31, 2018, from abdominal, thoracic, and emergency radiology subspecialty divisions. A previously validated natural language processing (NLP) tool identified 24,512 reports documenting pulmonary nodule(s), excluding reports NLP-positive for lung cancer. A second validated NLP tool identified reports with follow-up recommendations specifically for pulmonary nodules. Multivariable logistic regression was used to determine the likelihood of pulmonary nodule follow-up recommendation. Interradiologist variability was quantified within subspecialty divisions.

RESULTS

NLP classified 4,939 of 24,512 (20.1%) reports as having a follow-up recommendation for pulmonary nodule. Male patients comprised 45.3% (11,097) of the patient cohort; average patient age was 61.4 years (±14.1 years). The majority of reports were from outpatient studies (62.7%, 15,376 of 24,512), were chest CTs (75.9%, 18,615 of 24,512), and were interpreted by thoracic radiologists (63.7%, 15,614 of 24,512). In multivariable analysis, studies for male patients (odds ratio [OR]: 0.9 [0.8-0.9]) and abdominal CTs (OR: 0.6 [0.6-0.7] compared with chest CT) were less likely to have a pulmonary nodule follow-up recommendation. Older patients had higher rates of follow-up recommendation (OR: 1.01 for each additional year). Division-level analysis showed up to 4.3-fold difference between radiologists in the probability of making a follow-up recommendation for a pulmonary nodule.

DISCUSSION

Significant differences exist in the probability of making a follow-up recommendation for pulmonary nodules among radiologists within the same subspecialty division.

Epidemiological, clinical and diagnostic profile of breast cancer patients treated at Potchefstroom regional hospital, South Africa, 2012-2018: an open-cohort study

Introduction

Breast cancer is the second most diagnosed cancer worldwide. We aimed to depict the diagnostic approach as well as the epidemiological and clinical profile of patients with breast cancer at Potchefstroom regional hospital, South Africa.

Methods

This descriptive open-cohort study included patients with primary invasive breast cancer, confirmed by histology results and treated at the hospital from 01 January 2012 to 31 December 2018. Data such as demographics, patient history, histology, breast clinical findings, physical mass description and diagnostic investigations were captured from hospital registries and patient files.

Result

One-hundred thirty-eight patients (mean age 56.2 (SD: 14.4) (95% CI 54.6-59.7) years) met inclusion criteria. Most patients were female (98.6%), from African (67.4%) or Caucasian (23.9%) descent. Findings included mostly left-sided breast involvement (51.8%), lesions in the upper-outer quadrant (43.1%), extensions to the skin (25.6%, N = 39), and tumour size of 2 ≤ 5 cm (49.3%), or > 5 cm (39.1%). Most patients (57.9%, N = 135) were categorised as BIRADS-5, with a ductal pattern (89.6%) (p < 0.01). Patients mostly presented in stages II to IV of disease (89.1%; p < 0.05). Late-stage (stages III-IV) at time of diagnosis (n = 84) was significantly associated with mass location (p = 0.006; Cramér's V = 0.280), tumour size (p < 0.001, Cramér's V = 0.239), and skin changes (p = 0.027, Cramér's V = 0.492).

Conclusion

Most patients consulted at a late-stage of the disease, indicating a need for the promotion of breast awareness campaigns, early detection, and timeous referral.

Variation in Follow-up Imaging Recommendations in Radiology Reports: Patient, Modality, and Radiologist Predictors

Background Variation between radiologists when making recommendations for additional imaging and associated factors are, to the knowledge of the authors, unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose To determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Materials and Methods This retrospective study analyzed 318 366 reports obtained from diagnostic imaging examinations performed at a large urban quaternary care hospital from January 1 to December 31, 2016, excluding breast and US reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training-and-validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318 366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed, and intradivision and interradiologist variability was quantified. Results The machine learning algorithm classified 38 745 of 318 366 (12.2%) reports as containing follow-up recommendations. Average patient age was 59 years ± 17 (standard deviation); 45.2% (143 767 of 318 366) of reports were from male patients. Among 65 radiologists, 57% (37 of 65) were men. At multivariable analysis, older patients had higher rates of follow-up recommendations (odds ratio [OR], 1.01 [95% confidence interval {CI}: 1.01, 1.01] for each additional year), male patients had lower rates of follow-up recommendations (OR, 0.9; 95% CI: 0.9, 1.0), and follow-up recommendations were most common among CT studies (OR, 4.2 [95% CI: 4.0, 4.4] compared with radiography). Radiologist sex (P = .54), presence of a trainee (P = .45), and years in practice (P = .49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold interradiologist variation. Conclusion Substantial interradiologist variation exists in the probability of recommending a follow-up examination in a radiology report, after adjusting for patient, examination, and radiologist factors. © RSNA, 2019 See also the editorial by Russell in this issue.

Automatic inference of BI-RADS final assessment categories from narrative mammography report findings

We propose an efficient natural language processing approach for inferring the BI-RADS final assessment categories by analyzing only the mammogram findings reported by the mammographer in narrative form. The proposed hybrid method integrates semantic term embedding with distributional semantics, producing a context-aware vector representation of unstructured mammography reports. A large corpus of unannotated mammography reports (300,000) was used to learn the context of the key-terms using a distributional semantics approach, and the trained model was applied to generate context-aware vector representations of the reports annotated with BI-RADS category (22,091). The vectorized reports were utilized to train a supervised classifier to derive the BI-RADS assessment class. Even though the majority of the proposed embedding pipeline is unsupervised, the classifier was able to recognize substantial semantic information for deriving the BI-RADS categorization not only on a holdout internal testset and also on an external validation set (1900 reports). Our proposed method outperforms a recently published domain-specific rule-based system and could be relevant for evaluating concordance between radiologists. With minimal requirement for task specific customization, the proposed method can be easily transferable to a different domain to support large scale text mining or derivation of patient phenotype.

The breast lesion excision system procedure: An optimal solution for the management of indeterminate BI-RADS category 3 breast lesions in women with severe anxiety

Objectives:

To evaluate the efficacy of the Breast lesion excision system (BLES) procedure as a primary excisional biopsy for the management of breast imaging-reporting and data system (BI-RADS) category 3, small, and solid breast lesions in women having severe breast cancer anxiety.

Methods:

A retospective study was conducted on 68 patients who underwent a BLES procedure. The study protocol was approved by the local ethical committee of Yeni Yuzyıl University in Istanbul, Turkey. The study was carried out according to the principles of the Helsinki Declaration. Small breast lesions removed using a (12, 15 or 20 mm) wand from September 2011 to November 2014. These were category 3 lesions as determined by ultrasound (US) imaging according to BI-RADS. The radiological and pathological sizes of these lesions, the complete excision rates, the procedure durations, the pathological diagnosis, the complications, and the imaging findings before and after the procedure were all recorded.

Results:

All the patients had a benign pathology. The mean duration of procedure was 12 (range=8-22) minutes. There was no major complication during the procedure and in the following period. Only some small hematomas were determined in 3 (4.2%) patients, and no additional surgical intervention was performed.

Conclusion:

The BLES procedure is an optimal solution for the management of indeterminate BI-RADS category 3 breast lesions in women with severe anxiety.

BI-RADS 3: Current and Future Use of Probably Benign

PURPOSE OF REVIEW

Probably benign (BI-RADS 3) causes confusion for interpreting physicians and referring physicians and can induce significant patient anxiety. The best uses and evidence for using this assessment category in mammography, breast ultrasound, and breast MRI will be reviewed; the reader will have a better understanding of how and when to use BI-RADS 3.

RECENT FINDINGS

Interobserver variability in the use of BI-RADS 3 has been documented. The 5th edition of the BI-RADS atlas details the appropriate use of BI-RADS 3 for diagnostic mammography, ultrasound, and MRI, and discourages its use in screening mammography. Data mining, elastography, and diffusion weighted MRI have been evaluated to maximize the accuracy of BI-RADS 3.

SUMMARY

BI-RADS 3 is an evolving assessment category. When used properly, it reduces the number of benign biopsies while allowing the breast imager to maintain a high sensitivity for the detection of early stage breast cancer.