New mammography screening performance metrics based on the entire screening episode

Screening mammography performance according to breast density: a comparison between radiologists versus standalone intelligence detection

BACKGROUND

Artificial intelligence (AI) algorithms for the independent assessment of screening mammograms have not been well established in a large screening cohort of Asian women. We compared the performance of screening digital mammography considering breast density, between radiologists and AI standalone detection among Korean women.

METHODS

We retrospectively included 89,855 Korean women who underwent their initial screening digital mammography from 2009 to 2020. Breast cancer within 12 months of the screening mammography was the reference standard, according to the National Cancer Registry. Lunit software was used to determine the probability of malignancy scores, with a cutoff of 10% for breast cancer detection. The AI's performance was compared with that of the final Breast Imaging Reporting and Data System category, as recorded by breast radiologists. Breast density was classified into four categories (A-D) based on the radiologist and AI-based assessments. The performance metrics (cancer detection rate [CDR], sensitivity, specificity, positive predictive value [PPV], recall rate, and area under the receiver operating characteristic curve [AUC]) were compared across breast density categories.

RESULTS

Mean participant age was 43.5 ± 8.7 years; 143 breast cancer cases were identified within 12 months. The CDRs (1.1/1000 examination) and sensitivity values showed no significant differences between radiologist and AI-based results (69.9% [95% confidence interval [CI], 61.7-77.3] vs. 67.1% [95% CI, 58.8-74.8]). However, the AI algorithm showed better specificity (93.0% [95% CI, 92.9-93.2] vs. 77.6% [95% CI, 61.7-77.9]), PPV (1.5% [95% CI, 1.2-1.9] vs. 0.5% [95% CI, 0.4-0.6]), recall rate (7.1% [95% CI, 6.9-7.2] vs. 22.5% [95% CI, 22.2-22.7]), and AUC values (0.8 [95% CI, 0.76-0.84] vs. 0.74 [95% CI, 0.7-0.78]) (all P < 0.05). Radiologist and AI-based results showed the best performance in the non-dense category; the CDR and sensitivity were higher for radiologists in the heterogeneously dense category (P = 0.059). However, the specificity, PPV, and recall rate consistently favored AI-based results across all categories, including the extremely dense category.

CONCLUSIONS

AI-based software showed slightly lower sensitivity, although the difference was not statistically significant. However, it outperformed radiologists in recall rate, specificity, PPV, and AUC, with disparities most prominent in extremely dense breast tissue.

Supplemental magnetic resonance imaging plus mammography compared with magnetic resonance imaging or mammography by extent of breast density

BACKGROUND

Examining screening outcomes by breast density for breast magnetic resonance imaging (MRI) with or without mammography could inform discussions about supplemental MRI in women with dense breasts.

METHODS

We evaluated 52 237 women aged 40-79 years who underwent 2611 screening MRIs alone and 6518 supplemental MRI plus mammography pairs propensity score-matched to 65 810 screening mammograms. Rates per 1000 examinations of interval, advanced, and screen-detected early stage invasive cancers and false-positive recall and biopsy recommendation were estimated by breast density (nondense = almost entirely fatty or scattered fibroglandular densities; dense = heterogeneously/extremely dense) adjusting for registry, examination year, age, race and ethnicity, family history of breast cancer, and prior breast biopsy.

RESULTS

Screen-detected early stage cancer rates were statistically higher for MRI plus mammography vs mammography for nondense (9.3 vs 2.9; difference = 6.4, 95% confidence interval [CI] = 2.5 to 10.3) and dense (7.5 vs 3.5; difference = 4.0, 95% CI = 1.4 to 6.7) breasts and for MRI vs MRI plus mammography for dense breasts (19.2 vs 7.5; difference = 11.7, 95% CI = 4.6 to 18.8). Interval rates were not statistically different for MRI plus mammography vs mammography for nondense (0.8 vs 0.5; difference = 0.4, 95% CI = -0.8 to 1.6) or dense breasts (1.5 vs 1.4; difference = 0.0, 95% CI = -1.2 to 1.3), nor were advanced cancer rates. Interval rates were not statistically different for MRI vs MRI plus mammography for nondense (2.6 vs 0.8; difference = 1.8 (95% CI = -2.0 to 5.5) or dense breasts (0.6 vs 1.5; difference = -0.9, 95% CI = -2.5 to 0.7), nor were advanced cancer rates. False-positive recall and biopsy recommendation rates were statistically higher for MRI groups than mammography alone.

CONCLUSION

MRI screening with or without mammography increased rates of screen-detected early stage cancer and false-positives for women with dense breasts without a concomitant decrease in advanced or interval cancers.

Risk of cancer versus risk of cancer diagnosis? Accounting for diagnostic bias in predictions of breast cancer risk by race and ethnicity

OBJECTIVES

Cancer risk prediction may be subject to detection bias if utilization of screening is related to cancer risk factors. We examine detection bias when predicting breast cancer risk by race/ethnicity.

METHODS

We used screening and diagnosis histories from the Breast Cancer Surveillance Consortium to estimate risk of breast cancer onset and calculated relative risk of onset and diagnosis for each racial/ethnic group compared with non-Hispanic White women.

RESULTS

Of 104,073 women aged 40-54 receiving their first screening mammogram at a Breast Cancer Surveillance Consortium facility between 2000 and 2018, 10.2% (n = 10,634) identified as Asian, 10.9% (n = 11,292) as Hispanic, and 8.4% (n = 8719) as non-Hispanic Black. Hispanic and non-Hispanic Black women had slightly lower screening frequencies but biopsy rates following a positive mammogram were similar across groups. Risk of cancer diagnosis was similar for non-Hispanic Black and White women (relative risk vs non-Hispanic White = 0.90, 95% CI 0.65 to 1.14) but was lower for Asian (relative risk = 0.70, 95% CI 0.56 to 0.97) and Hispanic women (relative risk = 0.82, 95% CI 0.62 to 1.08). Relative risks of disease onset were 0.78 (95% CI 0.68 to 0.88), 0.70 (95% CI 0.59 to 0.83), and 0.95 (95% CI 0.84 to 1.09) for Asian, Hispanic, and non-Hispanic Black women, respectively.

CONCLUSIONS

Racial/ethnic differences in mammography and biopsy utilization did not induce substantial detection bias; relative risks of disease onset were similar to or modestly different than relative risks of diagnosis. Asian and Hispanic women have lower risks of developing breast cancer than non-Hispanic Black and White women, who have similar risks.

Breast cancer risk characteristics of women undergoing whole-breast ultrasound screening versus mammography alone

BACKGROUND

There are no consensus guidelines for supplemental breast cancer screening with whole-breast ultrasound. However, criteria for women at high risk of mammography screening failures (interval invasive cancer or advanced cancer) have been identified. Mammography screening failure risk was evaluated among women undergoing supplemental ultrasound screening in clinical practice compared with women undergoing mammography alone.

METHODS

A total of 38,166 screening ultrasounds and 825,360 screening mammograms without supplemental screening were identified during 2014-2020 within three Breast Cancer Surveillance Consortium (BCSC) registries. Risk of interval invasive cancer and advanced cancer were determined using BCSC prediction models. High interval invasive breast cancer risk was defined as heterogeneously dense breasts and BCSC 5-year breast cancer risk ≥2.5% or extremely dense breasts and BCSC 5-year breast cancer risk ≥1.67%. Intermediate/high advanced cancer risk was defined as BCSC 6-year advanced breast cancer risk ≥0.38%.

RESULTS

A total of 95.3% of 38,166 ultrasounds were among women with heterogeneously or extremely dense breasts, compared with 41.8% of 825,360 screening mammograms without supplemental screening (p < .0001). Among women with dense breasts, high interval invasive breast cancer risk was prevalent in 23.7% of screening ultrasounds compared with 18.5% of screening mammograms without supplemental imaging (adjusted odds ratio, 1.35; 95% CI, 1.30-1.39); intermediate/high advanced cancer risk was prevalent in 32.0% of screening ultrasounds versus 30.5% of screening mammograms without supplemental screening (adjusted odds ratio, 0.91; 95% CI, 0.89-0.94).

CONCLUSIONS

Ultrasound screening was highly targeted to women with dense breasts, but only a modest proportion were at high mammography screening failure risk. A clinically significant proportion of women undergoing mammography screening alone were at high mammography screening failure risk.

Digital Breast Tomosynthesis versus Digital Mammography Screening Performance on Successive Screening Rounds from the Breast Cancer Surveillance Consortium

Background Prior cross-sectional studies have observed that breast cancer screening with digital breast tomosynthesis (DBT) has a lower recall rate and higher cancer detection rate compared with digital mammography (DM). Purpose To evaluate breast cancer screening outcomes with DBT versus DM on successive screening rounds. Materials and Methods In this retrospective cohort study, data from 58 breast imaging facilities in the Breast Cancer Surveillance Consortium were collected. Analysis included women aged 40-79 years undergoing DBT or DM screening from 2011 to 2020. Absolute differences in screening outcomes by modality and screening round were estimated during the study period by using generalized estimating equations with marginal standardization to adjust for differences in women's risk characteristics across modality and round. Results A total of 523 485 DBT examinations (mean age of women, 58.7 years ± 9.7 [SD]) and 1 008 123 DM examinations (mean age, 58.4 years ± 9.8) among 504 863 women were evaluated. DBT and DM recall rates decreased with successive screening round, but absolute recall rates in each round were significantly lower with DBT versus DM (round 1 difference, -3.3% [95% CI: -4.6, -2.1] [P < .001]; round 2 difference, -1.8% [95% CI: -2.9, -0.7] [P = .003]; round 3 or above difference, -1.2% [95% CI: -2.4, -0.1] [P = .03]). DBT had significantly higher cancer detection (difference, 0.6 per 1000 examinations [95% CI: 0.2, 1.1]; P = .009) compared with DM only for round 3 and above. There were no significant differences in interval cancer rate (round 1 difference, 0.00 per 1000 examinations [95% CI: -0.24, 0.30] [P = .96]; round 2 or above difference, 0.04 [95% CI: -0.19, 0.31] [P = .76]) or total advanced cancer rate (round 1 difference, 0.00 per 1000 examinations [95% CI: -0.15, 0.19] [P = .94]; round 2 or above difference, -0.06 [95% CI: -0.18, 0.11] [P = .43]). Conclusion DBT had lower recall rates and could help detect more cancers than DM across three screening rounds, with no difference in interval or advanced cancer rates. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Skaane in this issue.

Independent evaluation of a multi-view multi-task convolutional neural network breast cancer classification model using Finnish mammography screening data

BACKGROUND

Development of deep convolutional neural networks for breast cancer classification has taken significant steps towards clinical adoption. It is though unclear how the models perform for unseen data, and what is required to adapt them to different demographic populations. In this retrospective study, we adopt an openly available pre-trained mammography breast cancer multi-view classification model and evaluate it by utilizing an independent Finnish dataset.

METHODS

Transfer learning was used, and the pre-trained model was finetuned with 8,829 examinations from the Finnish dataset (4,321 normal, 362 malignant and 4,146 benign examinations). Holdout dataset with 2,208 examinations from the Finnish dataset (1,082 normal, 70 malignant and 1,056 benign examinations) was used in the evaluation. The performance was also evaluated on a manually annotated malignant suspect subset. Receiver Operating Characteristic (ROC) and Precision-Recall curves were used to performance measures.

RESULTS

The Area Under ROC [95%CI] values for malignancy classification obtained with the finetuned model for the entire holdout set were 0.82 [0.76, 0.87], 0.84 [0.77, 0.89], 0.85 [0.79, 0.90], and 0.83 [0.76, 0.89] for R-MLO, L-MLO, R-CC and L-CC views respectively. Performance on the malignant suspect subset was slightly better. On the auxiliary benign classification task performance remained low.

CONCLUSIONS

The results indicate that the model performs well also in an out-of-distribution setting. Finetuning allowed the model to adapt to some of the underlying local demographics. Future research should concentrate to identify breast cancer subgroups adversely affecting performance, as it is a requirement for increasing the model's readiness level for a clinical setting.

Estimation of Breast Cancer Overdiagnosis in a U.S. Breast Screening Cohort

BACKGROUND

Mammography screening can lead to overdiagnosis-that is, screen-detected breast cancer that would not have caused symptoms or signs in the remaining lifetime. There is no consensus about the frequency of breast cancer overdiagnosis.

OBJECTIVE

To estimate the rate of breast cancer overdiagnosis in contemporary mammography practice accounting for the detection of nonprogressive cancer.

DESIGN

Bayesian inference of the natural history of breast cancer using individual screening and diagnosis records, allowing for nonprogressive preclinical cancer. Combination of fitted natural history model with life-table data to predict the rate of overdiagnosis among screen-detected cancer under biennial screening.

SETTING

Breast Cancer Surveillance Consortium (BCSC) facilities.

PARTICIPANTS

Women aged 50 to 74 years at first mammography screen between 2000 and 2018.

MEASUREMENTS

Screening mammograms and screen-detected or interval breast cancer.

RESULTS

The cohort included 35 986 women, 82 677 mammograms, and 718 breast cancer diagnoses. Among all preclinical cancer cases, 4.5% (95% uncertainty interval [UI], 0.1% to 14.8%) were estimated to be nonprogressive. In a program of biennial screening from age 50 to 74 years, 15.4% (UI, 9.4% to 26.5%) of screen-detected cancer cases were estimated to be overdiagnosed, with 6.1% (UI, 0.2% to 20.1%) due to detecting indolent preclinical cancer and 9.3% (UI, 5.5% to 13.5%) due to detecting progressive preclinical cancer in women who would have died of an unrelated cause before clinical diagnosis.

LIMITATIONS

Exclusion of women with first mammography screen outside BCSC.

CONCLUSION

On the basis of an authoritative U.S. population data set, the analysis projected that among biennially screened women aged 50 to 74 years, about 1 in 7 cases of screen-detected cancer is overdiagnosed. This information clarifies the risk for breast cancer overdiagnosis in contemporary screening practice and should facilitate shared and informed decision making about mammography screening.

PRIMARY FUNDING SOURCE

National Cancer Institute.

Prioritizing breast imaging services during the COVID pandemic: A survey of breast imaging facilities within the Breast Cancer Surveillance Consortium

The COVID-19 pandemic disrupted breast cancer screening and diagnostic imaging in the United States. We sought to evaluate how medical facilities prioritized breast imaging services during periods of reduced capacity or upon re-opening after closures. In fall 2020, we surveyed 77 breast imaging facilities within the Breast Cancer Surveillance Consortium in the United States. The survey ascertained the pandemic's impact on clinical practices during March-September 2020. Nearly all facilities (97%) reported closing or operating at reduced capacity at some point during this period. All facilities were open by August 2020, though 14% were still operating at reduced capacity in September 2020. During periods of re-opening or reduced capacity, 93% of facilities reported prioritizing diagnostic breast imaging over breast cancer screening. For diagnostic imaging, facilities prioritized based on rescheduling canceled appointments (89%), specific indication for diagnostic imaging (89%), patient demand (84%), individual characteristics and risk factors (77%), and time since last imaging examination (72%). For screening mammography, facilities prioritized based on rescheduled cancelations (96%), patient demand (83%), individual characteristics and risk factors (73%), and time since last mammogram (71%). For biopsy services, more than 90% of facilities reported prioritization based on rescheduling of canceled exams, patient demand, patient characteristics and risk factors and level of suspicion on imaging. The observed patterns from this large and geographically diverse sample of facilities in the United States indicate that multiple factors were commonly used to prioritize breast imaging services during periods of reduced capacity.

Digital Mammography and Breast Tomosynthesis Performance in Women with a Personal History of Breast Cancer, 2007-2016

Background Since 2007, digital mammography and digital breast tomosynthesis (DBT) replaced screen-film mammography. Whether these technologic advances have improved diagnostic performance has, to the knowledge of the authors, not yet been established. Purpose To evaluate the performance and outcomes of surveillance mammography (digital mammography and DBT) performed from 2007 to 2016 in women with a personal history of breast cancer and compare with data from 1996 to 2007 and the performance of digital mammography screening benchmarks. Materials and Methods In this observational cohort study, five Breast Cancer Surveillance Consortium registries provided prospectively collected mammography data linked with tumor registry and pathologic outcomes. This study identified asymptomatic women with American Joint Committee on Cancer anatomic stages 0-III primary breast cancer who underwent surveillance mammography from 2007 to 2016. The primary outcome was a second breast cancer diagnosis within 1 year of mammography. Performance measures included the recall rate, cancer detection rate, interval cancer rate, positive predictive value of biopsy recommendation, sensitivity, and specificity. Results Among 32 331 women who underwent 117 971 surveillance mammographic examinations (112 269 digital mammographic examinations and 5702 DBT examinations), the mean age at initial diagnosis was 59 years ± 12 (standard deviation). Of 1418 second breast cancers diagnosed, 998 were surveillance-detected cancers and 420 were interval cancers. The recall rate was 8.8% (10 365 of 117 971; 95% CI: 8.6%, 9.0%), the cancer detection rate was 8.5 per 1000 examinations (998 of 117 971; 95% CI: 8.0, 9.0), the interval cancer rate was 3.6 per 1000 examinations (420 of 117 971; 95% CI: 3.2, 3.9), the positive predictive value of biopsy recommendation was 31.0% (998 of 3220; 95% CI: 29.4%, 32.7%), the sensitivity was 70.4% (998 of 1418; 95% CI: 67.9%, 72.7%), and the specificity was 98.1% (114 331 of 116 553; 95% CI: 98.0%, 98.2%). Compared with previously published studies, interval cancer rate was comparable with rates from 1996 to 2007 in women with a personal history of breast cancer and was higher than the published digital mammography screening benchmarks. Conclusion In transitioning from screen-film to digital mammography and digital breast tomosynthesis, surveillance mammography performance demonstrated minimal improvement over time and remained inferior to the performance of screening mammography benchmarks. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Moy and Gao in this issue.

Receipt of Screening Mammography by Insured Women Diagnosed With Breast Cancer and Impact on Outcomes

BACKGROUND

The purpose of this study was to determine factors associated with receipt of screening mammography by insured women before breast cancer diagnosis, and subsequent outcomes.

PATIENTS AND METHODS

Using claims data from commercial and federal payers linked to a regional SEER registry, we identified women diagnosed with breast cancer from 2007 to 2017 and determined receipt of screening mammography within 1 year before diagnosis. We obtained patient and tumor characteristics from the SEER registry and assigned each woman a socioeconomic deprivation score based on residential address. Multivariable logistic regression models were used to evaluate associations of patient and tumor characteristics with late-stage disease and nonreceipt of mammography. We used multivariable Cox proportional hazards models to identify predictors of subsequent mortality.

RESULTS

Among 7,047 women, 69% (n=4,853) received screening mammography before breast cancer diagnosis. Compared with women who received mammography, those with no mammography had a higher proportion of late-stage disease (34% vs 10%) and higher 5-year mortality (18% vs 6%). In multivariable modeling, late-stage disease was most associated with nonreceipt of mammography (odds ratio [OR], 4.35; 95% CI, 3.80-4.98). The Cox model indicated that nonreceipt of mammography predicted increased risk of mortality (hazard ratio [HR], 2.00; 95% CI, 1.64-2.43), independent of late-stage disease at diagnosis (HR, 5.00; 95% CI, 4.10-6.10), Charlson comorbidity index score ≥1 (HR, 2.75; 95% CI, 2.26-3.34), and negative estrogen receptor/progesterone receptor status (HR, 2.09; 95% CI, 1.67-2.61). Nonreceipt of mammography was associated with younger age (40-49 vs 50-59 years; OR, 1.69; 95% CI, 1.45-1.96) and increased socioeconomic deprivation (OR, 1.05 per decile increase; 95% CI, 1.03-1.07).

CONCLUSIONS

In a cohort of insured women diagnosed with breast cancer, nonreceipt of screening mammography was significantly associated with late-stage disease and mortality, suggesting that interventions to further increase uptake of screening mammography may improve breast cancer outcomes.