Screen-detected and interval breast cancer after concordant and discordant interpretations in a population based screening program using independent double reading

Discordant and false-negative interpretations at digital breast tomosynthesis in the prospective Oslo Tomosynthesis Screening Trial (OTST) using independent double reading

OBJECTIVES

To analyze discordant and false-negatives of double reading digital breast tomosynthesis (DBT) versus digital mammography (DM) including reading times in the Oslo Tomosynthesis Screening Trial (OTST), and reclassify these in a retrospective reader study as missed, minimal sign, or true-negatives.

METHODS

The prospective OTST comparing double reading DBT vs. DM had paired design with four parallel arms: DM, DM + computer aided detection, DBT + DM, and DBT + synthetic mammography. Eight radiologists interpreted images in batches using a 5-point scale. Reading time was automatically recorded. A retrospective reader study including four radiologists classified screen-detected cancers with at least one false-negative score and screening examinations of interval cancers as negative, non-specific minimal sign, significant minimal sign, and missed; the two latter groups are defined "actionable." Statistics included chi-square, Fisher's exact, McNemar's, and Mann-Whitney U tests.

RESULTS

Discordant rate (cancer missed by one reader) for screen-detected cancers was overall comparable (DBT (31% [71/227]) and DM (30% [52/175]), p = .81), significantly lower at DBT for spiculated cancers (DBT, 19% [20/106] vs. DM, 36% [38/106], p = .003), but high (28/49 = 57%, p = 0.001) for DBT-only detected spiculated cancers. Reading time and sensitivity varied among readers. False-negative DBT-only detected spiculated cancers had shorter reading time than true-negatives in 46% (13/28). Retrospective evaluation classified the following DBT exams "actionable": three missed by both readers, 95% (39/41) of discordant cancers detected by both modes, all 30 discordant DBT-only cancers, 25% (13/51) of interval cancers.

CONCLUSIONS

Discordant rate was overall comparable for DBT and DM, significantly lower at DBT for spiculated cancers, but high for DBT-only detected spiculated lesions. Most false-negative screen-detected DBT were classified as "actionable."

CLINICAL RELEVANCE STATEMENT

Retrospective evaluation of false-negative interpretations from the Oslo Tomosynthesis Screening Trial shows that most discordant and several interval cancers could have been detected at screening. This underlines the potential for modern AI-based reading aids and triage, as high-volume screening is a demanding task.

KEY POINTS

• Digital breast tomosynthesis (DBT) screening is more sensitive and has higher specificity compared to digital mammography screening, but high-volume DBT screening is a demanding task which can result in high discordance rate among readers. • Independent double reading DBT screening had overall comparable discordance rate as digital mammography, lower for spiculated masses seen on both modalities, and higher for small spiculated cancer seen only on DBT. • Almost all discordant digital breast tomosynthesis-detected cancers (72 of 74) and 25% (13 of 51) of the interval cancers in the Oslo Tomosynthesis Screening Trial were retrospectively classified as actionable and could have been detected by the readers.

AI performance by mammographic density in a retrospective cohort study of 99,489 participants in BreastScreen Norway

OBJECTIVE

To explore the ability of artificial intelligence (AI) to classify breast cancer by mammographic density in an organized screening program.

MATERIALS AND METHOD

We included information about 99,489 examinations from 74,941 women who participated in BreastScreen Norway, 2013-2019. All examinations were analyzed with an AI system that assigned a malignancy risk score (AI score) from 1 (lowest) to 10 (highest) for each examination. Mammographic density was classified into Volpara density grade (VDG), VDG1-4; VDG1 indicated fatty and VDG4 extremely dense breasts. Screen-detected and interval cancers with an AI score of 1-10 were stratified by VDG.

RESULTS

We found 10,406 (10.5% of the total) examinations to have an AI risk score of 10, of which 6.7% (704/10,406) was breast cancer. The cancers represented 89.7% (617/688) of the screen-detected and 44.6% (87/195) of the interval cancers. 20.3% (20,178/99,489) of the examinations were classified as VDG1 and 6.1% (6047/99,489) as VDG4. For screen-detected cancers, 84.0% (68/81, 95% CI, 74.1-91.2) had an AI score of 10 for VDG1, 88.9% (328/369, 95% CI, 85.2-91.9) for VDG2, 92.5% (185/200, 95% CI, 87.9-95.7) for VDG3, and 94.7% (36/38, 95% CI, 82.3-99.4) for VDG4. For interval cancers, the percentages with an AI score of 10 were 33.3% (3/9, 95% CI, 7.5-70.1) for VDG1 and 48.0% (12/25, 95% CI, 27.8-68.7) for VDG4.

CONCLUSION

The tested AI system performed well according to cancer detection across all density categories, especially for extremely dense breasts. The highest proportion of screen-detected cancers with an AI score of 10 was observed for women classified as VDG4.

CLINICAL RELEVANCE STATEMENT

Our study demonstrates that AI can correctly classify the majority of screen-detected and about half of the interval breast cancers, regardless of breast density.

KEY POINTS

• Mammographic density is important to consider in the evaluation of artificial intelligence in mammographic screening. • Given a threshold representing about 10% of those with the highest malignancy risk score by an AI system, we found an increasing percentage of cancers with increasing mammographic density. • Artificial intelligence risk score and mammographic density combined may help triage examinations to reduce workload for radiologists.

AI Risk Score on Screening Mammograms Preceding Breast Cancer Diagnosis

Background Few studies have evaluated the role of artificial intelligence (AI) in prior screening mammography. Purpose To examine AI risk scores assigned to screening mammography in women who were later diagnosed with breast cancer. Materials and Methods Image data and screening information of examinations performed from January 2004 to December 2019 as part of BreastScreen Norway were used in this retrospective study. Prior screening examinations from women who were later diagnosed with cancer were assigned an AI risk score by a commercially available AI system (scores of 1-7, low risk of malignancy; 8-9, intermediate risk; and 10, high risk of malignancy). Mammographic features of the cancers based on the AI score were also assessed. The association between AI score and mammographic features was tested with a bivariate test. Results A total of 2787 prior screening examinations from 1602 women (mean age, 59 years ± 5.1 [SD]) with screen-detected (n = 1016) or interval (n = 586) cancers showed an AI risk score of 10 for 389 (38.3%) and 231 (39.4%) cancers, respectively, on the mammograms in the screening round prior to diagnosis. Among the screen-detected cancers with AI scores available two screening rounds (4 years) before diagnosis, 23.0% (122 of 531) had a score of 10. Mammographic features were associated with AI score for invasive screen-detected cancers (P < .001). Density with calcifications was registered for 13.6% (43 of 317) of screen-detected cases with a score of 10 and 4.6% (15 of 322) for those with a score of 1-7. Conclusion More than one in three cases of screen-detected and interval cancers had the highest AI risk score at prior screening, suggesting that the use of AI in mammography screening may lead to earlier detection of breast cancers. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Mehta in this issue.

Breast cancer missed at screening; hindsight or mistakes?

PURPOSE

To investigate radiologists' interpretation scores of screening mammograms prior to diagnosis of screen-detected and interval breast cancers retrospectively classified as missed or true negative.

METHODS

We included data on radiologists' interpretation scores at screening prior to diagnosis for 1223 screen-detected and 1007 interval cancer cases classified as missed or true negative in an informed consensus-based review. All prior screening examinations were independently scored 1-5 by two radiologists; score 1 by both was considered concordant negative, score ≥ 2 by one radiologist discordant, and score ≥ 2 by both concordant positive. We analyzed associations between interpretation, review categories, mammographic features and histopathological findings using descriptive statistics and logistic regression.

RESULTS

Among screen-detected cancers, 31% of missed and 10% of true negative cancers had discordant or concordant positive interpretation at prior screening. The corresponding percentages for interval cancer were 21% and 8%. Age-adjusted odds ratio (OR) and 95% confidence interval (CI) for missed screen-detected cancer was 3.8 (95% CI: 2.6-5.4) after discordant and 5.5 (95% CI: 3.2-9.5) after concordant positive interpretation, using concordant negative as reference. Corresponding ORs for missed interval cancer were 3.0 (95% CI: 2.0-4.5) for discordant and 6.3 (95% CI: 2.3-17.5) for concordant positive interpretation. Asymmetry was the dominating mammographic feature at prior screening for all, except concordant positive screen-detected cancers where a mass dominated. Histopathological characteristics did not vary statistically with interpretation.

CONCLUSIONS

Most cancers were interpreted negatively at screening prior to diagnosis. Increased risk for missed screen-detected or interval cancer was observed after positive interpretation at prior screening.

Possible strategies for use of artificial intelligence in screen-reading of mammograms, based on retrospective data from 122,969 screening examinations

OBJECTIVES

Artificial intelligence (AI) has shown promising results when used on retrospective data from mammographic screening. However, few studies have explored the possible consequences of different strategies for combining AI and radiologists in screen-reading.

METHODS

A total of 122,969 digital screening examinations performed between 2009 and 2018 in BreastScreen Norway were retrospectively processed by an AI system, which scored the examinations from 1 to 10; 1 indicated low suspicion of malignancy and 10 high suspicion. Results were merged with information about screening outcome and used to explore consensus, recall, and cancer detection for 11 different scenarios of combining AI and radiologists.

RESULTS

Recall was 3.2%, screen-detected cancer 0.61% and interval cancer 0.17% after independent double reading and served as reference values. In a scenario where examinations with AI scores 1-5 were considered negative and 6-10 resulted in standard independent double reading, the estimated recall was 2.6% and screen-detected cancer 0.60%. When scores 1-9 were considered negative and score 10 double read, recall was 1.2% and screen-detected cancer 0.53%. In these two scenarios, potential rates of screen-detected cancer could be up to 0.63% and 0.56%, if the interval cancers selected for consensus were detected at screening. In the former scenario, screen-reading volume would be reduced by 50%, while the latter would reduce the volume by 90%.

CONCLUSION

Several theoretical scenarios with AI and radiologists have the potential to reduce the volume in screen-reading without affecting cancer detection substantially. Possible influence on recall and interval cancers must be evaluated in prospective studies.

KEY POINTS

• Different scenarios using artificial intelligence in combination with radiologists could reduce the screen-reading volume by 50% and result in a rate of screen-detected cancer ranging from 0.59% to 0.60%, compared to 0.61% after standard independent double reading • The use of artificial intelligence in combination with radiologists has the potential to identify negative screening examinations with high precision in mammographic screening and to reduce the rate of interval cancer.