Artificial Intelligence Improves Detection of Supplemental Screening Ultrasound-detected Breast Cancers in Mammography

Artificial intelligence for breast cancer detection and its health technology assessment: A scoping review

BACKGROUND

Recent healthcare advancements highlight the potential of Artificial Intelligence (AI) - and especially, among its subfields, Machine Learning (ML) - in enhancing Breast Cancer (BC) clinical care, leading to improved patient outcomes and increased radiologists' efficiency. While medical imaging techniques have significantly contributed to BC detection and diagnosis, their synergy with AI algorithms has consistently demonstrated superior diagnostic accuracy, reduced False Positives (FPs), and enabled personalized treatment strategies. Despite the burgeoning enthusiasm for leveraging AI for early and effective BC clinical care, its widespread integration into clinical practice is yet to be realized, and the evaluation of AI-based health technologies in terms of health and economic outcomes remains an ongoing endeavor.

OBJECTIVES

This scoping review aims to investigate AI (and especially ML) applications that have been implemented and evaluated across diverse clinical tasks or decisions in breast imaging and to explore the current state of evidence concerning the assessment of AI-based technologies for BC clinical care within the context of Health Technology Assessment (HTA).

METHODS

We conducted a systematic literature search following the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) checklist in PubMed and Scopus to identify relevant studies on AI (and particularly ML) applications in BC detection and diagnosis. We limited our search to studies published from January 2015 to October 2023. The Minimum Information about CLinical Artificial Intelligence Modeling (MI-CLAIM) checklist was used to assess the quality of AI algorithms development, evaluation, and reporting quality in the reviewed articles. The HTA Core Model® was also used to analyze the comprehensiveness, robustness, and reliability of the reported results and evidence in AI-systems' evaluations to ensure rigorous assessment of AI systems' utility and cost-effectiveness in clinical practice.

RESULTS

Of the 1652 initially identified articles, 104 were deemed eligible for inclusion in the review. Most studies examined the clinical effectiveness of AI-based systems (78.84%, n= 82), with one study focusing on safety in clinical settings, and 13.46% (n=14) focusing on patients' benefits. Of the studies, 31.73% (n=33) were ethically approved to be carried out in clinical practice, whereas 25% (n=26) evaluated AI systems legally approved for clinical use. Notably, none of the studies addressed the organizational implications of AI systems in clinical practice. Of the 104 studies, only two of them focused on cost-effectiveness analysis, and were analyzed separately. The average percentage scores for the first 102 AI-based studies' quality assessment based on the MI-CLAIM checklist criteria were 84.12%, 83.92%, 83.98%, 74.51%, and 14.7% for study design, data and optimization, model performance, model examination, and reproducibility, respectively. Notably, 20.59% (n=21) of these studies relied on large-scale representative real-world breast screening datasets, with only 10.78% (n =11) studies demonstrating the robustness and generalizability of the evaluated AI systems.

CONCLUSION

In bridging the gap between cutting-edge developments and seamless integration of AI systems into clinical workflows, persistent challenges encompass data quality and availability, ethical and legal considerations, robustness and trustworthiness, scalability, and alignment with existing radiologists' workflow. These hurdles impede the synthesis of comprehensive, robust, and reliable evidence to substantiate these systems' clinical utility, relevance, and cost-effectiveness in real-world clinical workflows. Consequently, evaluating AI-based health technologies through established HTA methodologies becomes complicated. We also highlight potential significant influences on AI systems' effectiveness of various factors, such as operational dynamics, organizational structure, the application context of AI systems, and practices in breast screening or examination reading of AI support tools in radiology. Furthermore, we emphasize substantial reciprocal influences on decision-making processes between AI systems and radiologists. Thus, we advocate for an adapted assessment framework specifically designed to address these potential influences on AI systems' effectiveness, mainly addressing system-level transformative implications for AI systems rather than focusing solely on technical performance and task-level evaluations.

Screening Outcomes of Mammography with AI in Dense Breasts: A Comparative Study with Supplemental Screening US

Background Comparative performance between artificial intelligence (AI) and breast US for women with dense breasts undergoing screening mammography remains unclear. Purpose To compare the performance of mammography alone, mammography with AI, and mammography plus supplemental US for screening women with dense breasts, and to investigate the characteristics of the detected cancers. Materials and Methods A retrospective database search identified consecutive asymptomatic women (≥40 years of age) with dense breasts who underwent mammography plus supplemental whole-breast handheld US from January 2017 to December 2018 at a primary health care center. Sequential reading for mammography alone and mammography with the aid of an AI system was conducted by five breast radiologists, and their recall decisions were recorded. Results of the combined mammography and US examinations were collected from the database. A dedicated breast radiologist reviewed marks for mammography alone or with AI to confirm lesion identification. The reference standard was histologic examination and 1-year follow-up data. The cancer detection rate (CDR) per 1000 screening examinations, sensitivity, specificity, and abnormal interpretation rate (AIR) of mammography alone, mammography with AI, and mammography plus US were compared. Results Among 5707 asymptomatic women (mean age, 52.4 years ± 7.9 [SD]), 33 (0.6%) had cancer (median lesion size, 0.7 cm). Mammography with AI had a higher specificity (95.3% [95% CI: 94.7, 95.8], P = .003) and lower AIR (5.0% [95% CI: 4.5, 5.6], P = .004) than mammography alone (94.3% [95% CI: 93.6, 94.8] and 6.0% [95% CI: 5.4, 6.7], respectively). Mammography plus US had a higher CDR (5.6 vs 3.5 per 1000 examinations, P = .002) and sensitivity (97.0% vs 60.6%, P = .002) but lower specificity (77.6% vs 95.3%, P < .001) and higher AIR (22.9% vs 5.0%, P < .001) than mammography with AI. Supplemental US alone helped detect 12 cancers, mostly stage 0 and I (92%, 11 of 12). Conclusion Although AI improved the specificity of mammography interpretation, mammography plus supplemental US helped detect more node-negative early breast cancers that were undetected using mammography with AI. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Whitman and Destounis in this issue.

Artificial Intelligence for Breast Cancer Detection on Mammography: Factors Related to Cancer Detection

RATIONALE AND OBJECTIVES

Little is known about the factors affecting the Artificial Intelligence (AI) software performance on mammography for breast cancer detection. This study was to identify factors associated with abnormality scores assigned by the AI software.

MATERIALS AND METHODS

A retrospective database search was conducted to identify consecutive asymptomatic women who underwent breast cancer surgery between April 2016 and December 2019. A commercially available AI software (Lunit INSIGHT, MMG, Ver. 1.1.4.0) was used for preoperative mammography to assign individual abnormality scores to the lesions and score of 10 or higher was considered as positive detection by AI software. Radiologists without knowledge of the AI results retrospectively assessed the mammographic density and classified mammographic findings into positive and negative finding. General linear model (GLM) analysis was used to identify the clinical, pathological, and mammographic findings related to the abnormality scores, obtaining coefficient β values that represent the mean difference per unit or comparison with the reference value. Additionally, the reasons for non-detection by the AI software were investigated.

RESULTS

Among the 1001 index cancers (830 invasive cancers and 171 ductal carcinoma in situs) in 1001 patients, 717 (72%) were correctly detected by AI, while the remaining 284 (28%) were not detected. Multivariable GLM analysis showed that abnormal mammography findings (β = 77.0 for mass, β = 73.1 for calcification only, β = 49.4 for architectural distortion, and β = 47.6 for asymmetry compared to negative; all Ps < 0.001), invasive tumor size (β = 4.3 per 1 cm, P < 0.001), and human epidermal growth receptor type 2 (HER2) positivity (β = 9.2 compared to hormone receptor positive, HER2 negative, P = 0.004) were associated with higher mean abnormality score. AI failed to detect small asymmetries in extremely dense breasts, subcentimeter-sized or isodense lesions, and faint amorphous calcifications.

CONCLUSION

Cancers with positive abnormal mammographic findings on retrospective review, large invasive size, HER2 positivity had high AI abnormality scores. Understanding the patterns of AI software performance is crucial for effectively integrating AI into clinical practice.