Mammographic Density Assessment by Artificial Intelligence-Based Computer-Assisted Diagnosis: A Comparison with Automated Volumetric Assessment

Inter- and intra-observer variability of qualitative visual breast-composition assessment in mammography among Japanese physicians: a first multi-institutional observer performance study in Japan

BACKGROUND

Visual assessment of mammographic breast composition remains the most common worldwide, although subjective variability limits its reproducibility. This study aimed to investigate the inter- and intra-observer variability in qualitative visual assessment of mammographic breast composition through a multi-institutional observer performance study for the first time in Japan.

METHODS

This study enrolled 10 Japanese physicians from five different institutions. They used the new Japanese breast-composition classification system 4th edition to subjectively evaluate the breast composition in 200 pairs of right and left normal mediolateral oblique mammograms (number determined using precise sample size calculations) twice, with a 1-month interval (median patient age: 59 years [range 40-69 years]). The primary endpoint of this study was the inter-observer variability using kappa (κ) value.

RESULTS

Inter-observer variability for the four and two classes of breast-composition assessment revealed moderate agreement (Fleiss' κ: first and second reading = 0.553 and 0.587, respectively) and substantial agreement (Fleiss' κ: first and second reading = 0.689 and 0.70, respectively). Intra-observer variability for the four and two classes of breast-composition assessment demonstrated substantial agreement (Cohen's κ, median = 0.758) and almost perfect agreement (Cohen's κ, median = 0.813). Assessments of consensus between the 10 physicians and the automated software Volpara® revealed slight agreement (Cohen's κ; first and second reading: 0.104 and 0.075, respectively).

CONCLUSIONS

Qualitative visual assessment of mammographic breast composition using the new Japanese classification revealed excellent intra-observer reproducibility. However, persistent inter-observer variability, presenting a challenge in establishing it as the gold standard in Japan.

Mammographic density assessment: comparison of radiologists, automated volumetric measurement, and artificial intelligence-based computer-assisted diagnosis

BACKGROUND

Artificial intelligence-based computer-assisted diagnosis (AI-CAD) is increasingly used for mammographic exams, and its role in mammographic density assessment should be evaluated.

PURPOSE

To assess the inter-modality agreement between radiologists, automated volumetric density measurement program (Volpara), and AI-CAD system in breast density categorization using the Breast Imaging-Reporting and Data System (BI-RADS) density categories.

MATERIAL AND METHODS

A retrospective review was conducted on 1015 screening digital mammograms that were performed in Asian female patients (mean age = 56 years ± 10 years) in our health examination center between December 2022 and January 2023. Four radiologists with two different levels of experience (expert and general radiologists) performed density assessments. Agreement between the radiologists, Volpara, and AI-CAD (Lunit INSIGHT MMG) was evaluated using weighted kappa statistics and matched rates.

RESULTS

Inter-reader agreement between expert and general radiologists was substantial (k = 0.65) with a matched rate of 72.8%. The agreement was substantial between expert or general radiologists and Volpara (k = 0.64-0.67) with a matched rate of 72.0% but moderate between expert or general radiologists and AI-CAD (k = 0.45-0.58) with matched rates of 56.7%-67.0%. The agreement between Volpara and AI-CAD was moderate (k = 0.53) with a matched rate of 60.8%.

CONCLUSION

The agreement in breast density categorization between radiologists and automated volumetric density measurement program (Volpara) was higher than the agreement between radiologists and AI-CAD (Lunit INSIGHT MMG).

Performance of an Artificial Intelligence System for Breast Cancer Detection on Screening Mammograms from BreastScreen Norway

Purpose To explore the stand-alone breast cancer detection performance, at different risk score thresholds, of a commercially available artificial intelligence (AI) system. Materials and Methods This retrospective study included information from 661 695 digital mammographic examinations performed among 242 629 female individuals screened as a part of BreastScreen Norway, 2004-2018. The study sample included 3807 screen-detected cancers and 1110 interval breast cancers. A continuous examination-level risk score by the AI system was used to measure performance as the area under the receiver operating characteristic curve (AUC) with 95% CIs and cancer detection at different AI risk score thresholds. Results The AUC of the AI system was 0.93 (95% CI: 0.92, 0.93) for screen-detected cancers and interval breast cancers combined and 0.97 (95% CI: 0.97, 0.97) for screen-detected cancers. In a setting where 10% of the examinations with the highest AI risk scores were defined as positive and 90% with the lowest scores as negative, 92.0% (3502 of 3807) of the screen-detected cancers and 44.6% (495 of 1110) of the interval breast cancers were identified with AI. In this scenario, 68.5% (10 987 of 16 040) of false-positive screening results (negative recall assessment) were considered negative by AI. When 50% was used as the cutoff, 99.3% (3781 of 3807) of the screen-detected cancers and 85.2% (946 of 1110) of the interval breast cancers were identified as positive by AI, whereas 17.0% (2725 of 16 040) of the false-positive results were considered negative. Conclusion The AI system showed high performance in detecting breast cancers within 2 years of screening mammography and a potential for use to triage low-risk mammograms to reduce radiologist workload. Keywords: Mammography, Breast, Screening, Convolutional Neural Network (CNN), Deep Learning Algorithms Supplemental material is available for this article. © RSNA, 2024 See also commentary by Bahl and Do in this issue.

Screening mammography performance according to breast density: a comparison between radiologists versus standalone intelligence detection

BACKGROUND

Artificial intelligence (AI) algorithms for the independent assessment of screening mammograms have not been well established in a large screening cohort of Asian women. We compared the performance of screening digital mammography considering breast density, between radiologists and AI standalone detection among Korean women.

METHODS

We retrospectively included 89,855 Korean women who underwent their initial screening digital mammography from 2009 to 2020. Breast cancer within 12 months of the screening mammography was the reference standard, according to the National Cancer Registry. Lunit software was used to determine the probability of malignancy scores, with a cutoff of 10% for breast cancer detection. The AI's performance was compared with that of the final Breast Imaging Reporting and Data System category, as recorded by breast radiologists. Breast density was classified into four categories (A-D) based on the radiologist and AI-based assessments. The performance metrics (cancer detection rate [CDR], sensitivity, specificity, positive predictive value [PPV], recall rate, and area under the receiver operating characteristic curve [AUC]) were compared across breast density categories.

RESULTS

Mean participant age was 43.5 ± 8.7 years; 143 breast cancer cases were identified within 12 months. The CDRs (1.1/1000 examination) and sensitivity values showed no significant differences between radiologist and AI-based results (69.9% [95% confidence interval [CI], 61.7-77.3] vs. 67.1% [95% CI, 58.8-74.8]). However, the AI algorithm showed better specificity (93.0% [95% CI, 92.9-93.2] vs. 77.6% [95% CI, 61.7-77.9]), PPV (1.5% [95% CI, 1.2-1.9] vs. 0.5% [95% CI, 0.4-0.6]), recall rate (7.1% [95% CI, 6.9-7.2] vs. 22.5% [95% CI, 22.2-22.7]), and AUC values (0.8 [95% CI, 0.76-0.84] vs. 0.74 [95% CI, 0.7-0.78]) (all P < 0.05). Radiologist and AI-based results showed the best performance in the non-dense category; the CDR and sensitivity were higher for radiologists in the heterogeneously dense category (P = 0.059). However, the specificity, PPV, and recall rate consistently favored AI-based results across all categories, including the extremely dense category.

CONCLUSIONS

AI-based software showed slightly lower sensitivity, although the difference was not statistically significant. However, it outperformed radiologists in recall rate, specificity, PPV, and AUC, with disparities most prominent in extremely dense breast tissue.

Strengths and challenges of the artificial intelligence in the assessment of dense breasts

Objectives

High breast density is a risk factor for breast cancer and overlapping of glandular tissue can mask lesions thus lowering mammographic sensitivity. Also, dense breasts are more vulnerable to increase recall rate and false-positive results. New generations of artificial intelligence (AI) have been introduced to the realm of mammography. We aimed to assess the strengths and challenges of adopting artificial intelligence in reading mammograms of dense breasts.

Methods

This study included 6600 mammograms of dense patterns "c" and "d" and presented 4061 breast abnormalities. All the patients were subjected to full-field digital mammography, breast ultrasound, and their mammographic images were scanned by AI software (Lunit INSIGHT MMG).

Results

Diagnostic indices of the sono-mammography: a sensitivity of 98.71%, a specificity of 88.04%, a positive-predictive value of 80.16%, a negative-predictive value of 99.29%, and a diagnostic accuracy of 91.5%. AI-aided mammograms presented sensitivity of 88.29%, a specificity of 96.34%, a positive-predictive value of 92.2%, a negative-predictive value of 94.4%, and a diagnostic accuracy of 94.5% in its ability to read dense mammograms.

Conclusion

Dense breasts scanned with AI showed a notable reduction of mammographic misdiagnosis. Knowledge of such software challenges would enhance its application as a decision support tool to mammography in the diagnosis of cancer.

Advances in knowledge

Dense breast is challenging for radiologists and renders low sensitivity mammogram. Mammogram scanned by AI could be used to overcome such limitation, enhance the discrimination between benign and malignant breast abnormalities and the early detection of breast cancer.

Application of Computer 3D Modeling Technology in Graphic Image Design

In order to solve the problem that the manual input data work efficiency is low and the optimization cannot be achieved in the section modulus calculation, the author proposes a computer-aided 3D graphic design method for ship hull structure. First, complete the modeling of the ship hull, complete the statistical calculation of the size in the CAD platform, solve the static moment, and complete the three-dimensional modeling; Calculate the inertial load and wave load of the ship during work, and calculate the wave bending moment, and substitute it into the three-dimensional model for verification. Finally, according to the computer-aided three-dimensional graphic design process, the specification verification of the ship's scale is completed, the graphic file is associated with the database, and the three-dimensional graphic design is completed. Experimental results show that the maximum stress of the original three-dimensional integrated graphic design of the ship is 202 MPa, and the maximum deformation is 87.352 mm. Based on the computer-aided three-dimensional integrated graphic design of the ship, the maximum stress is 226 MPa, and the maximum deformation is 76.234 mm. In conclusion: the hull obtained by the computer-aided three-dimensional graphic design of the ship can effectively reduce the total area and materials of the midsection structure of the ship, reduce the weight of the hull, and realize the optimal configuration of the midsection material of the ship.

The Current Status of Breakthrough Devices Designation in the United States and Innovative Medical Devices Designation in Korea for Digital Health Software

INTRODUCTION

Artificial Intelligence (AI) is becoming increasingly utilized in the medical device industry as it can address unmet demands in clinical sites and provide more patient treatment options. This study aims to analyze the FDA's Breakthrough Device Program and MFDS' Innovative Medical Device Program, which support regulatory science for innovative medical devices today. Through this study, it is intended to enable prediction of current development trends of Software as a Medical Device (SaMD) and Digital Therapeutics (DTx), which combine AI and technologies to be used in the clinical field soon.

AREAS COVERED

A systematic search was conducted on the broad topics of "FDA and MFDS Program's SaMD, DTx". A parallel review and update of PubMed, and the official websites were conducted to investigate the regulator's databases, review official press releases of regulatory agencies, and provide detailed descriptions of researchers.

EXPERT OPINION

The efforts of related stakeholders are needed to expand AI technology to diagnosis, prevention, and treatment technologies for diseases that are difficult to diagnose early or are classified as clinical challenges. It is important to prepare regulatory policies suitable for the rapid pace of technological development and to create an environment where regulatory science can be realized by developers.