Persistent inter-observer variability of breast density assessment using BI-RADS® 5th edition guidelines

A systematic review and meta-analysis of correlation of automated breast density measurement

INTRODUCTION

Breast cancer is the most common cancer in women and a leading cause of mortality. This systematic review and meta-analysis aims to evaluate the correlation between breast density measurements obtained from various software and visual assessments by radiologists using full-field digital mammography (FFDM).

METHODS

Following the PRISMA 2020 guidelines, five databases (Pubmed, Google Scholar, Science Direct, Cochrane Library, and MEDLINE) were searched for studies correlating volumetric breast density with breast cancer risk. The Newcastle-Ottawa Scale and the Joanna Briggs Institute Checklist were used to assess the quality of the included studies. Meta-analysis of correlation was applied to aggregate correlation coefficients using a random-effects model using MedCalc Statistical Software version 19.2.6.

RESULTS

The review included 22 studies with a total of 58,491 women. The pooled correlation coefficient for volumetric breast density amongst Volpara™ and Quantra™ was found to be 0.755 (95% CI 0.496-0.890, p < 0.001), indicating a high positive correlation, albeit with a significant heterogeneity (I2 = 99.89%, p < 0.0001). Subgroup analyses based on study origin, quality, and methodology were performed but did not reveal the heterogeneity cause. Egger's and Begg's tests showed no significant publication bias.

CONCLUSION

Volumetric breast density is strongly correlated with breast cancer risk, underscoring the importance of accurate breast density assessment in screening programs. Automated volumetric measurement tools like Volpara™ and Quantra™ provide reliable assessments, potentially improving breast cancer risk prediction and management.

IMPLICATIONS FOR PRACTICE

Implementing fully automated breast density assessment tools could enhance consistency in clinical practice, minimizing observer variability and improving screening accuracy. These tools should be further validated against standardized criteria to ensure reliability in diverse clinical settings.

Breast density effect on the sensitivity of digital screening mammography in a UK cohort

OBJECTIVES

To assess the performance of breast cancer screening by category of breast density and age in a UK screening cohort.

METHODS

Raw full-field digital mammography data from a single site in the UK, forming a consecutive 3-year cohort of women aged 50 to 70 years from 2016 to 2018, were obtained retrospectively. Breast density was assessed using Volpara software. Examinations were grouped by density category and age group (50-60 and 61-70 years) to analyse screening performance. Statistical analysis was performed to determine the association between density categories and age groups. Volumetric breast density was assessed as a binary classifier of interval cancers (ICs) to find an optimal density threshold.

RESULTS

Forty-nine thousand nine-hundred forty-eight screening examinations (409 screen-detected cancers (SDCs) and 205 ICs) were included in the analysis. Mammographic sensitivity, SDC/(SDC + IC), decreased with increasing breast density from 75.0% for density a (p = 0.839, comparisons made to category b), to 73.5%, 59.8% (p = 0.001), and 51.3% (p < 0.001) in categories b, c, and d, respectively. IC rates were highest in the densest categories with rates of 1.8 (p = 0.039), 3.2, 5.7 (p < 0.001), and 7.9 (p < 0.001) per thousand for categories a, b, c, and d, respectively. The recall rate increased with breast density, leading to more false positive recalls, especially in the younger age group. There was no significant difference between the optimal density threshold found, 6.85, and that Volpara defined as the b/c boundary, 7.5.

CONCLUSIONS

The performance of screening is significantly reduced with increasing density with IC rates in the densest category four times higher than in women with fatty breasts. False positives are a particular issue for the younger subgroup without prior examinations.

CLINICAL RELEVANCE STATEMENT

In women attending screening there is significant underdiagnosis of breast cancer in those with dense breasts, most marked in the highest density category but still three times higher than in women with fatty breasts in the second highest category.

KEY POINTS

Breast density can mask cancers leading to underdiagnosis on mammography. Interval cancer rate increased with breast density categories 'a' to 'd'; 1.8 to 7.9 per thousand. Recall rates increased with increasing breast density, leading to more false positive recalls.

Inter-reader variability and reproducibility of the PI-QUAL score in a multicentre setting

PURPOSE

To assess the inter-reader reproducibility of the Prostate Imaging Quality (PI-QUAL) score between readers with varying clinical experience and its reproducibility at assessing imaging quality between different institutions.

METHODS

Following IRB approval, we assessed 60 consecutive prostate MRI scans performed at different academic teaching and non-academic hospitals uploaded to our institutes' PACS for second opinion or discussion in case conferences. Anonymized scans were independently reviewed using the PI-QUAL scoring sheet by three readers - two radiologists (with 1 and 12 years Prostate MRI reporting experience), and an experienced MRI technician with interest in image acquisition and quality. All readers were blinded to the site where scans were acquired.

RESULTS

Agreement coefficients between the 3 readers in paired comparison for each individual PI-QUAL score was moderate. When the scans were clustered into 2 groups according to their ability to rule in or rule out clinically significant prostate cancer [i.e., PI-QUAL score 1-3 vs PI-QUAL score 4-5], the Gwet AC1 coefficients between the three readers in paired comparison was good to very good [Gwet AC 1:0.77, 0.67, 0.836 respectively] with agreement percentage of 88.3%, 83.3% and 91.7% respectively. Agreement coefficient was higher between the experienced radiologist and the experienced MRI technician than between the less experienced trainee radiologist and the other two readers. The mean PI-QUAL score provided by each reader for the scans was significantly higher in the academic hospitals (n = 32) compared to the community hospital (n = 28) [experienced radiologist 4.6 vs 2.9; trainee radiologist 4.5 vs 2.4; experienced technologist 4.4 vs 2.4; p value < 0.001].

CONCLUSION

We observed good to very good reproducibility in the assessment of each MRI sequence and when scans were clustered into two groups [PI-QUAL 1-3 vs PI-QUAL 4-5] between readers with varying clinical experience. However, the reproducibility for each single PI-QUAL score between readers was moderate. Better definitions for each PI-QUAL score criteria may further improve reproducibility between readers. Additionally, the mean PI-QUAL score provided by all three readers was significantly higher for scans performed at academic teaching hospitals compared to community hospital.

Breast cancer risk assessment and risk distribution in 3,491 Slovenian women invited for screening at the age of 50; a population-based cross-sectional study

BACKGROUND

The evidence shows that risk-based strategy could be implemented to avoid unnecessary harm in mammography screening for breast cancer (BC) using age-only criterium. Our study aimed at identifying the uptake of Slovenian women to the BC risk assessment invitation and assessing the number of screening mammographies in case of risk-based screening.

PATIENTS AND METHODS

A cross-sectional population-based study enrolled 11,898 women at the age of 50, invited to BC screening. The data on BC risk factors, including breast density from the first 3,491 study responders was collected and BC risk was assessed using the Tyrer-Cuzick algorithm (version 8) to classify women into risk groups (low, population, moderately increased, and high risk group). The number of screening mammographies according to risk stratification was simulated.

RESULTS

57% (6,785) of women returned BC risk questionnaires. When stratifying 3,491 women into risk groups, 34.0% were assessed with low, 62.2% with population, 3.4% with moderately increased, and 0.4% with high 10-year BC risk. In the case of potential personalised screening, the number of screening mammographies would drop by 38.6% compared to the current screening policy.

CONCLUSIONS

The study uptake showed the feasibility of risk assessment when inviting women to regular BC screening. 3.8% of Slovenian women were recognised with higher than population 10-year BC risk. According to Slovenian BC guidelines they may be screened more often. Overall, personalised screening would decrease the number of screening mammographies in Slovenia. This information is to be considered when planning the pilot and assessing the feasibility of implementing population risk-based screening.

Mammographic Breast Density Model Using Semi-Supervised Learning Reduces Inter-/Intra-Reader Variability

Breast density is an important risk factor for breast cancer development; however, imager inconsistency in density reporting can lead to patient and clinician confusion. A deep learning (DL) model for mammographic density grading was examined in a retrospective multi-reader multi-case study consisting of 928 image pairs and assessed for impact on inter- and intra-reader variability and reading time. Seven readers assigned density categories to the images, then re-read the test set aided by the model after a 4-week washout. To measure intra-reader agreement, 100 image pairs were blindly double read in both sessions. Linear Cohen Kappa (κ) and Student's t-test were used to assess the model and reader performance. The model achieved a κ of 0.87 (95% CI: 0.84, 0.89) for four-class density assessment and a κ of 0.91 (95% CI: 0.88, 0.93) for binary non-dense/dense assessment. Superiority tests showed significant reduction in inter-reader variability (κ improved from 0.70 to 0.88, p ≤ 0.001) and intra-reader variability (κ improved from 0.83 to 0.95, p ≤ 0.01) for four-class density, and significant reduction in inter-reader variability (κ improved from 0.77 to 0.96, p ≤ 0.001) and intra-reader variability (κ improved from 0.89 to 0.97, p ≤ 0.01) for binary non-dense/dense assessment when aided by DL. The average reader mean reading time per image pair also decreased by 30%, 0.86 s (95% CI: 0.01, 1.71), with six of seven readers having reading time reductions.

International Interobserver Variability of Breast Density Assessment

PURPOSE

The aim of this study was to determine variability in visually assessed mammographic breast density categorization among radiologists practicing in Indonesia, the Netherlands, South Africa, and the United States.

METHODS

Two hundred consecutive 2-D full-field digital screening mammograms obtained from September to December 2017 were selected and retrospectively reviewed from four global locations, for a total of 800 mammograms. Three breast radiologists in each location (team) provided consensus density assessments of all 800 mammograms using BI-RADS® density categorization. Interreader agreement was compared using Gwet's AC2 with quadratic weighting across all four density categories and Gwet's AC1 for binary comparison of combined not dense versus dense categories. Variability of distribution among teams was calculated using the Stuart-Maxwell test of marginal homogeneity across all four categories and using the McNemar test for not dense versus dense categories. To compare readers from a particular country on their own 200 mammograms versus the other three teams, density distribution was calculated using conditional logistic regression.

RESULTS

For all 800 mammograms, interreader weighted agreement for distribution among four density categories was 0.86 (Gwet's AC2 with quadratic weighting; 95% confidence interval, 0.85-0.88), and for not dense versus dense categories, it was 0.66 (Gwet's AC1; 95% confidence interval, 0.63-0.70). Density distribution across four density categories was significantly different when teams were compared with one another and one team versus the other three teams combined (P < .001). Overall, all readers placed the largest number of mammograms in the scattered and heterogeneous categories.

CONCLUSIONS

Although reader teams from four different global locations had almost perfect interreader agreement in BI-RADS density categorization, variability in density distribution across four categories remained statistically significant.