Discrepant screening mammography assessments at blinded and non-blinded double reading: impact of arbitration by a third reader on screening outcome

Service and clinical impacts of reader bias in breast cancer screening: a retrospective study

OBJECTIVES

To determine factors influencing reader agreement in breast screening and investigate the relationship between agreement level and patient outcomes.

METHODS

Reader pair agreement for 83 265 sets of mammograms from the Scottish Breast Screening service (2015-2020) was evaluated using Cohen's kappa statistic. Each mammography examination was read by two readers, per routine screening practice, with the second initially blinded but able to choose to view the first reader's opinion. If the two readers disagreed, a third reader arbitrated. Variation in reader agreement was examined by: whether the reader acted as the first or second reader, reader experience, and recall, cancer detection and arbitration recall rate.

RESULTS

Readers' opinions varied by whether they acted as the first or second reader. Furthermore, reader 2 was more likely to agree with reader 1 if reader 1 was more experienced than they were, and less likely to agree if they themselves were more experienced than reader 1 (P < .001). Agreement was not significantly associated with cancer detection rate, overall recall rate or arbitration recall rates (P > .05). Lower agreement between readers led to a higher arbiter workload (P < .001).

CONCLUSIONS

In mammography screening, the second reader's opinion is influenced by the first reader's opinion, with the degree of influence dependent on the readers' relative experience levels.

ADVANCES IN KNOWLEDGE

While less-experienced readers relied on their more experienced reading partner, no adverse impact on service outcomes was observed. Allowing access to the first reader's opinion may benefit newly qualified readers, but reduces independent evaluation, which may lower cancer detection rates.

Screening outcome for interpretation by the first and second reader in a population-based mammographic screening program with independent double reading

BACKGROUND

Double reading of screening mammograms is associated with a higher rate of screen-detected cancer than single reading, but different strategies exist regarding reader pairing and blinding. Knowledge about these aspects is important when considering strategies for future use of artificial intelligence in mammographic screening.

PURPOSE

To investigate screening outcome, histopathological tumor characteristics, and mammographic features stratified by the first and the second reader in a population based screening program for breast cancer.

MATERIAL AND METHODS

The study sample consisted of data from 3,499,048 screening examinations from 834,691 women performed during 1996-2018 in BreastScreen Norway. All examinations were interpreted independently by two radiologists, 272 in total. We analyzed interpretation score, recall, and cancer detection, as well as histopathological tumor characteristics and mammographic features of the cancers, stratified by the first and second readers.

RESULTS

For Reader 1, the rate of positive interpretations was 4.8%, recall 2.3%, and cancer detection 0.5%. The corresponding percentages for Reader 2 were 4.9%, 2.5%, and 0.5% (P < 0.05 compared with Reader 1). No statistical difference was observed for histopathological tumor characteristics or mammographic features when stratified by Readers 1 and 2. Recall and cancer detection were statistically higher and histopathological tumor characteristics less favorable for cases detected after concordant positive compared with discordant interpretations.

CONCLUSION

Despite reaching statistical significance, mainly due to the large study sample, we consider the differences in interpretation scores, recall, and cancer detection between the first and second readers to be clinically negligible. For practical and clinical purposes, double reading in BreastScreen Norway is independent.

Breast cancer missed at screening; hindsight or mistakes?

PURPOSE

To investigate radiologists' interpretation scores of screening mammograms prior to diagnosis of screen-detected and interval breast cancers retrospectively classified as missed or true negative.

METHODS

We included data on radiologists' interpretation scores at screening prior to diagnosis for 1223 screen-detected and 1007 interval cancer cases classified as missed or true negative in an informed consensus-based review. All prior screening examinations were independently scored 1-5 by two radiologists; score 1 by both was considered concordant negative, score ≥ 2 by one radiologist discordant, and score ≥ 2 by both concordant positive. We analyzed associations between interpretation, review categories, mammographic features and histopathological findings using descriptive statistics and logistic regression.

RESULTS

Among screen-detected cancers, 31% of missed and 10% of true negative cancers had discordant or concordant positive interpretation at prior screening. The corresponding percentages for interval cancer were 21% and 8%. Age-adjusted odds ratio (OR) and 95% confidence interval (CI) for missed screen-detected cancer was 3.8 (95% CI: 2.6-5.4) after discordant and 5.5 (95% CI: 3.2-9.5) after concordant positive interpretation, using concordant negative as reference. Corresponding ORs for missed interval cancer were 3.0 (95% CI: 2.0-4.5) for discordant and 6.3 (95% CI: 2.3-17.5) for concordant positive interpretation. Asymmetry was the dominating mammographic feature at prior screening for all, except concordant positive screen-detected cancers where a mass dominated. Histopathological characteristics did not vary statistically with interpretation.

CONCLUSIONS

Most cancers were interpreted negatively at screening prior to diagnosis. Increased risk for missed screen-detected or interval cancer was observed after positive interpretation at prior screening.

Optimising breast cancer screening reading: blinding the second reader to the first reader's decisions

Objectives

In breast cancer screening, two readers separately examine each woman’s mammograms for signs of cancer. We examined whether preventing the two readers from seeing each other’s decisions (blinding) affects behaviour and outcomes.

Methods

This cohort study used data from the CO-OPS breast-screening trial (1,119,191 women from 43 screening centres in England) where all discrepant readings were arbitrated. Multilevel models were fitted using Markov chain Monte Carlo to measure whether reader 2 conformed to the decisions of reader 1 when they were not blinded, and the effect of blinding on overall rates of recall for further tests and cancer detection. Differences in positive predictive value (PPV) were assessed using Pearson’s chi-squared test.

Results

When reader 1 recalls, the probability of reader 2 also recalling was higher when not blinded than when blinded, suggesting readers may be influenced by the other’s decision. Overall, women were less likely to be recalled when reader 2 was blinded (OR 0.923; 95% credible interval 0.864, 0.986), with no clear pattern in cancer detection rate (OR 1.029; 95% credible interval 0.970, 1.089; Bayesian p value 0.832). PPV was 22.1% for blinded versus 20.6% for not blinded (p < 0.001).

Conclusions

Our results suggest that when not blinded, reader 2 is influenced by reader 1’s decisions to recall (alliterative bias) which would result in bypassing arbitration and negate some of the benefits of double-reading. We found a relationship between blinding the second reader and slightly higher PPV of breast cancer screening, although this analysis may be confounded by other centre characteristics.

Key Points

• In Europe, it is recommended that breast screening mammograms are analysed by two readers but there is little evidence on the effect of ‘blinding’ the readers so they cannot see each other’s decisions.

• We found evidence that when the second reader is not blinded, they are more likely to agree with a recall decision from the first reader and less likely to make an independent judgement (alliterative error). This may reduce overall accuracy through bypassing arbitration.

• This observational study suggests an association between blinding the second reader and higher positive predictive value of screening, but this may be confounded by centre characteristics.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-07965-z.

Characteristics of screen-detected cancers following concordant or discordant recalls at blinded double reading in biennial digital screening mammography

OBJECTIVES

To analyse which mammographic and tumour characteristics led to concordant versus discordant recalls at blinded double reading to further optimise our breast cancer screening programme.

METHODS

We included a consecutive series of 99,013 screening mammograms obtained between July 2013 and January 2015. All mammograms were double read in a blinded fashion. Discordant readings were routinely recalled without consensus or arbitration. During the 2-year follow-up, relevant data of the recalled women were collected. We compared mammographic characteristics, screening outcome and tumour characteristics between concordant and discordant recalls.

RESULTS

There were 2,543 concordant recalls (71.4%) and 997 discordant recalls (28.0%). The positive predictive value of a concordant recall was significantly higher (23.5% vs. 10.0%, p < 0.001). The proportion of BI-RADS 0 was significantly higher in the discordant recall group (75.7% vs. 56.3%, p < 0.001). Discordant recalls were more often an asymmetry or architectural distortion (21.8% vs. 13.2% and 9.3% vs. 6.5%, respectively, p < 0.001). There were no differences in the distribution of DCIS and invasive cancers and tumour characteristics were comparable for the two groups, except for a more favourable tumour grade in the discordant recall group (54.7% vs. 39.9% grade I tumours, p = 0.022).

CONCLUSIONS

Screen-detected cancers detected by a discordant reading show a more favourable tumour grade than cancers diagnosed after a concordant recall. The higher proportion of asymmetries and architectural distortions in this group provide a possible target for improving screening programmes by additional training of screening radiologists and the implementation of digital breast tomosynthesis.

KEY POINTS

• With blinded double reading of screening mammograms, screen-detected cancers detected by a discordant reading show a more favourable tumour grade than cancers diagnosed after a concordant recall. • The proportions of asymmetries and architectural distortions are higher in case of a discordant reading. • Possible improvement strategies could target additional training of screening radiologists and the implementation of digital breast tomosynthesis in breast cancer screening programmes.

Trends in incidence and tumour grade in screen-detected ductal carcinoma in situ and invasive breast cancer

PURPOSE

In a biennial screening mammography programme, we analysed the trends in incidence of screen-detected DCIS and invasive breast cancers in the era of screen-film mammography (SFM) screening, the period of the transition to full-field digital mammography (FFDM) screening and the period of FFDM screening. We also investigated a possible association between the incidence and grading of screen-detected DCIS and invasive breast cancer.

METHODS

In the southern part of the Netherlands, FFDM screening gradually replaced SFM screening between May 2009 and April 2010. We included a consecutive series of 484, 422 screens obtained between July 2005 and July 2015 and divided these screens into three groups; SFM-only cohort, transition cohort and FFDM-only cohort.

RESULTS

A total of 3059 referred women were diagnosed with DCIS (n = 623) or invasive breast cancer (n = 2436). The majority of DCIS were high-grade (48.2%), whereas the majority of the invasive breast cancers were low-grade (45.4%) or intermediate-grade (41.6%). The cancer detection rate (CDR) per 1000 screened women showed the same distribution by grade in both groups. The transition to FFDM was characterised by an increased overall detection rate of invasive cancers.

CONCLUSIONS

Screening mammography detects mostly high-grade DCIS and low- or intermediate-grade invasive cancers. The grade distribution as well as the CDR in the years after the introduction of FFDM remained stable compared to the era of SFM screening. By diagnosing and treating high-grade DCIS, which otherwise may develop into high-grade invasive carcinoma, our findings provide new evidence for the beneficial value of screening mammography programmes.

Double versus single reading of mammograms in a breast cancer screening programme: a cost-consequence analysis

OBJECTIVES

To assess the costs and health-related outcomes of double versus single reading of digital mammograms in a breast cancer screening programme.

METHODS

Based on data from 57,157 digital screening mammograms from women aged 50-69 years, we compared costs, false-positive results, positive predictive value and cancer detection rate using four reading strategies: double reading with and without consensus and arbitration, and single reading with first reader only and second reader only. Four highly trained radiologists read the mammograms.

RESULTS

Double reading with consensus and arbitration was 15 % (Euro 334,341) more expensive than single reading with first reader only. False-positive results were more frequent at double reading with consensus and arbitration than at single reading with first reader only (4.5 % and 4.2 %, respectively; p < 0.001). The positive predictive value (9.3 % and 9.1 %; p = 0.812) and cancer detection rate were similar for both reading strategies (4.6 and 4.2 per 1000 screens; p = 0.283).

CONCLUSIONS

Our results suggest that changing to single reading of mammograms could produce savings in breast cancer screening. Single reading could reduce the frequency of false-positive results without changing the cancer detection rate. These results are not conclusive and cannot be generalized to other contexts with less trained radiologists.

KEY POINTS

• Double reading of digital mammograms is more expensive than single reading. • Compared to single reading, double reading yields a higher proportion of false-positive results. • The cancer detection rate was similar for double and single readings. • Single reading may be a cost-effective strategy in breast cancer screening programmes.