Benefits of Independent Double Reading in Digital Mammography: A Theoretical Evaluation of All Possible Pairing Methodologies

Optimizing the Pairs of Radiologists That Double Read Screening Mammograms

Background Despite variation in performance characteristics among radiologists, the pairing of radiologists for the double reading of screening mammograms is performed randomly. It is unknown how to optimize pairing to improve screening performance. Purpose To investigate whether radiologist performance characteristics can be used to determine the optimal set of pairs of radiologists to double read screening mammograms for improved accuracy. Materials and Methods This retrospective study was performed with reading outcomes from breast cancer screening programs in Sweden (2008-2015), England (2012-2014), and Norway (2004-2018). Cancer detection rates (CDRs) and abnormal interpretation rates (AIRs) were calculated, with AIR defined as either reader flagging an examination as abnormal. Individual readers were divided into performance categories based on their high and low CDR and AIR. The performance of individuals determined the classification of pairs. Random pair performance, for which any type of pair was equally represented, was compared with the performance of specific pairing strategies, which consisted of pairs of readers who were either opposite or similar in AIR and/or CDR. Results Based on a minimum number of examinations per reader and per pair, the final study sample consisted of 3 592 414 examinations (Sweden, n = 965 263; England, n = 837 048; Norway, n = 1 790 103). The overall AIRs and CDRs for all specific pairing strategies (Sweden AIR range, 45.5-56.9 per 1000 examinations and CDR range, 3.1-3.6 per 1000; England AIR range, 68.2-70.5 per 1000 and CDR range, 8.9-9.4 per 1000; Norway AIR range, 81.6-88.1 per 1000 and CDR range, 6.1-6.8 per 1000) were not significantly different from the random pairing strategy (Sweden AIR, 54.1 per 1000 examinations and CDR, 3.3 per 1000; England AIR, 69.3 per 1000 and CDR, 9.1 per 1000; Norway AIR, 84.1 per 1000 and CDR, 6.3 per 1000). Conclusion Pairing a set of readers based on different pairing strategies did not show a significant difference in screening performance when compared with random pairing. © RSNA, 2023.

Global Radiomic Features from Mammography for Predicting Difficult-To-Interpret Normal Cases

This work aimed to investigate whether global radiomic features (GRFs) from mammograms can predict difficult-to-interpret normal cases (NCs). Assessments from 537 readers interpreting 239 normal mammograms were used to categorise cases as 120 difficult-to-interpret and 119 easy-to-interpret based on cases having the highest and lowest difficulty scores, respectively. Using lattice- and squared-based approaches, 34 handcrafted GRFs per image were extracted and normalised. Three classifiers were constructed: (i) CC and (ii) MLO using the GRFs from corresponding craniocaudal and mediolateral oblique images only, based on the random forest technique for distinguishing difficult- from easy-to-interpret NCs, and (iii) CC + MLO using the median predictive scores from both CC and MLO models. Useful GRFs for the CC and MLO models were recognised using a scree test. The CC and MLO models were trained and validated using the leave-one-out-cross-validation. The models' performances were assessed by the AUC and compared using the DeLong test. A Kruskal-Wallis test was used to examine if the 34 GRFs differed between difficult- and easy-to-interpret NCs and if difficulty level based on the traditional breast density (BD) categories differed among 115 low-BD and 124 high-BD NCs. The CC + MLO model achieved higher performance (0.71 AUC) than the individual CC and MLO model alone (0.66 each), but statistically non-significant difference was found (all p > 0.05). Six GRFs were identified to be valuable in describing difficult-to-interpret NCs. Twenty features, when compared between difficult- and easy-to-interpret NCs, differed significantly (p < 0.05). No statistically significant difference was observed in difficulty between low- and high-BD NCs (p = 0.709). GRF mammographic analysis can predict difficult-to-interpret NCs.

Screening outcome for interpretation by the first and second reader in a population-based mammographic screening program with independent double reading

BACKGROUND

Double reading of screening mammograms is associated with a higher rate of screen-detected cancer than single reading, but different strategies exist regarding reader pairing and blinding. Knowledge about these aspects is important when considering strategies for future use of artificial intelligence in mammographic screening.

PURPOSE

To investigate screening outcome, histopathological tumor characteristics, and mammographic features stratified by the first and the second reader in a population based screening program for breast cancer.

MATERIAL AND METHODS

The study sample consisted of data from 3,499,048 screening examinations from 834,691 women performed during 1996-2018 in BreastScreen Norway. All examinations were interpreted independently by two radiologists, 272 in total. We analyzed interpretation score, recall, and cancer detection, as well as histopathological tumor characteristics and mammographic features of the cancers, stratified by the first and second readers.

RESULTS

For Reader 1, the rate of positive interpretations was 4.8%, recall 2.3%, and cancer detection 0.5%. The corresponding percentages for Reader 2 were 4.9%, 2.5%, and 0.5% (P < 0.05 compared with Reader 1). No statistical difference was observed for histopathological tumor characteristics or mammographic features when stratified by Readers 1 and 2. Recall and cancer detection were statistically higher and histopathological tumor characteristics less favorable for cases detected after concordant positive compared with discordant interpretations.

CONCLUSION

Despite reaching statistical significance, mainly due to the large study sample, we consider the differences in interpretation scores, recall, and cancer detection between the first and second readers to be clinically negligible. For practical and clinical purposes, double reading in BreastScreen Norway is independent.

Breast cancer missed at screening; hindsight or mistakes?

PURPOSE

To investigate radiologists' interpretation scores of screening mammograms prior to diagnosis of screen-detected and interval breast cancers retrospectively classified as missed or true negative.

METHODS

We included data on radiologists' interpretation scores at screening prior to diagnosis for 1223 screen-detected and 1007 interval cancer cases classified as missed or true negative in an informed consensus-based review. All prior screening examinations were independently scored 1-5 by two radiologists; score 1 by both was considered concordant negative, score ≥ 2 by one radiologist discordant, and score ≥ 2 by both concordant positive. We analyzed associations between interpretation, review categories, mammographic features and histopathological findings using descriptive statistics and logistic regression.

RESULTS

Among screen-detected cancers, 31% of missed and 10% of true negative cancers had discordant or concordant positive interpretation at prior screening. The corresponding percentages for interval cancer were 21% and 8%. Age-adjusted odds ratio (OR) and 95% confidence interval (CI) for missed screen-detected cancer was 3.8 (95% CI: 2.6-5.4) after discordant and 5.5 (95% CI: 3.2-9.5) after concordant positive interpretation, using concordant negative as reference. Corresponding ORs for missed interval cancer were 3.0 (95% CI: 2.0-4.5) for discordant and 6.3 (95% CI: 2.3-17.5) for concordant positive interpretation. Asymmetry was the dominating mammographic feature at prior screening for all, except concordant positive screen-detected cancers where a mass dominated. Histopathological characteristics did not vary statistically with interpretation.

CONCLUSIONS

Most cancers were interpreted negatively at screening prior to diagnosis. Increased risk for missed screen-detected or interval cancer was observed after positive interpretation at prior screening.

Idiosyncratic biases in the perception of medical images

Introduction

Radiologists routinely make life-altering decisions. Optimizing these decisions has been an important goal for many years and has prompted a great deal of research on the basic perceptual mechanisms that underlie radiologists' decisions. Previous studies have found that there are substantial individual differences in radiologists' diagnostic performance (e.g., sensitivity) due to experience, training, or search strategies. In addition to variations in sensitivity, however, another possibility is that radiologists might have perceptual biases-systematic misperceptions of visual stimuli. Although a great deal of research has investigated radiologist sensitivity, very little has explored the presence of perceptual biases or the individual differences in these.

Methods

Here, we test whether radiologists' have perceptual biases using controlled artificial and Generative Adversarial Networks-generated realistic medical images. In Experiment 1, observers adjusted the appearance of simulated tumors to match the previously shown targets. In Experiment 2, observers were shown with a mix of real and GAN-generated CT lesion images and they rated the realness of each image.

Results

We show that every tested individual radiologist was characterized by unique and systematic perceptual biases; these perceptual biases cannot be simply explained by attentional differences, and they can be observed in different imaging modalities and task settings, suggesting that idiosyncratic biases in medical image perception may widely exist.

Discussion

Characterizing and understanding these biases could be important for many practical settings such as training, pairing readers, and career selection for radiologists. These results may have consequential implications for many other fields as well, where individual observers are the linchpins for life-altering perceptual decisions.

A machine learning model based on readers' characteristics to predict their performances in reading screening mammograms

OBJECTIVES

Proposing a machine learning model to predict readers' performances, as measured by the area under the receiver operating characteristics curve (AUC) and lesion sensitivity, using the readers' characteristics.

METHODS

Data were collected from 905 radiologists and breast physicians who completed at least one case-set of 60 mammographic images containing 40 normal and 20 biopsy-proven cancer cases. Nine different case-sets were available. Using a questionnaire, we collected radiologists' demographic details, such as reading volume and years of experience. These characteristics along with a case set difficulty measure were fed into two ensemble of regression trees to predict the readers' AUCs and lesion sensitivities. We calculated the Pearson correlation coefficient between the predicted values by the model and the actual AUC and lesion sensitivity. The usefulness of the model to categorize readers as low and high performers based on different criteria was also evaluated. The performances of the models were evaluated using leave-one-out cross-validation.

RESULTS

The Pearson correlation coefficient between the predicted AUC and actual one was 0.60 (p < 0.001). The model's performance for differentiating the reader in the first and fourth quartile based on the AUC values was 0.86 (95% CI 0.83-0.89). The model reached an AUC of 0.91 (95% CI 0.88-0.93) for distinguishing the readers in the first quartile from the fourth one based on the lesion sensitivity.

CONCLUSION

A machine learning model can be used to categorize readers as high- or low-performing. Such model could be useful for screening programs for designing a targeted quality assurance and optimizing the double reading practice.

Diagnostic Efficacy across Dense and Non-Dense Breasts during Digital Breast Tomosynthesis and Ultrasound Assessment for Recalled Women

Background: To compare the diagnostic efficacy of digital breast tomosynthesis (DBT) and ultrasound across breast densities in women recalled for assessment. Methods: A total of 482 women recalled for assessment from January 2017 to December 2019 were selected for the study. Women met the inclusion criteria if they had undergone DBT, ultrasound and had confirmed biopsy results. We calculated sensitivity, specificity, PPV, and AUC for DBT and ultrasound. Results: In dense breasts, DBT showed significantly higher sensitivity than ultrasound (98.2% vs. 80%; p < 0.001), but lower specificity (15.4% vs. 55%; p < 0.001), PPV (61.3% vs. 71%; p = 0.04) and AUC (0.568 vs. 0.671; p = 0.001). In non-dense breasts, DBT showed significantly higher sensitivity than ultrasound (99.2% vs. 84%; p < 0.001), but no differences in specificity (22% vs. 33%; p = 0.14), PPV (69.2% vs. 68.8%; p = 0.93) or AUC (0.606 vs. 0.583; p = 0.57). Around 73% (74% dense and 71% non-dense) and 77% (81% dense and 72% non-dense) of lesions assigned a RANZCR 3 by DBT and ultrasound, respectively, were benign. Conclusion: DBT has higher sensitivity, but lower specificity and PPV than ultrasound in women with dense breasts recalled for assessment. Most lesions rated RANZCR 3 on DBT and ultrasound are benign and may benefit from short interval follow-up rather than biopsy.

True and Missed Interval Cancer in Organized Mammographic Screening: A Retrospective Review Study of Diagnostic and Prior Screening Mammograms

RATIONALE AND OBJECTIVES

To explore radiological aspects of interval breast cancer in a population-based screening program.

MATERIALS AND METHODS

We performed a consensus-based informed review of mammograms from diagnosis and prior screening from women diagnosed with interval cancer 2004-2016 in BreastScreen Norway. Cases were classified as true (no findings on prior screening mammograms), occult (no findings at screening or diagnosis), minimal signs (minor/non-specific findings) and missed (obvious findings). We analyzed mammographic findings, density, time since prior screening, and histopathological characteristics between the classification groups.

RESULTS

The study included 1010 interval cancer cases. Mean age at diagnosis was 61 years (SD = 6), mean time between screening and diagnosis 14 months (SD = 7). A total of 48% (479/1010) were classified as true or occult, 28% (285/1010) as minimal signs and 24% (246/1010) as missed. We observed no differences in mammographic density between the groups, except from a higher percentage of dense breasts in women with occult cancer. Among cancers classified as missed, about 1/3 were masses and 1/3 asymmetries at prior screening. True interval cancers were diagnosed later in the screening interval than the other classification categories. No differences in histopathological characteristics were observed between true, minimal signs and missed cases.

CONCLUSION

In an informed review, 24% of the interval cancers were classified as missed based on visibility and mammographic findings on prior screening mammograms. Three out of four true interval cancers were diagnosed in the second year of the screening interval. We observed no statistical differences in histopathological characteristics between true and missed interval cancers.

SCREENING MAMMOGRAPHY: DIAGNOSTIC EFFICACY-ISSUES AND CONSIDERATIONS FOR THE 2020S

Diagnostic efficacy in medical imaging is ultimately a reflection of radiologist performance. This can be influenced by numerous factors, some of which are patient related, such as the physical size and density of the breast, and machine related, where some lesions are difficult to visualise on traditional imaging techniques. Other factors are human reader errors that occur during the diagnostic process, which relate to reader experience and their perceptual and cognitive oversights. Given the large-scale nature of breast cancer screening, even small increases in diagnostic performance equate to large numbers of women saved. It is important to identify the causes of diagnostic errors and how detection efficacy can be improved. This narrative review will therefore explore the various factors that influence mammographic performance and the potential solutions used in an attempt to ameliorate the errors made.

Public Health Radiography: A Scoping Review of Benefits, and Growth Opportunities for Radiographers

INTRODUCTION

There is growing adoption of radiographic techniques in public health to improve outcomes of chronic and communicable diseases. This review examines the applications, benefits, and implications of radiography in public health. It also examines the challenges and potential advanced practice roles for radiographers in public health radiography (PHR).

METHODOLOGY

Preferred Reporting Items for Systematic Reviews and Meta-Analyses - Scoping review extension (PRISMA- ScR) checklist was employed, and the search was conducted using PubMed, Medline, Web of Science, ScienceDirect, and Google Scholar to identify relevant articles that explored the concept of radiography in public health. Evidence was analysed using an inductive iterative approach.

RESULTS

Radiographic imaging modalities such as ultrasound, computed tomography, and plain X-ray had wide applicability in public health fields of preventive cardiology, preventive oncology, maternal health, infectious disease epidemiology, and radiographic informatics. PHR effectively reduced mortality, improved outcomes, informed lifestyle changes to mitigate the risk of impending disease. PHR also helped in monitoring disease progression and predicting treatment outcomes. However, evidence establishing a competency framework that supports PHR is scarce.

CONCLUSION

Radiography makes a significant contribution to public health in reducing mortality and morbidity. Therefore, developing a PHR competency framework can accentuate the contribution Radiographers make to solving public health issues.

Evaluation of deep learning-based artificial intelligence techniques for breast cancer detection on mammograms: Results from a retrospective study using a BreastScreen Victoria dataset

Introduction

This study aims to evaluate deep learning (DL)‐based artificial intelligence (AI) techniques for detecting the presence of breast cancer on a digital mammogram image.

Methods

We evaluated several DL‐based AI techniques that employ different approaches and backbone DL models and tested the effect on performance of using different data‐processing strategies on a set of digital mammographic images with annotations of pathologically proven breast cancer.

Results

Our evaluation uses the area under curve (AUC) and accuracy (ACC) for performance measurement. The best evaluation result, based on 349 test cases (930 test images), was an AUC of 0.8979 [95% confidence interval (CI) 0.873, 0.923] and ACC of 0.8178 [95% CI 0.785, 0.850]. This was achieved by an AI technique that utilises a certain family of DL models, namely ResNet, as its backbone, combines the global features extracted from the whole mammogram and the local features extracted from the automatically detected cancer and non‐cancer local regions in the whole image, and leverages background cropping and text removal, contrast adjustment and more training data.

Conclusion

DL‐based AI techniques have shown promising results in retrospective studies for many medical image analysis applications. Our study demonstrates a significant opportunity to boost the performance of such techniques applied to breast cancer detection by exploring different types of approaches, backbone DL models and data‐processing strategies. The promising results we have obtained suggest further development of AI reading services could transform breast cancer screening in the future.

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme

OBJECTIVES

Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance.

METHODS

We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance.

RESULTS

In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted).

CONCLUSION

Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected.

KEY POINTS

• In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

Batch Reading and Interrupted Interpretation of Digital Screening Mammograms Without and With Tomosynthesis

OBJECTIVE

To compare batch reading and interrupted interpretation for modern screening mammography.

METHODS

We retrospectively reviewed digital mammograms without and with tomosynthesis that were originally interpreted with batch reading or interrupted interpretation between January 2015 and June 2017. The following performance metrics were compared: recall rate (per 100 examinations), cancer detection rate (per 1,000 examinations), and positive predictive values for recall and biopsy.

RESULTS

In all, 9,832 digital mammograms were batch read, yielding a recall rate of 9.98%, cancer detection rate of 4.27, and positive predictive values for recall and biopsy of 4.40% and 35.5%, respectively. There were 49,496 digital mammograms that were read with interrupted interpretation, yielding a recall rate of 11.3%, cancer detection rate of 4.44, and positive predictive values for recall and biopsy of 3.92% and 30.1%, respectively. Of the digital mammograms with tomosynthesis, 7,075 were batch read, yielding a recall rate of 6.98%, cancer detection rate of 5.37, and positive predictive values for recall and biopsy of 7.69% and 38.0%, respectively. Of the digital mammograms with tomosynthesis, 24,380 were read with interrupted interpretation, yielding a recall rate of 8.30%, cancer detection rate of 5.41, and positive predictive values for recall and biopsy of 6.52% and 33.3%, respectively. For both digital mammograms without and with tomosynthesis, recall rates improved with batch reading compared with interrupted interpretation (P < .001), but no significant differences were seen for other metrics.

DISCUSSION

Batch reading digital mammograms without and with tomosynthesis improves recall rates while maintaining cancer detection rates and positive predictive values compared with interrupted interpretation.

Double reading in breast cancer screening: considerations for policy-making

In this article, we explore the evidence around the relative benefits and harms of breast cancer screening using a single radiologist to examine each female's mammograms for signs of cancer (single reading), or two radiologists (double reading). First, we briefly explore the historical evidence using film-screen mammography, before providing an in-depth description of evidence using digital mammography. We classify studies according to which exact version of double reading they use, because the evidence suggests that effectiveness of double reading is contingent on whether the two radiologists are blinded to one another's decisions, and how the decisions of the two radiologists are integrated. Finally, we explore the implications for future mammography, including using artificial intelligence as the second reader, and applications to more complex three-dimensional imaging techniques such as tomosynthesis.