Initial interpretation scores of screening mammograms and cancer detection in BreastScreen Norway

PURPOSE

To explore the association between radiologists' interpretation scores, early performance measures and cumulative reading volume in mammographic screening.

METHOD

We analyzed 1,689,731 screening examinations (3,379,462 breasts) from BreastScreen Norway 2012-2020, all breasts scored 1-5 by two independent radiologists. Score 1 was considered negative/benign and score ≥2 positive in this scoring system. We performed descriptive analyses of recall, screen-detected cancer, positive predictive value (PPV) 1, mammographic features and histopathological characteristics by breast-based interpretation scores, and cumulative reading volume by examination-based interpretation scores.

RESULTS

Counting breasts and not women, 3.9 % (132,570/3,379,462) had a score of ≥2 by one or both radiologists. Of these, 84.8 % (112,440/132,570) were given a maximum score 2. Total recall rate was 1.6 % (53,735/3,379,462), 69.3 % (37,220/53,735) given maximum score 2. Among the 0.3 % (9733/3,379,462) diagnosed with screen-detected cancer, 34.6 % (3369/9733) had maximum score 3. The percentages of recall, screen-detected cancer and PPV-1 increased by increasing the sum of scores assigned by two radiologists (p < 0.001 for trend). Higher proportions of masses were observed among recalls and screen-detected cancers with low scores, and higher proportions of spiculated masses were observed for high scores (p < 0.001). Proportions of invasive carcinoma, histological grade 3 and lymph node positive tumors were higher for high versus low scores (p < 0.001). The proportion of examinations scored 1 increased by cumulative reading volume.

CONCLUSIONS

We observed higher rates of recall and screen-detected cancer and less favorable histopathological tumor characteristics for high versus low interpretation scores. However, a considerable number of recalls and screen-detected cancers had low interpretation scores.

An initial exploration of factors that may impact radiographer performance in reporting mammograms

OBJECTIVES

In the United Kingdom, radiographers with a qualification in image interpretation have interpreted mammograms since 1995. These radiographers work under the title of radiography advanced practitioners (RAP) or Consultant Radiographer. This study extends upon what has been very recently published by exploring further clinical, non-clinical and experiential factors that may impact the reporting performance of RAPs.

METHODS

Fifteen RAPs interpreted an image test set of 60 2D mammograms of known truth using the Detected-X software platform. Unknown to the reader, twenty cases contained a malignancy. Sensitivity, specificity, lesion sensitivity, receiver operating characteristic (ROC) and jack-knife free response operating characteristic (AFROC) values were established for each RAP. Specific features that had significant impact on accuracy were identified using Student's-T and Mann Whitney tests.

RESULTS

RAPs with more than 10 years' experience in image interpretation, compared to those with less than 10 years' experience, demonstrated lower specificity (51.3% vs 84.8%, p = 0.0264), ROC (0.83 vs 0.91, p = 0.0264) and AFROC (0.75 vs 0.87, p = 0.0037) values. Further, higher sensitivity values of 90.7% were seen in those RAPs who had an eye test in the last year compared to those who had not, 82% (p = 0.021). Other changes are presented in the paper.

CONCLUSION

These data reveal previously unidentified factors that impact the diagnostic efficacy of RAPs when interpreting mammographic images. Highlighting such findings will empower screening authorities to better examine ways of standardising performance and offer a baseline for performance benchmarks.

IMPLICATIONS FOR PRACTICE

This study for the first time performs an initial exploration of the factors that may be associated with RAP performance when interpreting screening mammograms.

A survey by the European Society of Breast Imaging on radiologists' preferences regarding quality assurance measures of image interpretation in screening and diagnostic mammography

OBJECTIVES

Quality assurance (QA) of image interpretation plays a key role in screening and diagnostic mammography, maintaining minimum standards and supporting continuous improvement in interpreting images. However, the QA structure across Europe shows considerable variation. The European Society of Breast Imaging (EUSOBI) conducted a survey among the members to collect information on radiologists' preferences regarding QA measures in mammography.

MATERIALS AND METHODS

An anonymous online survey consisting of 25 questions was distributed to all EUSOBI members and national breast radiology bodies in Europe. The questions were designed to collect demographic characteristics, information on responders' mammography workload and data about QA measures currently used in their country. Data was analysed using descriptive statistical analysis, the χ2 test, linear regression, and Durbin-Watson statistic test.

RESULTS

In total, 251 breast radiologists from 34 countries completed the survey. Most respondents were providing both screening and symptomatic services (137/251, 54.6%), working in an academic hospital (85/251, 33.9%) and reading 1000-4999 cases per year (109/251, 43.4%). More than half of them (133/251, 53%) had established QA measures in their workplace. Although less than one-third (71/251, 28.3%) had to participate in regular performance testing, the vast majority (190/251, 75.7%) agreed that a mandatory test would be helpful to improve their skills.

CONCLUSION

QA measures were in place for more than half of the respondents working in screening and diagnostic mammography to evaluate their breast imaging performance. Although there were substantial differences between countries, the importance of having QA in the workplace and implemented was widely acknowledged by radiologists.

CLINICAL RELEVANCE STATEMENT

Although several quality assurance (QA) measures of image interpretation are recommended by European bodies or national organisations, the QA in mammography is quite heterogenous between countries and reporting settings, and not always actively implemented across Europe.

KEY POINTS

The first survey that presents radiologists' preferences regarding QA measures of image interpretation in mammography. Quality assurance measures in the workplace are better-established for breast screening compared to diagnostic mammography. Radiologists consider that performance tests would help to improve their mammography interpretation skills.

Artificial Intelligence Improves Detection of Supplemental Screening Ultrasound-detected Breast Cancers in Mammography

Despite recent advances in artificial intelligence (AI) software with improved performance in mammography screening for breast cancer, insufficient data are available on its performance in detecting cancers that were initially missed on mammography. In this study, we aimed to determine whether AI software-aided mammography could provide additional value in identifying cancers detected through supplemental screening ultrasound. We searched our database from 2017 to 2018 and included 238 asymptomatic patients (median age, 50 years; interquartile range, 45-57 years) diagnosed with breast cancer using supplemental ultrasound. Two unblinded radiologists retrospectively reviewed the mammograms using commercially available AI software and identified the reasons for missed detection. Clinicopathological characteristics of AI-detected and AI-undetected cancers were compared using univariate and multivariate logistic regression analyses. A total of 253 cancers were detected in 238 patients using ultrasound. In an unblinded review, the AI software failed to detect 187 of the 253 (73.9%) mammography cases with negative findings in retrospective observations. The AI software detected 66 cancers (26.1%), of which 42 (63.6%) exhibited indiscernible findings obscured by overlapping dense breast tissues, even with the knowledge of magnetic resonance imaging and post-wire localization mammography. The remaining 24 cases (36.4%) were considered interpretive errors by the radiologists. Invasive tumor size was associated with AI detection after multivariable analysis (odds ratio, 2.2; 95% confidence intervals, 1.5-3.3; p < 0.001). In the control group of 160 women without cancer, the AI software identified 19 false positives (11.9%, 19/160). Although most ultrasound-detected cancers were not detected on mammography with the use of AI, the software proved valuable in identifying breast cancers with indiscernible abnormalities or those that clinicians may have overlooked.

Do Reader Characteristics Affect Diagnostic Efficacy in Screening Mammography? A Systematic Review

To examine reader characteristics associated with diagnostic efficacy in the interpretation of screening mammograms. A systematic search of the literature was conducted using databases such as Cochrane, Scopus, Medline, Embase, Web of Science, and PubMed. Search terms were combined with "AND" or "OR" and included: "Radiologist's characteristics AND performance"; "radiologist experience AND screening mammography"; "annual volume read AND diagnostic efficacy"; "screening mammography performance OR diagnostic efficacy". Studies were included if they assessed reader performance in screening mammography interpretation, breast readers, used a reference standard to assess the performance, and were published in the English language. Twenty-eight studies were reviewed. Increasing reader's age was associated with lower false positive rates. No association was found between gender and performance. Half of the studies showed no association between years of reading mammograms and performance. Most studies showed that high reading volume was more likely to be associated with increased sensitivity, cancer detection rates (CDR), lower recall rate, and lower false positive rates. Inconsistent associations were found between fellowship training in breast imaging and reader performance. Specialization in breast imaging was associated with better CDR, sensitivity, and specificity. Limited studies were available to establish the association between performance and factors such as time spent in breast imaging (n = 2), screening focus (n = 1), formal rotation in mammography (n = 1), owner of practice (n = 1), and practice type (n = 1). No individual characteristics is associated with versatility in diagnostic efficacy, albeit reading volume and specialization in breast imaging appear to be associated with with increased sensitivity and CDR without significantly affecting other performance metrics.

Test-set training improves the detection rates of invasive cancer in screening mammography

AIM

To investigate if mammographic test-set participation affects routine breast cancer screening performance.

MATERIALS AND METHODS

Clinical audit data between 2008 and 2018 were collected for 35 breast screen readers who participated in the BreastScreen Reader Assessment Strategy (BREAST) and 22 readers with no history of test-set participation. For BREAST readers, the annual audit data were divided according to the year they completed their first test set, and the same years were used randomly to align and divide the data of non-BREAST readers into pre- and post-test set periods. Multiple audit parameters were inspected retrospectively for the two cohorts to identify how their reading performance has evolved in screening mammography.

RESULTS

Investigating 2 calendar years before and after test-set participation, BREAST and non-BREAST readers recalled lower rates of women in the latter period (p=0.03 and p=0.02, respectively). They also improved their positive predictive value (PPV; p=0.01 and p=0.02, respectively). BREAST readers additionally improved their detection rates of invasive cancer (p=0.02) and all cancers (p=0.01). In an extended 3-year comparison, similar improvements occurred in the recall rate for BREAST (p=0.02) and non-BREAST readers (p=0.02) and in PPV (p=0.001, 0.01, respectively); however, improvements in detection rates also occurred exclusively in BREAST readers' performance for invasive cancer (p=0.04), DCIS (p=0.05), and all cancers (p=0.02); however, significant improvements in detection did not involve <15 mm invasive cancers in both periods. Meanwhile, non-BREAST readers demonstrated a decrease in sensitivity (p=0.02).

CONCLUSION

Participation in test sets is linked to over-time improvements in most audit-measured cancer detection rates.

Identification of factors associated with diagnostic performance variation in reporting of mammograms: A review

OBJECTIVES

This narrative review aims to identify what factors are linked to diagnostic performance variation for those who interpret mammograms. Identification of influential factors has potential to contribute to the optimisation of breast cancer diagnosis. PubMed, ScienceDirect and Google Scholar databases were searched using the following terms: 'Radiology', 'Radiologist', 'Radiographer', 'Radiography', 'Mammography', 'Interpret', 'read', 'observe' 'report', 'screen', 'image', 'performance' and 'characteristics.' Exclusion criteria included articles published prior to 2000 as digital mammography was introduced at this time. Non-English articles language were also excluded. 38 of 2542 studies identified were analysed.

KEY FINDINGS

Influencing factors included, new technology, volume of reads, experience and training, availability of prior images, social networking, fatigue and time-of-day of interpretation. Advancements in breast imaging such as digital breast tomosynthesis and volume of mammograms are primary factors that affect performance as well as tiredness, time-of-day when images are interpreted, stages of training and years of experience. Recent studies emphasised the importance of social networking and knowledge sharing if breast cancer diagnosis is to be optimised.

CONCLUSION

It was demonstrated that data on radiologist performance variability is widely available but there is a paucity of data on radiographers who interpret mammographic images.

IMPLICATIONS FOR PRACTICE

This scarcity of research needs to be addressed in order to optimise radiography-led reporting and set baseline values for diagnostic efficacy.

A machine learning model based on readers' characteristics to predict their performances in reading screening mammograms

OBJECTIVES

Proposing a machine learning model to predict readers' performances, as measured by the area under the receiver operating characteristics curve (AUC) and lesion sensitivity, using the readers' characteristics.

METHODS

Data were collected from 905 radiologists and breast physicians who completed at least one case-set of 60 mammographic images containing 40 normal and 20 biopsy-proven cancer cases. Nine different case-sets were available. Using a questionnaire, we collected radiologists' demographic details, such as reading volume and years of experience. These characteristics along with a case set difficulty measure were fed into two ensemble of regression trees to predict the readers' AUCs and lesion sensitivities. We calculated the Pearson correlation coefficient between the predicted values by the model and the actual AUC and lesion sensitivity. The usefulness of the model to categorize readers as low and high performers based on different criteria was also evaluated. The performances of the models were evaluated using leave-one-out cross-validation.

RESULTS

The Pearson correlation coefficient between the predicted AUC and actual one was 0.60 (p < 0.001). The model's performance for differentiating the reader in the first and fourth quartile based on the AUC values was 0.86 (95% CI 0.83-0.89). The model reached an AUC of 0.91 (95% CI 0.88-0.93) for distinguishing the readers in the first quartile from the fourth one based on the lesion sensitivity.

CONCLUSION

A machine learning model can be used to categorize readers as high- or low-performing. Such model could be useful for screening programs for designing a targeted quality assurance and optimizing the double reading practice.

Artificial Intelligence Evaluation of 122 969 Mammography Examinations from a Population-based Screening Program

Background Artificial intelligence (AI) has shown promising results for cancer detection with mammographic screening. However, evidence related to the use of AI in real screening settings remain sparse. Purpose To compare the performance of a commercially available AI system with routine, independent double reading with consensus as performed in a population-based screening program. Furthermore, the histopathologic characteristics of tumors with different AI scores were explored. Materials and Methods In this retrospective study, 122 969 screening examinations from 47 877 women performed at four screening units in BreastScreen Norway from October 2009 to December 2018 were included. The data set included 752 screen-detected cancers (6.1 per 1000 examinations) and 205 interval cancers (1.7 per 1000 examinations). Each examination had an AI score between 1 and 10, where 1 indicated low risk of breast cancer and 10 indicated high risk. Threshold 1, threshold 2, and threshold 3 were used to assess the performance of the AI system as a binary decision tool (selected vs not selected). Threshold 1 was set at an AI score of 10, threshold 2 was set to yield a selection rate similar to the consensus rate (8.8%), and threshold 3 was set to yield a selection rate similar to an average individual radiologist (5.8%). Descriptive statistics were used to summarize screening outcomes. Results A total of 653 of 752 screen-detected cancers (86.8%) and 92 of 205 interval cancers (44.9%) were given a score of 10 by the AI system (threshold 1). Using threshold 3, 80.1% of the screen-detected cancers (602 of 752) and 30.7% of the interval cancers (63 of 205) were selected. Screen-detected cancer with AI scores not selected using the thresholds had favorable histopathologic characteristics compared to those selected; opposite results were observed for interval cancer. Conclusion The proportion of screen-detected cancers not selected by the artificial intelligence (AI) system at the three evaluated thresholds was less than 20%. The overall performance of the AI system was promising according to cancer detection. © RSNA, 2022.

Breast cancer detection across dense and non-dense breasts: Markers of diagnostic confidence and efficacy

Background

The impact of radiologists’ characteristics has become a major focus of recent research. However, the markers of diagnostic efficacy and confidence in dense and non-dense breasts are poorly understood.

Purpose

This study aims to assess the relationship between radiologists’ characteristics and diagnostic performance across dense and non-dense breasts.

Materials and methods

Radiologists specialising in breast imaging (n = 128) who had 0.5–40 (13±10.6) years of experience reading mammograms were recruited. Participants independently interpreted a test set containing 60 digital mammograms (40 normal and 20 abnormal) with similarly distributed breast densities. Diagnostic performance measures were analysed via Jamovi software (version 1.6.22).

Results

In dense breasts, breast-imaging fellowship completion significantly improved specificity (p = 0.004), location sensitivity (p = 0.01) and the area under the curve (AUC) of the receiver operating characteristic (p = 0.03). Only participation in BreastScreen reading significantly improved all performance metrics: specificity (p = 0.04), sensitivity (p = 0.005), location sensitivity (p < 0.001) and AUC (p < 0.001). Reading > 100 mammograms weekly significantly improved sensitivity (p = 0.03), location sensitivity (p = 0.001), and AUC (p = 0.03).In non-dense breasts, breast fellowship completion significantly improved sensitivity (p = 0.02), location sensitivity (p = 0.04) and AUC (p = 0.002). Participation in BreastScreen reading and reading > 100 mammograms weekly significantly improved only sensitivity (p = 0.002 and p = 0.003, respectively) and location sensitivity (p < 0.001 and p < 0.001, respectively).

Conclusion

Participating in screening programs, breast fellowships and reading > 100 mammograms weekly are important indicators of the diagnostic performance of radiologists across dense and non-dense breasts. In dense breasts, optimal performance resulted from participation in a breast screening program.

True and Missed Interval Cancer in Organized Mammographic Screening: A Retrospective Review Study of Diagnostic and Prior Screening Mammograms

RATIONALE AND OBJECTIVES

To explore radiological aspects of interval breast cancer in a population-based screening program.

MATERIALS AND METHODS

We performed a consensus-based informed review of mammograms from diagnosis and prior screening from women diagnosed with interval cancer 2004-2016 in BreastScreen Norway. Cases were classified as true (no findings on prior screening mammograms), occult (no findings at screening or diagnosis), minimal signs (minor/non-specific findings) and missed (obvious findings). We analyzed mammographic findings, density, time since prior screening, and histopathological characteristics between the classification groups.

RESULTS

The study included 1010 interval cancer cases. Mean age at diagnosis was 61 years (SD = 6), mean time between screening and diagnosis 14 months (SD = 7). A total of 48% (479/1010) were classified as true or occult, 28% (285/1010) as minimal signs and 24% (246/1010) as missed. We observed no differences in mammographic density between the groups, except from a higher percentage of dense breasts in women with occult cancer. Among cancers classified as missed, about 1/3 were masses and 1/3 asymmetries at prior screening. True interval cancers were diagnosed later in the screening interval than the other classification categories. No differences in histopathological characteristics were observed between true, minimal signs and missed cases.

CONCLUSION

In an informed review, 24% of the interval cancers were classified as missed based on visibility and mammographic findings on prior screening mammograms. Three out of four true interval cancers were diagnosed in the second year of the screening interval. We observed no statistical differences in histopathological characteristics between true and missed interval cancers.

Cumulative risk of a false-positive screening result: A retrospective cohort study using empirical data from 10 biennial screening rounds in BreastScreen Norway

BACKGROUND

False-positive screening results are an inevitable and commonly recognized disadvantage of mammographic screening. This study estimated the cumulative probability of experiencing a first false-positive screening result in women attending 10 biennial screening rounds in BreastScreen Norway, which targets women aged 50 to 69 years.

METHODS

This retrospective cohort study analyzed screening outcomes from 421,545 women who underwent 1,894,523 screening examinations during 1995-2019. Empirical data were used to calculate the cumulative risk of experiencing a first false-positive screening result and a first false-positive screening result that involved an invasive procedure over 10 screening rounds. Logistic regression was used to evaluate the effect of adjusting for irregular attendance, age at screening, and number of screens attended.

RESULTS

The cumulative risk of experiencing a first false-positive screening result was 18.04% (95% confidence interval [CI], 18.00%-18.07%). It was 5.01% (95% CI, 5.01%-5.02%) for experiencing a false-positive screening result that involved an invasive procedure. Adjusting for irregular attendance or age at screening did not appreciably affect these estimates. After adjustments for the number of screens attended, the cumulative risk of a first false-positive screening result was 18.28% (95% CI, 18.24%-18.32%), and the risk of a false-positive screening result including an invasive procedure was 5.11% (95% CI, 5.11%-5.22%). This suggested that there was minimal bias from dependent censoring.

CONCLUSIONS

Nearly 1 in 5 women will experience a false-positive screening result if they attend 10 biennial screening rounds in BreastScreen Norway. One in 20 will experience a false-positive screening result with an invasive procedure.

LAY SUMMARY

A false-positive screening result occurs when a woman attending mammographic screening is called back for further assessment because of suspicious findings, but the assessment does not detect breast cancer. Further assessment includes additional imaging. Usually, it involves ultrasound, and sometimes, it involves a biopsy. This study has evaluated the chance of experiencing a false-positive screening result among women attending 10 screening examinations over 20 years in BreastScreen Norway. Nearly 1 in 5 women will experience a false-positive screening result over 10 screening rounds. One in 20 women will experience a false-positive screening result involving a biopsy.

The Impact of Radiologist Screening Mammogram Reading Volume on Performance in the Ontario Breast Screening Program

PURPOSE

Although some studies have shown increasing radiologists' mammography volumes improves performance, there is a lack of evidence specific to digital mammography and breast screening program performance targets. This study evaluates the relationship between digital screening volume and meeting performance targets.

METHODS

This retrospective cohort study included 493 radiologists in the Ontario Breast Screening Program who interpreted 1,762,173 screening mammograms in participants ages 50-90 between 2014 and 2016. Associations between annual screening volume and meeting performance targets for abnormal call rate, positive predictive value (PPV), invasive cancer detection rate (CDR), sensitivity, and specificity were modeled using mixed-effects multivariate logistic regression.

RESULTS

Most radiologists read 500-999 (36.7%) or 1,000-1,999 (31.0%) screens annually, and 18.5% read ≥2,000. Radiologists who read ≥2,000 annually were more likely to meet abnormal call rate (OR = 3.85; 95% CI: 1.17-12.61), PPV (OR = 5.36; 95% CI: 2.53-11.34), invasive CDR (OR = 4.14; 95% CI: 1.50-11.46), and specificity (OR = 4.07; 95% CI: 1.89-8.79) targets versus those who read 100-499 screens. Radiologists reading 1,000-1,999 screens annually were more likely to meet PPV (OR = 2.32; 95% CI: 1.22-4.40), invasive CDR (OR = 3.36; 95% CI: 1.49-7.59) and specificity (OR = 2.00; 95% CI: 1.04-3.84) targets versus those who read 100-499 screens. No significant differences were observed for sensitivity.

CONCLUSIONS

Annual reading volume requirements of 1,000 in Canada are supported as screening volume above 1,000 was strongly associated with achieving performance targets for nearly all measures. Increasing the minimum volume to 2,000 may further reduce the potential limitations of screening due to false positives, leading to improvements in overall breast screening program quality.

Visualization of the Nipple in Profile: Does It Really Affect Selected Outcomes in Organized Mammographic Screening?

OBJECTIVE

To investigate whether having the nipple imaged in profile was associated with breast characteristics or compression parameters, and whether it affected selected outcomes in screening with standard digital mammography or digital breast tomosynthesis.

METHODS

In this IRB-approved retrospective study, results from 87 450 examinations (174 900 breasts) performed as part of BreastScreen Norway, 2016-2019, were compared by nipple in profile status and screening technique using descriptive statistics and generalized estimating equations. Unadjusted and adjusted odds ratios with 95% confidence intervals (95% CIs) were estimated for outcomes of interest, including age, breast volume, volumetric breast density, and compression force as covariates.

RESULTS

Achieving the nipple in profile versus not in profile was associated with lower breast volume (845.1 cm3 versus 1059.9 cm3, P < 0.01) and higher mammographic density (5.6% versus 4.4%, P < 0.01). Lower compression force and higher compression pressure were applied to breasts with the nipple in profile (106.6 N and 11.5 kPa) compared to the nipple not in profile (110.8 N and 10.5 kPa, P < 0.01 for both). The adjusted odds ratio was 0.95 (95% CI: 0.88-1.02; P = 0.15) for recall and 0.92 (95% CI: 0.77-1.10; P = 0.36) for screen-detected cancer for nipple in profile versus not in profile.

CONCLUSION

Breast characteristics and compression parameters might hamper imaging of the nipple in profile. However, whether the nipple was in profile or not on the screening mammograms did not influence the odds of recall or screen-detected cancer, regardless of screening technique.

The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme

PURPOSE

This retrospective study determined whether a test-set based assessment scheme (PERFORMS) used in a national breast screening programme could be used to predict real-life performance by investigating if the number of cancers missed by mammography readers in real-life related to the number of cancers missed in the PERFORMS test-set and whether real-life reading volumes affected performance.

METHOD

Data was obtained from consenting readers in the screening programme in England (NHSBSP) where double reading is standard. The rate of cancers missed by individual first readers but correctly identified by second readers was compared with the number of cancers missed in the PERFORMS test-set over a 3-year period. NHSBSP readers are required to interpret at least 1500 cases per year as a first reader, so results were compared between readers who exceeded this target and those that did not. Parametric and non-parametric correlations were calculated.

RESULTS

Amongst the 536 readers, there was a highly significant positive correlation between the real-life and PERFORMS test-set missed cancer metrics (Pearson Correlation = 0.228, n = 536, p < .0001, Spearman's rho = 0.265, n = 536, p < .0001). There was no significant difference in rates of missed cancers between the 452 readers who exceeded the 1500 first read per year target and those who did not (t(94.2) = -1.87, p = .0643, r = 0.19).

CONCLUSIONS

The use of a test-set based assessment scheme accurately reflects real-life mammography reading performance, indicating that it can be a useful tool in identifying poor reader performance.

Interval and Subsequent Round Breast Cancer in a Randomized Controlled Trial Comparing Digital Breast Tomosynthesis and Digital Mammography Screening

Background Prevalent digital breast tomosynthesis (DBT) has shown higher cancer detection rates and lower recall rates compared with those of digital mammography (DM). However, data are limited on rates and histopathologic tumor characteristics of interval and subsequent round screen-detected cancers for DBT. Purpose To follow women randomized to screening with DBT or DM and to investigate rates and tumor characteristics of interval and subsequent round screen-detected cancers. Materials and Methods To-Be is a randomized controlled trial comparing the outcome of DBT and DM in organized breast cancer screening. The trial included 28 749 women, with 22 306 women returning for subsequent DBT screening 2 years later (11 201 and 11 105 originally screened with DBT and DM, respectively). Differences in rates, means, and distribution of histopathologic tumor characteristics between women prevalently screened with DBT versus DM were evaluated with Z tests, t tests, and χ² tests. Relative risk (RR) with 95% CIs was calculated for the cancer rates. Results Interval cancer rates were 1.4 per 1000 screens (20 of 14 380; 95% CI: 0.9, 2.1) for DBT versus 2.0 per 1000 screens (29 of 14 369; 95% CI: 1.4, 2.9; P = .20) for DM. The rates of subsequent round screen-detected cancer were 8.1 per 1000 (95% CI: 6.6, 10.0) for women originally screened with DBT and 9.1 per 1000 (95% CI: 7.4, 11.0; P = .43) for women screened with DM. The distribution of tumor characteristics did not differ between groups for either interval or subsequent screen-detected cancer. The RR of interval cancer was 0.69 (95% CI: 0.39, 1.22; P = .20) for DBT versus DM, whereas RR of subsequent screen-detected cancer for women prevalently screened with DBT versus DM was 0.89 (95% CI: 0.67, 1.19; P = .43). Conclusion Rates of interval or subsequent round screen-detected cancers and their tumor characteristics did not differ between women originally screened with digital breast tomosynthesis (DBT) versus digital mammography. The analysis suggests that the benefits of prevalent DBT screening did not come at the expense of worse downstream screening performance measures in a population-based screening program. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Taourel in this issue.

Radiological review of prior screening mammograms of screen-detected breast cancer

OBJECTIVE

To perform a radiological review of mammograms from prior screening and diagnosis of screen-detected breast cancer in BreastScreen Norway, a population-based screening program.

METHODS

We performed a consensus-based informed review of mammograms from prior screening and diagnosis for screen-detected breast cancers. Mammographic density and findings on screening and diagnostic mammograms were classified according to the Breast Imaging-Reporting and Data System®. Cases were classified based on visible findings on prior screening mammograms as true (no findings), missed (obvious findings), minimal signs (minor/non-specific findings), or occult (no findings at diagnosis). Histopathologic tumor characteristics were extracted from the Cancer Registry of Norway. The Bonferroni correction was used to adjust for multiple testing; p < 0.001 was considered statistically significant.

RESULTS

The study included mammograms for 1225 women with screen-detected breast cancer. Mean age was 62 years ± 5 (SD); 46% (567/1225) were classified as true, 22% (266/1225) as missed, and 32% (392/1225) as minimal signs. No difference in mammographic density was observed between the classification categories. At diagnosis, 59% (336/567) of true and 70% (185/266) of missed cancers were classified as masses (p = 0.004). The percentage of histological grade 3 cancers was higher for true (30% (138/469)) than for missed (14% (33/234)) cancers (p < 0.001). Estrogen receptor positivity was observed in 86% (387/469) of true and 95% (215/234) of missed (p < 0.001) cancers.

CONCLUSIONS

We classified 22% of the screen-detected cancers as missed based on a review of prior screening mammograms with diagnostic images available. One main goal of the study was quality improvement of radiologists' performance and the program. Visible findings on prior screening mammograms were not necessarily indicative of screening failure.

KEY POINTS

• After a consensus-based informed review, 46% of screen-detected breast cancers were classified as true, 22% as missed, and 32% as minimal signs. • Less favorable prognostic and predictive tumor characteristics were observed in true screen-detected breast cancer compared with missed. • The most frequent mammographic finding for all classification categories at the time of diagnosis was mass, while the most frequent mammographic finding on prior screening mammograms was a mass for missed cancers and asymmetry for minimal signs.

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme

OBJECTIVES

Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance.

METHODS

We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance.

RESULTS

In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted).

CONCLUSION

Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected.

KEY POINTS

• In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

Understanding the Mammography Audit

Screening mammography aims to identify small, node-negative breast cancers when they are still curable while maintaining an acceptable range of false-positive recalls and biopsies. The mammography audit is a powerful tool to help radiologists understand their performance with respect to that goal. This article defines audit terms and describes how to use collected and derived data to perform a mammography audit. Accepted benchmarks are discussed as well as their applicability to radiologists and breast imaging practices in the United States. Special considerations regarding volumes and radiologist characteristics are explored, because these factors may affect audit results.

Expert radiologist review at a hepatobiliary multidisciplinary tumor board: impact on patient management

PURPOSE

To identify the frequency, source, and management impact of discrepancies between the initial radiology report and expert reinterpretation occurring in the context of a hepatobiliary multidisciplinary tumor board (MTB).

METHODS

This retrospective study included 974 consecutive patients discussed at a weekly MTB at a large tertiary care academic medical center over a 2-year period. A single radiologist with dedicated hepatobiliary imaging expertise attended all conferences to review and discuss the relevant liver imaging and rated the concordance between original and re-reads based on RADPEER scoring criteria. Impact on management was based on the conference discussion and reflected changes in follow-up imaging, recommendations for biopsy/surgery, or liver transplant eligibility.

RESULTS

Image reinterpretation was discordant with the initial report in 19.9% (194/974) of cases (59.8%, 34.5%, 5.7% RADPEER 2/3/4 discrepancies, respectively). A change in LI-RADS category occurred in 59.8% of discrepancies. Most common causes of discordance included re-classification of a lesion as benign rather than malignant (16.0%) and missed tumor recurrence (13.9%). Impact on management occurred in 99.0% of discordant cases and included loco-regional therapy instead of follow-up imaging (19.1%), follow-up imaging instead of treatment (17.5%), and avoidance of biopsy (12.4%). 11.3% received OPTN exception scores due to the revised interpretation, and 8.8% were excluded from listing for orthotopic liver transplant.

CONCLUSION

Even in a sub-specialized abdominal imaging academic practice, expert radiologist review in the MTB setting identified discordant interpretations and impacted management in a substantial fraction of patients, potentially impacting transplant allocation. The findings may impact how abdominal imaging sections best staff advanced MTBs.

Impact of radiomics on the breast ultrasound radiologist's clinical practice: From lumpologist to data wrangler

OBJECTIVE

The study aims to assess the impact of radiomics in the clinical practice of breast ultrasound, to determine which lesions are undetermined by the software, and to discuss the future of the radiologist's role.

METHODS

Consecutive analyses of 207 ultrasound masses from January 2018 to April 2019 referred for percutaneous breast biopsy. Breast masses were classified using dedicated ultrasound software (AI). The AI software automatically classified the masses on a scale of 0-100, where 100 is the most suspicious. We adopt the histology results as the gold standard. The cut-off point of malignancy by radiomics was determined, with ±10 % of margin error according to the Youden's index. We considered these lesions as undetermined masses. The performance of the AI software and the radiologist classification was compared using the area under roc curves (AUROC). We also discuss the impact of radiologist validation of AI results, especially in undetermined lesions.

RESULTS

Of the 207 evaluated masses, 143 were benign, and 64 were malignant. The Youden's index was 0.516, including undetermined masses with a varied range of 10 % (0.464-0.567). Twenty-one (14.58 %) benign and twelve (19.05 %) malignant masses were in this range. The best accuracy performance to classify masses was the combination of the reader and AI (0.829). The most common undetermined masses in AI were fibroadenoma, followed by phyllodes tumor, steatonecrosis as benign. Whereas, low-grade, and high-grade invasive ductal carcinoma represents the malignant lesions.

CONCLUSIONS

Artificial Intelligence has a reliable performance in ultrasound breast masses classification. Radiologist validation is critical to determine the final BI-RADS assessment, especially in undetermined masses to obtain the best classification performance.