Effect of radiologists' diagnostic work-up volume on interpretive performance

Racial and Ethnic Variation in Diagnostic Mammography Performance among Women Reporting a Breast Lump

BACKGROUND

We evaluated diagnostic mammography among women with a breast lump to determine whether performance varied across racial and ethnic groups.

METHODS

This study included 51,014 diagnostic mammograms performed between 2005 and 2018 in the Breast Cancer Surveillance Consortium among Asian/Pacific Islander (12%), Black (7%), Hispanic/Latina (6%), and White (75%) women reporting a lump. Breast cancers occurring within 1 year were ascertained from cancer registry linkages. Multivariable regression was used to adjust performance statistic comparisons for breast cancer risk factors, mammogram modality, demographics, additional imaging, and imaging facility.

RESULTS

Cancer detection rates were highest among Asian/Pacific Islander [per 1,000 exams, 84.2 (95% confidence interval (CI): 72.0-98.2)] and Black women [81.4 (95% CI: 69.4-95.2)] and lowest among Hispanic/Latina women [42.9 (95% CI: 34.2-53.6)]. Positive predictive values (PPV) were higher among Black [37.0% (95% CI: 31.2-43.3)] and White [37.0% (95% CI: 30.0-44.6)] women and lowest among Hispanic/Latina women [22.0% (95% CI: 17.2-27.7)]. False-positive results were most common among Asian/Pacific Islander women [per 1,000 exams, 183.9 (95% CI: 126.7-259.2)] and lowest among White women [112.4 (95% CI: 86.1-145.5)]. After adjustment, false-positive and cancer detection rates remained higher for Asian/Pacific Islander and Black women (vs. Hispanic/Latina and White). Adjusted PPV was highest among Asian/Pacific Islander women.

CONCLUSIONS

Among women with a lump, Asian/Pacific Islander and Black women were more likely to have cancer detected and more likely to receive a false-positive result compared with White and Hispanic/Latina women.

IMPACT

Strategies for optimizing diagnostic mammography among women with a lump may vary by racial/ethnic group, but additional factors that influence performance differences need to be identified. See related In the Spotlight, p. 1479.

Reducing ' probably benign ' assessments in normal mammograms: The role of radiologist experience

Rationale and objectives

to investigate the relationship between radiologists' experience in reporting mammograms, their caseloads, and the classification of category '3' or 'Probably Benign' on normal mammograms.

Materials and Methods

A total of 92 board-certified radiologists participated. Self-reported parameters related to experience, including age, years since qualifying as a radiologist, years of experience reading mammograms, number of mammograms read per year, and hours spent reading mammograms per week, were documented. To assess the radiologists' accuracy, "Probably Benign fractions" was calculated by dividing the number of "Probably Benign findings" given by each radiologist in the normal cases by the total number of normal cases Probably Benign fractions were correlated with various factors, such as the radiologists' experience.

Results

The results of the statistical analysis revealed a significant negative correlation between radiologist experience and 'Probably Benign' fractions for normal images. Specifically, for normal cases, the number of mammograms read per year (r = -0.29, P = 0.006) and the number of mammograms read over the radiologist's lifetime (r = -0.21, P = 0.049) were both negatively correlated with 'Probably Benign' fractions.

Conclusion

The results indicate that a relationship exists between increased reading volumes and reduced assessments of 'Probably Benign' in normal mammograms. The implications of these findings extend to the effectiveness of screening programs and the recall rates.

Do Reader Characteristics Affect Diagnostic Efficacy in Screening Mammography? A Systematic Review

To examine reader characteristics associated with diagnostic efficacy in the interpretation of screening mammograms. A systematic search of the literature was conducted using databases such as Cochrane, Scopus, Medline, Embase, Web of Science, and PubMed. Search terms were combined with "AND" or "OR" and included: "Radiologist's characteristics AND performance"; "radiologist experience AND screening mammography"; "annual volume read AND diagnostic efficacy"; "screening mammography performance OR diagnostic efficacy". Studies were included if they assessed reader performance in screening mammography interpretation, breast readers, used a reference standard to assess the performance, and were published in the English language. Twenty-eight studies were reviewed. Increasing reader's age was associated with lower false positive rates. No association was found between gender and performance. Half of the studies showed no association between years of reading mammograms and performance. Most studies showed that high reading volume was more likely to be associated with increased sensitivity, cancer detection rates (CDR), lower recall rate, and lower false positive rates. Inconsistent associations were found between fellowship training in breast imaging and reader performance. Specialization in breast imaging was associated with better CDR, sensitivity, and specificity. Limited studies were available to establish the association between performance and factors such as time spent in breast imaging (n = 2), screening focus (n = 1), formal rotation in mammography (n = 1), owner of practice (n = 1), and practice type (n = 1). No individual characteristics is associated with versatility in diagnostic efficacy, albeit reading volume and specialization in breast imaging appear to be associated with with increased sensitivity and CDR without significantly affecting other performance metrics.

Prioritizing Screening Mammograms for Immediate Interpretation and Diagnostic Evaluation on the Basis of Risk for Recall

PURPOSE

The aim of this study was to develop a prioritization strategy for scheduling immediate screening mammographic interpretation and possible diagnostic evaluation.

METHODS

A population-based cohort with screening mammograms performed from 2012 to 2020 at 126 radiology facilities from 7 Breast Cancer Surveillance Consortium registries was identified. Classification trees identified combinations of clinical history (age, BI-RADS® density, time since prior mammogram, history of false-positive recall or biopsy result), screening modality (digital mammography, digital breast tomosynthesis), and facility characteristics (profit status, location, screening volume, practice type, academic affiliation) that grouped screening mammograms by recall rate, with ≥12/100 considered high and ≥16/100 very high. An efficiency ratio was estimated as the percentage of recalls divided by the percentage of mammograms.

RESULTS

The study cohort included 2,674,051 screening mammograms in 925,777 women, with 235,569 recalls. The most important predictor of recall was time since prior mammogram, followed by age, history of false-positive recall, breast density, history of benign biopsy, and screening modality. Recall rates were very high for baseline mammograms (21.3/100; 95% confidence interval, 19.7-23.0) and high for women with ≥5 years since prior mammogram (15.1/100; 95% confidence interval, 14.3-16.1). The 9.2% of mammograms in subgroups with very high and high recall rates accounted for 19.2% of recalls, an efficiency ratio of 2.1 compared with a random approach. Adding women <50 years of age with dense breasts accounted for 20.3% of mammograms and 33.9% of recalls (efficiency ratio = 1.7). Results including facility-level characteristics were similar.

CONCLUSIONS

Prioritizing women with baseline mammograms or ≥5 years since prior mammogram for immediate interpretation and possible diagnostic evaluation could considerably reduce the number of women needing to return for diagnostic imaging at another visit.

Using the Medical Audit to Improve Practice Performance

Feedback to physicians on their clinical performance is critical to continuous learning and maintenance of skills as well as maintaining patient safety. However, it is fraught with challenges around both implementation and acceptance. Additionally, rewarding of performance improvement is not often done, putting into question the efficacy of the process. Physician audit and feedback have been studied extensively and shown to be beneficial in many fields of medicine. Documenting physician performance and sharing individual and group data have been positively linked to changing physician behavior, ultimately leading to improved patient outcomes. Although casual review of one's own performance is often the easiest approach, it is frequently over- or underestimated by self-evaluation. Objective measures are therefore important to provide concrete data on which physicians can act. A fundamental question remains in mammography: Is reporting the information to the physician and accreditation bodies enough, or should there be consequences for the radiologist and/or facility if there is outlier behavior?

Proportion of Thick versus Thin Melanomas as a Benchmarking Tool

BACKGROUND

The ratio of benign moles excised for each malignant melanoma diagnosed (number-needed-to-excise (NNE)) is a metric used to express the efficiency of diagnostic accuracy of melanoma. The literature suggests a progressive effort to reduce the NNE, thus raising concerns about missing early melanoma because the NNE does not capture the most significant outcome for melanoma prognosis, which is linked to the Breslow thickness. A lower NNE could reduce health costs related to melanoma diagnosis only if doing so does not increase the proportion of thicker melanomas.

OBJECTIVES

The diagnostic performance by two tertiary referral centres using the NNE and proportion of thick (Breslow thickness > 1 mm) versus thin (Breslow thickness ≤ 1 mm) excised melanoma (thick/thin ratio: TTR) was compared to determine if a lower NNE is associated with a greater proportion of thicker melanoma. Combining TTR with NNE allows a better estimate of the effectiveness in melanoma diagnosis, assessing both the overall cost for a given pool of excised melanomas and costs due to unnecessary nevi excision at a particular dermatology centre.

METHODS

Demographic data and Breslow thickness of excised melanoma were extracted from patient histologic records at two referral centres for melanoma (Parma Dermatology Unit and Ravenna and Meldola Skin Cancer Unit, Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori. IRCCS (IRST)) on all skin tumours excised between 2002 and 2011 and diagnosed as melanoma or melanocytic nevus. NNE and TTR were calculated and compared among the considered variables. Logistic regression was used to assess the contribution of each variable in predicting a higher TTR.

RESULTS

Data from 16,738 excised lesions were analysed. The IRST Unit reported a mean NNE of 4.6, whereas the Parma Unit excised 10.6 nevi for each melanoma. No statistically significant differences existed in the mean (IRST Unit, 0.56 ± 0.89 mm; Parma Unit, 1.07 ± 2.2 mm) and median (range) Breslow thickness (IRST Unit, 0.4 (9) mm; Parma Unit 0.4 (30) mm). The TTR between centres was significantly different (Parma Unit, 24%; IRST Unit, 12%; p < 0.001). Based on logistic regression, the diagnosing centre was the most powerful factor in determining a thickness of >1 mm among diagnosed melanomas (OR = 1.8; 95% CI, 1.2-2.7; p < 0.01), with all other factors being equal. The NNE decreased at both centres from younger-to-older patients, whereas the TTR increased simultaneously; however, the increase in TTR was non-significantly related to NNE reduction after adjusting for confounders (age, gender, and localization).

CONCLUSIONS

A better diagnostic performance is capable of reducing the NNE and TTR, i.e., unnecessary excisions of melanocytic nevi can be reduced without increasing the risk of overlooking melanomas. The TTR, in addition to the NNE, allows stakeholders to better estimate the effectiveness in melanoma diagnosis because both overall costs for a given pool of excised melanomas and costs due for unnecessary nevi excision at a particular dermatology centre can be compared.

RADIAL: leveraging a learning management system to support radiology education

Increasing volume and complexity of cases in academic radiology and the drive toward pediatric sub-specialization have threatened knowledge assimilation for radiologists. There is a clear need for a system that retrieves vetted information from the excess available on the internet. Partnered with an interdisciplinary team from the Graduate School of Education, the authors created the first comprehensive learning management system (LMS) for radiology, implemented in the reading room to augment image interpretation and point-of-care education. The LMS supports quantitative analysis using a robust analytics platform to evaluate user statistics, facilitating improved quality of patient care by revolutionizing the way radiologists assimilate knowledge. This integration promises to enhance workflow and point-of-care teaching and to support the highest quality of care.

Radiologist Characteristics Associated with Interpretive Performance of Screening Mammography: A National Mammography Database (NMD) Study

Background Factors affecting radiologists' performance in screening mammography interpretation remain poorly understood. Purpose To identify radiologists characteristics that affect screening mammography interpretation performance. Materials and Methods This retrospective study included 1223 radiologists in the National Mammography Database (NMD) from 2008 to 2019 who could be linked to Centers for Medicare & Medicaid Services (CMS) datasets. NMD screening performance metrics were extracted. Acceptable ranges were defined as follows: recall rate (RR) between 5% and 12%; cancer detection rate (CDR) of at least 2.5 per 1000 screening examinations; positive predictive value of recall (PPV1) between 3% and 8%; positive predictive value of biopsies recommended (PPV2) between 20% and 40%; positive predictive value of biopsies performed (PPV3) between the 25th and 75th percentile of study sample; invasive CDR of at least the 25th percentile of the study sample; and percentage of ductal carcinoma in situ (DCIS) of at least the 25th percentile of the study sample. Radiologist characteristics extracted from CMS datasets included demographics, subspecialization, and clinical practice patterns. Multivariable stepwise logistic regression models were performed to identify characteristics independently associated with acceptable performance for the seven metrics. The most influential characteristics were defined as those independently associated with the majority of the metrics (at least four). Results Relative to radiologists practicing in the Northeast, those in the Midwest were more likely to achieve acceptable RR, PPV1, PPV2, and CDR (odds ratio [OR], 1.4-2.5); those practicing in the West were more likely to achieve acceptable RR, PPV2, and PPV3 (OR, 1.7-2.1) but less likely to achieve acceptable invasive CDR (OR, 0.6). Relative to general radiologists, breast imagers were more likely to achieve acceptable PPV1, invasive CDR, percentage DCIS, and CDR (OR, 1.4-4.4). Those performing diagnostic mammography were more likely to achieve acceptable PPV1, PPV2, PPV3, invasive CDR, and CDR (OR, 1.9-2.9). Those performing breast US were less likely to achieve acceptable PPV1, PPV2, percentage DCIS, and CDR (OR, 0.5-0.7). Conclusion The geographic location of the radiology practice, subspecialization in breast imaging, and performance of diagnostic mammography are associated with better screening mammography performance; performance of breast US is associated with lower performance. ^©RSNA, 2021 Online supplemental material is available for this article.

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme

OBJECTIVES

Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance.

METHODS

We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance.

RESULTS

In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted).

CONCLUSION

Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected.

KEY POINTS

• In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

The Adoption and Impact on Performance of an Automated Outcomes Feedback Application for Tomosynthesis Screening Mammography

OBJECTIVE

To evaluate a tomosynthesis screening mammography automated outcomes feedback application's adoption and impact on performance.

METHODS

This prospective intervention study evaluated a feedback application that provided mammographers subsequent imaging and pathology results for patients that radiologists had personally recalled from screening. Deployed to 13 academic and 5 private practice attending radiologists, adoption was studied from March 29, 2018, to March 20, 2019. Radiologists indicated if reviewed feedback would influence future clinical decisions. For a subset of eight academic radiologists consistently interpreting screening mammograms during the study, performance metrics were compared pre-intervention (January 1, 2016, to September 30, 2017) and post-intervention (October 1, 2017 to June 30, 2018). Abnormal interpretation rate, positive predictive value of biopsies performed, sensitivity, specificity, and cancer detection rate were compared using Pearson's χ² test. Logistic regression models were fit, adjusting for age, race, breast density, prior comparison, breast cancer history, and radiologist.

RESULTS

The 18 radiologists reviewed 68.5% (1,398 of 2,042) of available feedback cases and indicated that 17.4% of cases (243 of 1,398) could influence future decisions. For the eight academic radiologist subset, after multivariable adjustment with comparison to pre-intervention, average abnormal interpretation rate decreased (from 7.5% to 6.7%, adjusted odds ratio [aOR] 0.86, P < .01), positive predictive value of biopsies performed increased (from 40.6% to 51.3%, aOR 1.48, P = .011), and specificity increased (from 93.0% to 93.9%, aOR 1.17, P < .01) post-intervention. There was no difference in cancer detection rate per 1,000 examinations (from 5.8 to 6.1, aOR 1.01, P = .91) or sensitivity (from 81.2% to 78.7%, aOR 0.84, P = .47).

CONCLUSIONS

Radiologists used a screening mammography automated outcomes feedback application. Its use decreased false-positive examinations, without evidence of reduced cancer detection.

New mammography screening performance metrics based on the entire screening episode

BACKGROUND

Established mammography screening performance metrics use the initial screening mammography assessment because they were developed for radiologist performance auditing, yet these metrics are frequently used to inform health policy and screening decision making. The authors have developed new performance metrics based on the final assessment that consider the entire screening episode, including diagnostic workup.

METHODS

The authors used data from 2,512,577 screening episodes during 2005-2017 at 146 facilities in the United States participating in the Breast Cancer Surveillance Consortium. Screening performance metrics based on the final assessment of the screening episode were compared with conventional metrics defined with the initial assessment. Results were also stratified by breast density and breast cancer risk.

RESULTS

The cancer detection rates were similar for the final assessment (4.1 per 1000; 95% confidence interval [CI], 3.8-4.3 per 1000) and the initial assessment (4.1 per 1000; 95% CI, 3.9-4.3 per 1000). The interval cancer rate was 12% higher when it was based on the final assessment (0.77 per 1000; 95% CI, 0.71-0.83 per 1000) versus the initial assessment (0.69 per 1000; 95% CI, 0.64-0.74 per 1000), and this resulted in a modest difference in sensitivity (84.1% [95% CI, 83.0%-85.1%] vs 85.7% [95% CI, 84.8%-86.6%], respectively). Absolute differences in the interval cancer rate between final and initial assessments increased with breast density and breast cancer risk (eg, a difference of 0.29 per 1000 for women with extremely dense breasts and a 5-year risk >2.49%).

CONCLUSIONS

Established screening performance metrics underestimate the interval cancer rate of a mammography screening episode, particularly for women with dense breasts or an elevated breast cancer risk. Women, clinicians, policymakers, and researchers should use final-assessment performance metrics to support informed screening decisions.

Association of volume of self-directed versus assigned interpretive work with diagnostic performance of radiologists: an observational study

OBJECTIVES

To understand the sources of variability in diagnostic performance among experienced radiologists.

DESIGN

All prostate MRI examinations performed between 2016 and 2018 were retrospectively reviewed.

SETTING

University hospital in Japan.

PARTICIPANTS

Data derived from 334 pathology-proven cases (male, mean age: 70 years; range: 35-90 years) that were interpreted by 10 experienced radiologists were subjected to the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

Diagnostic performance measures of the radiologists were compared with candidate factors, including interpretive volume of prostate MRIs, volume of self-directed and assigned total annual interpretive work, and years of experience. The potential influence of fatigue was also evaluated by examining the effect of the report's issue time.

RESULTS

There were 186 prostate cancer cases. Performance was based on accuracy, sensitivity and specificity (86%, 85% and 84%, respectively). While performance was not correlated with the volume of prostate MRIs, per se (ρ=-0.15, p=0.69; ρ=-0.01, p=0.99; ρ=-0.33, p=0.36) or the total MRIs assigned for each radiologist (p>0.6) or years of experience (p>0.4), all measures were strongly correlated with voluntary work represented by the interpretive volume of abdominal CTs (r=0.79, p<0.01; r=0.80, p<0.01; r=0.64, p=0.048). The performance did not differ based on the issue time of the report (morning, afternoon and evening) (χ²(2)=3.65, p=0.16).

CONCLUSIONS

Greater autonomy, represented as enhanced self-directed interpretive work, was most significantly correlated with the performance of prostate MRI interpretation. The lack of a correlation between the performance and assigned volume confirms the complexity of human learning. Together, these findings support the hypothesis that successful promotion of internal drivers could have a pervasive positive impact on improving diagnostic performance.

Digital Breast Tomosynthesis: Radiologist Learning Curve

Background There is growing evidence that digital breast tomosynthesis (DBT) results in lower recall rates and higher cancer detection rates when compared with digital mammography. However, whether DBT interpretative performance changes with experience (learning curve effect) is unknown. Purpose To evaluate screening DBT performance by cumulative DBT volume within 2 years after adoption relative to digital mammography (DM) performance 1 year before DBT adoption. Materials and Methods This prospective study included 106 126 DBT and 221 248 DM examinations in 271 362 women (mean age, 57.5 years) from 2010 to 2017 that were interpreted by 104 radiologists from 53 facilities in the Breast Cancer Surveillance Consortium. Conditional logistic regression was used to estimate within-radiologist effects of increasing cumulative DBT volume on recall and cancer detection rates relative to DM and was adjusted for examination-level characteristics. Changes were also evaluated by subspecialty and breast density. Results Before DBT adoption, DM recall rate was 10.4% (95% confidence interval [CI]: 9.5%, 11.4%) and cancer detection rate was 4.0 per 1000 screenings (95% CI: 3.6 per 1000 screenings, 4.5 per 1000 screenings); after DBT adoption, DBT recall rate was lower (9.4%; 95% CI: 8.2%, 10.6%; P = .02) and cancer detection rate was similar (4.6 per 1000 screenings; 95% CI: 4.0 per 1000 screenings, 5.2 per 1000 screenings; P = .12). Relative to DM, DBT recall rate decreased for a cumulative DBT volume of fewer than 400 studies (odds ratio [OR] = 0.83; 95% CI: 0.78, 0.89) and remained lower as volume increased (400-799 studies, OR = 0.8 [95% CI: 0.75, 0.85]; 800-1199 studies, OR = 0.81 [95% CI: 0.76, 0.87]; 1200-1599 studies, OR = 0.78 [95% CI: 0.73, 0.84]; 1600-2000 studies, OR = 0.81 [95% CI: 0.75, 0.88]; P < .001). Improvements were sustained for breast imaging subspecialists (OR range, 0.67-0.85; P < .02) and readers who were not breast imaging specialists (OR range, 0.80-0.85; P < .001). Recall rates decreased more in women with nondense breasts (OR range, 0.68-0.76; P < .001) than in those with dense breasts (OR range, 0.86-0.90; P ≤ .05; P interaction < .001). Cancer detection rates for DM and DBT were similar, regardless of DBT volume (P ≥ .10). Conclusion Early performance improvements after digital breast tomosynthesis (DBT) adoption were sustained regardless of DBT volume, radiologist subspecialty, or breast density. © RSNA, 2019 See also the editorial by Hooley in this issue.

Impact of Immediate Interpretation of Screening Tomosynthesis Mammography on Performance Metrics

RATIONALE AND OBJECTIVES

This study aimed to compare performance metrics for immediate and delayed batch interpretation of screening tomosynthesis mammograms.

MATERIALS AND METHODS

This HIPAA compliant study was approved by institutional review board with a waiver of consent. A retrospective analysis of screening performance metrics for tomosynthesis mammograms interpreted in 2015 when mammograms were read immediately was compared to historical controls from 2013 to 2014 when mammograms were batch interpreted after the patient had departed. A total of 5518 screening tomosynthesis mammograms (n = 1212 for batch interpretation and n = 4306 for immediate interpretation) were evaluated. The larger sample size for the latter group reflects a group practice shift to performing tomosynthesis for the majority of patients. Age, breast density, comparison examinations, and high-risk status were compared. An asymptotic proportion test and multivariable analysis were used to compare performance metrics.

RESULTS

There was no statistically significant difference in recall or cancer detection rates for the batch interpretation group compared to immediate interpretation group with respective recall rate of 6.5% vs 5.3% = +1.2% (95% confidence interval -0.3 to 2.7%; P = .101) and cancer detection rate of 6.6 vs 7.2 per thousand = -0.6 (95% confidence interval -5.9 to 4.6; P = .825). There was no statistically significant difference in positive predictive values (PPVs) including PPV1 (screening recall), PPV2 (biopsy recommendation), or PPV 3 (biopsy performed) with batch interpretation (10.1%, 42.1%, and 40.0%, respectively) and immediate interpretation (13.6%, 39.2%, and 39.7%, respectively). After adjusting for age, breast density, high-risk status, and comparison mammogram, there was no difference in the odds of being recalled or cancer detection between the two groups.

CONCLUSIONS

There is no statistically significant difference in interpretation performance metrics for screening tomosynthesis mammograms interpreted immediately compared to those interpreted in a delayed fashion.

A blueprint for cancer screening and early detection: Advancing screening's contribution to cancer control

From the mid-20th century, accumulating evidence has supported the introduction of screening for cancers of the cervix, breast, colon and rectum, prostate (via shared decisions), and lung. The opportunity to detect and treat precursor lesions and invasive disease at a more favorable stage has contributed substantially to reduced incidence, morbidity, and mortality. However, as new discoveries portend advancements in technology and risk-based screening, we fail to fulfill the greatest potential of the existing technology, in terms of both full access among the target population and the delivery of state-of-the art care at each crucial step in the cascade of events that characterize successful cancer screening. There also is insufficient commitment to invest in the development of new technologies, incentivize the development of new ideas, and rapidly evaluate promising new technology. In this report, the authors summarize the status of cancer screening and propose a blueprint for the nation to further advance the contribution of screening to cancer control.

Errors in Mammography Cannot be Solved Through Technology Alone

Mammography has been the frontline screening tool for breast cancer for decades. However, high error rates in the form of false negatives (FNs) and false positives (FPs) have persisted despite technological improvements. Radiologists still miss between 10% and 30% of cancers while 80% of woman recalled for additional views have normal outcomes, with 40% of biopsied lesions being benign. Research show that the majority of cancers missed is actually visible and looked at, but either go unnoticed or are deemed to be benign. Causal agents for these errors include human related characteristics resulting in contributory search, perception and decision-making behaviours. Technical, patient and lesion factors are also important relating to positioning, compression, patient size, breast density and presence of breast implants as well as the nature and subtype of the cancer itself, where features such as architectural distortion and triple-negative cancers remain challenging to detect on screening. A better understanding of these causal agents as well as the adoption of technological and educational interventions, which audits reader performance and provide immediate perceptual feedback, should help. This paper reviews the current status of our knowledge around error rates in mammography and explores the factors impacting it. It also presents potential solutions for maximizing diagnostic efficacy thus benefiting the millions of women who undergo this procedure each year.

Strategies for Decreasing Screening Mammography Recall Rates While Maintaining Performance Metrics

RATIONALE AND OBJECTIVE

This study aimed to determine the impact of interventions designed to reduce screening mammography recall rates on screening performance metrics.

MATERIALS AND METHODS

We assessed baseline performance for full-field digital mammography (FFDM) and digital breast tomosynthesis mammography (DBT) for a 3-year period before intervention. The first intervention sought to increase awareness of recalls from screening mammography. Breast imagers discussed their perceptions regarding screening recalls and were required to review their own recalled cases, including outcomes of diagnostic evaluation and biopsy. The second intervention implemented consensus double reading of all recalls, requiring two radiologists to agree if recall was necessary. Recall rates, cancer detection rates, and positive predictive value 1 (PPV1) were compared before and after each intervention.

RESULTS

The baseline recall rate, cancer detection rate, and PPV1 were 11.1%, 3.8/1000, and 3.4%, respectively, for FFDM, and 7.6%, 4.8/1000, and 6.0%, respectively, for DBT. Recall rates decreased significantly to 9.2% for FFDM and to 6.6% for DBT after the first intervention promoting awareness, as well as to 9.9% for FFDM after the second intervention implementing group consensus. PPV1 increased significantly to 5.7% for FFDM and to 9.0% for DBT after the second intervention. Cancer detection rate did not significantly change with the implementation of these interventions. An average of 2.3 minutes was spent consulting for each recall.

CONCLUSION

Reduction in recall rates is desirable, provided performance metrics remain favorable. Our interventions improved performance and could be implemented in other breast imaging settings.

Correlation Between Screening Mammography Interpretive Performance on a Test Set and Performance in Clinical Practice

RATIONALE AND OBJECTIVES

Evidence is inconsistent about whether radiologists' interpretive performance on a screening mammography test set reflects their performance in clinical practice. This study aimed to estimate the correlation between test set and clinical performance and determine if the correlation is influenced by cancer prevalence or lesion difficulty in the test set.

MATERIALS AND METHODS

This institutional review board-approved study randomized 83 radiologists from six Breast Cancer Surveillance Consortium registries to assess one of four test sets of 109 screening mammograms each; 48 radiologists completed a fifth test set of 110 mammograms 2 years later. Test sets differed in number of cancer cases and difficulty of lesion detection. Test set sensitivity and specificity were estimated using woman-level and breast-level recall with cancer status and expert opinion as gold standards. Clinical performance was estimated using women-level recall with cancer status as the gold standard. Spearman rank correlations between test set and clinical performance with 95% confidence intervals (CI) were estimated.

RESULTS

For test sets with fewer cancers (N = 15) that were more difficult to detect, correlations were weak to moderate for sensitivity (woman level = 0.46, 95% CI = 0.16, 0.69; breast level = 0.35, 95% CI = 0.03, 0.61) and weak for specificity (0.24, 95% CI = 0.01, 0.45) relative to expert recall. Correlations for test sets with more cancers (N = 30) were close to 0 and not statistically significant.

CONCLUSIONS

Correlations between screening performance on a test set and performance in clinical practice are not strong. Test set performance more accurately reflects performance in clinical practice if cancer prevalence is low and lesions are challenging to detect.

Radiologists' interpretive skills in screening vs. diagnostic mammography: are they related?

PURPOSE

This study aims to determine whether radiologists who perform well in screening also perform well in interpreting diagnostic mammography.

MATERIALS AND METHODS

We evaluated the accuracy of 468 radiologists interpreting 2,234,947 screening and 196,164 diagnostic mammograms. Adjusting for site, radiologist, and patient characteristics, we identified radiologists with performance in the highest tertile and compared to those with lower performance.

RESULTS

A moderate correlation was noted for radiologists' accuracy when interpreting screening versus their accuracy on diagnostic examinations: sensitivity (r_spearman=0.51, 95% CI: 0.22, 0.80; P=.0006) and specificity (r_spearman=0.40, 95% CI: 0.30, 0.49; P<.0001).

CONCLUSION

Different educational approaches to screening and diagnostic imaging should be considered.

[An audit of breast cancer screening mammograms and the variability of radiological practice]

OBJECTIVE

To audit the breast cancer screening mammograms performed in a general hospital and to assess the variation in medical practice in the diagnostic process.

MATERIAL AND METHODS

A review was carried out on the screening mammograms performed between 1 May 2010 and 30 April 2011, with clinical follow up for two years, and a comparison with the published standards.

RESULTS

Of the 3,878 women examined, 368 (9.48%) were called back to complete the study (97 [16.1%] in the initial screening and 271 [8.2%] in revisions). Forty three biopsies (1.1%) were indicated, of which 24 were diagnosed with cancer. The positive predictive value (PPV) in screening studies (PPV1) was 6.52%. For the recommended biopsy (PPV2) it was 55%, with a sensitivity of 100%, a specificity of 91% and a cancer detection rate of 6.1/1,000. There were no false negatives. Twenty tumours were invasive; with no axillary lymph node infiltration was observed 15 of them. In 6 cases, the size of the tumour was less than or equal to 10mm, and in 17 it was less than 15mm. There were a higher percentage of new appointments by two radiologists (12% and 17.2% versus 7.3%) (P<.001). In 217 cases (58.96%; P<.001) only one radiologist indicated new appointments. Of this group, 73% were discharged in the first visit, compared to 47.6% in the non-discrepant group (P<.001). Four of the cancers were detected in these 217 patients.

CONCLUSIONS

The observed results are adjusted to the reference values. The discordant data are the new appointments rate, both in the initial screening and in the review, with a significant variation depending on the radiology reader.

Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice

As a part of a special collection in this issue of Academic Medicine, which is focused on mastery learning in medical education, this Perspective describes how the expert-performance approach with deliberate practice is consistent with many characteristics of mastery learning. Importantly, this Perspective also explains how the expert-performance approach provides a very different perspective on the acquisition of skill. Whereas traditional education with mastery learning focuses on having students attain an adequate level of performance that is based on goals set by the existing curricula, the expert-performance approach takes an empirical approach and first identifies the final goal of training-namely, reproducibly superior objective performance (superior patient outcomes) for individuals in particular medical specialties. Analyzing this superior complex performance reveals three types of mental representations that permit expert performers to plan, execute, and monitor their own performance. By reviewing research on medical performance and education, the author describes evidence for these representations and their development within the expert-performance framework. He uses the research to generate suggestions for improved training of medical students and professionals. Two strategies-designing learning environments with libraries of cases and creating opportunities for individualized teacher-guided training-should enable motivated individuals to acquire a full set of refined mental representations. Providing the right resources to support the expert-performance approach will allow such individuals to become self-regulated learners-that is, members of the medical community who have the tools to improve their own and their team members' performances throughout their entire professional careers.

Breast Cancer Screening for Women at Average Risk: 2015 Guideline Update From the American Cancer Society

IMPORTANCE

Breast cancer is a leading cause of premature mortality among US women. Early detection has been shown to be associated with reduced breast cancer morbidity and mortality.

OBJECTIVE

To update the American Cancer Society (ACS) 2003 breast cancer screening guideline for women at average risk for breast cancer.

PROCESS

The ACS commissioned a systematic evidence review of the breast cancer screening literature to inform the update and a supplemental analysis of mammography registry data to address questions related to the screening interval. Formulation of recommendations was based on the quality of the evidence and judgment (incorporating values and preferences) about the balance of benefits and harms.

EVIDENCE SYNTHESIS

Screening mammography in women aged 40 to 69 years is associated with a reduction in breast cancer deaths across a range of study designs, and inferential evidence supports breast cancer screening for women 70 years and older who are in good health. Estimates of the cumulative lifetime risk of false-positive examination results are greater if screening begins at younger ages because of the greater number of mammograms, as well as the higher recall rate in younger women. The quality of the evidence for overdiagnosis is not sufficient to estimate a lifetime risk with confidence. Analysis examining the screening interval demonstrates more favorable tumor characteristics when premenopausal women are screened annually vs biennially. Evidence does not support routine clinical breast examination as a screening method for women at average risk.

RECOMMENDATIONS

The ACS recommends that women with an average risk of breast cancer should undergo regular screening mammography starting at age 45 years (strong recommendation). Women aged 45 to 54 years should be screened annually (qualified recommendation). Women 55 years and older should transition to biennial screening or have the opportunity to continue screening annually (qualified recommendation). Women should have the opportunity to begin annual screening between the ages of 40 and 44 years (qualified recommendation). Women should continue screening mammography as long as their overall health is good and they have a life expectancy of 10 years or longer (qualified recommendation). The ACS does not recommend clinical breast examination for breast cancer screening among average-risk women at any age (qualified recommendation).

CONCLUSIONS AND RELEVANCE

These updated ACS guidelines provide evidence-based recommendations for breast cancer screening for women at average risk of breast cancer. These recommendations should be considered by physicians and women in discussions about breast cancer screening.