Criteria for identifying radiologists with acceptable screening mammography interpretive performance on basis of multiple performance measures

The Importance of Outcomes Ascertainment for Accurate Assessment of the Mammography Screening Cancer Detection Rate: A Simulation Study

PURPOSE

Cancer detection rate (CDR), an important metric in the mammography screening audit, is designed to ensure adequate sensitivity. Most practices use biopsy results as the reference standard; however, commonly ascertainment of biopsy results is incomplete. We used simulation to determine the relationship between the cancer ascertainment rate of biopsy (AR-biopsy), CDR estimation, and associated error rates in classifying whether practices and radiologists meet the established ACR benchmark of 2.5 per 1,000.

MATERIALS AND METHODS

We simulated screening mammography volume, number of cancers detected, and CDR, using negative binomial and beta-binomial distributions, respectively. Simulations were performed at both the practice and radiologist level. Average CDR was based on linearly rescaling a published CDR by the AR-biopsy. CDR distributions were simulated for AR-biopsy between 5% and 100% in steps of five percentage points and were summarized with boxplots and smoothed histograms over the range of AR-biopsy, to quantify the proportion of practices and radiologists meeting the ACR benchmark at each level of AR-biopsy.

RESULTS

Decreasing AR-biopsy led to an increasing probability of categorizing CDR performance as being below the ACR benchmark. Our simulation predicts that at the practice level, an AR-biopsy of 65% categorizes 17.6% below the benchmark (compared to 1.6% at an AR-biopsy of 100%), and at the radiologist level, an AR-biopsy of 65% categorizes 34.7% as being below the benchmark (compared to 11.6% at an AR-biopsy of 100%).

CONCLUSIONS

Our simulation demonstrates that decreasing the AR-biopsy (in currently clinically relevant ranges) has the potential to artifactually lower the assessed CDR on both the practice and radiologist levels and may, in turn, increase the chance of erroneous categorization of underperformance per the ACR benchmark.

[Breast Imaging Reporting and Data System (BI-RADS): Advantages and Limitations]

Breast Imaging Reporting and Data System (BI-RADS) is a communication and data tracking system that standardizes and controls the quality of reporting by presenting lexicon descriptors, assessment categories, and recommendations for managing breast lesions. Using standardized terminology recommended by BI-RADS, radiologists can concisely and reproducibly communicate breast imaging results to clinicians. They can also provide the estimated malignant probability of the lesions found and guide management for them by determining the final assessment category. The limitations of BI-RADS 5th edition currently in use are that there are some areas for which standardized terminologies still need to be established, and that the diagnostic criteria of MRI assessment categories 3 and 4 are ambiguous compared to those for mammography or ultrasound. The next revision of BI-RADS is expected to include solutions for overcoming current limitations.

Using the Medical Audit to Improve Practice Performance

Feedback to physicians on their clinical performance is critical to continuous learning and maintenance of skills as well as maintaining patient safety. However, it is fraught with challenges around both implementation and acceptance. Additionally, rewarding of performance improvement is not often done, putting into question the efficacy of the process. Physician audit and feedback have been studied extensively and shown to be beneficial in many fields of medicine. Documenting physician performance and sharing individual and group data have been positively linked to changing physician behavior, ultimately leading to improved patient outcomes. Although casual review of one's own performance is often the easiest approach, it is frequently over- or underestimated by self-evaluation. Objective measures are therefore important to provide concrete data on which physicians can act. A fundamental question remains in mammography: Is reporting the information to the physician and accreditation bodies enough, or should there be consequences for the radiologist and/or facility if there is outlier behavior?

The Benefits of Early Detection: Evidence From Modern International Mammography Service Screening Programs

Research from randomized controlled trials initiated up to 60 years ago consistently confirms that regular screening with mammography significantly reduces breast cancer mortality. Despite this success, there is ongoing debate regarding the efficacy of screening, which is confounded by technologic advances and concerns about cost, overdiagnosis, overtreatment, and equitable care of diverse patient populations. More recent screening research, designed to quell the debates, derives data from variable study designs, each with unique strengths and weaknesses. This article reviews observational population-based screening research that has followed the early initial long-term randomized controlled trials that are no longer practical or ethical to perform. The advantages and disadvantages of observational data and study design are outlined, including the three subtypes of population-based observational studies: cohort/case-control, trend, and incidence-based mortality/staging. The most recent research, typically performed in countries that administer screening mammography to women through centralized health service programs and directly track patient-specific outcomes and detection data, is summarized. These data are essential to understand and inform construction of effective new databases that facilitate continuous assessment of optimal screening techniques in the current era of rapidly developing medical technology, combined with a focus on health care that is both personal and equitable.

Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists

RATIONALE AND OBJECTIVES

Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers.

MATERIALS AND METHODS

This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1).

RESULTS

Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer.

CONCLUSION

Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

The Effects of Prior Mammography Screening on the Performance of Breast Cancer Detection in Taiwan

The aim of this study was to investigate the influence of previous mammography screening on the performance of breast cancer detection. The screened women were divided into first-visit and follow-up groups for breast cancer screening. The positive predictive value (PPV), cancer detection rate (CDR), and recall rate were used to evaluate and analyze the overall screening performance among the two groups. Among them, 10,040 screenings (67.2%) were first visits and 4895 screenings (32.8%) were follow-up visits. The proportion of positive screening results for first-visit participants was higher than that for their follow-up counterparts (9.3% vs. 4.0%). A total of 98 participants (74 first-visit and 24 follow-up visit) were confirmed to have breast cancer. The PPV for positive mammography for women who underwent biopsy confirmation was 28.7% overall, reaching 35.8% for the follow-up visit group and 27.0% for the first-visit group. The CDR was 6.6 per 1000 overall, reaching 7.4 per 1000 for first-visit group and 4.9 per 1000 for the follow-up group. The overall recall rate was 7.9%, reaching 9.7% for the first-visit group and 4.2% for the follow-up group. The PPV is improved and the recall rate is decreased if prior mammography images are available for comparison when conducting mammography screening for breast cancer. By this study, we concluded that prior mammography plays an important role for breast cancer screening, while follow-up mammography may increase the diagnostic rate when compared to the prior mammography. We suggest that the public health authority can encourage subjects to undergo screenings in the same health institute where they regularly visit.

Radiologist Characteristics Associated with Interpretive Performance of Screening Mammography: A National Mammography Database (NMD) Study

Background Factors affecting radiologists' performance in screening mammography interpretation remain poorly understood. Purpose To identify radiologists characteristics that affect screening mammography interpretation performance. Materials and Methods This retrospective study included 1223 radiologists in the National Mammography Database (NMD) from 2008 to 2019 who could be linked to Centers for Medicare & Medicaid Services (CMS) datasets. NMD screening performance metrics were extracted. Acceptable ranges were defined as follows: recall rate (RR) between 5% and 12%; cancer detection rate (CDR) of at least 2.5 per 1000 screening examinations; positive predictive value of recall (PPV1) between 3% and 8%; positive predictive value of biopsies recommended (PPV2) between 20% and 40%; positive predictive value of biopsies performed (PPV3) between the 25th and 75th percentile of study sample; invasive CDR of at least the 25th percentile of the study sample; and percentage of ductal carcinoma in situ (DCIS) of at least the 25th percentile of the study sample. Radiologist characteristics extracted from CMS datasets included demographics, subspecialization, and clinical practice patterns. Multivariable stepwise logistic regression models were performed to identify characteristics independently associated with acceptable performance for the seven metrics. The most influential characteristics were defined as those independently associated with the majority of the metrics (at least four). Results Relative to radiologists practicing in the Northeast, those in the Midwest were more likely to achieve acceptable RR, PPV1, PPV2, and CDR (odds ratio [OR], 1.4-2.5); those practicing in the West were more likely to achieve acceptable RR, PPV2, and PPV3 (OR, 1.7-2.1) but less likely to achieve acceptable invasive CDR (OR, 0.6). Relative to general radiologists, breast imagers were more likely to achieve acceptable PPV1, invasive CDR, percentage DCIS, and CDR (OR, 1.4-4.4). Those performing diagnostic mammography were more likely to achieve acceptable PPV1, PPV2, PPV3, invasive CDR, and CDR (OR, 1.9-2.9). Those performing breast US were less likely to achieve acceptable PPV1, PPV2, percentage DCIS, and CDR (OR, 0.5-0.7). Conclusion The geographic location of the radiology practice, subspecialization in breast imaging, and performance of diagnostic mammography are associated with better screening mammography performance; performance of breast US is associated with lower performance. ^©RSNA, 2021 Online supplemental material is available for this article.

A Review of Breast Imaging for Timely Diagnosis of Disease

Breast cancer (BC) is the cancer with the highest incidence in women in the world. In this last period, the COVID-19 pandemic has caused in many cases a drastic reduction of routine breast imaging activity due to the combination of various factors. The survival of BC is directly proportional to the earliness of diagnosis, and especially during this period, it is at least fundamental to remember that a diagnostic delay of even just three months could affect BC outcomes. In this article we will review the state of the art of breast imaging, starting from morphological imaging, i.e., mammography, tomosynthesis, ultrasound and magnetic resonance imaging and contrast-enhanced mammography, and their most recent evolutions; and ending with functional images, i.e., magnetic resonance imaging and contrast enhanced mammography.

Current Status and Future of BI-RADS in Multimodality Imaging, From the AJR Special Series on Radiology Reporting and Data Systems

BI-RADS is a communication and data tracking system that has evolved since its inception as a brief mammography lexicon and reporting guide into a robust structured reporting platform and comprehensive quality assurance tool for mammography, ultrasound, and MRI. Consistent and appropriate use of the BI-RADS lexicon terminology and assessment categories effectively communicates findings, estimates the risk of malignancy, and provides management recommendations to patients and referring clinicians. The impact of BI-RADS currently extends internationally through six language translations. A condensed version has been proposed to facilitate a phased implementation of BI-RADS in resource-constrained regions. The primary advance of the 5th edition of BI-RADS is harmonization of the lexicon terms across mammography, ultrasound, and MRI. Harmonization has also been achieved across these modalities for the reporting structure, assessment categories, management recommendations, and data tracking system. Areas for improvement relate to certain common findings that lack lexicon descriptors and a need for further clarification of proper use of category 3. BI-RADS is anticipated to continue to evolve for application to a range of emerging breast imaging modalities.

Breast Magnetic Resonance Imaging Audit: Pitfalls, Challenges, and Future Considerations

Breast magnetic resonance (MR) imaging is the most sensitive imaging modality for breast cancer detection and guidelines recommend its use, in addition to screening mammography, for high-risk women. The most recent American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS) manual coordinated cross-modality BI-RADS terminology and established an outcome monitoring section that helps guide a medical imaging outcomes audit. This article provides a framework for performing a breast MR imaging audit in clinical practice, incorporating ACR BI-RADS guidance and more recently published data, clarifies common pitfalls, and discusses audit challenges related to evolving clinical practice.

Ultra-low-dose CT reconstructed with ASiR-V using SmartmA for pulmonary nodule detection and Lung-RADS classifications compared with low-dose CT

AIM

To evaluate the accuracy of ultra-low-dose computed tomography (ULDCT) with ASiR-V using a noise index (SmartmA) for pulmonary nodule detection and Lung CT Screening Reporting And Data System (Lung-RADS) classifications compared with low-dose CT (LDCT).

MATERIALS AND METHODS

Two-hundred and ten patients referred for lung cancer screening underwent conventional chest LDCT (0.80 ± 0.28 mSv) followed immediately by ULDCT (0.16 ± 0.03 mSv). ULDCT was scanned using 120 kV/SmartmA with a noise index of 28 HU and reconstructed with ASiR-V70%. The types and diameters of all nodules were recorded. The attenuation of pure ground-glass nodules (pGGNs) was measured on LDCT. All nodules were further classified using Lung-RADS. Sensitivities of nodule detection on ULDCT were analysed using LDCT as the reference standard. Logistic regression was used to establish a prediction model for the sensitivity of nodules.

RESULTS

LDCT revealed 362 nodules and the overall sensitivity on ULDCT was 90.1%. The sensitivity for solid nodules (SNs) of ≥1 mm diameter was 96.6% (228/236) and 100% (26/26) for SNs of ≥6 mm diameter. For pGGNs of ≥6 mm, the overall sensitivity was 93% (40/43) and 100% (29/29) for nodules with a attenuation value -700 HU or more. The agreement of Lung-RADS classification between two scans was good. On logistic regression, diameter was the only independent predictor for sensitivity of SNs (p<0.05). Diameter and attenuation value were predictors for pGGNs (p<0.05).

CONCLUSION

ULDCT with ASiR-V using SmartmA is suitable for lung-cancer screening in people with a BMI ≤35 kg/m² as it has a low radiation dose of 0.16 mSv, high sensitivity for nodule detection and good performance of Lung-RADS classifications.

Understanding the Mammography Audit

Screening mammography aims to identify small, node-negative breast cancers when they are still curable while maintaining an acceptable range of false-positive recalls and biopsies. The mammography audit is a powerful tool to help radiologists understand their performance with respect to that goal. This article defines audit terms and describes how to use collected and derived data to perform a mammography audit. Accepted benchmarks are discussed as well as their applicability to radiologists and breast imaging practices in the United States. Special considerations regarding volumes and radiologist characteristics are explored, because these factors may affect audit results.

BI-RADS 3 (short-interval follow-up) assessment rate at diagnostic mammography: Correlation with recall rates and utilization as a performance benchmark

The purpose of this study was to identify a correlation between the screening BI-RADS 0 (recall) rates and diagnostic BI-RADS 3 (short-interval follow-up) rates of individual interpreting radiologists, with the goal of utilizing the BI-RADS 3 rate as an acceptable performance metric in the diagnostic population. A multicenter retrospective analysis of medical audit statistics was conducted on annual radiologist performance data collected over a 14-year period in a community hospital-based practice. Mixed regression models were used to estimate the association between screening BI-RADS 0 and diagnostic BI-RADS 3 examinations while adjusting for calendar year, annual radiologist screening volume, annual radiologist diagnostic volume, and diagnostic examination indication. A moderate statistically significant positive correlation was established between the screening BI-RADS 0 rates and Diagnostic BI-RADS 3 rates (Pearson correlation coefficient + 0.349, P ≤ .001). Furthermore, when utilizing a national benchmark range of 8%-12% as an acceptable BI-RADS 0 rate within a screening population, the correlative BI-RADS 3 assessment rate was demonstrated to be approximately 16%. We propose that this BI-RADS category 3 rate may represent an additional acceptable performance metric in the diagnostic population. Routine inclusion of an interpreting mammographer's diagnostic BI-RADS 3 rate in the annual medical audit may help reduce inappropriate and/or excess use of the BI-RADS 3 category, which may lead to significant potential reductions in follow-up examinations with their associated healthcare-related costs, resource expenditure, and induced patient anxiety.

A feasibility study of pulmonary nodule detection by ultralow-dose CT with adaptive statistical iterative reconstruction-V technique

PURPOSE

To evaluate the clinical value of ultralow-dose CT (ULDCT) with adaptive statistical iterative reconstruction-V (ASiR-V) in the detection of pulmonary nodules in a Chinese population.

METHOD

One hundred eighty-eight patients (16.41 ≤ BMI ≤ 29.87 kg/m²) with pulmonary nodules detected on low-dose chest CT (LDCT) underwent local ULDCT at the center of the chosen nodule with a scan length of 3 cm. LDCT was performed using the Assist kV (120/100 kV)/Smart mA mode and at 120 kV/2.8 mAs for ULDCT. After scanning, CT images were reconstructed with ASiR-V 50%. For both scans, nodule diameters were measured and reference standards were established for the presence and types of lung nodules found on LDCT. The sensitivity of ULDCT was compared against the standard, and logistic regression analysis was used to determine the independent predictors for nodule detection.

RESULTS

Compared with LDCT (0.93 ± 0.32 mSv), a 89.7% dose decrease was seen with ULDCT, for which the calculated effective dose was 0.096 ± 0.006 mSv (P < 0.001). LDCT showed 188 nodules, including 123 solid and 65 subsolid nodules. The overall sensitivity for nodule detection in ULDCT was 90.4% (170/188), and 98.2% (54/55) for nodules ≥ 6 mm. In multivariate analysis, nodule types and diameters were independent predictors of sensitivity (P < 0.05). However, patients' BMI had no effect on nodule detection (P > 0.05).

CONCLUSIONS

ULDCT can be used in the management of pulmonary nodules for people with BMI ≤ 30 kg/m² at 10% radiation dose of LDCT.

Mammographic detection of breast cancer in a non-screening country

OBJECTIVE:

To compare the diagnostic accuracy between radiologists' from a country with and without breast cancer screening.

METHODS:

All participating radiologists gave informed consent. A test-set involving 60 mammographic cases (20 cancer and 40 non-cancer) were read by 11 radiologists from a non-screening (NS) country during a workshop in July 2016. 52 radiologists from a screening country read the same test-set at the Royal Australian and New Zealand College of Radiologists' meetings in July 2015. The screening radiologists were classified into two groups: those with less than or equal to 5 years of experience; those with more than 5 years of experience, and each group was compared to the group of NS radiologists. A Kruskal-Wallis test followed by post-hoc multiple comparisons test were used to compare measures of diagnostic accuracy among the reader groups.

RESULTS:

The diagnostic accuracy of the NS radiologists was significantly lower in terms of sensitivity [mean = 54.0; 95% confidence interval (CI) (40.0-67.0)], location sensitivity [mean = 26.0; 95% CI (16.0-37.0)], receive roperating characteristic area under curve [mean = 73.0; 95% CI (66.5-81.0)] and Jackknifefree-response receiver operating characteristics figure-of-merit [mean = 45.0; 95% CI (40.0-50.0)] when compared with the less and more experienced screening radiologists, whilst no difference in specificity [mean = 75.0; 95% CI (70.0- 81.0)] was found. No significant differences in all measured diagnostic accuracy were found between the two groups of screening radiologists.

CONCLUSION:

The mammographic performance of a group of radiologists from a country without screening program was suboptimal compared with radiologists from Australia.

ADVANCES IN KNOWLEDGE:

Identifying mammographic performance in developing countries is required to optimize breast cancer diagnosis.

Association between Screening Mammography Recall Rate and Interval Cancers in the UK Breast Cancer Service Screening Program: A Cohort Study

Purpose To determine whether low levels of recall lead to increased interval cancers and the magnitude of this effect. Materials and Methods The authors retrospectively analyzed prospectively collected data from the UK National Health Service Breast Screening Programme during a 36-month period (April 1, 2005 to March 31, 2008), with 3-year follow-up in women aged 50-70 years. Data on recall, cancers detected at screening, and interval cancers were available for each of the 84 breast screening units and for each year (n = 252). The association between interval cancers and recalls was modeled by using Poisson regression on aggregated data and according to age (5-year intervals) and screening type (prevalent vs incident). Results The authors analyzed 5 126 689 screening episodes, demonstrating an average recall to assessment rate (RAR) of 4.56% (range, 1.64%-8.42%; standard deviation, 1.15%), cancer detection rate of 8.1 per 1000 women screened, and interval cancer rate (ICR) of 3.1 per 1000 women screened. Overall, a significant negative association was found between RAR and ICR (Poisson regression coefficient: -0.039 [95% confidence interval: -0.062, -0.017]; P = .001), with approximately one fewer interval cancer for every additional 80-84 recalls. Subgroup analysis revealed similar negative correlations in women aged 50-54 years (P = .002), 60-64 years (P = .01), and 65-69 years (P = .008) as well as in incident screens (P = .001) and prevalent screens (P = .04). No significant relationship was found in women aged 55-59 years (P = .46). Conclusion There was a statistically significant negative correlation between RAR and ICR, which suggests the merit of a minimum threshold for RAR. © RSNA, 2018 Online supplemental material is available for this article.

Assessing the Recall Rate for Screening Mammography: Comparing the Medicare Hospital Compare Dataset With the National Mammography Database

OBJECTIVE

High-quality screening mammography has been shown to substantially reduce mortality from breast cancer. Recall rate is a principal performance metric for screening mammography because it directly relates to the rate of false-positive examinations. This study aims to compare the recall rate derived using two sources-the claims-based Hospital Compare (HC) dataset from the Centers for Medicare & Medicaid Services versus the National Mammography Database (NMD) from the American College of Radiology-to understand the implications in pay-for-performance and quality improvement activities.

MATERIALS AND METHODS

This study retrospectively compared the recall rate reported by NMD facilities with that reported in the HC dataset. Site matching was performed by facility name and zip code, followed by manual verification. Scatterplots, correlations, a paired t test, and Bland-Altman analysis were performed to assess association between the two measures.

RESULTS

During the period from October 1 to December 1, 2016, 92 facilities were unambiguously matched using 2014-2015 records in both datasets. The recall rates were positively correlated (r = 0.428, p < 0.001), but the mean HC recall rate (8.5% ± 2.86% [SD]) was significantly (p < 0.001) lower than the mean NMD recall rate (10.6% ± 3.90%).

CONCLUSION

The NMD and HC are two commonly used datasets for measuring screening mammography recall rate. Although recall rates are correlated at the individual facility level, there are important differences that have implications for quality improvement and pay-for-performance.

Integrating Customer Intimacy Into Radiology to Improve the Patient Perspective: The Case of Breast Cancer Screening

OBJECTIVE

The customer intimacy business model has emerged as a key operational approach for health care organizations as they move toward patient-centered care. The question arises how the customer intimacy approach can be implemented in the clinical setting and whether it can help practitioners address problems and improve quality of care.

CONCLUSION

Breast cancer screening and its emphasis on the patient perspective provides an interesting case study for understanding how the customer intimacy approach can be integrated into radiologic practice to improve the patient experience.