Variability in interpretive performance at screening mammography and radiologists' characteristics associated with accuracy

Modeling Radiologists' Assessments to Explore Pairing Strategies for Optimized Double Reading of Screening Mammograms

PURPOSE

To develop a model that simulates radiologist assessments and use it to explore whether pairing readers based on their individual performance characteristics could optimize screening performance.

METHODS

Logistic regression models were designed and used to model individual radiologist assessments. For model evaluation, model-predicted individual performance metrics and paired disagreement rates were compared against the observed data using Pearson correlation coefficients. The logistic regression models were subsequently used to simulate different screening programs with reader pairing based on individual true-positive rates (TPR) and/or false-positive rates (FPR). For this, retrospective results from breast cancer screening programs employing double reading in Sweden, England, and Norway were used. Outcomes of random pairing were compared against those composed of readers with similar and opposite TPRs/FPRs, with positive assessments defined by either reader flagging an examination as abnormal.

RESULTS

The analysis data sets consisted of 936,621 (Sweden), 435,281 (England), and 1,820,053 (Norway) examinations. There was good agreement between the model-predicted and observed radiologists' TPR and FPR (r ≥ 0.969). Model-predicted negative-case disagreement rates showed high correlations (r ≥ 0.709), whereas positive-case disagreement rates had lower correlation levels due to sparse data (r ≥ 0.532). Pairing radiologists with similar FPR characteristics (Sweden: 4.50% [95% confidence interval: 4.46%-4.54%], England: 5.51% [5.47%-5.56%], Norway: 8.03% [7.99%-8.07%]) resulted in significantly lower FPR than with random pairing (Sweden: 4.74% [4.70%-4.78%], England: 5.76% [5.71%-5.80%], Norway: 8.30% [8.26%-8.34%]), reducing examinations sent to consensus/arbitration while the TPR did not change significantly. Other pairing strategies resulted in equal or worse performance than random pairing.

CONCLUSIONS

Logistic regression models accurately predicted screening mammography assessments and helped explore different radiologist pairing strategies. Pairing readers with similar modeled FPR characteristics reduced the number of examinations unnecessarily sent to consensus/arbitration without significantly compromising the TPR.

HIGHLIGHTS

A logistic-regression model can be derived that accurately predicts individual and paired reader performance during mammography screening reading.Pairing screening mammography radiologists with similar false-positive characteristics reduced false-positive rates with no significant loss in true positives and may reduce the number of examinations unnecessarily sent to consensus/arbitration.

Improving mammography interpretation for both novice and experienced readers: a comparative study of two commercial artificial intelligence software

OBJECTIVES

To evaluate the improvement of mammography interpretation for novice and experienced radiologists assisted by two commercial AI software.

METHODS

We compared the performance of two AI software (AI-1 and AI-2) in two experienced and two novice readers for 200 mammographic examinations (80 cancer cases). Two reading sessions were conducted within 4 weeks. The readers rated the likelihood of malignancy (range, 1-7) and the percentage probability of malignancy (range, 0-100%), with and without AI assistance. Differences in AUROC, sensitivity, and specificity were analyzed.

RESULTS

Mean AUROC increased in both novice (0.86 to 0.90 with AI-1 [p = 0.005]; 0.91 with AI-2 [p < 0.001]) and experienced readers (0.87 to 0.92 with AI-1 [p < 0.001]; 0.90 with AI-2 [p = 0.004]). Sensitivities increased from 81.3 to 88.8% with AI-1 (p = 0.027) and to 91.3% with AI-2 (p = 0.005) in novice readers, and from 81.9 to 90.6% with AI-1 (p = 0.001) and to 87.5% with AI-2 (p = 0.016) in experienced readers. Specificity did not decrease significantly in both novice (p > 0.999, both) and experienced readers (p > 0.999 with AI-1 and 0.282 with AI-2). There was no significant difference in the performance change depending on the type of AI software (p > 0.999).

CONCLUSION

Commercial AI software improved the diagnostic performance of both novice and experienced readers. The type of AI software used did not significantly impact performance changes. Further validation with a larger number of cases and readers is needed.

CLINICAL RELEVANCE STATEMENT

Commercial AI software effectively aided mammography interpretation irrespective of the experience level of human readers.

KEY POINTS

• Mammography interpretation remains challenging and is subject to a wide range of interobserver variability. • In this multi-reader study, two commercial AI software improved the sensitivity of mammography interpretation by both novice and experienced readers. The type of AI software used did not significantly impact performance changes. • Commercial AI software may effectively support mammography interpretation irrespective of the experience level of human readers.

Quantitative evaluation of Saliency-Based Explainable artificial intelligence (XAI) methods in Deep Learning-Based mammogram analysis

BACKGROUND

Explainable Artificial Intelligence (XAI) is prominent in the diagnostics of opaque deep learning (DL) models, especially in medical imaging. Saliency methods are commonly used, yet there's a lack of quantitative evidence regarding their performance.

OBJECTIVES

To quantitatively evaluate the performance of widely utilized saliency XAI methods in the task of breast cancer detection on mammograms.

METHODS

Three radiologists drew ground-truth boxes on a balanced mammogram dataset of women (n = 1496 cancer-positive and negative scans) from three centers. A modified, pre-trained DL model was employed for breast cancer detection, using MLO and CC images. Saliency XAI methods, including Gradient-weighted Class Activation Mapping (Grad-CAM), Grad-CAM++, and Eigen-CAM, were evaluated. We utilized the Pointing Game to assess these methods, determining if the maximum value of a saliency map aligned with the bounding boxes, representing the ratio of correctly identified lesions among all cancer patients, with a value ranging from 0 to 1.

RESULTS

The development sample included 2,244 women (75%), with the remaining 748 women (25%) in the testing set for unbiased XAI evaluation. The model's recall, precision, accuracy, and F1-Score in identifying cancer in the testing set were 69%, 88%, 80%, and 0.77, respectively. The Pointing Game Scores for Grad-CAM, Grad-CAM++, and Eigen-CAM were 0.41, 0.30, and 0.35 in women with cancer and marginally increased to 0.41, 0.31, and 0.36 when considering only true-positive samples.

CONCLUSIONS

While saliency-based methods provide some degree of explainability, they frequently fall short in delineating how DL models arrive at decisions in a considerable number of instances.

Impact of a Categorical AI System for Digital Breast Tomosynthesis on Breast Cancer Interpretation by Both General Radiologists and Breast Imaging Specialists

Purpose To evaluate performance improvements of general radiologists and breast imaging specialists when interpreting a set of diverse digital breast tomosynthesis (DBT) examinations with the aid of a custom-built categorical artificial intelligence (AI) system. Materials and Methods A fully balanced multireader, multicase reader study was conducted to compare the performance of 18 radiologists (nine general radiologists and nine breast imaging specialists) reading 240 retrospectively collected screening DBT mammograms (mean patient age, 59.8 years ± 11.3 [SD]; 100% women), acquired between August 2016 and March 2019, with and without the aid of a custom-built categorical AI system. The area under the receiver operating characteristic curve (AUC), sensitivity, and specificity across general radiologists and breast imaging specialists reading with versus without AI were assessed. Reader performance was also analyzed as a function of breast cancer characteristics and patient subgroups. Results Every radiologist demonstrated improved interpretation performance when reading with versus without AI, with an average AUC of 0.93 versus 0.87, demonstrating a difference in AUC of 0.06 (95% CI: 0.04, 0.08; P < .001). Improvement in AUC was observed for both general radiologists (difference of 0.08; P < .001) and breast imaging specialists (difference of 0.04; P < .001) and across all cancer characteristics (lesion type, lesion size, and pathology) and patient subgroups (race and ethnicity, age, and breast density) examined. Conclusion A categorical AI system helped improve overall radiologist interpretation performance of DBT screening mammograms for both general radiologists and breast imaging specialists and across various patient subgroups and breast cancer characteristics. Keywords: Computer-aided Diagnosis, Screening Mammography, Digital Breast Tomosynthesis, Breast Cancer, Screening, Convolutional Neural Network (CNN), Artificial Intelligence Supplemental material is available for this article. © RSNA, 2024.

A Preliminary Investigation into Search and Matching for Tumor Discrimination in World Health Organization Breast Taxonomy Using Deep Networks

Breast cancer is one of the most common cancers affecting women worldwide. It includes a group of malignant neoplasms with a variety of biological, clinical, and histopathologic characteristics. There are more than 35 different histologic forms of breast lesions that can be classified and diagnosed histologically according to cell morphology, growth, and architecture patterns. Recently, deep learning, in the field of artificial intelligence, has drawn a lot of attention for the computerized representation of medical images. Searchable digital atlases can provide pathologists with patch-matching tools, allowing them to search among evidently diagnosed and treated archival cases, a technology that may be regarded as computational second opinion. In this study, we indexed and analyzed the World Health Organization breast taxonomy (Classification of Tumors fifth ed.) spanning 35 tumor types. We visualized all tumor types using deep features extracted from a state-of-the-art deep-learning model, pretrained on millions of diagnostic histopathology images from the Cancer Genome Atlas repository. Furthermore, we tested the concept of a digital "atlas" as a reference for search and matching with rare test cases. The patch similarity search within the World Health Organization breast taxonomy data reached >88% accuracy when validating through "majority vote" and >91% accuracy when validating using top n tumor types. These results show for the first time that complex relationships among common and rare breast lesions can be investigated using an indexed digital archive.

State-of-the-Art of Breast Cancer Diagnosis in Medical Images via Convolutional Neural Networks (CNNs)

Early detection of breast cancer is crucial for a better prognosis. Various studies have been conducted where tumor lesions are detected and localized on images. This is a narrative review where the studies reviewed are related to five different image modalities: histopathological, mammogram, magnetic resonance imaging (MRI), ultrasound, and computed tomography (CT) images, making it different from other review studies where fewer image modalities are reviewed. The goal is to have the necessary information, such as pre-processing techniques and CNN-based diagnosis techniques for the five modalities, readily available in one place for future studies. Each modality has pros and cons, such as mammograms might give a high false positive rate for radiographically dense breasts, while ultrasounds with low soft tissue contrast result in early-stage false detection, and MRI provides a three-dimensional volumetric image, but it is expensive and cannot be used as a routine test. Various studies were manually reviewed using particular inclusion and exclusion criteria; as a result, 91 recent studies that classify and detect tumor lesions on breast cancer images from 2017 to 2022 related to the five image modalities were included. For histopathological images, the maximum accuracy achieved was around 99 % , and the maximum sensitivity achieved was 97.29 % by using DenseNet, ResNet34, and ResNet50 architecture. For mammogram images, the maximum accuracy achieved was 96.52 % using a customized CNN architecture. For MRI, the maximum accuracy achieved was 98.33 % using customized CNN architecture. For ultrasound, the maximum accuracy achieved was around 99 % by using DarkNet-53, ResNet-50, G-CNN, and VGG. For CT, the maximum sensitivity achieved was 96 % by using Xception architecture. Histopathological and ultrasound images achieved higher accuracy of around 99 % by using ResNet34, ResNet50, DarkNet-53, G-CNN, and VGG compared to other modalities for either of the following reasons: use of pre-trained architectures with pre-processing techniques, use of modified architectures with pre-processing techniques, use of two-stage CNN, and higher number of studies available for Artificial Intelligence (AI)/machine learning (ML) researchers to reference. One of the gaps we found is that only a single image modality is used for CNN-based diagnosis; in the future, a multiple image modality approach can be used to design a CNN architecture with higher accuracy.

Artificial intelligence-based computer-assisted detection/diagnosis (AI-CAD) for screening mammography: Outcomes of AI-CAD in the mammographic interpretation workflow

Purpose

To evaluate the stand-alone diagnostic performances of AI-CAD and outcomes of AI-CAD detected abnormalities when applied to the mammographic interpretation workflow.

Methods

From January 2016 to December 2017, 6499 screening mammograms of 5228 women were collected from a single screening facility. Historic reads of three radiologists were used as radiologist interpretation. A commercially-available AI-CAD was used for analysis. One radiologist not involved in interpretation had retrospectively reviewed the abnormality features and assessed the significance (negligible vs. need recall) of the AI-CAD marks. Ground truth in terms of cancer, benign or absence of abnormality was confirmed according to histopathologic diagnosis or negative results on the next-round screen.

Results

Of the 6499 mammograms, 6282 (96.7%) were in the negative, 189 (2.9%) were in the benign, and 28 (0.4%) were in the cancer group. AI-CAD detected 5 (17.9%, 5 of 28) of the 9 cancers that were intially interpreted as negative. Of the 648 AI-CAD recalls, 89.0% (577 of 648) were marks seen on examinations in the negative group, and 267 (41.2%) of the AI-CAD marks were considered to be negligible. Stand-alone AI-CAD has significantly higher recall rates (10.0% vs. 3.4%, P < 0.001) with comparable sensitivity and cancer detection rates (P = 0.086 and 0.102, respectively) when compared to the radiologists' interpretation.

Conclusion

AI-CAD detected 17.9% additional cancers on screening mammography that were initially overlooked by the radiologists. In spite of the additional cancer detection, AI-CAD had significantly higher recall rates in the clinical workflow, in which 89.0% of AI-CAD marks are on negative mammograms.

Improving Radiology Oncologic Imaging Trainee Case Diversity through Automatic Examination Assignment: Retrospective Study from a Tertiary Cancer Center

In a retrospective single-center study, the authors assessed the efficacy of an automated imaging examination assignment system for enhancing the diversity of subspecialty examinations reported by oncologic imaging fellows. The study aimed to mitigate traditional biases of manual case selection and ensure equitable exposure to various case types. Methods included evaluating the proportion of "uncommon" to "common" cases reported by fellows before and after system implementation and measuring the weekly Shannon Diversity Index to determine case distribution equity. The proportion of reported uncommon cases more than doubled from 8.6% to 17.7% in total, at the cost of a concurrent 9.0% decrease in common cases from 91.3% to 82.3%. The weekly Shannon Diversity Index per fellow increased significantly from 0.66 (95% CI: 0.65, 0.67) to 0.74 (95% CI: 0.72, 0.75; P < .001), confirming a more balanced case distribution among fellows after introduction of the automatic assignment. © RSNA, 2023 Keywords: Computer Applications, Education, Fellows, Informatics, MRI, Oncologic Imaging.

Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine

Breast cancer is a significant cause of cancer-related mortality in women worldwide. Early and precise diagnosis is crucial, and clinical outcomes can be markedly enhanced. The rise of artificial intelligence (AI) has ushered in a new era, notably in image analysis, paving the way for major advancements in breast cancer diagnosis and individualized treatment regimens. In the diagnostic workflow for patients with breast cancer, the role of AI encompasses screening, diagnosis, staging, biomarker evaluation, prognostication, and therapeutic response prediction. Although its potential is immense, its complete integration into clinical practice is challenging. Particularly, these challenges include the imperatives for extensive clinical validation, model generalizability, navigating the "black-box" conundrum, and pragmatic considerations of embedding AI into everyday clinical environments. In this review, we comprehensively explored the diverse applications of AI in breast cancer care, underlining its transformative promise and existing impediments. In radiology, we specifically address AI in mammography, tomosynthesis, risk prediction models, and supplementary imaging methods, including magnetic resonance imaging and ultrasound. In pathology, our focus is on AI applications for pathologic diagnosis, evaluation of biomarkers, and predictions related to genetic alterations, treatment response, and prognosis in the context of breast cancer diagnosis and treatment. Our discussion underscores the transformative potential of AI in breast cancer management and emphasizes the importance of focused research to realize the full spectrum of benefits of AI in patient care.

Spatially localized sparse approximations of deep features for breast mass characterization

We propose a deep feature-based sparse approximation classification technique for classification of breast masses into benign and malignant categories in film screen mammographs. This is a significant application as breast cancer is a leading cause of death in the modern world and improvements in diagnosis may help to decrease rates of mortality for large populations. While deep learning techniques have produced remarkable results in the field of computer-aided diagnosis of breast cancer, there are several aspects of this field that remain under-studied. In this work, we investigate the applicability of deep-feature-generated dictionaries to sparse approximation-based classification. To this end we construct dictionaries from deep features and compute sparse approximations of Regions Of Interest (ROIs) of breast masses for classification. Furthermore, we propose block and patch decomposition methods to construct overcomplete dictionaries suitable for sparse coding. The effectiveness of our deep feature spatially localized ensemble sparse analysis (DF-SLESA) technique is evaluated on a merged dataset of mass ROIs from the CBIS-DDSM and MIAS datasets. Experimental results indicate that dictionaries of deep features yield more discriminative sparse approximations of mass characteristics than dictionaries of imaging patterns and dictionaries learned by unsupervised machine learning techniques such as K-SVD. Of note is that the proposed block and patch decomposition strategies may help to simplify the sparse coding problem and to find tractable solutions. The proposed technique achieves competitive performances with state-of-the-art techniques for benign/malignant breast mass classification, using 10-fold cross-validation in merged datasets of film screen mammograms.

Reducing ' probably benign ' assessments in normal mammograms: The role of radiologist experience

Rationale and objectives

to investigate the relationship between radiologists' experience in reporting mammograms, their caseloads, and the classification of category '3' or 'Probably Benign' on normal mammograms.

Materials and Methods

A total of 92 board-certified radiologists participated. Self-reported parameters related to experience, including age, years since qualifying as a radiologist, years of experience reading mammograms, number of mammograms read per year, and hours spent reading mammograms per week, were documented. To assess the radiologists' accuracy, "Probably Benign fractions" was calculated by dividing the number of "Probably Benign findings" given by each radiologist in the normal cases by the total number of normal cases Probably Benign fractions were correlated with various factors, such as the radiologists' experience.

Results

The results of the statistical analysis revealed a significant negative correlation between radiologist experience and 'Probably Benign' fractions for normal images. Specifically, for normal cases, the number of mammograms read per year (r = -0.29, P = 0.006) and the number of mammograms read over the radiologist's lifetime (r = -0.21, P = 0.049) were both negatively correlated with 'Probably Benign' fractions.

Conclusion

The results indicate that a relationship exists between increased reading volumes and reduced assessments of 'Probably Benign' in normal mammograms. The implications of these findings extend to the effectiveness of screening programs and the recall rates.

An International Non-Inferiority Study for the Benchmarking of AI for Routine Radiology Cases: Chest X-ray, Fluorography and Mammography

An international reader study was conducted to gauge an average diagnostic accuracy of radiologists interpreting chest X-ray images, including those from fluorography and mammography, and establish requirements for stand-alone radiological artificial intelligence (AI) models. The retrospective studies in the datasets were labelled as containing or not containing target pathological findings based on a consensus of two experienced radiologists, and the results of a laboratory test and follow-up examination, where applicable. A total of 204 radiologists from 11 countries with various experience performed an assessment of the dataset with a 5-point Likert scale via a web platform. Eight commercial radiological AI models analyzed the same dataset. The AI AUROC was 0.87 (95% CI:0.83-0.9) versus 0.96 (95% CI 0.94-0.97) for radiologists. The sensitivity and specificity of AI versus radiologists were 0.71 (95% CI 0.64-0.78) versus 0.91 (95% CI 0.86-0.95) and 0.93 (95% CI 0.89-0.96) versus 0.9 (95% CI 0.85-0.94) for AI. The overall diagnostic accuracy of radiologists was superior to AI for chest X-ray and mammography. However, the accuracy of AI was noninferior to the least experienced radiologists for mammography and fluorography, and to all radiologists for chest X-ray. Therefore, an AI-based first reading could be recommended to reduce the workload burden of radiologists for the most common radiological studies such as chest X-ray and mammography.

Breast cancer missed at screening; hindsight or mistakes?

PURPOSE

To investigate radiologists' interpretation scores of screening mammograms prior to diagnosis of screen-detected and interval breast cancers retrospectively classified as missed or true negative.

METHODS

We included data on radiologists' interpretation scores at screening prior to diagnosis for 1223 screen-detected and 1007 interval cancer cases classified as missed or true negative in an informed consensus-based review. All prior screening examinations were independently scored 1-5 by two radiologists; score 1 by both was considered concordant negative, score ≥ 2 by one radiologist discordant, and score ≥ 2 by both concordant positive. We analyzed associations between interpretation, review categories, mammographic features and histopathological findings using descriptive statistics and logistic regression.

RESULTS

Among screen-detected cancers, 31% of missed and 10% of true negative cancers had discordant or concordant positive interpretation at prior screening. The corresponding percentages for interval cancer were 21% and 8%. Age-adjusted odds ratio (OR) and 95% confidence interval (CI) for missed screen-detected cancer was 3.8 (95% CI: 2.6-5.4) after discordant and 5.5 (95% CI: 3.2-9.5) after concordant positive interpretation, using concordant negative as reference. Corresponding ORs for missed interval cancer were 3.0 (95% CI: 2.0-4.5) for discordant and 6.3 (95% CI: 2.3-17.5) for concordant positive interpretation. Asymmetry was the dominating mammographic feature at prior screening for all, except concordant positive screen-detected cancers where a mass dominated. Histopathological characteristics did not vary statistically with interpretation.

CONCLUSIONS

Most cancers were interpreted negatively at screening prior to diagnosis. Increased risk for missed screen-detected or interval cancer was observed after positive interpretation at prior screening.

Do Reader Characteristics Affect Diagnostic Efficacy in Screening Mammography? A Systematic Review

To examine reader characteristics associated with diagnostic efficacy in the interpretation of screening mammograms. A systematic search of the literature was conducted using databases such as Cochrane, Scopus, Medline, Embase, Web of Science, and PubMed. Search terms were combined with "AND" or "OR" and included: "Radiologist's characteristics AND performance"; "radiologist experience AND screening mammography"; "annual volume read AND diagnostic efficacy"; "screening mammography performance OR diagnostic efficacy". Studies were included if they assessed reader performance in screening mammography interpretation, breast readers, used a reference standard to assess the performance, and were published in the English language. Twenty-eight studies were reviewed. Increasing reader's age was associated with lower false positive rates. No association was found between gender and performance. Half of the studies showed no association between years of reading mammograms and performance. Most studies showed that high reading volume was more likely to be associated with increased sensitivity, cancer detection rates (CDR), lower recall rate, and lower false positive rates. Inconsistent associations were found between fellowship training in breast imaging and reader performance. Specialization in breast imaging was associated with better CDR, sensitivity, and specificity. Limited studies were available to establish the association between performance and factors such as time spent in breast imaging (n = 2), screening focus (n = 1), formal rotation in mammography (n = 1), owner of practice (n = 1), and practice type (n = 1). No individual characteristics is associated with versatility in diagnostic efficacy, albeit reading volume and specialization in breast imaging appear to be associated with with increased sensitivity and CDR without significantly affecting other performance metrics.

Proposal and Definition of an Intelligent Clinical Decision Support System Applied to the Screening and Early Diagnosis of Breast Cancer

Breast cancer is the most frequently diagnosed tumor pathology on a global scale, being the leading cause of mortality in women. In light of this problem, screening programs have been implemented on the population at risk in the form of mammograms, starting in the 20th century. This has considerably reduced the associated deaths, as well as improved the prognosis of the patients who suffer from this disease. In spite of this, the evaluation of mammograms is not without certain variability and depends, to a large extent, on the experience and training of the medical team carrying out the assessment. With the aim of supporting the evaluation process of mammogram images and improving the diagnosis process, this work presents the design, development and proof of concept of a novel intelligent clinical decision support system, grounded on two predictive approaches that work concurrently. The first of them applies a series of expert systems based on fuzzy inferential engines, geared towards the treatment of the characteristics associated with the main findings present in mammograms. This allows the determination of a series of risk indicators, the Symbolic Risks, related to the risk of developing breast cancer according to the different findings. The second one implements a classification machine learning algorithm, which using data related to mammography findings as well as general patient information determines another metric, the Statistical Risk, also linked to the risk of developing breast cancer. These risk indicators are then combined, resulting in a new indicator, the Global Risk. This could then be corrected using a weighting factor according to the BI-RADS category, allocated to each patient by the medical team in charge. Thus, the Corrected Global Risk is obtained, which after interpretation can be used to establish the patient’s status as well as generate personalized recommendations. The proof of concept and software implementation of the system were carried out using a data set with 130 patients from a database from the School of Medicine and Public Health of the University of Wisconsin-Madison. The results obtained were encouraging, highlighting the potential use of the application, albeit pending intensive clinical validation in real environments. Moreover, its possible integration in hospital computer systems is expected to improve diagnostic processes as well as patient prognosis.

Mucins as Potential Biomarkers for Early Detection of Cancer

Early detection significantly correlates with improved survival in cancer patients. So far, a limited number of biomarkers have been validated to diagnose cancers at an early stage. Considering the leading cancer types that contribute to more than 50% of deaths in the USA, we discuss the ongoing endeavors toward early detection of lung, breast, ovarian, colon, prostate, liver, and pancreatic cancers to highlight the significance of mucin glycoproteins in cancer diagnosis. As mucin deregulation is one of the earliest events in most epithelial malignancies following oncogenic transformation, these high-molecular-weight glycoproteins are considered potential candidates for biomarker development. The diagnostic potential of mucins is mainly attributed to their deregulated expression, altered glycosylation, splicing, and ability to induce autoantibodies. Secretory and shed mucins are commonly detected in patients' sera, body fluids, and tumor biopsies. For instance, CA125, also called MUC16, is one of the biomarkers implemented for the diagnosis of ovarian cancer and is currently being investigated for other malignancies. Similarly, MUC5AC, a secretory mucin, is a potential biomarker for pancreatic cancer. Moreover, anti-mucin autoantibodies and mucin-packaged exosomes have opened new avenues of biomarker development for early cancer diagnosis. In this review, we discuss the diagnostic potential of mucins in epithelial cancers and provide evidence and a rationale for developing a mucin-based biomarker panel for early cancer detection.

Identification of factors associated with diagnostic performance variation in reporting of mammograms: A review

OBJECTIVES

This narrative review aims to identify what factors are linked to diagnostic performance variation for those who interpret mammograms. Identification of influential factors has potential to contribute to the optimisation of breast cancer diagnosis. PubMed, ScienceDirect and Google Scholar databases were searched using the following terms: 'Radiology', 'Radiologist', 'Radiographer', 'Radiography', 'Mammography', 'Interpret', 'read', 'observe' 'report', 'screen', 'image', 'performance' and 'characteristics.' Exclusion criteria included articles published prior to 2000 as digital mammography was introduced at this time. Non-English articles language were also excluded. 38 of 2542 studies identified were analysed.

KEY FINDINGS

Influencing factors included, new technology, volume of reads, experience and training, availability of prior images, social networking, fatigue and time-of-day of interpretation. Advancements in breast imaging such as digital breast tomosynthesis and volume of mammograms are primary factors that affect performance as well as tiredness, time-of-day when images are interpreted, stages of training and years of experience. Recent studies emphasised the importance of social networking and knowledge sharing if breast cancer diagnosis is to be optimised.

CONCLUSION

It was demonstrated that data on radiologist performance variability is widely available but there is a paucity of data on radiographers who interpret mammographic images.

IMPLICATIONS FOR PRACTICE

This scarcity of research needs to be addressed in order to optimise radiography-led reporting and set baseline values for diagnostic efficacy.

Periapical Radiography versus Cone Beam Computed Tomography in Endodontic Disease Detection: A Free-response, Factorial Study

AIM

To assess and compare reader performance in interpreting digital periapical (PA) radiography and cone beam computed tomography (CBCT) in endodontic disease detection, using a free-response, factorial model.

MATERIALS AND METHODS

A reader performance study of 2 image test sets was undertaken using a factorial, free-response design, accounting for the independent variables: case type, case severity, reader type, and imaging modality. Twenty-two readers interpreted 60 PA and 60 CBCT images divided into 5 categories: diseased-subtle, diseased-moderate, diseased-obvious, nondiseased-subtle, and nondiseased-obvious. Lesion localization fraction, specificity, false positive (FP) marks, and the weighted alternative free-response receiver operating characteristic figure of merit were calculated.

RESULTS

CBCT had greater specificity than PA in the obvious nondiseased cases (P = .01) and no significant difference in the subtle nondiseased category. Weighted alternative free-response receiver operating characteristic values were higher for PA than CBCT in the subtle diseased (P = .02) and moderate diseased (P = .01) groups with no significant difference between in the obvious diseased groups. CBCT had higher mean FPs than PA (P < .05) in subtle diseased cases. Mean lesion localization fraction in the moderate diseased group was higher in PA than CBCT (P = .003). No relationships were found between clinical experience and all diagnostic performance measures, except for in the obvious diseased CBCT group, where increasing experience was associated mean FP marks (P = .04).

CONCLUSIONS

Reader performance in the detection of endodontic disease is better with PA radiography than CBCT. Clinical experience does not impact upon the accuracy of interpretation of both PA radiography and CBCT.

The effect of clinical history on diagnostic performance of endodontic cone-beam CT interpretation

AIM

To assess the effect of clinical history on the interpretation of endodontic disease in dental cone-beam computed tomography (CBCT).

MATERIALS AND METHODS

A reader performance study of an image test-set was undertaken using a factorial, free-response, crossover design, accounting for the independent variables: case type, case severity, reader type, and reading modality. Twenty-three readers interpreted 60 CBCT images twice over two reading sessions using a balanced design, once with access to clinical history and once without, where 30 in each session included history. Lesion localisations, specificity, false-positive marks and the weighted alternative free-response receiver operating characteristic (wAFROC1) figure of merit were calculated.

RESULTS

Clinical history had no significant effect on specificity and false-positive rates in non-diseased cases (p>0.05), but improved lesion localisation in subtle and obvious diseased cases (p<0.01). wAFROC1 values were higher with clinical history for subtle (0.58 versus 0.48; p<0.001) and obvious (0.77 versus 0.71; p=0.006) diseased categories. No associations were observed between clinical history and both readers' years of experience and reading volume in the non-diseased categories. Readers with fewer (p=0.03) and moderate (p=0.008) years of experience and low (p=0.002) CBCT reading volume demonstrated better lesion localisation in subtle diseased cases when clinical history was available.

CONCLUSIONS

Clinical history improved the interpretation of CBCT images with disease without affecting the interpretation of images without disease. Less and moderately experienced readers and low-volume readers benefitted more from availability of clinical history.

Idiosyncratic biases in the perception of medical images

Introduction

Radiologists routinely make life-altering decisions. Optimizing these decisions has been an important goal for many years and has prompted a great deal of research on the basic perceptual mechanisms that underlie radiologists' decisions. Previous studies have found that there are substantial individual differences in radiologists' diagnostic performance (e.g., sensitivity) due to experience, training, or search strategies. In addition to variations in sensitivity, however, another possibility is that radiologists might have perceptual biases-systematic misperceptions of visual stimuli. Although a great deal of research has investigated radiologist sensitivity, very little has explored the presence of perceptual biases or the individual differences in these.

Methods

Here, we test whether radiologists' have perceptual biases using controlled artificial and Generative Adversarial Networks-generated realistic medical images. In Experiment 1, observers adjusted the appearance of simulated tumors to match the previously shown targets. In Experiment 2, observers were shown with a mix of real and GAN-generated CT lesion images and they rated the realness of each image.

Results

We show that every tested individual radiologist was characterized by unique and systematic perceptual biases; these perceptual biases cannot be simply explained by attentional differences, and they can be observed in different imaging modalities and task settings, suggesting that idiosyncratic biases in medical image perception may widely exist.

Discussion

Characterizing and understanding these biases could be important for many practical settings such as training, pairing readers, and career selection for radiologists. These results may have consequential implications for many other fields as well, where individual observers are the linchpins for life-altering perceptual decisions.

Beyond fine-tuning: Classifying high resolution mammograms using function-preserving transformations

The task of classifying mammograms is very challenging because the lesion is usually small in the high resolution image. The current state-of-the-art approaches for medical image classification rely on using the de-facto method for convolutional neural networks-fine-tuning. However, there are fundamental differences between natural images and medical images, which based on existing evidence from the literature, limits the overall performance gain when designed with algorithmic approaches. In this paper, we propose to go beyond fine-tuning by introducing a novel framework called MorphHR, in which we highlight a new transfer learning scheme. The idea behind the proposed framework is to integrate function-preserving transformations, for any continuous non-linear activation neurons, to internally regularise the network for improving mammograms classification. The proposed solution offers two major advantages over the existing techniques. Firstly and unlike fine-tuning, the proposed approach allows for modifying not only the last few layers but also several of the first ones on a deep ConvNet. By doing this, we can design the network front to be suitable for learning domain specific features. Secondly, the proposed scheme is scalable to hardware. Therefore, one can fit high resolution images on standard GPU memory. We show that by using high resolution images, one prevents losing relevant information. We demonstrate, through numerical and visual experiments, that the proposed approach yields to a significant improvement in the classification performance over state-of-the-art techniques, and is indeed on a par with radiology experts. Moreover and for generalisation purposes, we show the effectiveness of the proposed learning scheme on another large dataset, the ChestX-ray14, surpassing current state-of-the-art techniques.

Evolution of research trends in artificial intelligence for breast cancer diagnosis and prognosis over the past two decades: A bibliometric analysis

Objective

In recent years, among the available tools, the concurrent application of Artificial Intelligence (AI) has improved the diagnostic performance of breast cancer screening. In this context, the present study intends to provide a comprehensive overview of the evolution of AI for breast cancer diagnosis and prognosis research using bibliometric analysis.

Methodology

Therefore, in the present study, relevant peer-reviewed research articles published from 2000 to 2021 were downloaded from the Scopus and Web of Science (WOS) databases and later quantitatively analyzed and visualized using Bibliometrix (R package). Finally, open challenges areas were identified for future research work.

Results

The present study revealed that the number of literature studies published in AI for breast cancer detection and survival prediction has increased from 12 to 546 between the years 2000 to 2021. The United States of America (USA), the Republic of China, and India are the most productive publication-wise in this field. Furthermore, the USA leads in terms of the total citations; however, hungry and Holland take the lead positions in average citations per year. Wang J is the most productive author, and Zhan J is the most relevant author in this field. Stanford University in the USA is the most relevant affiliation by the number of published articles. The top 10 most relevant sources are Q1 journals with PLOS ONE and computer in Biology and Medicine are the leading journals in this field. The most trending topics related to our study, transfer learning and deep learning, were identified.

Conclusion

The present findings provide insight and research directions for policymakers and academic researchers for future collaboration and research in AI for breast cancer patients.

CoroNet: Deep Neural Network-Based End-to-End Training for Breast Cancer Diagnosis

In 2020, according to the publications of both the Global Cancer Observatory (GCO) and the World Health Organization (WHO), breast cancer (BC) represents one of the highest prevalent cancers in women worldwide. Almost 47% of the world’s 100,000 people are diagnosed with breast cancer, among females. Moreover, BC prevails among 38.8% of Egyptian women having cancer. Current deep learning developments have shown the common usage of deep convolutional neural networks (CNNs) for analyzing medical images. Unlike the randomly initialized ones, pre-trained natural image database (ImageNet)-based CNN models may become successfully fine-tuned to obtain improved findings. To conduct the automatic detection of BC by the CBIS-DDSM dataset, a CNN model, namely CoroNet, is proposed. It relies on the Xception architecture, which has been pre-trained on the ImageNet dataset and has been fully trained on whole-image BC according to mammograms. The convolutional design method is used in this paper, since it performs better than the other methods. On the prepared dataset, CoroNet was trained and tested. Experiments show that in a four-class classification, it may attain an overall accuracy of 94.92% (benign mass vs. malignant mass) and (benign calcification vs. malignant calcification). CoroNet has a classification accuracy of 88.67% for the two-class cases (calcifications and masses). The paper concluded that there are promising outcomes that could be improved because more training data are available.

A machine learning model based on readers' characteristics to predict their performances in reading screening mammograms

OBJECTIVES

Proposing a machine learning model to predict readers' performances, as measured by the area under the receiver operating characteristics curve (AUC) and lesion sensitivity, using the readers' characteristics.

METHODS

Data were collected from 905 radiologists and breast physicians who completed at least one case-set of 60 mammographic images containing 40 normal and 20 biopsy-proven cancer cases. Nine different case-sets were available. Using a questionnaire, we collected radiologists' demographic details, such as reading volume and years of experience. These characteristics along with a case set difficulty measure were fed into two ensemble of regression trees to predict the readers' AUCs and lesion sensitivities. We calculated the Pearson correlation coefficient between the predicted values by the model and the actual AUC and lesion sensitivity. The usefulness of the model to categorize readers as low and high performers based on different criteria was also evaluated. The performances of the models were evaluated using leave-one-out cross-validation.

RESULTS

The Pearson correlation coefficient between the predicted AUC and actual one was 0.60 (p < 0.001). The model's performance for differentiating the reader in the first and fourth quartile based on the AUC values was 0.86 (95% CI 0.83-0.89). The model reached an AUC of 0.91 (95% CI 0.88-0.93) for distinguishing the readers in the first quartile from the fourth one based on the lesion sensitivity.

CONCLUSION

A machine learning model can be used to categorize readers as high- or low-performing. Such model could be useful for screening programs for designing a targeted quality assurance and optimizing the double reading practice.

Breast cancer detection across dense and non-dense breasts: Markers of diagnostic confidence and efficacy

Background

The impact of radiologists’ characteristics has become a major focus of recent research. However, the markers of diagnostic efficacy and confidence in dense and non-dense breasts are poorly understood.

Purpose

This study aims to assess the relationship between radiologists’ characteristics and diagnostic performance across dense and non-dense breasts.

Materials and methods

Radiologists specialising in breast imaging (n = 128) who had 0.5–40 (13±10.6) years of experience reading mammograms were recruited. Participants independently interpreted a test set containing 60 digital mammograms (40 normal and 20 abnormal) with similarly distributed breast densities. Diagnostic performance measures were analysed via Jamovi software (version 1.6.22).

Results

In dense breasts, breast-imaging fellowship completion significantly improved specificity (p = 0.004), location sensitivity (p = 0.01) and the area under the curve (AUC) of the receiver operating characteristic (p = 0.03). Only participation in BreastScreen reading significantly improved all performance metrics: specificity (p = 0.04), sensitivity (p = 0.005), location sensitivity (p < 0.001) and AUC (p < 0.001). Reading > 100 mammograms weekly significantly improved sensitivity (p = 0.03), location sensitivity (p = 0.001), and AUC (p = 0.03).In non-dense breasts, breast fellowship completion significantly improved sensitivity (p = 0.02), location sensitivity (p = 0.04) and AUC (p = 0.002). Participation in BreastScreen reading and reading > 100 mammograms weekly significantly improved only sensitivity (p = 0.002 and p = 0.003, respectively) and location sensitivity (p < 0.001 and p < 0.001, respectively).

Conclusion

Participating in screening programs, breast fellowships and reading > 100 mammograms weekly are important indicators of the diagnostic performance of radiologists across dense and non-dense breasts. In dense breasts, optimal performance resulted from participation in a breast screening program.

Design and Development of an Intelligent Clinical Decision Support System Applied to the Evaluation of Breast Cancer Risk

Breast cancer is currently one of the main causes of death and tumoral diseases in women. Even if early diagnosis processes have evolved in the last years thanks to the popularization of mammogram tests, nowadays, it is still a challenge to have available reliable diagnosis systems that are exempt of variability in their interpretation. To this end, in this work, the design and development of an intelligent clinical decision support system to be used in the preventive diagnosis of breast cancer is presented, aiming both to improve the accuracy in the evaluation and to reduce its uncertainty. Through the integration of expert systems (based on Mamdani-type fuzzy-logic inference engines) deployed in cascade, exploratory factorial analysis, data augmentation approaches, and classification algorithms such as k-neighbors and bagged trees, the system is able to learn and to interpret the patient’s medical-healthcare data, generating an alert level associated to the danger she has of suffering from cancer. For the system’s initial performance tests, a software implementation of it has been built that was used in the diagnosis of a series of patients contained into a 130-cases database provided by the School of Medicine and Public Health of the University of Wisconsin-Madison, which has been also used to create the knowledge base. The obtained results, characterized as areas under the ROC curves of 0.95–0.97 and high success rates, highlight the huge diagnosis and preventive potential of the developed system, and they allow forecasting, even when a detailed and contrasted validation is still pending, its relevance and applicability within the clinical field.

A review of screening mammography: The benefits and radiation risks put into perspective

INTRODUCTION/BACKGROUND

In medical imaging a benefit to risk analysis is required when justifying or implementing diagnostic procedures. Screening mammography is no exception and in particular concerns around the use of radiation to help diagnose cancer must be addressed.

METHODS

The Medline database and various established reports on breast screening and radiological protection were utilised to explore this review.

RESULTS/DISCUSSION

The benefit of screening is well argued; the ability to detect and treat breast cancer has led to a 91% 5-year survival rate and 497 deaths prevented from breast cancer amongst 100,000 screened women. Subsequently, screening guidelines by various countries recommend annual, biennial or triennial screening from ages somewhere between 40-74 years. Whilst the literature presents different perspectives on screening younger and older women, the current evidence of benefit for screening women <40 and ≥75 years is currently not strong. The radiation dose and associated risk delivered to each woman for a single examination is dependent upon age, breast density and breast thickness, however the average mean glandular dose is around 2.5-3 mGy, and this would result in 65 induced cancers and 8 deaths per 100,000 women over a screening lifetime from 40-74 years. This results in a ratio of lives saved to deaths from induced cancer of 62:1.

CONCLUSION

Therefore, compared to the potential mortality reduction achievable with screening mammography, the risk is small.

A Comprehensive Survey on Deep-Learning-Based Breast Cancer Diagnosis

Breast cancer is now the most frequently diagnosed cancer in women, and its percentage is gradually increasing. Optimistically, there is a good chance of recovery from breast cancer if identified and treated at an early stage. Therefore, several researchers have established deep-learning-based automated methods for their efficiency and accuracy in predicting the growth of cancer cells utilizing medical imaging modalities. As of yet, few review studies on breast cancer diagnosis are available that summarize some existing studies. However, these studies were unable to address emerging architectures and modalities in breast cancer diagnosis. This review focuses on the evolving architectures of deep learning for breast cancer detection. In what follows, this survey presents existing deep-learning-based architectures, analyzes the strengths and limitations of the existing studies, examines the used datasets, and reviews image pre-processing techniques. Furthermore, a concrete review of diverse imaging modalities, performance metrics and results, challenges, and research directions for future researchers is presented.

Classification of Breast Cancer in Mammograms with Deep Learning Adding a Fifth Class

Breast cancer is one of the diseases of most profound concern, with the most prevalence worldwide, where early detections and diagnoses play the leading role against this disease achieved through imaging techniques such as mammography. Radiologists tend to have a high false positive rate for mammography diagnoses and an accuracy of around 82%. Currently, deep learning (DL) techniques have shown promising results in the early detection of breast cancer by generating computer-aided diagnosis (CAD) systems implementing convolutional neural networks (CNNs). This work focuses on applying, evaluating, and comparing the architectures: AlexNet, GoogLeNet, Resnet50, and Vgg19 to classify breast lesions after using transfer learning with fine-tuning and training the CNN with regions extracted from the MIAS and INbreast databases. We analyzed 14 classifiers, involving 4 classes as several researches have done it before, corresponding to benign and malignant microcalcifications and masses, and as our main contribution, we also added a 5th class for the normal tissue of the mammary parenchyma increasing the correct detection; in order to evaluate the architectures with a statistical analysis based on the received operational characteristics (ROC), the area under the curve (AUC), F1 Score, accuracy, precision, sensitivity, and specificity. We generate the best results with the CNN GoogLeNet trained with five classes on a balanced database with an AUC of 99.29%, F1 Score of 91.92%, the accuracy of 91.92%, precision of 92.15%, sensitivity of 91.70%, and specificity of 97.66%, concluding that GoogLeNet is optimal as a classifier in a CAD system to deal with breast cancer.

Transitioning to Practice: Getting up to Speed in Efficiency and Accuracy

The transition from trainee to breast radiologist is challenging. The many new responsibilities that breast radiologists acquire while establishing themselves as clinicians may increase stress and anxiety. Taking inventory of existing knowledge and skills and addressing deficits toward the end of one's training can be beneficial. New breast radiologists should expect to be slower and gain proficiency in the first several years out of training. Having realistic expectations for oneself with respect to screening mammography interpretation and following up on the subsequent diagnostic imaging workup of screening callback examinations can increase competence and confidence. Familiarity with the available literature to guide management in the diagnostic setting can increase efficiency. Planning ahead for localizations and biopsies also allows for efficiency while alleviating anxiety. Ultimately, adapting to a new work environment using a collaborative approach with primary healthcare providers, pathologists, and surgeons while remembering to have mentors within and beyond the field of radiology allows for a more successful transition.

AAWS-Net: Anatomy-aware weakly-supervised learning network for breast mass segmentation

Accurate segmentation of breast masses is an essential step in computer aided diagnosis of breast cancer. The scarcity of annotated training data greatly hinders the model’s generalization ability, especially for the deep learning based methods. However, high-quality image-level annotations are time-consuming and cumbersome in medical image analysis scenarios. In addition, a large amount of weak annotations is under-utilized which comprise common anatomy features. To this end, inspired by teacher-student networks, we propose an Anatomy-Aware Weakly-Supervised learning Network (AAWS-Net) for extracting useful information from mammograms with weak annotations for efficient and accurate breast mass segmentation. Specifically, we adopt a weakly-supervised learning strategy in the Teacher to extract anatomy structure from mammograms with weak annotations by reconstructing the original image. Besides, knowledge distillation is used to suggest morphological differences between benign and malignant masses. Moreover, the prior knowledge learned from the Teacher is introduced to the Student in an end-to-end way, which improves the ability of the student network to locate and segment masses. Experiments on CBIS-DDSM have shown that our method yields promising performance compared with state-of-the-art alternative models for breast mass segmentation in terms of segmentation accuracy and IoU.

The Impact of Radiologist Screening Mammogram Reading Volume on Performance in the Ontario Breast Screening Program

PURPOSE

Although some studies have shown increasing radiologists' mammography volumes improves performance, there is a lack of evidence specific to digital mammography and breast screening program performance targets. This study evaluates the relationship between digital screening volume and meeting performance targets.

METHODS

This retrospective cohort study included 493 radiologists in the Ontario Breast Screening Program who interpreted 1,762,173 screening mammograms in participants ages 50-90 between 2014 and 2016. Associations between annual screening volume and meeting performance targets for abnormal call rate, positive predictive value (PPV), invasive cancer detection rate (CDR), sensitivity, and specificity were modeled using mixed-effects multivariate logistic regression.

RESULTS

Most radiologists read 500-999 (36.7%) or 1,000-1,999 (31.0%) screens annually, and 18.5% read ≥2,000. Radiologists who read ≥2,000 annually were more likely to meet abnormal call rate (OR = 3.85; 95% CI: 1.17-12.61), PPV (OR = 5.36; 95% CI: 2.53-11.34), invasive CDR (OR = 4.14; 95% CI: 1.50-11.46), and specificity (OR = 4.07; 95% CI: 1.89-8.79) targets versus those who read 100-499 screens. Radiologists reading 1,000-1,999 screens annually were more likely to meet PPV (OR = 2.32; 95% CI: 1.22-4.40), invasive CDR (OR = 3.36; 95% CI: 1.49-7.59) and specificity (OR = 2.00; 95% CI: 1.04-3.84) targets versus those who read 100-499 screens. No significant differences were observed for sensitivity.

CONCLUSIONS

Annual reading volume requirements of 1,000 in Canada are supported as screening volume above 1,000 was strongly associated with achieving performance targets for nearly all measures. Increasing the minimum volume to 2,000 may further reduce the potential limitations of screening due to false positives, leading to improvements in overall breast screening program quality.

Radiologist Characteristics Associated with Interpretive Performance of Screening Mammography: A National Mammography Database (NMD) Study

Background Factors affecting radiologists' performance in screening mammography interpretation remain poorly understood. Purpose To identify radiologists characteristics that affect screening mammography interpretation performance. Materials and Methods This retrospective study included 1223 radiologists in the National Mammography Database (NMD) from 2008 to 2019 who could be linked to Centers for Medicare & Medicaid Services (CMS) datasets. NMD screening performance metrics were extracted. Acceptable ranges were defined as follows: recall rate (RR) between 5% and 12%; cancer detection rate (CDR) of at least 2.5 per 1000 screening examinations; positive predictive value of recall (PPV1) between 3% and 8%; positive predictive value of biopsies recommended (PPV2) between 20% and 40%; positive predictive value of biopsies performed (PPV3) between the 25th and 75th percentile of study sample; invasive CDR of at least the 25th percentile of the study sample; and percentage of ductal carcinoma in situ (DCIS) of at least the 25th percentile of the study sample. Radiologist characteristics extracted from CMS datasets included demographics, subspecialization, and clinical practice patterns. Multivariable stepwise logistic regression models were performed to identify characteristics independently associated with acceptable performance for the seven metrics. The most influential characteristics were defined as those independently associated with the majority of the metrics (at least four). Results Relative to radiologists practicing in the Northeast, those in the Midwest were more likely to achieve acceptable RR, PPV1, PPV2, and CDR (odds ratio [OR], 1.4-2.5); those practicing in the West were more likely to achieve acceptable RR, PPV2, and PPV3 (OR, 1.7-2.1) but less likely to achieve acceptable invasive CDR (OR, 0.6). Relative to general radiologists, breast imagers were more likely to achieve acceptable PPV1, invasive CDR, percentage DCIS, and CDR (OR, 1.4-4.4). Those performing diagnostic mammography were more likely to achieve acceptable PPV1, PPV2, PPV3, invasive CDR, and CDR (OR, 1.9-2.9). Those performing breast US were less likely to achieve acceptable PPV1, PPV2, percentage DCIS, and CDR (OR, 0.5-0.7). Conclusion The geographic location of the radiology practice, subspecialization in breast imaging, and performance of diagnostic mammography are associated with better screening mammography performance; performance of breast US is associated with lower performance. ^©RSNA, 2021 Online supplemental material is available for this article.

The influence of breast density and key demographics of radiographers on mammography reporting performance - a pilot study

Introduction

A high demand has been placed on radiologists to perform screen reads due to higher number of women undergoing mammography. This study aims to examine radiographer performance in reporting low compared with high‐mammographic density (MD) images; and to assess the influence of key demographics of Jordanian radiographers on their performance.

Methods

Thirty mammograms with varied MD were reported by 12 radiographers using the Breast Imaging‐Reporting and Data System (BI‐RADS). Radiographer performance was measured using sensitivity, specificity, positive (PPV) and negative predictive values (NPV), and area under the receiver operating characteristic curve (ROC AUC). Performance measures were compared between cases with low‐ and high‐MD and between subgroups of radiographers according to key demographics.

Results

All performance measures were significantly higher in low‐ compared to high‐MD cases (P value < 0.0). The mean sensitivity, specificity, PPV, NPV and ROC AUC were 0.58, 0.68, 0.67, 0.63 and 0.69 respectively. PPV was significantly different for readers who had different years of experience in mammography, hours and cases per week P value = 0.023, 0.01, 0.017 respectively. ROC AUC was significantly different for radiographers with different number of hours and cases performed per week (P value = 0.001 and 0.004 respectively).

Conclusions

The results of this pilot study are encouraging however a more extensive study is required to determine if Jordanian radiographers are capable of successfully taking part in breast screen reading. The lack of skills and knowledge required for correct and consistent reporting of high‐MD images highlights the need for any formal training in mammographic interpretation to focus on the dense breast.

Improving radiologist's ability in identifying particular abnormal lesions on mammograms through training test set with immediate feedback

It has been shown that there are differences in diagnostic accuracy of cancer detection on mammograms, from below 50% in developing countries to over 80% in developed world. One previous study reported that radiologists from a population in Asia displayed a low mammographic cancer detection of 48% compared with over 80% in developed countries, and more importantly, that most lesions missed by these radiologists were spiculated masses or stellate lesions. The aim of this study was to explore the performance of radiologists after undertaking a training test set which had been designed to improve the capability in detecting a specific type of cancers on mammograms. Twenty-five radiologists read two sets of 60 mammograms in a standardized mammogram reading room. The first test set focused on stellate or spiculated masses. When radiologists completed the first set, the system displayed immediate feedback to the readers comparing their performances in each case with the truth of cancer cases and cancer types so that the readers could identify individual-based errors. Later radiologists were asked to read the second set of mammograms which contained different types of cancers including stellate/spiculated masses, asymmetric density, calcification, discrete mass and architectural distortion. Case sensitivity, lesion sensitivity, specificity, receiver operating characteristics (ROC) and Jackknife alternative free-response receiver operating characteristics (JAFROC) were calculated for each participant and their diagnostic accuracy was compared between two sessions. Results showed significant improvement among radiologists in case sensitivity (+ 11.4%; P < 0.05), lesion sensitivity (+ 18.7%; P < 0.01) and JAFROC (+ 11%; P < 0.01) in the second set compared with the first set. The increase in diagnostic accuracy was also recorded in the detection of stellate/spiculated mass (+ 20.6%; P < 0.05). This indicated that the performance of radiologists in detecting malignant lesions on mammograms can be improved if an appropriate training intervention is applied after the readers' weakness and strength are identified.

A review on recent advancements in diagnosis and classification of cancers using artificial intelligence

Artificial intelligence has illustrated drastic changes in radiology and medical imaging techniques which in turn led to tremendous changes in screening patterns. In particular, advancements in these techniques led to the development of computer aided detection (CAD) strategy. These approaches provided highly accurate diagnostic reports which served as a "second-opinion" to the radiologists. However, with significant advancements in artificial intelligence strategy, the diagnostic and classifying capabilities of CAD system are meeting the levels of radiologists and clinicians. Thus, it shifts the CAD system from second opinion approach to a high utility tool. This article reviews the strategies and algorithms developed using artificial intelligence for the foremost cancer diagnosis and classification which overcomes the challenges in the traditional method. In addition, the possible direction of AI in medical aspects is also discussed in this study.

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme

OBJECTIVES

Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance.

METHODS

We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance.

RESULTS

In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted).

CONCLUSION

Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected.

KEY POINTS

• In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

Narrowing the Gap: Imaging Disparities in Radiology

It may seem unlikely that the field of radiology perpetuates disparities in health care, as most radiologists never interact directly with patients, and racial bias is not an obvious factor when interpreting images. However, a closer look reveals that imaging plays an important role in the propagation of disparities. For example, many advanced and resource-intensive imaging modalities, such as MRI and PET/CT, are generally less available in the hospitals frequented by people of color, and when they are available, access is impeded due to longer travel and wait times. Furthermore, their images may be of lower quality, and their interpretations may be more error prone. The aggregate effect of these imaging acquisition and interpretation disparities in conjunction with social factors is insufficiently recognized as part of the wide variation in disease outcomes seen between races in America. Understanding the nature of disparities in radiology is important to effectively deploy the resources and expertise necessary to mitigate disparities through diversity and inclusion efforts, research, and advocacy. In this article, the authors discuss disparities in access to imaging, examine their causes, and propose solutions aimed at addressing these disparities.

Nonpalpable breast lesions: impact of a second-opinion review at a breast unit on BI-RADS classification

OBJECTIVE

To compare BI-RADS classification, management, and outcome of nonpalpable breast lesions assessed both by community practices and by a multidisciplinary tumor board (MTB) at a breast unit.

METHODS

All nonpalpable lesions that were first assigned a BI-RADS score by community practices and then reassessed by an MTB at a single breast unit from 2009 to 2017 were retrospectively reviewed. Inter-review agreement was assessed with Cohen's kappa statistic. Changes in biopsy recommendation were calculated. The percentage of additional tumor lesions detected by the MTB was obtained. The sensitivity, AUC, and cancer rates for BI-RADS category 3, 4, and 5 lesions were computed for both reviews.

RESULTS

A total of 1909 nonpalpable lesions in 1732 patients were included. For BI-RADS scores in the whole cohort, a fair agreement was found (κ = 0.40 [0.36-0.45]) between the two reviews. Agreement was higher when considering only mammography combined with ultrasound (κ = 0.53 [0.44-0.62]), masses (κ = 0.50 [0.44-0.56]), and architectural distortion (κ = 0.44 [0.11-0.78]). Changes in biopsy recommendation occurred in 589 cases (31%). Ninety of 345 additional biopsies revealed high-risk or malignant lesions. Overall, the MTB identified 27% additional high-risk and malignant lesions compared to community practices. The BI-RADS classification AUCs for detecting malignant lesions were 0.66 (0.63-0.69) for community practices and 0.76 (0.75-0.78) for the MTB (p < 0.001).

CONCLUSION

Agreement between community practices and MTB reviews for BI-RADS classification in nonpalpable lesions is only fair. MTB review improves diagnostic performances of breast imaging and patient management.

KEY POINTS

• The inter-review agreement for BI-RADS classification between community practices and the multidisciplinary board was only fair (κ = 0.40). • Disagreements resulted in changes of biopsy recommendation in 31% of the lesions. • The multidisciplinary board identified 27% additional high-risk and malignant lesions compared to community practices.

Understanding the Mammography Audit

Screening mammography aims to identify small, node-negative breast cancers when they are still curable while maintaining an acceptable range of false-positive recalls and biopsies. The mammography audit is a powerful tool to help radiologists understand their performance with respect to that goal. This article defines audit terms and describes how to use collected and derived data to perform a mammography audit. Accepted benchmarks are discussed as well as their applicability to radiologists and breast imaging practices in the United States. Special considerations regarding volumes and radiologist characteristics are explored, because these factors may affect audit results.

Reader characteristics and mammogram features associated with breast imaging reporting scores

Objectives:

This study aims to explore the reading performances of radiologists in detecting cancers on mammograms using Tabar Breast Imaging Reporting and Data System (BIRADS) classification and identify factors related to breast imaging reporting scores.

Methods:

117 readings of five different mammogram test sets with each set containing 20 cancer and 40 normal cases were performed by Australian radiologists. Each radiologist evaluated the mammograms using the BIRADS lexicon with category 1 - negative, category 2 - benign findings, category 3 - equivocal findings (Recall), category 4 - suspicious findings (Recall), and category 5 - highly suggestive of malignant findings (Recall). Performance metrics (true positive, false positive, true negative, and false negative) were calculated for each radiologist and the distribution of reporting categories was analyzed in reader-based and case-based groups. The association of reader characteristics and case features among categories was examined using Mann-Whitney U and Kruskal-Wallis tests.

Results:

38% of cancer-containing mammograms were reported with category 3 which decreased to 32.3% with category 4 and 16.2% with category 5 while 16.6 and 10.3% of cancer cases were marked with categories 1 and 2. Female readers had less false-negative rates when using categories 1 and 2 for cancer cases than male readers (p < 0.01). A similar pattern as gender category was also found in Breast Screen readers and readers completed breast reading fellowships compared with non-Breast Screen and non-fellowship readers (p < 0.05). Radiologists with low number of cases read per week were more likely to record the cancer cases with category 4 while the ones with high number of cases were with category 3 (p < 0.01). Discrete mass and asymmetric density were the two types of abnormalities reported mostly as equivocal findings with category 3 (47–50%; p = 0.005) while spiculated mass or stellate lesions were mostly selected as highly suggestive of malignancy with category 5 (26%, p = 0.001).

Conclusions:

Most radiologists used category 3 when reporting cancer mammograms. Gender, working for BreastScreen, fellowship completion, and number of cases read per week were factors associated with scoring selection. Radiologists reported higher Tabar BIRADS category for specific types of abnormalities on mammograms than others.

Advances in knowledge:

The study identified factors associated with the decision of radiologists in assigning a BIRADS Tabar score for mammograms with abnormality. These findings will be useful for individual training programs to improve the confidence of radiologists in recognizing abnormal lesions on screening mammograms.

External Evaluation of 3 Commercial Artificial Intelligence Algorithms for Independent Assessment of Screening Mammograms

Importance

A computer algorithm that performs at or above the level of radiologists in mammography screening assessment could improve the effectiveness of breast cancer screening.

Objective

To perform an external evaluation of 3 commercially available artificial intelligence (AI) computer-aided detection algorithms as independent mammography readers and to assess the screening performance when combined with radiologists.

Design, Setting, and Participants

This retrospective case-control study was based on a double-reader population-based mammography screening cohort of women screened at an academic hospital in Stockholm, Sweden, from 2008 to 2015. The study included 8805 women aged 40 to 74 years who underwent mammography screening and who did not have implants or prior breast cancer. The study sample included 739 women who were diagnosed as having breast cancer (positive) and a random sample of 8066 healthy controls (negative for breast cancer).

Main Outcomes and Measures

Positive follow-up findings were determined by pathology-verified diagnosis at screening or within 12 months thereafter. Negative follow-up findings were determined by a 2-year cancer-free follow-up. Three AI computer-aided detection algorithms (AI-1, AI-2, and AI-3), sourced from different vendors, yielded a continuous score for the suspicion of cancer in each mammography examination. For a decision of normal or abnormal, the cut point was defined by the mean specificity of the first-reader radiologists (96.6%).

Results

The median age of study participants was 60 years (interquartile range, 50-66 years) for 739 women who received a diagnosis of breast cancer and 54 years (interquartile range, 47-63 years) for 8066 healthy controls. The cases positive for cancer comprised 618 (84%) screen detected and 121 (16%) clinically detected within 12 months of the screening examination. The area under the receiver operating curve for cancer detection was 0.956 (95% CI, 0.948-0.965) for AI-1, 0.922 (95% CI, 0.910-0.934) for AI-2, and 0.920 (95% CI, 0.909-0.931) for AI-3. At the specificity of the radiologists, the sensitivities were 81.9% for AI-1, 67.0% for AI-2, 67.4% for AI-3, 77.4% for first-reader radiologist, and 80.1% for second-reader radiologist. Combining AI-1 with first-reader radiologists achieved 88.6% sensitivity at 93.0% specificity (abnormal defined by either of the 2 making an abnormal assessment). No other examined combination of AI algorithms and radiologists surpassed this sensitivity level.

Conclusions and Relevance

To our knowledge, this study is the first independent evaluation of several AI computer-aided detection algorithms for screening mammography. The results of this study indicated that a commercially available AI computer-aided detection algorithm can assess screening mammograms with a sufficient diagnostic performance to be further evaluated as an independent reader in prospective clinical trials. Combining the first readers with the best algorithm identified more cases positive for cancer than combining the first readers with second readers.

Interpretative characteristics and case features associated with the performances of radiologists in reading mammograms: A study from a non-screening population in Asia

AIMS

To explore radiologist characteristics and case features associated with diagnostic performances in cancer detection on mammograms in a South East Asian population.

METHODS

Fifty-three radiologists reported 60 mammographic examinations which consisted of 40 normal and 20 cancer-containing cases at the BREAST workshops. Radiologists were asked to examine each mammogram using the BIRADS on diagnostic monitors. Differences in reader characteristics and case features between correct and incorrect decisions were assessed separately for cancer and normal cases. Univariate and multivariate logistic regressions were applied to generate odds ratios (OR) for significant factors related to correct decisions.

RESULTS

Radiologists who spent ≥10 hours/week reporting mammograms had a higher possibility of detecting cancer lesions (OR = 1.6; P = 0.01). A higher rate of accuracy in reporting negative cases was associated with female radiologists (OR = 1.4; P = 0.002), radiologists who read ≤20 mammograms per week (OR = 1.5; P < 0.0001), had completed training course (OR = 1.7; P < 0.0001) or wore eyeglasses (OR = 1.4; P = 0.01). Cancer cases with breast density >50% (OR = 2.1; P < 0.0001), having abnormal lesions ≥9 mm (OR = 1.8; P < 0.0001), or displaying calcifications, a discrete mass or nonspecific density (OR = 1.6; P < 0.0001) were recorded with a higher detection rate by radiologists than other cases. Lesions located on the right breasts (OR = 1.8; P < 0.0001) or found in the lower inner, upper outer or mixed locations (OR = 2.7; P < 0.0001) were also recorded with a better diagnostic possibility compared with other lesions.

CONCLUSION

This work identified key features related to diagnostic accuracy of breast cancer on mammograms in a nonscreening population, which is helpful for developing appropriate strategies to improve breast cancer detectability of radiologists.

Reading High Breast Density Mammograms: Differences in Diagnostic Performance between Radiologists from Hong Kong SAR/Guangdong Province in China and Australia

Background:

Variations in the performance of radiologists reading mammographic images are well reported, but key parameters explaining such variations in different countries are not fully explored. The main aim of this study is to investigate performances of Chinese (Hong Kong SAR and Guangdong Province) and Australian radiologists in interpreting dense breast mammographic images.

Methods:

A test set, contained 60 mammographic examinations with high breast density, was used to assess radiologists’ performance. Twelve Chinese and thirteen Australian radiologists read all the cases independently and were asked to identify all lesions and provide a grade from 1 to 5 to each lesion. Case sensitivity, specificity, lesion sensitivity, AUC and JAFROC were used to assess radiologists’ performances. Demographic information and reading experience were also collected from the readers. Performance scores were compared between the two populations and the relationships between performance scores and their reading experience were discovered.

Results:

For radiologists who were less than 40-year-old, lesion sensitivity, AUC and JAFROC were significantly lower in Chinese radiologists than those in Australian (52.10% vs 71.45%, p=0.043; 0.76 vs 0.84, p=0.031; 0.59 vs 0.72, p=0.045; respectively). Australian radiologists with less than 10 years of reading experience had higher AUC and JAFROC scores compared with their Chinese counterparts (0.83 vs 0.76, p=0.039; 0.70 vs 0.56, p=0.020, respectively).

Conclusions:

We found that younger Australian radiologists performed better at reading dense breast cases which is likely to be linked to intensive fellowship training, immersion in a screening program and exposure to the benefits of a performance-measuring education tool.

Range of Radiologist Performance in a Population-based Screening Cohort of 1 Million Digital Mammography Examinations

Background There is great interest in developing artificial intelligence (AI)-based computer-aided detection (CAD) systems for use in screening mammography. Comparative performance benchmarks from true screening cohorts are needed. Purpose To determine the range of human first-reader performance measures within a population-based screening cohort of 1 million screening mammograms to gauge the performance of emerging AI CAD systems. Materials and Methods This retrospective study consisted of all screening mammograms in women aged 40-74 years in Stockholm County, Sweden, who underwent screening with full-field digital mammography between 2008 and 2015. There were 110 interpreting radiologists, of whom 24 were defined as high-volume readers (ie, those who interpreted more than 5000 annual screening mammograms). A true-positive finding was defined as the presence of a pathology-confirmed cancer within 12 months. Performance benchmarks included sensitivity and specificity, examined per quartile of radiologists' performance. First-reader sensitivity was determined for each tumor subgroup, overall and by quartile of high-volume reader sensitivity. Screening outcomes were examined based on the first reader's sensitivity quartile with 10 000 screening mammograms per quartile. Linear regression models were fitted to test for a linear trend across quartiles of performance. Results A total of 418 041 women (mean age, 54 years ± 10 [standard deviation]) were included, and 1 186 045 digital mammograms were evaluated, with 972 899 assessed by high-volume readers. Overall sensitivity was 73% (95% confidence interval [CI]: 69%, 77%), and overall specificity was 96% (95% CI: 95%, 97%). The mean values per quartile of high-volume reader performance ranged from 63% to 84% for sensitivity and from 95% to 98% for specificity. The sensitivity difference was very large for basal cancers, with the least sensitive and most sensitive high-volume readers detecting 53% and 89% of cancers, respectively (P < .001). Conclusion Benchmarks showed a wide range of performance differences between high-volume readers. Sensitivity varied by tumor characteristics. © RSNA, 2020 Online supplemental material is available for this article.