Global processing provides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection

AI for interpreting screening mammograms: implications for missed cancer in double reading practices and challenging-to-locate lesions

Although the value of adding AI as a surrogate second reader in various scenarios has been investigated, it is unknown whether implementing an AI tool within double reading practice would capture additional subtle cancers missed by both radiologists who independently assessed the mammograms. This paper assesses the effectiveness of two state-of-the-art Artificial Intelligence (AI) models in detecting retrospectively-identified missed cancers within a screening program employing double reading practices. The study also explores the agreement between AI and radiologists in locating the lesions, considering various levels of concordance among the radiologists in locating the lesions. The Globally-aware Multiple Instance Classifier (GMIC) and Global-Local Activation Maps (GLAM) models were fine-tuned for our dataset. We evaluated the sensitivity of both models on missed cancers retrospectively identified by a panel of three radiologists who reviewed prior examinations of 729 cancer cases detected in a screening program with double reading practice. Two of these experts annotated the lesions, and based on their concordance levels, cases were categorized as 'almost perfect,' 'substantial,' 'moderate,' and 'poor.' We employed Similarity or Histogram Intersection (SIM) and Kullback-Leibler Divergence (KLD) metrics to compare saliency maps of malignant cases from the AI model with annotations from radiologists in each category. In total, 24.82% of cancers were labeled as "missed." The performance of GMIC and GLAM on the missed cancer cases was 82.98% and 79.79%, respectively, while for the true screen-detected cancers, the performances were 89.54% and 87.25%, respectively (p-values for the difference in sensitivity < 0.05). As anticipated, SIM and KLD from saliency maps were best in 'almost perfect,' followed by 'substantial,' 'moderate,' and 'poor.' Both GMIC and GLAM (p-values < 0.05) exhibited greater sensitivity at higher concordance. Even in a screening program with independent double reading, adding AI could potentially identify missed cancers. However, the challenging-to-locate lesions for radiologists impose a similar challenge for AI.

Computer-extracted global radiomic features can predict the radiologists' first impression about the abnormality of a screening mammogram

OBJECTIVE

Radiologists can detect the gist of abnormal based on their rapid initial impression on a mammogram (ie, global gist signal [GGS]). This study explores (1) whether global radiomic (ie, computer-extracted) features can predict the GGS; and if so, (ii) what features are the most important drivers of the signals.

METHODS

The GGS of cases in two extreme conditions was considered: when observers detect a very strong gist (high-gist) and when the gist of abnormal was not/poorly perceived (low-gist). Gist signals/scores from 13 observers reading 4191 craniocaudal mammograms were collected. As gist is a noisy signal, the gist scores from all observers were averaged and assigned to each image. The high-gist and low-gist categories contained all images in the fourth and first quartiles, respectively. One hundred thirty handcrafted global radiomic features (GRFs) per mammogram were extracted and utilized to construct eight separate machine learning random forest classifiers (All, Normal, Cancer, Prior-1, Prior-2, Missed, Prior-Visible, and Prior-Invisible) for characterizing high-gist from low-gist images. The models were trained and validated using the 10-fold cross-validation approach. The models' performances were evaluated by the area under receiver operating characteristic curve (AUC). Important features for each model were identified through a scree test.

RESULTS

The Prior-Visible model achieved the highest AUC of 0.84 followed by the Prior-Invisible (0.83), Normal (0.82), Prior-1 (0.81), All (0.79), Prior-2 (0.77), Missed (0.75), and Cancer model (0.69). Cluster shade, standard deviation, skewness, kurtosis, and range were identified to be the most important features.

CONCLUSIONS

Our findings suggest that GRFs can accurately classify high- from low-gist images.

ADVANCES IN KNOWLEDGE

Global mammographic radiomic features can accurately predict high- from low-gist images with five features identified to be valuable in describing high-gist images. These are critical in providing better understanding of the mammographic image characteristics that drive the strength of the GGSs which could be exploited to advance breast cancer (BC) screening and risk prediction, enabling early detection and treatment of BC thereby further reducing BC-related deaths.

Global Radiomic Features from Mammography for Predicting Difficult-To-Interpret Normal Cases

This work aimed to investigate whether global radiomic features (GRFs) from mammograms can predict difficult-to-interpret normal cases (NCs). Assessments from 537 readers interpreting 239 normal mammograms were used to categorise cases as 120 difficult-to-interpret and 119 easy-to-interpret based on cases having the highest and lowest difficulty scores, respectively. Using lattice- and squared-based approaches, 34 handcrafted GRFs per image were extracted and normalised. Three classifiers were constructed: (i) CC and (ii) MLO using the GRFs from corresponding craniocaudal and mediolateral oblique images only, based on the random forest technique for distinguishing difficult- from easy-to-interpret NCs, and (iii) CC + MLO using the median predictive scores from both CC and MLO models. Useful GRFs for the CC and MLO models were recognised using a scree test. The CC and MLO models were trained and validated using the leave-one-out-cross-validation. The models' performances were assessed by the AUC and compared using the DeLong test. A Kruskal-Wallis test was used to examine if the 34 GRFs differed between difficult- and easy-to-interpret NCs and if difficulty level based on the traditional breast density (BD) categories differed among 115 low-BD and 124 high-BD NCs. The CC + MLO model achieved higher performance (0.71 AUC) than the individual CC and MLO model alone (0.66 each), but statistically non-significant difference was found (all p > 0.05). Six GRFs were identified to be valuable in describing difficult-to-interpret NCs. Twenty features, when compared between difficult- and easy-to-interpret NCs, differed significantly (p < 0.05). No statistically significant difference was observed in difficulty between low- and high-BD NCs (p = 0.709). GRF mammographic analysis can predict difficult-to-interpret NCs.

Using Radiomics-Based Machine Learning to Create Targeted Test Sets to Improve Specific Mammography Reader Cohort Performance: A Feasibility Study

Mammography interpretation is challenging with high error rates. This study aims to reduce the errors in mammography reading by mapping diagnostic errors against global mammographic characteristics using a radiomics-based machine learning approach. A total of 36 radiologists from cohort A (n = 20) and cohort B (n = 16) read 60 high-density mammographic cases. Radiomic features were extracted from three regions of interest (ROIs), and random forest models were trained to predict diagnostic errors for each cohort. Performance was evaluated using sensitivity, specificity, accuracy, and AUC. The impact of ROI placement and normalization on prediction was investigated. Our approach successfully predicted both the false positive and false negative errors of both cohorts but did not consistently predict location errors. The errors produced by radiologists from cohort B were less predictable compared to those in cohort A. The performance of the models did not show significant improvement after feature normalization, despite the mammograms being produced by different vendors. Our novel radiomics-based machine learning pipeline focusing on global radiomic features could predict false positive and false negative errors. The proposed method can be used to develop group-tailored mammographic educational strategies to help improve future mammography reader performance.

Reliability of radiologists' first impression when interpreting a screening mammogram

Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists' initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen's Kappa was 0.478 (interquartile range = 0.419-0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen's Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26-0.37). The Fleiss Kappa score of 0.106 (CI = 0.105-0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists' initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search.

Multiple expressions of "expert" abnormality gist in novices following perceptual learning

With a brief half-second presentation, a medical expert can determine at above chance levels whether a medical scan she sees is abnormal based on a first impression arising from an initial global image process, termed "gist." The nature of gist processing is debated but this debate stems from results in medical experts who have years of perceptual experience. The aim of the present study was to determine if gist processing for medical images occurs in naïve (non-medically trained) participants who received a brief perceptual training and to tease apart the nature of that gist signal. We trained 20 naïve participants on a brief perceptual-adaptive training of histology images. After training, naïve observers were able to obtain abnormality detection and abnormality categorization above chance, from a brief 500 ms masked presentation of a histology image, hence showing "gist." The global signal demonstrated in perceptually trained naïve participants demonstrated multiple dissociable components, with some of these components relating to how rapidly naïve participants learned a normal template during perceptual learning. We suggest that multiple gist signals are present when experts view medical images derived from the tens of thousands of images that they are exposed to throughout their training and careers. We also suggest that a directed learning of a normal template may produce better abnormality detection and identification in radiologists and pathologists.

Using global feedback to induce learning of gist of abnormality in mammograms

Extraction of global structural regularities provides general 'gist' of our everyday visual environment as it does the gist of abnormality for medical experts reviewing medical images. We investigated whether naïve observers could learn this gist of medical abnormality. Fifteen participants completed nine adaptive training sessions viewing four categories of unilateral mammograms: normal, obvious-abnormal, subtle-abnormal, and global signals of abnormality (mammograms with no visible lesions but from breasts contralateral to or years prior to the development of cancer) and receiving only categorical feedback. Performance was tested pre-training, post-training, and after a week's retention on 200 mammograms viewed for 500 ms without feedback. Performance measured as d' was modulated by mammogram category, with the highest performance for mammograms with visible lesions. Post-training, twelve observed showed increased d' for all mammogram categories but a subset of nine, labelled learners also showed a positive correlation of d' across training. Critically, learners learned to detect abnormality in mammograms with only the global signals, but improvements were poorly retained. A state-of-the-art breast cancer classifier detected mammograms with lesions but struggled to detect cancer in mammograms with the global signal of abnormality. The gist of abnormality can be learned through perceptual/incidental learning in mammograms both with and without visible lesions, subject to individual differences. Poor retention suggests perceptual tuning to gist needs maintenance, converging with findings that radiologists' gist performance correlates with the number of cases reviewed per year, not years of experience. The human visual system can tune itself to complex global signals not easily captured by current deep neural networks.

Spotting lesions in thorax X-rays at a glance: holistic processing in radiology

Radiologists often need only a glance to grasp the essence of complex medical images. Here, we use paradigms and manipulations from perceptual learning and expertise fields to elicit mechanisms and limits of holistic processing in radiological expertise. In the first experiment, radiologists were significantly better at categorizing thorax X-rays when they were presented for 200 ms in an upright orientation than when they were presented upside-down. Medical students, in contrast, were guessing in both situations. When the presentation time was increased to 500 ms, allowing for a couple more glances, the radiologists improved their performance on the upright stimuli, but remained at the same level on the inverted presentation. The second experiment circumvented the holistic processing by immediately cueing a tissue within the X-rays, which may or may not contain a nodule. Radiologists were again better than medical students at recognizing whether the cued tissue was a nodule, but this time neither the inverted presentation nor additional time affected their performance. Our study demonstrates that holistic processing is most likely a continuous recurring process which is just as susceptible to the inversion effect as in other expertise domains. More importantly, our study also indicates that holistic-like processing readily occurs in complex stimuli (e.g., whole thorax X-rays) but is more difficult to find in uniform single parts of such stimuli (e.g., nodules).