From "satisfaction of search" to "subsequent search misses": a review of multiple-target search errors across radiology and cognitive science

Three scans are better than two for follow-up: An automatic method for finding missed and misidentified lesions in cross-sectional follow-up of oncology patients

PURPOSE

Missed and misidentified neoplastic lesions in longitudinal studies of oncology patients are pervasive and may affect the evaluation of the disease status. Two newly identified patterns of lesion changes, lone lesions and non-consecutive lesion changes, may help radiologists to detect these lesions. This study evaluated a new interpretation revision workflow of lesion annotations in three or more consecutive scans based on these suspicious patterns.

METHODS

The interpretation revision workflow was evaluated on manual and computed lesion annotations in longitudinal oncology patient studies. For the manual revision, a senior radiologist and a senior neurosurgeon (the readers) manually annotated the lesions in each scan and later revised their annotations to identify missed and misidentified lesions with the workflow using the automatically detected patterns. For the computerized revision, lesion annotations were first computed with a previously trained nnU-Net and were then automatically revised with an AI-based method that automates the workflow readers' decisions. The evaluation included 67 patient studies with 2295 metastatic lesions in lung (19 patients, 83 CT scans, 1178 lesions), liver (18 patients, 77 CECT scans, 800 lesions) and brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions).

RESULTS

Revision of the manual lesion annotations revealed 120 missed lesions and 20 misidentified lesions in 31 out of 67 (46%) studies. The automatic revision reduced the number of computed missed lesions by 55 and computed misidentified lesions by 164 in 51 out of 67 (76%) studies.

CONCLUSION

Automatic analysis of three or more consecutive volumetric scans helps find missed and misidentified lesions and may improve the evaluation of temporal changes of oncological lesions.

Inattentional blindness in medicine

People often fail to notice unexpected stimuli when their attention is directed elsewhere. Most studies of this "inattentional blindness" have been conducted using laboratory tasks with little connection to real-world performance. Medical case reports document examples of missed findings in radiographs and CT images, unintentionally retained guidewires following surgery, and additional conditions being overlooked after making initial diagnoses. These cases suggest that inattentional blindness might contribute to medical errors, but relatively few studies have directly examined inattentional blindness in realistic medical contexts. We review the existing literature, much of which focuses on the use of augmented reality aids or inspection of medical images. Although these studies suggest a role for inattentional blindness in errors, most of the studies do not provide clear evidence that these errors result from inattentional blindness as opposed to other mechanisms. We discuss the design, analysis, and reporting practices that can make the contributions of inattentional blindness unclear, and we describe guidelines for future research in medicine and similar contexts that could provide clearer evidence for the role of inattentional blindness.

Clinical history and incidental abnormality detection in endodontic cone beam computed tomography

Purpose

To assess the effect of clinical history on incidental abnormality detection, false positive (FP) marks, and diagnostic confidence in endodontic cone beam computed tomography (CBCT) imaging.

Approach

A reader performance study using a free-response, factorial study design was undertaken, which accounted for changes in the independent variables: native case type, native case severity, reader type, and reading modality. Twenty-three readers interpreted 26 cases (18 diseased and 8 non-diseased) twice, once with and once without access to clinical history. Each case had at least one incidental abnormality that was not a native endodontic finding. Lesion localization (LL), non-localizations (FPs), and diagnostic confidence (rating 2, 3, or 4: lowest, middle, and highest, respectively) of incidental abnormalities were analyzed.

Results

Clinical history increased the detection of incidental abnormalities in non-diseased subtle cases (76 versus 59, p = 0.04 ). Reader experience and monthly CBCT reading volume did not affect incidental abnormality detection. FPs were neither affected by clinical history nor reader characteristics. The highest confidence rating was most often used in each case type when clinical history was available. For this rating, history had significantly greater LLs in subtle diseased (53 versus 41, p = 0.03 ) and non-diseased images (53 versus 33, p = 0.02 ).

Conclusions

Clinical history improved the detection of incidental endodontic abnormalities in non-diseased subtle CBCT images and did not affect the number of FP marks. Reader confidence in correctly identified abnormalities was higher with clinical history when disease and non-disease were subtle but was not associated with an improvement in diagnostic performance.

Assessing satisfaction of search in virtual mammograms for experienced and novice searchers

Purpose

Satisfaction of search (SOS) is a phenomenon where searchers are more likely to miss a lesion/target after detecting a first lesion/target. Here, we investigated SOS for masses and calcifications in virtual mammograms with experienced and novice searchers to determine the extent to which: (1) SOS affects breast lesion detection, (2) similarity between lesions impacts detection, and (3) experience impacts SOS rates.

Approach

The open virtual clinical trials framework was used to simulate the breast anatomy of patients, and up to two simulated masses and/or single-calcifications were inserted into the breast models. Experienced searchers (residents, fellows, and radiologists with breast imaging experience) and novice searchers (undergraduates who had no breast imaging experience) were instructed to search for up to two lesions (masses and calcifications) per image.

Results

2 × 2 mixed factors analysis of variances (ANOVAs) were run with: (1) single versus second lesion hit rates, (2) similar versus dissimilar second-lesion hit rates, and (3) similar versus dissimilar second-lesion response times as within-subject factors and experience as the between subject's factor. The ANOVAs demonstrated that: (1) experienced and novice searchers made a significant amount of SOS errors, (2) similarity had little impact on experienced searchers, but novice searchers were more likely to miss a dissimilar second lesion compared to when it was similar to a detected first lesion, (3) experienced and novice searchers were faster at finding similar compared to dissimilar second lesions.

Conclusions

We demonstrated that SOS is a significant cause of lesion misses in virtual mammograms and that reader experience impacts detection rates for similar compared to dissimilar abnormalities. These results suggest that experience may impact strategy and/or recognition with theoretical implications for determining why SOS occurs.

Target detection and discrimination in pop-out visual search with two targets

To successfully interact with objects in complex and crowded environments, we often perform visual search to detect or identify a relevant target (or targets) among distractors. Previous studies have reported a redundancy gain when two targets instead of one are presented in a simple target detection task. However, research is scant about the role of multiple targets in target discrimination tasks, especially in the context of visual search. Here, we address this question and investigate its underlying mechanisms in a pop-out search paradigm. In Experiment 1, we directly compared visual search performance for one or two targets for detection or discrimination tasks. We found that two targets led to a redundancy gain for detection, whereas it led to a redundancy cost for discrimination. To understand the basis for the redundancy cost observed in discrimination tasks for multiple targets, we further investigated the role of perceptual grouping (Experiment 2) and stimulus-response feature compatibility (Experiment 3). We determined that the strength of perceptual grouping among homogenous distractors was attenuated when two targets were present compared with one. We also found that response compatibility between two targets contributed more to the redundancy cost compared with perceptual compatibility. Taken together, our results show how pop-out search involving two targets is modulated by the level of feature processing, perceptual grouping, and compatibility of perceptual and response features.