A taxonomy of errors in multiple-target visual search

In simple but challenging search tasks, most errors are stochastic

In visual search tasks in the lab and in the real world, people routinely miss targets that are clearly visible: so-called look but fail to see (LBFTS) errors. If search displays are shown to the same observer twice, we can ask about the probability of joint errors, where the target is missed both times. If errors are "deterministic," then the probability of a second error on the same display-given that the target was missed the first time-should be high. If errors are "stochastic," the probability of joint errors should be the product of the error rate for first and second appearances. Here, we report on two versions of a T among Ls search with somewhat degraded letters to make search more difficult. In Experiment 1, Ts could either appear amidst crowded "clumps" of Ls or more in isolation. Observers made more errors when the T was in a clump, but these errors were mainly stochastic. In Experiment 2, the task was made harder by making Ts and Ls more similar. Again, errors were predominantly stochastic. If other, socially important errors are also stochastic, this would suggest that "double reading," where two observers (human or otherwise) look at each stimulus, could reduce overall error rates.

A review and perspective on the neural basis of radiological expertise

Radiological expertise requires tremendous time, effort, and training. While there has been a myriad of studies focusing on radiological expertise and error, the precise underlying neural mechanism still remains largely unexplored. In this article, we review potential neural mechanisms, namely, the fusiform face area, working memory, and predictive coding and propose experiments to test the predictive coding framework.

Don't hide the instruction manual: A dynamic trade-off between using internal and external templates during visual search

Visual search is typically studied by requiring participants to memorize a template initially, for which they subsequently search in a crowded display. Search in daily life, however, often involves templates that remain accessible externally, and may therefore be (re)attended for just-in-time encoding or to refresh internal template representations. Here, we show that participants indeed use external templates during search when given the chance. This behavior was observed during both simple and complex search, scaled with task difficulty, and was associated with improved performance. Furthermore, we show that participants used external sampling not only to offload memory, but also as a means of verifying whether the template was remembered correctly at the end of trials. We conclude that the external world may not only provide the challenge (e.g., distractors), but may dynamically ease search. These results argue for extensions of state-of-the-art models of search, because external sampling seems to be used frequently, in at least two ways and is actually beneficial for task performance. Our findings support a model of visual working memory that emphasizes a resource-efficient trade-off between storing and (re)attending external information.

Assessing satisfaction of search in virtual mammograms for experienced and novice searchers

Purpose

Satisfaction of search (SOS) is a phenomenon where searchers are more likely to miss a lesion/target after detecting a first lesion/target. Here, we investigated SOS for masses and calcifications in virtual mammograms with experienced and novice searchers to determine the extent to which: (1) SOS affects breast lesion detection, (2) similarity between lesions impacts detection, and (3) experience impacts SOS rates.

Approach

The open virtual clinical trials framework was used to simulate the breast anatomy of patients, and up to two simulated masses and/or single-calcifications were inserted into the breast models. Experienced searchers (residents, fellows, and radiologists with breast imaging experience) and novice searchers (undergraduates who had no breast imaging experience) were instructed to search for up to two lesions (masses and calcifications) per image.

Results

2 × 2 mixed factors analysis of variances (ANOVAs) were run with: (1) single versus second lesion hit rates, (2) similar versus dissimilar second-lesion hit rates, and (3) similar versus dissimilar second-lesion response times as within-subject factors and experience as the between subject's factor. The ANOVAs demonstrated that: (1) experienced and novice searchers made a significant amount of SOS errors, (2) similarity had little impact on experienced searchers, but novice searchers were more likely to miss a dissimilar second lesion compared to when it was similar to a detected first lesion, (3) experienced and novice searchers were faster at finding similar compared to dissimilar second lesions.

Conclusions

We demonstrated that SOS is a significant cause of lesion misses in virtual mammograms and that reader experience impacts detection rates for similar compared to dissimilar abnormalities. These results suggest that experience may impact strategy and/or recognition with theoretical implications for determining why SOS occurs.

Visual search as an embodied process: The effects of perspective change and external reference on search performance

Traditional visual search tasks in the laboratories typically involve looking for targets in 2D displays with exemplar views of objects. In real life, visual search commonly entails 3D objects in 3D spaces with nonperpendicular viewing and relative motions between observers and search array items, both of which lead to transformations of objects' projected images in lawful but unpredicted ways. Furthermore, observers often do not have to memorize a target before searching, but may refer to it while searching, for example, holding a picture of someone while looking for them from a crowd. Extending the traditional visual search task, in this study, we investigated the effects of image transformation as a result of perspective change yielded by discrete viewing angle change (Experiment 1) or continuous rotation of the search array (Experiment 2) and of having external references on visual search performance. Results showed that when searching from 3D objects with a non-zero viewing angle, performance was similar to searching from 2D exemplar views of objects; when searching for 3D targets from rotating arrays in virtual reality, performance was similar to searching from stationary arrays. In general, discrete or continuous perspective change did not affect the search outcomes in terms of accuracy, response time, and self-rated confidence, or the search process in terms of eye movement patterns. Therefore, visual search does not require the exact match of retinal images. Additionally, being able to see the target during the search improved search accuracy and observers' confidence. It increased search time because, as revealed by the eye movements, observers actively checked back on the reference target. Thus, visual search is an embodied process that involves real-time information exchange between the observers and the environment.

Satisfaction of Search Can Be Ameliorated by Perceptual Learning: A Proof-of-Principle Study

When searching a visual image that contains multiple target objects of interest, human subjects often show a satisfaction of search (SOS) effect, whereby if the subjects find one target, they are less likely to find additional targets in the image. Reducing SOS or, equivalently, subsequent search miss (SSM), is of great significance in many real-world situations where it is of paramount importance to find all targets in a given image, not just one. However, studies have shown that even highly trained and experienced subjects, such as expert radiologists, are subject to SOS. Here, using the detection of camouflaged objects (or camouflage-breaking) as an illustrative case, we demonstrate that when naïve subjects are trained to detect camouflaged objects more effectively, it has the side effect of reducing subjects’ SOS. We tested subjects in the SOS task before and after they were trained in camouflage-breaking. During SOS testing, subjects viewed naturalistic scenes that contained zero, one, or two targets, depending on the image. As expected, before camouflage-training, subjects showed a strong SOS effect, whereby if they had found a target with relatively high visual saliency in a given image, they were less likely to have also found a lower-saliency target when one existed in the image. Subjects were then trained in the camouflage-breaking task to criterion using non-SOS images, i.e., camouflage images that contained zero or one target. Surprisingly, the trained subjects no longer showed significant levels of SOS. This reduction was specific to the particular background texture in which the subjects received camouflage training; subjects continued to show significant SOS when tested using a different background texture in which they did not receive camouflage training. A separate experiment showed that the reduction in SOS was not attributable to non-specific exposure or practice effects. Together, our results demonstrate that perceptual expertise can, in principle, reduce SOS, even when the perceptual training does not specifically target SOS reduction.

Are all real-world objects created equal? Estimating the "set-size" of the search target in visual working memory

Are all real-world objects created equal? Visual search difficulty increases with the number of targets and as target-related visual working memory (VWM) load increases. Our goal was to investigate the load imposed by individual real-world objects held in VWM in the context of search. Measures of visual clutter attempt to quantify real-world set-size in the context of scenes. We applied one of these measures, the number of proto-objects, to individual real-world objects and used contralateral delay activity (CDA) to measure the resulting VWM load. The current study presented a real-world object as a target cue, followed by a delay where CDA was measured. This was followed by a four-object search array. We compared CDA and later search performance from target cues containing a high or low number of proto-objects. High proto-object target cues resulted in greater CDA, longer search RTs, target dwell times, and reduced search guidance, relative to low proto-object targets. These findings demonstrate that targets with more proto-objects result in a higher VWM load and reduced search performance. This shows that the number of proto-objects contained within individual objects produce set-size like effects in VWM and suggests proto-objects may be a viable unit of measure of real-world VWM load. Importantly, this demonstrates that not all real-world objects are created equal.

Salience matters: Distractors may, or may not, speed target-absent searches

Attention is often captured by irrelevant but salient changes in the environment, and usually results in slowed search speeds and increased errors during a typical visual search task. Nonetheless, a recent study conducted by Moher (2020, Psychological Science, 31[1], 31-42) found that the effect of a highly salient distractor on visual search depended on whether or not a target was also present in the display. While the distractor slowed search and increased errors for target-present trials, it speeded search for target-absent trials. Here, we aimed to replicate this finding and explore a potential boundary condition to the effect by manipulating the overall salience of the distractor. We did this by changing the size of the distractor to make it more or less salient. In Experiment, participants conducted a target-present and target-absent visual search task in the presence of a large, delayed-onset color distractor similar to that used in Moher's Study. In Experiment 2, a distractor that was much smaller than that used in the original Moher study was utilized. Critically, when a large distractor was used, the original findings of Moher were largely replicated; large salient distractors speeded target-absent visual search and increased errors for target-present visual search. However, when a smaller distractor was used, the results differed. For target-absent trials, search speeds were slower when the distractor was present compared with when it was absent. Thus, it appears that a highly salient distractor might be needed to trigger a shift in visual search strategy, and subsequently, lower quitting thresholds.

Benchmarking Human Performance for Visual Search of Aerial Images

Aerial images are frequently used in geospatial analysis to inform responses to crises and disasters but can pose unique challenges for visual search when they contain low resolution, degraded information about color, and small object sizes. Aerial image analysis is often performed by humans, but machine learning approaches are being developed to complement manual analysis. To date, however, relatively little work has explored how humans perform visual search on these tasks, and understanding this could ultimately help enable human-machine teaming. We designed a set of studies to understand what features of an aerial image make visual search difficult for humans and what strategies humans use when performing these tasks. Across two experiments, we tested human performance on a counting task with a series of aerial images and examined the influence of features such as target size, location, color, clarity, and number of targets on accuracy and search strategies. Both experiments presented trials consisting of an aerial satellite image; participants were asked to find all instances of a search template in the image. Target size was consistently a significant predictor of performance, influencing not only accuracy of selections but the order in which participants selected target instances in the trial. Experiment 2 demonstrated that the clarity of the target instance and the match between the color of the search template and the color of the target instance also predicted accuracy. Furthermore, color also predicted the order of selecting instances in the trial. These experiments establish not only a benchmark of typical human performance on visual search of aerial images but also identify several features that can influence the task difficulty level for humans. These results have implications for understanding human visual search on real-world tasks and when humans may benefit from automated approaches.

Assessment of the Effect of Cleanliness on the Visual Inspection of Aircraft Engine Blades: An Eye Tracking Study

Background-The visual inspection of aircraft parts such as engine blades is crucial to ensure safe aircraft operation. There is a need to understand the reliability of such inspections and the factors that affect the results. In this study, the factor 'cleanliness' was analysed among other factors. Method-Fifty industry practitioners of three expertise levels inspected 24 images of parts with a variety of defects in clean and dirty conditions, resulting in a total of N = 1200 observations. The data were analysed statistically to evaluate the relationships between cleanliness and inspection performance. Eye tracking was applied to understand the search strategies of different levels of expertise for various part conditions. Results-The results show an inspection accuracy of 86.8% and 66.8% for clean and dirty blades, respectively. The statistical analysis showed that cleanliness and defect type influenced the inspection accuracy, while expertise was surprisingly not a significant factor. In contrast, inspection time was affected by expertise along with other factors, including cleanliness, defect type and visual acuity. Eye tracking revealed that inspectors (experts) apply a more structured and systematic search with less fixations and revisits compared to other groups. Conclusions-Cleaning prior to inspection leads to better results. Eye tracking revealed that inspectors used an underlying search strategy characterised by edge detection and differentiation between surface deposits and other types of damage, which contributed to better performance.

From "satisfaction of search" to "subsequent search misses": a review of multiple-target search errors across radiology and cognitive science

For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target 'T' if a different target 'T' was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science. Here we review the SSM literature in both radiology and cognitive science and discuss: (1) the current SSM theories (i.e., satisfaction, perceptual set, and resource depletion theories), (2) the eye movement errors that underlie the SSM effect, (3) the existing efforts tested to alleviate SSM errors, and (4) the evolution of methodologies and analyses used when calculating the SSM effect. Finally, we present the attentional template theory, a novel mechanistic explanation for SSM errors, which ties together our current understanding of SSM errors and the attentional template literature.

Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.

Avoiding potential pitfalls in visual search and eye-movement experiments: A tutorial review

Examining eye-movement behavior during visual search is an increasingly popular approach for gaining insights into the moment-to-moment processing that takes place when we look for targets in our environment. In this tutorial review, we describe a set of pitfalls and considerations that are important for researchers – both experienced and new to the field – when engaging in eye-movement and visual search experiments. We walk the reader through the research cycle of a visual search and eye-movement experiment, from choosing the right predictions, through to data collection, reporting of methodology, analytic approaches, the different dependent variables to analyze, and drawing conclusions from patterns of results. Overall, our hope is that this review can serve as a guide, a talking point, a reflection on the practices and potential problems with the current literature on this topic, and ultimately a first step towards standardizing research practices in the field.

Improved X-ray baggage screening sensitivity with 'targetless' search training

When searching for a known target, mental representations of target features, or templates, guide attention towards matching objects and facilitate recognition. When only distractor features are known, distractor templates allow irrelevant objects to be recognised and attention to be shifted away. This is particularly true in X-ray baggage search, a challenging real-world visual search task with implications for public safety, where targets may be unknown, difficult to predict and concealed by an adversary, but distractors are typically benign and easier to identify. In the present study, we draw on basic principles of distractor suppression and rejection to investigate a counterintuitive ‘targetless’ approach to training baggage search. In a simulated X-ray baggage search task, we observed significant benefits to target detection sensitivity (d′) for targetless relative to target-based training, but no effects of performance-contingent rewards or the inclusion of superordinate semantic categories during training. The benefits of targetless search training were most apparent for stimuli involving less spatial overlap (occlusion), which likely represents the difficulty and greater individual variation involved in searching more visually complex images. Together, these results demonstrate the effectiveness of a counterintuitive targetless approach to training target detection in X-ray baggage search, based on basic principles of distractor suppression and rejection, with potential for use as a real-world training tool.

The invisible breast cancer: Experience does not protect against inattentional blindness to clinically relevant findings in radiology

Retrospectively obvious events are frequently missed when attention is engaged in another task-a phenomenon known as inattentional blindness. Although the task characteristics that predict inattentional blindness rates are relatively well understood, the observer characteristics that predict inattentional blindness rates are largely unknown. Previously, expert radiologists showed a surprising rate of inattentional blindness to a gorilla photoshopped into a CT scan during lung-cancer screening. However, inattentional blindness rates were higher for a group of naïve observers performing the same task, suggesting that perceptual expertise may provide protection against inattentional blindness. Here, we tested whether expertise in radiology predicts inattentional blindness rates for unexpected abnormalities that were clinically relevant. Fifty radiologists evaluated CT scans for lung cancer. The final case contained a large (9.1 cm) breast mass and lymphadenopathy. When their attention was focused on searching for lung nodules, 66% of radiologists did not detect breast cancer and 30% did not detect lymphadenopathy. In contrast, only 3% and 10% of radiologists (N = 30), respectively, missed these abnormalities in a follow-up study when searching for a broader range of abnormalities. Neither experience, primary task performance, nor search behavior predicted which radiologists missed the unexpected abnormalities. These findings suggest perceptual expertise does not protect against inattentional blindness, even for unexpected stimuli that are within the domain of expertise.

A novel, unbiased approach to evaluating subsequent search misses in dual target visual search

Research in radiology and visual cognition suggest that finding one target during visual search may result in increased misses for a second target, an effect known as subsequent search misses (SSM). Here, we demonstrate that the common method of calculating second-target detection performance is biased and could produce spurious SSM effects. We describe the source of that bias and document factors that influence its magnitude. We use a modification of signal-detection theory to develop a novel, unbiased method of calculating the expected value for dual-target performance under the null hypothesis. We then apply our novel method to two of our data sets that showed modest SSM effects when calculated in the traditional manner. Our correction reduced the effect size to the point that there was no longer a significant SSM effect. We then applied our method to a published data set that had a larger effect size when calculated using the traditional calculation as well as when using an alternative calculation that was recently proposed to account for biases in the traditional method. We find that both the traditional method and the recently proposed alternative substantially overestimate the magnitude of the SSM effect in these data, but a significant SSM effect persisted even with our calculation. We recommend that future SSM studies use our method to ensure accurate effect-size estimates, and suggest that the method be applied to reanalyze published results, particularly those with small effect sizes, to rule out the possibility that they were spurious.

Experience with searching in displays containing depth improves search performance by training participants to search more exhaustively

In a typical visual search task, participants search for single targets amongst displays containing non-overlapping objects that are presented on a single depth plane. Recent work has begun to examine displays containing overlapping objects that are presented on different depth planes to one another. It has been found that searching displays containing depth improves response accuracy by making participants more likely to fixate targets and to identify targets after fixating them. Here we extended this previous research by seeking first of all to replicate the previous pattern of results, and then to determine whether extensive training using depth in search transfers to two-dimensional displays. We provided participants with sixteen sessions of training with displays containing transparent overlapping objects presented in depth, and found a similar pattern of results to our previous study. We also found evidence that some performance improvements from the depth training transferred to search of two-dimensional displays that did not contain depth. Further examinations revealed that participants learn to search more exhaustively (i.e., search for longer) in displays containing depth. We conclude that depth does influence search performance but the influences depend very much on the stimuli and the degree of overlap within them.

Visual Search: How Do We Find What We Are Looking For?

In visual search tasks, observers look for targets among distractors. In the lab, this often takes the form of multiple searches for a simple shape that may or may not be present among other items scattered at random on a computer screen (e.g., Find a red T among other letters that are either black or red.). In the real world, observers may search for multiple classes of target in complex scenes that occur only once (e.g., As I emerge from the subway, can I find lunch, my friend, and a street sign in the scene before me?). This article reviews work on how search is guided intelligently. I ask how serial and parallel processes collaborate in visual search, describe the distinction between search templates in working memory and target templates in long-term memory, and consider how searches are terminated.

Visual search errors are persistent in a laboratory analog of the incidental finding problem

When radiologists search for a specific target (e.g., lung cancer), they are also asked to report any other clinically significant “incidental findings” (e.g., pneumonia). These incidental findings are missed at an undesirably high rate. In an effort to understand and reduce these errors, Wolfe et al. (Cognitive Research: Principles and Implications 2:35, 2017) developed “mixed hybrid search” as a model system for incidental findings. In this task, non-expert observers memorize six targets: half of these targets are specific images (analogous to the suspected diagnosis in the clinical task). The other half are broader, categorically defined targets, like “animals” or “cars” (analogous to the less well-specified incidental findings). In subsequent search through displays for any instances of any of the targets, observers miss about one third of the categorical targets, mimicking the incidental finding problem. In the present paper, we attempted to reduce the number of errors in the mixed hybrid search task with the goal of finding methods that could be deployed in a clinical setting. In Experiments 1a and 1b, we reminded observers about the categorical targets by inserting non-search trials in which categorical targets were clearly marked. In Experiment 2, observers responded twice on each trial: once to confirm the presence or absence of the specific targets, and once to confirm the presence or absence of the categorical targets. In Experiment 3, observers were required to confirm the presence or absence of every target on every trial using a checklist procedure. Only Experiment 3 produced a marked decline in categorical target errors, but at the cost of a substantial increase in response time.

Component processes in free-viewing visual search: Insights from fixation-aligned pupillary response averaging

Pupil size changes during a visual search may reflect cognitive processes, such as effort and memory accumulation, but methodological confounds and the general lack of literature in this area leave the reliability of findings open to question. We used a novel synthesis of experimental methods and averaging techniques to explore how cognitive processing unfolds during free-viewing visual search for multiple targets. Twenty-seven participants completed 152 searches across two separate 1-hour sessions. The number of targets present (Targets: 0, 1, 2, and 3) in each trial was the main manipulation and the task was to "find all of the targets" and report the total via mouse-click at the end of the trial. Search time lasted for 10 seconds or until the participant purported to have found all of the targets, in which case they could terminate the search via keypress. Whole-trial pupil analysis revealed a significant effect of button pressing as well as a significant main effect of targets for trials that were not self-terminated via button press. Fixation-aligned pupil responses revealed transient modulations in pupil size following initial fixations on targets but not distractors and refixations on both targets and distractors. Owing to rigorous control over experimental confounds and a detailed analysis and correction of eye-movement-related measurement error, we confidently discuss these findings in terms of task-related processing and underlying brain activity.

Distracting Objects Induce Early Quitting in Visual Search

Task-irrelevant objects can sometimes capture attention and increase the time it takes an observer to find a target. However, less is known about how these distractors impact visual search strategies. Here, I found that salient distractors reduced rather than increased response times on target-absent trials (Experiment 1; N = 200). Combined with higher error rates on target-present trials, these results indicate that distractors can induce observers to quit search earlier than they otherwise would. These effects were replicated when target prevalence was low (Experiment 2; N = 200) and with different stimuli that elicited shallower search slopes (Experiment 3; N = 75). These results demonstrate that salient distractors can produce at least two consequences in visual search: They can capture attention, and they can cause observers to quit searching early. This novel finding has implications both for understanding visual attention and for examining distraction in real-world domains where targets are often absent, such as medical image screening.

Dual-target hazard perception: Could identifying one hazard hinder a driver's capacity to find a second?

Low-level cognitive processes like visual search are crucial for hazard detection. In dual-target searches, subsequent search misses (SSMs) are known to occur when the identification of one target impedes detection of another that is concurrently presented. Despite the high likelihood of concurrent hazards in busy driving environments, SSMs have not been empirically investigated in driving. In three studies, participants were asked to identify safety-related target(s) in simulated traffic scenes that contained zero, one, or two target(s) of low or high perceptual saliency. These targets were defined as objects or events that would have prevented safe travel in the direction indicated by an arrow preceding the traffic scene. Findings from the pilot study (n = 20) and Experiment 1 (n = 29) demonstrated that detecting one target hindered drivers' abilities to find a second from the same scene. In Experiment 2 (n = 30), explicit instructions regarding the level of risk were manipulated. It was found that search times were affected by the instructions, though SSMs persisted. Implications of SSMs in understanding the causes of some crashes are discussed, as well as future directions to improve ecological and criterion validity and to explore the roles of expertise and cognitive capabilities in multi-hazard detection.

Visual search in breast imaging

Breast cancer is the most common cancer among females worldwide and large volumes of breast images are produced and interpreted annually. As long as radiologists interpret these images, the diagnostic accuracy will be limited by human factors and both false-positive and false-negative errors might occur. By understanding visual search in breast images, we may be able to identify causes of diagnostic errors, find ways to reduce them, and also provide a better education to radiology residents. Many visual search studies in breast radiology have been devoted to mammography. These studies showed that 70% of missed lesions on mammograms attract radiologists' visual attention and that a plethora of different reasons, such as satisfaction of search, incorrect background sampling, and incorrect first impression can cause diagnostic errors in the interpretation of mammograms. Recently, highly accurate tools, which rely on both eye-tracking data and the content of the mammogram, have been proposed to provide feedback to the radiologists. Improving these tools and determining the optimal pathway to integrate them in the radiology workflow could be a possible line of future research. Moreover, in the past few years deep learning has led to improving diagnostic accuracy of computerized diagnostic tools and visual search studies will be required to understand how radiologists interact with the prompts from these tools, and to identify the best way to utilize them. Visual search in other breast imaging modalities, such as breast ultrasound and digital breast tomosynthesis, have so far received less attention, probably due to associated complexities of eye-tracking monitoring and analysing the data. For example, in digital breast tomosynthesis, scrolling through the image results in longer trials, adds a new factor to the study's complexity and makes calculation of gaze parameters more difficult. However, considering the wide utilization of three-dimensional imaging modalities, more visual search studies involving reading stack-view examinations are required in the future. To conclude, in the past few decades visual search studies provided extensive understanding about underlying reasons for diagnostic errors in breast radiology and characterized differences between experts' and novices' visual search patterns. Further visual search studies are required to investigate radiologists' interaction with relatively newer imaging modalities and artificial intelligence tools.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Detection measures for visual inspection of X-ray images of passenger baggage

In visual inspection tasks, such as airport security and medical screening, researchers often use the detection measures d' or A' to analyze detection performance independent of response tendency. However, recent studies that manipulated the frequency of targets (target prevalence) indicate that da with a slope parameter of 0.6 is more valid for such tasks than d' or A'. We investigated the validity of detection measures (d', A', and da) using two experiments. In the first experiment, 31 security officers completed a simulated X-ray baggage inspection task while response tendency was manipulated directly through instruction. The participants knew half of the prohibited items used in the study from training, whereas the other half were novel, thereby establishing two levels of task difficulty. The results demonstrated that for both levels, d' and A' decreased when the criterion became more liberal, whereas da with a slope parameter of 0.6 remained constant. Eye-tracking data indicated that manipulating response tendency affected the decision component of the inspection task rather than search errors. In the second experiment, 124 security officers completed another simulated X-ray baggage inspection task. Receiver operating characteristic (ROC) curves based on confidence ratings provided further support for da, and the estimated slope parameter was 0.5. Consistent with previous findings, our results imply that d' and A' are not valid measures of detection performance in X-ray image inspection. We recommend always calculating da with a slope parameter of 0.5 in addition to d' to avoid potentially wrong conclusions if ROC curves are not available.

"Satisfaction" in search: Individuals' own search expectations predict their errors in multiple-target search

When people search for multiple targets in a display, finding one target hinders their ability to find additional targets. These errors were originally proposed to stem from a "satisfaction" with finding a first target that leads people to prematurely stop searching. However, empirical evidence for this premise has been elusive, prompting consideration of other theories. We returned to the satisfaction proposal and assessed whether people generate expectations regarding the likelihood of multiple targets that lead to search biases that, in turn, predict the rates at which additional targets are missed. Participants searched for one or two targets among distractors. To measure accuracy, most trials allowed search to progress to completion. The remaining trials terminated when participants had found their first target. In these cases, participants guessed whether an additional (unfound) target was present. The time needed to find a first target was inversely related to the searchers' expectations that a second target would be present. These expectations underestimated objective reality, and the strength of an individual's one-target bias was directly related to his or her likelihood of missing subsequent targets. Thus, people's expectations-based on their own behavior-likely impacted search performance, providing a novel mechanistic explanation for the previously posited "satisfaction-of-search" errors.

Eye Movements in Medical Image Perception: A Selective Review of Past, Present and Future

The eye movements of experts, reading medical images, have been studied for many years. Unlike topics such as face perception, medical image perception research needs to cope with substantial, qualitative changes in the stimuli under study due to dramatic advances in medical imaging technology. For example, little is known about how radiologists search through 3D volumes of image data because they simply did not exist when earlier eye tracking studies were performed. Moreover, improvements in the affordability and portability of modern eye trackers make other, new studies practical. Here, we review some uses of eye movements in the study of medical image perception with an emphasis on newer work. We ask how basic research on scene perception relates to studies of medical 'scenes' and we discuss how tracking experts' eyes may provide useful insights for medical education and screening efficiency.

Using Eye Movements to Understand how Security Screeners Search for Threats in X-Ray Baggage

There has been an increasing drive to understand failures in searches for weapons and explosives in X-ray baggage screening. Tracking eye movements during the search has produced new insights into the guidance of attention during the search, and the identification of targets once they are fixated. Here, we review the eye-movement literature that has emerged on this front over the last fifteen years, including a discussion of the problems that real-world searchers face when trying to detect targets that could do serious harm to people and infrastructure.

A review of eye tracking for understanding and improving diagnostic interpretation

Inspecting digital imaging for primary diagnosis introduces perceptual and cognitive demands for physicians tasked with interpreting visual medical information and arriving at appropriate diagnoses and treatment decisions. The process of medical interpretation and diagnosis involves a complex interplay between visual perception and multiple cognitive processes, including memory retrieval, problem-solving, and decision-making. Eye-tracking technologies are becoming increasingly available in the consumer and research markets and provide novel opportunities to learn more about the interpretive process, including differences between novices and experts, how heuristics and biases shape visual perception and decision-making, and the mechanisms underlying misinterpretation and misdiagnosis. The present review provides an overview of eye-tracking technology, the perceptual and cognitive processes involved in medical interpretation, how eye tracking has been employed to understand medical interpretation and promote medical education and training, and some of the promises and challenges for future applications of this technology.

Visual search for changes in scenes creates long-term, incidental memory traces

Humans are very good at remembering large numbers of scenes over substantial periods of time. But how good are they at remembering changes to scenes? In this study, we tested scene memory and change detection two weeks after initial scene learning. In Experiments 1-3, scenes were learned incidentally during visual search for change. In Experiment 4, observers explicitly memorized scenes. At test, after two weeks observers were asked to discriminate old from new scenes, to recall a change that they had detected in the study phase, or to detect a newly introduced change in the memorization experiment. Next, they performed a change detection task, usually looking for the same change as in the study period. Scene recognition memory was found to be similar in all experiments, regardless of the study task. In Experiment 1, more difficult change detection produced better scene memory. Experiments 2 and 3 supported a "depth-of-processing" account for the effects of initial search and change detection on incidental memory for scenes. Of most interest, change detection was faster during the test phase than during the study phase, even when the observer had no explicit memory of having found that change previously. This result was replicated in two of our three change detection experiments. We conclude that scenes can be encoded incidentally as well as explicitly and that changes in those scenes can leave measurable traces even if they are not explicitly recalled.

Getting satisfied with "satisfaction of search": How to measure errors during multiple-target visual search

Visual search studies are common in cognitive psychology, and the results generally focus upon accuracy, response times, or both. Most research has focused upon search scenarios where no more than 1 target will be present for any single trial. However, if multiple targets can be present on a single trial, it introduces an additional source of error because the found target can interfere with subsequent search performance. These errors have been studied thoroughly in radiology for decades, although their emphasis in cognitive psychology studies has been more recent. One particular issue with multiple-target search is that these subsequent search errors (i.e., specific errors which occur following a found target) are measured differently by different studies. There is currently no guidance as to which measurement method is best or what impact different measurement methods could have upon various results and conclusions. The current investigation provides two efforts to address these issues. First, the existing literature is reviewed to clarify the appropriate scenarios where subsequent search errors could be observed. Second, several different measurement methods are used with several existing datasets to contrast and compare how each method would have affected the results and conclusions of those studies. The evidence is then used to provide appropriate guidelines for measuring multiple-target search errors in future studies.

Minds treating brains: understanding the interpretation of non-contrast CT ASPECTS in acute ischemic stroke

INTRODUCTION

The Alberta Stroke Program Early CT Score on non-contrast CT is a key component of prognostication and treatment selection in acute stroke care. Previous findings show that the reliability of this scale must be improved to maximize its clinical utility. Areas covered: This review discusses technical, patient-level, and reader-level sources of variability in ASPECTS reading; relevant concepts in the psychology of medical image perception; and potential interventions likely to improve inter- and intra-rater reliability. Expert commentary: Approaching variability in medical decision making from a psychological perspective will afford cognitively informed insights into the development of interventions and training techniques aimed at improving this issue.

How did I miss that? Developing mixed hybrid visual search as a 'model system' for incidental finding errors in radiology

In a real world search, it can be important to keep ‘an eye out’ for items of interest that are not the primary subject of the search. For instance, you might look for the exit sign on the freeway, but you should also respond to the armadillo crossing the road. In medicine, these items are known as “incidental findings,” findings of possible clinical significance that were not the main object of search. These errors (e.g., missing a broken rib while looking for pneumonia) have medical consequences for the patient and potential legal consequences for the physician. Here we report three experiments intended to develop a ‘model system’ for incidental findings – a paradigm that could be used in the lab to develop strategies to reduce incidental finding errors in the clinic. All the experiments involve ‘hybrid’ visual search for any of several targets held in memory. In this ‘mixed hybrid search task,’ observers search for any of three specific targets (e.g., this rabbit, this truck, and this spoon) and three categorical targets (e.g., masks, furniture, and plants). The hypothesis is that the specific items are like the specific goals of a real world search and the categorical targets are like the less well-defined incidental findings that might be present and that should be reported. In all these experiments, varying target prevalence, number of targets, etc., the categorical targets are missed at a much higher rate than the specific targets. This paradigm shows promise as a model of the incidental finding problem.

An individual differences approach to multiple-target visual search errors: How search errors relate to different characteristics of attention

A persistent problem in visual search is that searchers are more likely to miss a target if they have already found another in the same display. This phenomenon, the Subsequent Search Miss (SSM) effect, has remained despite being a known issue for decades. Increasingly, evidence supports a resource depletion account of SSM errors-a previously detected target consumes attentional resources leaving fewer resources available for the processing of a second target. However, "attention" is broadly defined and is composed of many different characteristics, leaving considerable uncertainty about how attention affects second-target detection. The goal of the current study was to identify which attentional characteristics (i.e., selection, limited capacity, modulation, and vigilance) related to second-target misses. The current study compared second-target misses to an attentional blink task and a vigilance task, which both have established measures that were used to operationally define each of four attentional characteristics. Second-target misses in the multiple-target search were correlated with (1) a measure of the time it took for the second target to recovery from the blink in the attentional blink task (i.e., modulation), and (2) target sensitivity (d') in the vigilance task (i.e., vigilance). Participants with longer recovery and poorer vigilance had more second-target misses in the multiple-target visual search task. The results add further support to a resource depletion account of SSM errors and highlight that worse modulation and poor vigilance reflect a deficit in attentional resources that can account for SSM errors.

Use-inspired basic research in medical image perception

This journal is dedicated to "use-inspired basic research" where a problem in the world shapes the hypotheses for a study in the laboratory. This brief review presents several examples of "use-inspired basic research" in the area of medical image perception. These are cases where the field of radiology raises an interesting issue in visual cognition. Basic research on those issues may then lead to proposals to improve performance on clinical tasks in medical image perception. Of the six examples given here, the first three ask essentially perceptual questions: How can stereopsis improve medical image perception? How shall we assess the tradeoff between radiation dose and image quality? How does the choice of colors change the interpretation of medical images? The second three examples address attentional issues in those aspects of radiology that can be described as visual search problems: Can eye tracking help us understand errors in radiologic search? What happens if the number of targets in an image is unknown? What happens if, as in radiology screening programs, the target of search is very rare?

Modeling Lag-2 Revisits to Understand Trade-Offs in Mixed Control of Fixation Termination During Visual Search

An important question about eye-movement behavior is when the decision is made to terminate a fixation and program the following saccade. Different approaches have found converging evidence in favor of a mixed-control account, in which there is some overlap between processing information at fixation and planning the following saccade. We examined one interesting instance of mixed control in visual search: lag-2 revisits, during which observers fixate a stimulus, move to a different stimulus, and then revisit the first stimulus on the next fixation. Results show that the probability of lag-2 revisits occurring increased with the number of target-similar stimuli, and revisits were preceded by a brief fixation on the intervening distractor stimulus. We developed the Efficient Visual Sampling (EVS) computational model to simulate our findings (fixation durations and fixation locations) and to provide insight into mixed control of fixations and the perceptual, cognitive, and motor processes that produce lag-2 revisits.

Multiple-Target Visual Search Errors

Visual search—the ability to locate visual targets among distractors—is a fundamental part of professional performance for many careers, including radiology, airport security screening, cytology, lifeguarding, and more. Successful execution of visual search in these settings is critically important because the consequences of a missed target can be horrific. Unfortunately, many of these professions place high demands on the people performing the searches, and either the task or the environment (or both) could lead to significant errors. One known source of error that exists across many fields is “multiple-target visual search” errors—a target is less likely to be detected if another target was already found in the same search than if the target was the only one present. These errors have proven to be stubborn and not easily eliminated. This article offers a brief overview of the existing research on multiple-target visual search errors and discusses possible policy implications of the errors for airport security screening. The policy suggestions are based on empirical research, with the hope of providing food for thought on using scientific data and theory to improve performance. Specifically, three policy suggestions are raised: shift screening to a remote location away from the checkpoint, reduce the number of prohibited items to lessen the searchers’ cognitive burden, and emphasize search consistency in the training process. Note that the focus here is on airport security screening, as this is a domain most readers can relate to, but the suggestions can equally apply to many search environments.

Targets Need Their Own Personal Space: Effects of Clutter on Multiple-Target Search Accuracy

Visual search is an essential task for many lifesaving professions; airport security personnel search baggage X-ray images for dangerous items and radiologists examine radiographs for tumors. Accuracy is critical for such searches; however, there are potentially negative influences that can affect performance; for example, the displays can be cluttered and can contain multiple targets. Previous research has demonstrated that clutter can hurt search performance and a second target is less likely to be detected in a multiple-target search after a first target has been found, which raises a concern-how does clutter affect multiple-target search performance? The current study explored clutter in a multiple-target search paradigm, where there could be one or two targets present, and targets appeared in varying levels of clutter. There was a significant interaction between clutter and target number: Increasing levels of clutter did not affect single-target detection but did reduce detection of a second target. Multiple-target search accuracy is known to be sensitive to contextual influences, and the current results reveal a specific effect wherein clutter disproportionally affected multiple-target search accuracy. These results suggest that the detection and processing of a first target might enhance the masking effects of clutter around a second target.

Perceptual failures in the selection and identification of low-prevalence targets in relative prevalence visual search

Previous research has shown that during visual search tasks target prevalence (the proportion of trials in which a target appears) influences both the probability that a target will be detected, and the speed at which participants will quit searching and provide an 'absent' response. When prevalence is low (e.g., target presented on 2 % of trials), participants are less likely to detect the target than when prevalence is higher (e.g., 50 % of trials). In the present set of experiments, we examined perceptual failures to detect low prevalence targets in visual search. We used a relative prevalence search task in order to be able to present an overall 50 % target prevalence and thereby prevent the results being accounted for by early quitting behavior. Participants searched for two targets, one of which appeared on 45 % of trials and another that appeared on 5 % of trials, leaving overall target prevalence at 50 %. In the first experiment, participants searched for two dissimilar targets; in the second experiment, participants searched for two similar targets. Overall, the results supported the notion that a reduction in prevalence primarily influenced perceptual failures of identification, rather than of selection. Together, these experiments add to a growing body of research exploring how and why observers fail to detect low prevalence targets, especially in real-world tasks in which some targets are more likely to appear than others.

Rare, but obviously there: effects of target frequency and salience on visual search accuracy

Accuracy can be extremely important for many visual search tasks. However, numerous factors work to undermine successful search. Several negative influences on search have been well studied, yet one potentially influential factor has gone almost entirely unexplored-namely, how is search performance affected by the likelihood that a specific target might appear? A recent study demonstrated that when specific targets appear infrequently (i.e., once in every thousand trials) they were, on average, not often found. Even so, some infrequently appearing targets were actually found quite often, suggesting that the targets' frequency is not the only factor at play. Here, we investigated whether salience (i.e., the extent to which an item stands out during search) could explain why some infrequent targets are easily found whereas others are almost never found. Using the mobile application Airport Scanner, we assessed how individual target frequency and salience interacted in a visual search task that included a wide array of targets and millions of trials. Target frequency and salience were both significant predictors of search accuracy, although target frequency explained more of the accuracy variance. Further, when examining only the rarest target items (those that appeared on less than 0.15% of all trials), there was a significant relationship between salience and accuracy such that less salient items were less likely to be found. Beyond implications for search theory, these data suggest significant vulnerability for real-world searches that involve targets that are both infrequent and hard-to-spot.