Search for multiple targets: Evidence for memory-based control of attention

From "satisfaction of search" to "subsequent search misses": a review of multiple-target search errors across radiology and cognitive science

For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target 'T' if a different target 'T' was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science. Here we review the SSM literature in both radiology and cognitive science and discuss: (1) the current SSM theories (i.e., satisfaction, perceptual set, and resource depletion theories), (2) the eye movement errors that underlie the SSM effect, (3) the existing efforts tested to alleviate SSM errors, and (4) the evolution of methodologies and analyses used when calculating the SSM effect. Finally, we present the attentional template theory, a novel mechanistic explanation for SSM errors, which ties together our current understanding of SSM errors and the attentional template literature.

Target processing in overt serial visual search involves the dorsal attention network: A fixation-based event-related fMRI study

In serial visual search we shift attention successively from location to location in search for the target. Although such search has been investigated using fMRI, overt attention (i.e., eye movements) was usually neglected or discouraged. As a result, it is unclear what happens in the instant when our gaze falls upon a target as compared to a distractor. In the present experiment, we used a multiple target search task that required eye movements and employed an analysis based on fixations as events of interest to investigate differences between target and distractor processing. Twenty young healthy adults indicated the number of targets (0-3) among distractors in a 20-item display. Compared to distractor fixations, we found that target fixations gave rise to wide-spread activation in the dorsal attention system, as well as in the visual cortex. Targets that were found later during the search activated the left inferior frontal gyrus and the left supramarginal gyrus more strongly than those that were found earlier. Finally, areas associated with visual and verbal working memory showed increased activation with a larger number of targets in the display.

Visual working memory load does not affect the overall stimulus processing time in visual search

The dual-task paradigm is widely used in studying the interaction between visual search and working memory. A number of studies showed that holding items in working memory delays the overall response time (RT) in visual search, but it does not affect the efficiency of search (i.e., the slope of the RT × set size function). Why the memory load merely affects the overall RT? Some researchers proposed that this load-effect on overall RT may be caused by factors that only affect response selection processes, while others argued that it may reflect the effect of visual working memory load on visual search. This study investigated the two competing hypotheses by measuring the threshold stimulus exposure duration (TSED) for successfully fulfilling a search task. Experiment 1 replicated the large overall RT difference with the RT method but only found a small though reliable overall TSED difference with the TSED method. Experiment 2, with better controls, found no TSED difference by manipulating the visual working memory load. Experiment 3 showed that the TSED is not influenced by processes in the response selection stage. The present findings suggest that the overall stimulus processing time in visual search is not affected by visual working memory load and that the effect of memory load on overall RT is largely due to factors affecting response selection alone.

Visual Search May Not Require Target Representation in Working Memory or Long-Term Memory

We spend a lot of time searching for things. If we know what we are looking for in advance, a memory representation of the target will be created to guide search. But if the identity of the search target is revealed simultaneously with the presentation of the search array, is a similar memory representation formed? In the present study, 96 observers determined whether a central target was present in a peripheral search array. The results revealed that as long as the central target remained available for inspection (even if only in iconic memory), observers reinspected it after each distractor was checked, apparently forgoing consolidation of the target into working memory. The present findings challenged the assumption that evaluating items in a search array must involve comparison with a template in working memory.

Active search for multiple targets is inefficient

This study examines saccade strategy in a novel task where observers actively search a display to find multiple targets in a limited time. Theory predicts that the relative merit of different saccade strategies depends on the prior probability of the target at a location: when the target prior is low and multiple-target trials are rare, making a saccade to the most likely target location is close to the optimal strategy, but when the target prior is high and multiple-target trials are frequent, selecting uncertain locations is more informative. The prior probability of the target was varied from 0.17 to 0.67 to determine whether observers adjusted their saccades strategies to maximize information. Observers actively searched a noisy display with six potential target locations. Each location had an independent probability of a target, so the number of targets in a trial ranged from 0 to 6. For all target priors ranging from low to high, a trial-by-trial analysis of saccade strategy indicated that observers made saccades to the most likely target location more often than the most uncertain location. Fixating likely locations is efficient only when multiple targets are rare, as in the case of a low target prior, or in the case of the more standard single-target search task. Yet it is the preferred saccade strategy in all our conditions, even when multiple targets are frequent. These findings indicate that humans are far from ideal searchers in multiple-target search.

Generalized "satisfaction of search": adverse influences on dual-target search accuracy

The successful detection of a target in a radiological search can reduce the detectability of a second target, a phenomenon termed satisfaction of search (SOS). Given the potential consequences, here we investigate the generality of SOS with the goal of simultaneously informing radiology, cognitive psychology, and nonmedical searches such as airport luggage screening. Ten experiments utilizing nonmedical searches and untrained searchers suggest that SOS is affected by a diverse array of factors, including (1) the relative frequency of different target types, (2) external pressures (reward and time), and (3) expectations about the number of targets present. Collectively, these experiments indicate that SOS arises when searchers have a biased expectation about the low likelihood of specific targets or events, and when they are under pressure to perform efficiently. This first demonstration of SOS outside of radiology implicates a general heuristic applicable to many kinds of searches. In an example like airport luggage screening, the current data suggest that the detection of an easy-to-spot target (e.g., a water bottle) might reduce detection of a hard-to-spot target (e.g., a box cutter).

Memory for where, but not what, is used during visual search

Although the role of memory in visual search is debatable, most researchers agree with a limited-capacity model of memory in visual search. The authors demonstrate the role of memory by replicating previous findings showing that visual search is biased away from old items (previously examined items) and toward new items (nonexamined items). Furthermore, the authors examined the type of memory representations used to bias search by changing an item's individuating feature or location during search. Changing the individuating feature of an item did not disrupt normal search biases. However, when the location of an item changed, normal search biases were disrupted. These results suggest that memory used in visual search is based on items' locations rather than their identity.

Effect of previously fixated locations on saccade trajectory during free visual search

Recent studies have shown that the saccade trajectory often curved away from an object that was previously attended but irrelevant to the current saccade goal. We investigated whether such curved saccades occur during serial visual search, which requires sequential saccades possibly controlled by inhibition to multiple locations. The results show that the saccade trajectories were affected by at least three previous fixations. Furthermore, the effect of the previous fixations on saccade trajectories decreased exponentially with time or the number of intervening saccades. The relationship between the curved saccade trajectory and inhibition of return during serial visual search was discussed.

An inter-item similarity model unifying feature and conjunction search

We propose a model of visual search to address the hitherto unresolved issue of reconciling serial deployment of attention accounts with inter-item similarity effects. Target-distractor and distractor-distractor similarity were systematically varied in 85 (17x5) set type-size conditions over seven experiments, including univariate feature and bivariate conjunction search. The model, a power (square root) function of dimension-specific target-distractor and distractor-distractor similarity in linear combination with set size, accounted for 98% of the variance on type-size means. It suggests that much of efficient and inefficient search can be unified under a single theory involving item similarity.

Rapid onset and long-term inhibition of return in the multiple cuing paradigm

Inhibition of return (IOR) refers to the finding that targets at cued locations are responded to more slowly than targets at uncued locations when a relatively long temporal interval occurs between the two events. In studies which have examined the time course of IOR (e.g., Samuel & Kat in Psychonomic Bulletin & Review, 10, 897-906, 2003), the effect is generally shown to develop at around 200 ms and dissipate at around 3,000 ms following a cue. A number of recent studies, however, have demonstrated that IOR can develop much more quickly (up to 50 ms following a cue) and last much longer (up to 13 min following a cue) in certain tasks. The present study uses the multiple cuing paradigm to determine whether IOR can be observed outside the normally reported temporal boundaries (300-3,000 ms) when attention is shifted very quickly (every 15 ms) or very slowly (every 1,500 ms) throughout the visual field. IOR was observed as quickly as 30 ms following cue onset and as long as 6,000 ms following cue onset. Implications for the role of IOR in visual search are discussed.