Sit-and-wait strategies in dynamic visual search

Testing a relational account of search templates in visual foraging

Search templates guide human visual attention toward relevant targets. Templates are often seen as encoding exact target features, but recent studies suggest that templates rather contain "relational properties" (e.g., they facilitate "redder" stimuli instead of specific hues of red). Such relational guidance seems helpful in naturalistic searches where illumination or perspective renders exact feature values unreliable. So far relational guidance has only been demonstrated in rather artificial single-target search tasks with briefly flashed displays. Here, we investigate whether relational guidance also occurs when humans interact with the search environment for longer durations to collect multiple target elements. In a visual foraging task, participants searched for and collected multiple targets among distractors of different relationships to the target colour. Distractors whose colour differed from the environment in the same direction as the targets reduced foraging efficiency to the same amount as distractors whose colour matched the target colour. Distractors that differed by the same colour distance but in the opposite direction of the target colour did not reduce efficiency. These findings provide evidence that search templates encode relational target features in naturalistic search tasks and suggest that attention guidance based on relational features is a common mode in dynamic, real-world search environments.

Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.

Maintaining rejected distractors in working memory during visual search depends on search stimuli: Evidence from contralateral delay activity

The presence of memory for rejected distractors during visual search has been heavily debated in the literature and has proven challenging to investigate behaviorally. In this research, we used an electrophysiological index of working memory (contralateral delay activity) to passively measure working memory activity during visual search. Participants were asked to indicate whether a novel target was present or absent in a lateralized search array with three visual set sizes (2, 4, or 6). If rejected distractors are maintained in working memory during search, working memory activity should increase with the number of distractors that need to be evaluated. Therefore, we predicted the amplitude of the contralateral delay activity would be larger for target-absent trials and would increase with visual set size until WM capacity was reached. In Experiment 1, we found no evidence for distractor maintenance in working memory during search for real-world stimuli. In Experiment 2, we found partial evidence in support of distractor maintenance during search for stimuli with high target/distractor similarity. In both experiments, working memory capacity did not appear to be a limiting factor during visual search. These results suggest the role of working memory during search may depend on the visual search task in question. Maintaining distractors in working memory appears to be unnecessary during search for realistic stimuli. However, there appears to be a limited role for distractor maintenance during search for artificial stimuli with a high degree of feature overlap.

Visual Search: How Do We Find What We Are Looking For?

In visual search tasks, observers look for targets among distractors. In the lab, this often takes the form of multiple searches for a simple shape that may or may not be present among other items scattered at random on a computer screen (e.g., Find a red T among other letters that are either black or red.). In the real world, observers may search for multiple classes of target in complex scenes that occur only once (e.g., As I emerge from the subway, can I find lunch, my friend, and a street sign in the scene before me?). This article reviews work on how search is guided intelligently. I ask how serial and parallel processes collaborate in visual search, describe the distinction between search templates in working memory and target templates in long-term memory, and consider how searches are terminated.

Visual Search Within a Limited Window Area: Scrolling Versus Moving Window

Every day we perceive pictures on our mobile phones and scroll through images within a limited space. At present, however, visual perception via image scrolling is not well understood. This study investigated the nature of visual perception within a small window frame. It compared visual search efficiency using three modes: scrolling, moving-window, and free-viewing. The item number and stimulus size varied. Results showed variations in search efficiency depending on search mode. The slowest search occurred under the scrolling condition, followed by the moving-window condition, and the fastest search occurred under the no-window condition. For the scrolling condition, the response time increased the least sharply in proportion to item number but most sharply in proportion to the stimulus size compared to the other two conditions. Analysis of the trace of scan revealed frequent pauses interjected with small and fast stimulus shifts for the scrolling condition, but slow and continuous window movements interjected with a few pauses for the moving-window condition. We concluded that searching via scrolling was less efficient than searching via a moving-window, reflecting differences in dynamic properties of participants' scan.

What do eye movements reveal about the role of memory in visual search?

Horowitz and Wolfe (1998, 2003) have challenged the view that serial visual search involves memory processes that keep track of already inspected locations. The present study used a search paradigm similar to Horowitz and Wolfe's (1998), comparing a standard static search condition with a dynamic condition in which display elements changed locations randomly every 111 ms. In addition to measuring search reaction times, observers’ eye movements were recorded. For target-present trials, the search rates were near-identical in the two search conditions, replicating Horowitz and Wolfe's findings. However, the number of fixations and saccade amplitude were larger in the static than in the dynamic condition, whereas fixation duration and the latency of the first saccade were longer in the dynamic condition. These results indicate that an active, memory-guided search strategy was adopted in the static condition, and a passive “sit-and-wait” strategy in the dynamic condition.

Losing the trees for the forest in dynamic visual search

Representing temporally continuous objects across change (e.g., in position) requires integration of newly sampled visual information with existing object representations. We asked what consequences representational updating has for visual search. In this dynamic visual search task, bars rotated around their central axis. Observers searched for a single episodic target state (oblique bar among vertical and horizontal bars). Search was efficient when the target display was presented as an isolated static display. Performance declined to near chance, however, when the same display was a single state of a dynamically changing scene (Experiment 1), as though temporal selection of the target display from the stream of stimulation failed entirely (Experiment 3). The deficit is attributable neither to masking (Experiment 2), nor to a lack of temporal marker for the target display (Experiment 4). The deficit was partially reduced by visually marking the target display with unique feature information (Experiment 5). We suggest that representational updating causes a loss of access to instantaneous state information in search. Similar to spatially crowded displays that are perceived as textures (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001), we propose a temporal version of the trees (instantaneous orientation information) being lost for the forest (rotating bars).

Ability to use the wait-and-see strategy in pathological gamblers

Pathological gamblers (PGs) perform differently on neurocognitive tests than do healthy controls (HC). The aim of this study was to assess "waiting ability" - a major components of inhibition control-using a modified Stop Signal Task (SST) in a population of male PGs (N=55), and HCs (N=53). Results indicated no differences between PGs and HCs in reaction times, intra-individual response variability, or number of false alarms and misses. In conclusion, PGs were not impaired in their ability to manipulate their on-line response strategy during the experimental task and were instead able to change their strategy to decrease the number of false alarms. However, much more empirical and theoretical work needs to be carried out in order to understand the key neural basis of impulsivity among PGs.

New Evidence for Strategic Differences between Static and Dynamic Search Tasks: An Individual Observer Analysis of Eye Movements

Two experiments are reported that further explore the processes underlying dynamic search. In Experiment 1, observers’ oculomotor behavior was monitored while they searched for a randomly oriented T among oriented L distractors under static and dynamic viewing conditions. Despite similar search slopes, eye movements were less frequent and more spatially constrained under dynamic viewing relative to static, with misses also increasing more with target eccentricity in the dynamic condition. These patterns suggest that dynamic search involves a form of sit-and-wait strategy in which search is restricted to a small group of items surrounding fixation. To evaluate this interpretation, we developed a computational model of a sit-and-wait process hypothesized to underlie dynamic search. In Experiment 2 we tested this model by varying fixation position in the display and found that display positions optimized for a sit-and-wait strategy resulted in higher d′ values relative to a less optimal location. We conclude that different strategies, and therefore underlying processes, are used to search static and dynamic displays.

Visual and memory search in complex environments: determinants of eye movements and search performance

Previous research on visual and memory search revealed various top down and bottom up factors influencing performance. However, utilising abstract stimuli (e.g. geometrical shapes or letters) and focussing on individual factors has often limited the applicability of research findings. Two experiments were designed to analyse which attributes of a product facilitate search in an applied environment. Participants scanned displays containing juice packages while their eye movements were recorded. The familiarity, saliency, and position of search targets were systematically varied. Experiment 1 involved a visual search task, whereas Experiment 2 focussed on memory search. The results showed that bottom up (target saliency) and top down (target familiarity) factors strongly interacted. Overt visual attention was influenced by cultural habits, purposes, and current task demands. The results provide a solid database for assessing the impact and interplay of fundamental top down and bottom up determinants of search processes in applied fields of psychology. Practitioner Summary: Our study demonstrates how a product (or a visual item in general) needs to be designed and placed to ensure that it can be found effectively and efficiently within complex environments. Corresponding product design should result in faster and more accurate visual and memory based search processes.

When do I quit? The search termination problem in visual search

In visual search tasks, observers look for targets in displays or scenes containing distracting, non-target items. Most of the research on this topic has concerned the finding of those targets. Search termination is a less thoroughly studied topic. When is it time to abandon the current search? The answer is fairly straight forward when the one and only target has been found (There are my keys.). The problem is more vexed if nothing has been found (When is it time to stop looking for a weapon at the airport checkpoint?) or when the number of targets is unknown (Have we found all the tumors?). This chapter reviews the development of ideas about quitting time in visual search and offers an outline of our current theory.

Why is visual search superior in autism spectrum disorder?

This study investigated the possibility that enhanced memory for rejected distractor locations underlies the superior visual search skills exhibited by individuals with autism spectrum disorder (ASD). We compared the performance of 21 children with ASD and 21 age- and IQ-matched typically developing (TD) children in a standard static search task and a dynamic search task, in which targets and distractors randomly changed locations every 500 ms, precluding the use of memory in search. Children with ASD exhibited overall faster reaction time (RT) relative to TD children, and showed no disruption in search efficiency in the dynamic condition, discounting the possibility that memory for rejected distractors augments their visual search abilities. Analyses of RT x set size functions showed no group differences in slopes but lower intercepts for the ASD group in both static and dynamic search, suggesting that the ASD advantage derived from non-search processes, such as an enhanced ability to discriminate between targets and distractors at the locus of attention. Eye-movement analyses revealed that the ASD and TD groups were similar in the number and spatial distribution of fixations across the search array, but that fixation duration was significantly shorter among children with ASD. Lower intercepts in static search were related to increased symptom severity in children with ASD. In summary, ASD search superiority did not derive from differences in the manner in which individuals with ASD deployed their attention while searching, but from anomalously enhanced perception of stimulus features, which was in turn positively associated with autism symptom severity.

The speed of free will

Do voluntary and task-driven shifts of attention have the same time course? In order to measure the time needed to voluntarily shift attention, we devised several novel visual search tasks that elicited multiple sequential attentional shifts. Participants could only respond correctly if they attended to the right place at the right time. In control conditions, search tasks were similar but participants were not required to shift attention in any order. Across five experiments, voluntary shifts of attention required 200-300 ms. Control conditions yielded estimates of 35-100 ms for task-driven shifts. We suggest that the slower speed of voluntary shifts reflects the "clock speed of free will". Wishing to attend to something takes more time than shifting attention in response to sensory input.

Visual search is guided by prospective and retrospective memory

Although there has been some controversy as to whether attention is guided by memory during visual search, recent findings have suggested that memory helps to prevent attention from needlessly reinspecting examined items. Until now, it has been assumed that some form of retrospective memory is responsible for keeping track of examined items and preventing revisitations. Alternatively, some form of prospective memory, such as strategic scanpath planning, could be responsible for guiding attention away from examined items. We used a new technique that allowed us to selectively prevent retrospective or prospective memory from contributing to search. We demonstrated that both retrospective and prospective memory guide attention during visual search.

Effect of previously fixated locations on saccade trajectory during free visual search

Recent studies have shown that the saccade trajectory often curved away from an object that was previously attended but irrelevant to the current saccade goal. We investigated whether such curved saccades occur during serial visual search, which requires sequential saccades possibly controlled by inhibition to multiple locations. The results show that the saccade trajectories were affected by at least three previous fixations. Furthermore, the effect of the previous fixations on saccade trajectories decreased exponentially with time or the number of intervening saccades. The relationship between the curved saccade trajectory and inhibition of return during serial visual search was discussed.

Marking rejected distractors: a gaze-contingent technique for measuring memory during search

There is a debate among search theorists as to whether search exploits a memory for rejected distractors. We addressed this question by monitoring eye movements and explicitly marking objects visited by gaze during search. If search is memoryless, markers might be used to reduce distractor reinspections and improve search efficiency, relative to a no-marking baseline. However, if search already uses distractor memory, there should be no differences between marking and no-marking conditions. In four experiments, with stimuli ranging from Os and Qs to realistic scenes, two consistent data patterns emerged: (1) Marking rejected distractors produced no systematic benefit for search efficiency, as measured by reinspections, reaction times, or errors, and (2) distractor reinspection rates were, overall, extremely low. These results suggest that search uses a memory for rejected distractors, at least in those many real-world search tasks in which gaze is free to move.

Inhibition of return: Sensitivity and criterion as a function of response time

Inhibition of return (IOR) refers to a mechanism that results in a performance disadvantage typically observed when targets are presented at a location once occupied by a cue. Although the time course of the phenomenon--from the cue to the target--has been well studied, the time course of the effect--from target to response--is unknown. In 2 experiments, the effect of IOR upon sensitivity and response criterion under different levels of speed stress was examined. In go/no-go and choice reaction time tasks, IOR had at least 2 distinct effects on information processing. Early in target processing, before sufficient target information has accrued, there is a bias against responding to cued targets. Later, as target information is allowed to accrue, IOR reduces sensitivity to the target's nonspatial feature. Three accounts relating to the early bias effect of IOR and the late effect of IOR on sensitivity are offered.

Visual-memory search: an integrative perspective

A large, single-frame, visual-memory search experiment is reported in which memory and display loads of 1, 2, and 4 alphanumeric characters were factorially combined. In addition to the usual Consistent Mapping and Varied Mapping conditions, the experiment also involved a Categorical Varied Mapping condition in which different sets of stimuli switched roles as targets and distractors over trials. The stimuli used in these various mapping conditions were either digits, letters, or digits and letters. Analyses of the response time means obtained early and late in training suggest that the presence of categorical distinctions among the stimuli is the most important determinant of search efficiency. Comparison of the load effects on the response time means and on their standard deviations revealed a fairly constant ratio throughout the experimental conditions, which suggests that similar search processes may have been involved. A feature-based comparison model is indeed shown to account for the response time means obtained after extensive training under just about all training conditions, as well as for the ratios of load effects on means and standard deviations. According to the model, improvement in search efficiency results from a reduction in the number of features considered. The model's performance questions the necessity to postulate qualitative differences between controlled and automatic processing, while the experiment forces a reassessment of the importance of the consistent mapping that underlies dual-process theories.