Visual search has memory

Prioritization of new objects in real-world scenes: evidence from eye movements

The authors examined the prioritization of abruptly appearing objects in real-world scenes by measuring the eyes' propensity to be directed to the new object. New objects were fixated more often than chance whether they appeared during fixations (transient onsets) or saccades (nontransient onsets). However, onsets that appeared during fixations were fixated sooner and more often than those coincident with saccades. Prioritization of onsets during saccades, but not fixations, were affected by manipulations of memory: Reducing scene viewing time prior to the onset eliminated prioritization, whereas prior study of the scenes increased prioritization. Transient objects draw attention quickly and do not depend on memory, but without a transient signal, new objects are prioritized over several saccades as memory is used to explicitly identify the change. These effects were not modulated by observers' expectations concerning the appearance of new objects, suggesting the prioritization of a transient is automatic and that memory-guided prioritization is implicit.

Inhibition of return: Sensitivity and criterion as a function of response time

Inhibition of return (IOR) refers to a mechanism that results in a performance disadvantage typically observed when targets are presented at a location once occupied by a cue. Although the time course of the phenomenon--from the cue to the target--has been well studied, the time course of the effect--from target to response--is unknown. In 2 experiments, the effect of IOR upon sensitivity and response criterion under different levels of speed stress was examined. In go/no-go and choice reaction time tasks, IOR had at least 2 distinct effects on information processing. Early in target processing, before sufficient target information has accrued, there is a bias against responding to cued targets. Later, as target information is allowed to accrue, IOR reduces sensitivity to the target's nonspatial feature. Three accounts relating to the early bias effect of IOR and the late effect of IOR on sensitivity are offered.

Memory hazard functions: A vehicle for theory development and test

A framework is developed to rigorously test an entire class of memory retention functions by examining hazard properties. Evidence is provided that the memory hazard function is not monotonically decreasing. Yet most of the proposals for retention functions, which have emerged from the psychological literature, imply that memory hazard is monotonically decreasing over the entire temporal range. Furthermore, the few remaining proposals, that do not have monotonically decreasing hazard, have difficulty fitting data over both short-term and long-term intervals. A new 2-trace hazard model is developed that successfully circumvents these difficulties. This new model is used to account for the size of memory span and the time course of proactive and retroactive interference effects. The model can fit the retention characteristics of H. M., the famous amnesic patient, as well as normal experimental participants. The model is also used to account for the time course of the misinformation effect.

Eye scanning of multi-element displays: I. Scanpath planning

We recorded oculomotor scanpaths in a task that required individuals to scan through displays consisting of a small number (between 3 and 12) of near-identical items. The task required each item to be fixated at least once and our objective was to explore the principles governing the generation of scanpaths. In general the observers carried out the task efficiently, although omissions occurred quite frequently (about 25% of trials) in the 12-item case. Backtracking occurred rarely except in the case of immediate rescanning back to the previously fixated item. Such immediate backtracking occurred on about 4% of fixations and, in contrast to more distant backtracking, was not associated with increased errors. Evidence was found for both directional (raster-like) scanning strategies and scanning strategies based on the global external contour.

Search, enumeration, and aging: eye movement requirements cause age-equivalent performance in enumeration but not in search tasks

Response time (RT) and fixation frequency were measured for young and older adults in search and enumeration tasks under high- and low-attentional competition conditions. There was an age-related decrement in search rates for single targets both for RT and fixation frequency, but there was no deficit in enumeration rates either with or without distractors even though serial enumeration rates were much slower than single target search rates. Fixation frequency per item in serial enumeration was much greater than in serial visual search for a single target. Enumerating targets with distractors produced an overall increase in RT and fixation frequency that was greater for older adults. The data are contrary to a generalized slowing account, and an alternative is proposed on the basis of the need to make eye movements in enumeration but not in search tasks.

Associative learning improves visual working memory performance

The ability to remember visual stimuli over a short delay period is limited by the small capacity of visual working memory (VWM). Here the authors investigate the role of learning in enhancing VWM. Participants saw 2 spatial arrays separated by a 1-s interval. The 2 arrays were identical except for 1 location. Participants had to detect the difference. Unknown to the participants, some spatial arrays would repeat once every dozen trials or so for up to 32 repetitions. Spatial VWM performance increased significantly when the same location changed across display repetitions, but not at all when different locations changed from one display repetition to another. The authors suggest that a major role of learning in VWM is to mediate which information gets retained, rather than to directly increase VWM capacity.

Rapid Resumption of Interrupted Visual Search: New Insights on the Interaction Between Vision and Memory

A modified visual search task demonstrates that humans are very good at resuming a search after it has been momentarily interrupted. This is shown by exceptionally rapid response time to a display that reappears after a brief interruption, even when an entirely different visual display is seen during the interruption and two different visual searches are performed simultaneously. This rapid resumption depends on the stability of the visual scene and is not due to display or response anticipations. These results are consistent with the existence of an iterative hypothesis-testing mechanism that compares information stored in short-term memory (the perceptual hypothesis) with information about the display (the sensory pattern). In this view, rapid resumption occurs because a hypothesis based on a previous glance of the scene can be tested very rapidly in a subsequent glance, given that the initial hypothesis-generation step has already been performed.

Incidental visual memory for targets and distractors in visual search

We explored incidental retention of visual details of encountered objects during search. Participants searched for conjunction targets in 32 arrays of 12 pictures of real-world objects and then performed a token discrimination task that examined their memory for visual details of the targets and distractors from the search task. The results indicate that even though participants had not been instructed to memorize the objects, the visual details of search targets and distractor objects related to the targets were retained after the search. Distractor objects unrelated to the search target were remembered more poorly. Eye-movement measures indicated that the objects that were remembered were looked at more frequently during search than those that were not remembered. These results provide support that detailed visual information is included incidentally in the visual representation of an object after the object is no longer in view.

Oculomotor correlates of context-guided learning in visual search

Previous studies have shown that context-facilitated visual search can occur through implicit learning. In the present study, we have explored its oculomotor correlates as a step toward unraveling the mechanisms that underlie such learning. Specifically, we examined a number of oculomotor parameters that might accompany the learning of context-guided search. The results showed that a decrease in the number of saccades occurred along with a fall in search time. Furthermore, we identified an effective search period in which each saccade monotonically brought the fixation closer to the target. Most important, the speed with which eye fixation approached the target did not change as a result of learning. We discuss the general implications of these results for visual search.

Automatic and intentional memory processes in visual search

Previous research has indicated that saccade target selection during visual search is influenced by scanning history. Already inspected items are less likely to be chosen as saccade targets as long as the number intervening saccades is small. Here, we adapted Jacoby's (1991) process dissociation procedure to assess the role of intentional and automatic processes in saccade target selection. Results indicate a large automatic component biasing participants to move their eyes to unexamined locations. However, an intentional component allowed participants to both reinspect old items and aid their selection of new items. A second experiment examined inhibition of return (IOR) as a candidate for the observed automatic component. IOR was found for items that had been previously examined. It is concluded that both automatic and intentional memory traces are available to guide the eyes during search.

Inhibition of return with rapid serial shifts of attention: implications for memory and visual search

Horowitz and Wolfe (2001) suggested that inhibition of return (IOR) should not be observed in tasks that involve rapid deployments of attention. To examine this issue, five of six possible locations were sequentially cued with either short-duration peripheral cues (50 msec) or long-duration peripheral cues (500 msec). As was expected, IOR was observed in the first two experiments at every cued location with the long-duration cues, with the magnitude of IOR decreasing for earlier cued locations relative to later cued locations. In the short-cue condition, IOR was observed at only one cued location (the second to last). The pattern of results for the short-duration cues was found regardless of whether the fixation cue was of a short (Experiment 1) or a long (Experiment 2) duration. In Experiment 3, the final fixation cue was removed, and IOR was again observed at virtually all locations in both the short- and the long-cue conditions. These findings indicate that IOR can be observed at multiple locations when attention is shifted rapidly between locations.

Search for multiple targets: Evidence for memory-based control of attention

There are two opposing models with regard to the function of memory in visual search: a memory-driven model and a memory-free model. Recently, Horowitz and Wolfe (2001) investigated a multiple-target search task. Participants were required to decide whether or not there were at least n targets present. They demonstrated that the reaction time x n function has a positive and accelerated curve. They argued that the memory-free model predicts this curve, whereas the memory-driven model predicts a linear function. In this study, I varied the total set sizes of a multiple-target search task and fitted the models separately for each n condition. The model fit indicated that the memory-driven model is more appropriate than the memory-free model in each n condition. These results suggest that an amnesic process does not cause the positive accelerated curve of the reaction time x n function but that it is the result of the time needed to examine each additional n item.

The multi-item localization (MILO) task: Measuring the spatiotemporal context of vision for action

We describe a new multi-item localization task that can be used to probe the temporal and spatial contexts of search-like behaviors. A sequence of four target letters (e.g., E, F, G, and H) was presented among four distractor letters. Observers located the targets in order. Both retrospective and prospective components of performance were examined. The retrospective component was assessed by having target items either vanish or remain once they had been located. This manipulation had little effect on search performance, suggesting that old target items can be efficiently ignored. The prospective component was assessed by shuffling future target and distractor locations after each response. This manipulation revealed that observers typically plan ahead at least one target into the future. However, even when observers cannot plan ahead, they are still able to ignore old targets. These findings suggest that both "what you did" and "what you intend to do" can influence the localization and selection of targets.

Partitioning visual displays aids task-directed visual search

We reduced time to detect target symbols in mock radar screens by adding perceptual boundaries that partitioned displays in accordance with task instructions. Targets appeared among distractor symbols either close to or far from the display center, and participants were instructed to find the target closest to the center. Search time increased with both number of distractors and distance of target from center. However, when close and far regions were delineated by a centrally presented "range ring," the distractor effect was substantially reduced. In addition, eye movement patterns more closely resembled a task-efficient spiral when displays contained a range ring. Results suggest that the addition of perceptual boundaries to visual displays can help to guide search in accordance with task-directed constraints. Actual or potential applications of this research include the incorporation of perceptual boundaries into display designs in order to encourage task-efficient scanpaths (as identified via task analysis and/or empirical testing).

How much memory does oculomotor search have?

Research has demonstrated that oculomotor visual search is guided by memory for which items or locations within a display have already been inspected. In the study reported here, we used a gaze-contingent search paradigm to examine properties of this memory. Data revealed a memory buffer for search history of three to four items. This buffer was effected in part by a space-based trace attached to a location independently of whether the object that had been seen at that position remained visible, and was subject to interference from other stimuli seen in the course of a trial.

The role of spatial working memory in inhibition of return: Evidence from divided attention tasks

Inhibition of return (IOR) refers to a bias against returning attention to a location that has been recently attended. In the present experiments, we examined the role of working memory in IOR by introducing secondary tasks (in the temporal interval between the cue and the target) that involved a working memory component. When the secondary task was nonspatial in nature (monitoring odd digits or adding digits), IOR was present, although overall reaction times were greater in the presence of the secondary task. When the task involved a spatial working memory load (remembering the directionality of arrows or the orientation of objects), IOR was eliminated. However, when the participants had incentive to process the directionality of an arrow but did not have to use any memory system, IOR persisted at peripheral locations. Overall, the results suggest that IOR is partially mediated by a spatial working memory system.

Sit-and-wait strategies in dynamic visual search

The role of memory in visual search has lately become a controversial issue. Horowitz and Wolfe (1998) observed that performance in a visual search task was little affected by whether the stimuli were static or randomly relocated every 111 ms. Because a memory-based mechanism, such as inhibition of return, would be of no use in the dynamic condition, Horowitz and Wolfe concluded that memory is likewise not involved in the static condition. However, Horowitz and Wolfe could not effectively rule out the possibility that observers adopted a different strategy in the dynamic condition than in the static condition. That is, in the dynamic condition observers may have attended to a subregion of the display and waited for the target to appear there (sit-and-wait strategy). This hypothesis is supported by experimental data showing that performance in their dynamic condition does not differ from performance in another dynamic condition in which observers are forced to adopt a sit-and-wait strategy by being presented with a limited region of the display only.

Contingent visual marking by transients

Preview search is a phenomenon in which a set of new items can be searched with seemingly no interference from items already present in the display. The preview effect has been shown to occur only when the presentation of the new items is accompanied by a luminance change (Donk & Theeuwes, 2001). In a series of experiments, we extend the type of transients that can lead to a preview benefit to offsets and motion, and confirm Donk and Theeuwes's finding that equiluminant color changes do not lead to a preview effect. Like Donk and Theeuwes, we find that preview search does not occur when only the old items undergo a transient change, suggesting that the processes responsible for preview search are triggered when the new items undergo a change detectable by the magnocellular system. In addition, we find that irrelevant transients interfere with preview search only when they match the current attentional set (e.g., luminance change or motion). Results suggest that preview search is not the automatic capture of attention by transients, but rather is contingent on top-down control settings.

Visual search is facilitated by scene and sequence familiarity in rhesus monkeys

Whether searching for targets in a familiar scene leads to improved performance was tested in monkeys. We found that search performance improved for a familiar scene when target locations were always randomized. However, when target locations repeatedly followed a predictable sequence, performance improvement for a familiar scene was manifested only for targets presented in a familiar sequence, suggesting that scene memory might be masked by the learning of target sequences. These results suggest that information about a visual scene can facilitate the performance of visual search, and that this memory is coupled to the learned sequence of target locations.

Failures of retrieval and comparison constrain change detection in natural scenes

In a change detection paradigm, a target object in a natural scene either rotated in depth, was replaced by another object token, or remained the same. Change detection performance was reliably higher when a target postcue allowed participants to restrict retrieval and comparison processes to the target object (Experiment 1). Change detection performance remained excellent when the target object was not attended at change (Experiment 2) and when a concurrent verbal working memory load minimized the possibility of verbal encoding (Experiment 3). Together, these data demonstrate that visual representations accumulate in memory from attended objects as the eyes and attention are oriented within a scene and that change blindness derives, at least in part, from retrieval and comparison failure.

Serial deployment of attention during visual search

This study examined whether objects are attended in serial or in parallel during a demanding visual search task. A component of the event-related potential waveform, the N2pc wave, was used as a continuous measure of the allocation of attention to possible targets in the search arrays. Experiment 1 demonstrated that the relative allocation of attention shifts rapidly, favoring one item and then another. In Experiment 2, a paradigm was used that made it possible to track the absolute allocation of attention to individual items. This experiment showed that attention was allocated to one object for 100-150 ms before attention began to be allocated to the next object. These findings support models of attention that posit serial processing in demanding visual search tasks.

Moving towards solutions to some enduring controversies in visual search

How do we find a target item in a visual world filled with distractors? A quarter of a century ago, in her influential 'Feature Integration Theory (FIT)', Treisman proposed a two-stage solution to the problem of visual search: a preattentive stage that could process a limited number of basic features in parallel and an attentive stage that could perform more complex acts of recognition, one object at a time. The theory posed a series of problems. What is the nature of that preattentive stage? How do serial and parallel processes interact? How does a search unfold over time? Recent work has shed new light on these issues.

Visual search in long-term cannabis users with early age of onset

The present research tested the hypothesis that there is a specific deficit in visual scanning in chronic users of cannabis with early onset of their drug consumption (age 14 to 16). 17 users and 20 control participants were asked to search for targets on a 5 x 5 stimulus array while their eye movements were monitored. Cannabis users showed less effective search behavior, including longer response times and more fixations at about the same error level. Search patterns were more conservative and included more frequent reinspections of previously fixated areas. In sum, the results point to two loci of adverse effects: an impairment in visual short-term memory, and less effective visual processing at a more strategic, top down controlled level.

Visual search in patients with left visual hemineglect

In patients with hemi-spatial neglect eye movement patterns during visual search reflect not only inattention for the contralesional hemi-field, but interacting deficits of multiple visuo-spatial and cognitive functions, even in the ipsilesional hemi-field. Evidence for these deficits is presented from the literature and from saccadic scan-path analysis during feature and conjunction search in 10 healthy subjects and in 10 patients with manifest or recovered left visual neglect due to right-hemispheric stroke. Deficits include (1) a rightward shift of spatial representation, (2) deficient spatial working memory and failure of systematic search strategies, leading to multiple re-fixations, more after frontal lesions, and (3) a reduced spotlight of attention and a deficient pop-out effect of color, more after temporo-parietal lesions.

Probability cuing of target location facilitates visual search implicitly in normal participants and patients with hemispatial neglect

We explored how variability in the probability of target locations affects visual search in normal individuals and in patients with hemispatial neglect, a deficit in attending to the contralesional side of space. Young and elderly normal participants responded faster when targets appeared in the more probable region than when targets appeared in the less probable region. Similarly, patients were sensitive to the distribution of targets, even in the neglected field. Although the attentional gradient that characterizes neglect was not eliminated, the response facilitation due to the probability distribution was proportionate to that of control participants and equal in magnitude across the neglected field. All participants exploited the uneven distribution of targets to enhance task performance without explicit instructions to do so or awareness of biases in their behavior. These results suggest that attentional orientation and sensitivity to external probabilities are possibly dissociable. An early sensory and a late motor mechanism are postulated as possibly being involved in the observed probability-matching behavior of participants.

Temporal integration between visual images and visual percepts

Using a temporal integration task, subjects in 5 experiments were expected to combine information from temporally separated visual presentations. Evidence from these experiments indicated that perceptual information can be integrated with previously generated and currently maintained visual images to form a representation that contains information from each source. Properties and limitations of this integration process were also explored, including the time required to generated the image, the speed at which percepts were integrated with images, and the capacity of the representation. Implications for theories of visual processing and memory are discussed.