How much memory does oculomotor search have?

Skills and cautiousness predict performance in difficult search

People differ in how well they search. What are the factors that might contribute to this variability? We tested the contribution of two cognitive abilities: visual working memory (VWM) capacity and object recognition ability. Participants completed three tasks: a difficult inefficient visual search task, where they searched for a target letter T among skewed L distractors; a VWM task, where they memorized a color array and then identified whether a probed color belonged to the previous array; and the Novel Object Memory Test (NOMT), where they learnt complex novel objects and then identified them amongst objects that closely resembled them. Exploratory and confirmatory factor analyses revealed that there are two latent factors that explain the shared variance among these three tasks: a factor indicative of the level of caution participants exercised during the challenging visual search task, and a factor representing their visual cognitive abilities. People who score high on the search cautiousness tend to perform a more accurate but slower search. People who score high on the visual cognitive ability factor tend to have a higher VWM capacity, a better object recognition ability, and a faster search speed. The results reflect two points: (1) Visual search tasks share components with visual working memory and object recognition tasks. (2) Search performance is influenced not only by the search display's properties but also by individual predispositions such as caution and general visual abilities. This study introduces new factors for consideration when interpreting variations in visual search behaviors.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Humans represent the precision and utility of information acquired across fixations

Our environment contains an abundance of objects which humans interact with daily, gathering visual information using sequences of eye-movements to choose which object is best-suited for a particular task. This process is not trivial, and requires a complex strategy where task affordance defines the search strategy, and the estimated precision of the visual information gathered from each object may be used to track perceptual confidence for object selection. This study addresses the fundamental problem of how such visual information is metacognitively represented and used for subsequent behaviour, and reveals a complex interplay between task affordance, visual information gathering, and metacogntive decision making. People fixate higher-utility objects, and most importantly retain metaknowledge about how much information they have gathered about these objects, which is used to guide perceptual report choices. These findings suggest that such metacognitive knowledge is important in situations where decisions are based on information acquired in a temporal sequence.

Maintaining rejected distractors in working memory during visual search depends on search stimuli: Evidence from contralateral delay activity

The presence of memory for rejected distractors during visual search has been heavily debated in the literature and has proven challenging to investigate behaviorally. In this research, we used an electrophysiological index of working memory (contralateral delay activity) to passively measure working memory activity during visual search. Participants were asked to indicate whether a novel target was present or absent in a lateralized search array with three visual set sizes (2, 4, or 6). If rejected distractors are maintained in working memory during search, working memory activity should increase with the number of distractors that need to be evaluated. Therefore, we predicted the amplitude of the contralateral delay activity would be larger for target-absent trials and would increase with visual set size until WM capacity was reached. In Experiment 1, we found no evidence for distractor maintenance in working memory during search for real-world stimuli. In Experiment 2, we found partial evidence in support of distractor maintenance during search for stimuli with high target/distractor similarity. In both experiments, working memory capacity did not appear to be a limiting factor during visual search. These results suggest the role of working memory during search may depend on the visual search task in question. Maintaining distractors in working memory appears to be unnecessary during search for realistic stimuli. However, there appears to be a limited role for distractor maintenance during search for artificial stimuli with a high degree of feature overlap.

Human visual search follows a suboptimal Bayesian strategy revealed by a spatiotemporal computational model and experiment

There is conflicting evidence regarding whether humans can make spatially optimal eye movements during visual search. Some studies have shown that humans can optimally integrate information across fixations and determine the next fixation location, however, these models have generally ignored the control of fixation duration and memory limitation, and the model results do not agree well with the details of human eye movement metrics. Here, we measured the temporal course of the human visibility map and performed a visual search experiment. We further built a continuous-time eye movement model that considers saccadic inaccuracy, saccadic bias, and memory constraints. We show that this model agrees better with the spatial and temporal properties of human eye movements and predict that humans have a memory capacity of around eight previous fixations. The model results reveal that humans employ a suboptimal eye movement strategy to find a target, which may minimize costs while still achieving sufficiently high search performance.

Visual Search: How Do We Find What We Are Looking For?

In visual search tasks, observers look for targets among distractors. In the lab, this often takes the form of multiple searches for a simple shape that may or may not be present among other items scattered at random on a computer screen (e.g., Find a red T among other letters that are either black or red.). In the real world, observers may search for multiple classes of target in complex scenes that occur only once (e.g., As I emerge from the subway, can I find lunch, my friend, and a street sign in the scene before me?). This article reviews work on how search is guided intelligently. I ask how serial and parallel processes collaborate in visual search, describe the distinction between search templates in working memory and target templates in long-term memory, and consider how searches are terminated.

Neural correlates of task-related refixation behavior

Eye movement research has shown that attention shifts from the currently fixated location to the next before a saccade is executed. We investigated whether the cost of the attention shift depends on higher-order processing at the time of fixation, in particular on visual working memory load differences between fixations and refixations on task-relevant items. The attention shift is reflected in EEG activity in the saccade-related potential (SRP). In a free viewing task involving visual search and memorization of multiple targets amongst distractors, we compared the SRP in first fixations versus refixations on targets and distractors. The task-relevance of targets implies that more information will be loaded in memory (e.g. both identity and location) than for distractors (e.g. location only). First fixations will involve greater memory load than refixations, since first fixations involve loading of new items, while refixations involve rehearsal of previously visited items. The SRP in the interval preceding the saccade away from a target or distractor revealed that saccade preparation is affected by task-relevance and refixation behavior. For task-relevant items only, we found longer fixation duration and higher SRP amplitudes for first fixations than for refixations over the occipital region and the opposite effect over the frontal region. Our findings provide first neurophysiological evidence that working memory loading of task-relevant information at fixation affects saccade planning.

Component processes in free-viewing visual search: Insights from fixation-aligned pupillary response averaging

Pupil size changes during a visual search may reflect cognitive processes, such as effort and memory accumulation, but methodological confounds and the general lack of literature in this area leave the reliability of findings open to question. We used a novel synthesis of experimental methods and averaging techniques to explore how cognitive processing unfolds during free-viewing visual search for multiple targets. Twenty-seven participants completed 152 searches across two separate 1-hour sessions. The number of targets present (Targets: 0, 1, 2, and 3) in each trial was the main manipulation and the task was to "find all of the targets" and report the total via mouse-click at the end of the trial. Search time lasted for 10 seconds or until the participant purported to have found all of the targets, in which case they could terminate the search via keypress. Whole-trial pupil analysis revealed a significant effect of button pressing as well as a significant main effect of targets for trials that were not self-terminated via button press. Fixation-aligned pupil responses revealed transient modulations in pupil size following initial fixations on targets but not distractors and refixations on both targets and distractors. Owing to rigorous control over experimental confounds and a detailed analysis and correction of eye-movement-related measurement error, we confidently discuss these findings in terms of task-related processing and underlying brain activity.

Medium versus difficult visual search: How a quantitative change in the functional visual field leads to a qualitative difference in performance

The dominant theories of visual search assume that search is a process involving comparisons of individual items against a target description that is based on the properties of the target in isolation. Here, we present four experiments that demonstrate that this holds true only in difficult search. In medium search it seems that the relation between the target and neighbouring items is also part of the target description. We used two sets of oriented lines to construct the search items. The cardinal set contained horizontal and vertical lines, the diagonal set contained left diagonal and right diagonal lines. In all experiments, participants knew the identity of the target and the line set used to construct it. In difficult search this knowledge allowed performance to improve in displays where only half of the search items came from the same line set as the target (50% eligibility), relative to displays where all items did (100% eligibility). However, in medium search, performance was actually poorer for 50% eligibility, especially on target-absent trials. This opposite effect of ineligible items in medium search and difficult search is hard to reconcile with theories based on individual items. It is more in line with theories that conceive search as a sequence of fixations where the number of items processed during a fixation depends on the difficulty of the search task: When search is medium, multiple items are processed per fixation. But when search is difficult, only a single item is processed.

Novelty competes with saliency for attention

A highly debated question in attention research is to what extent attention is biased by bottom-up factors such as saliency versus top-down factors as governed by the task. Visual search experiments in which participants are briefly familiarized with the task and then see a novel stimulus unannounced and for the first time support yet another factor, showing that novel and surprising features attract attention. In the present study, we tested whether gaze behavior as an indicator for attentional prioritization can be predicted accurately within displays containing both salient and novel stimuli by means of a priority map that assumes novelty as an additional source of activation. To that aim, we conducted a visual search experiment where a color singleton was presented for the first time in the surprise trial and manipulated the color-novelty of the remaining non-singletons between participants. In one group, the singleton was the only novel stimulus ("one-new"), whereas in another group, the non-singleton stimuli were likewise novel ("all-new"). The surprise trial was always target absent and designed such that top-down prioritization of any color was unlikely. The results show that the singleton in the all-new group captured the gaze less strongly, with more early fixations being directed to the novel non-singletons. Overall, the fixation pattern can accurately be explained by noisy priority maps where saliency and novelty compete for gaze control.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Refixation patterns reveal memory-encoding strategies in free viewing

We investigated visual working memory encoding across saccadic eye movements, focusing our analysis on refixation behavior. Over 10-s periods, participants performed a visual search for three, four, or five targets and remembered their orientations for a subsequent change-detection task. In 50% of the trials, one of the targets had its orientation changed. From the visual search period, we scored three types of refixations and applied measures for quantifying eye-fixation recurrence patterns. Repeated fixations on the same regions as well as repeated fixation patterns increased with memory load. Correct change detection was associated with more refixations on targets and less on distractors, with increased frequency of recurrence, and with longer intervals between refixations. The results are in accordance with the view that patterns of eye movement are an integral part of visual working memory representation.

Refixation control in free viewing: a specialized mechanism divulged by eye-movement-related brain activity

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eye-movement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. NEW & NOTEWORTHY A substantial proportion of eye fixations in human natural viewing behavior are revisits of recently visited locations, i.e., refixations. Our recently developed methods enabled us to study refixations in a free viewing visual search task, using combined eye movement and EEG recording. We identified in the EEG a distinctive refixation-related signal, signifying a control mechanism specific to refixations as opposed to ordinary eye fixations.

What do eye movements reveal about the role of memory in visual search?

Horowitz and Wolfe (1998, 2003) have challenged the view that serial visual search involves memory processes that keep track of already inspected locations. The present study used a search paradigm similar to Horowitz and Wolfe's (1998), comparing a standard static search condition with a dynamic condition in which display elements changed locations randomly every 111 ms. In addition to measuring search reaction times, observers’ eye movements were recorded. For target-present trials, the search rates were near-identical in the two search conditions, replicating Horowitz and Wolfe's findings. However, the number of fixations and saccade amplitude were larger in the static than in the dynamic condition, whereas fixation duration and the latency of the first saccade were longer in the dynamic condition. These results indicate that an active, memory-guided search strategy was adopted in the static condition, and a passive “sit-and-wait” strategy in the dynamic condition.

Semantic and Syntactic Associations During Word Search Modulate the Relationship Between Attention and Subsequent Memory

Two experiments were conducted to investigate how linguistic information influences attention allocation in visual search and memory for words. In Experiment 1, participants searched for the synonym of a cue word among five words. The distractors included one antonym and three unrelated words. In Experiment 2, participants were asked to judge whether the five words presented on the screen comprise a valid sentence. The relationships among words were sentential, semantically related or unrelated. A memory recognition task followed. Results in both experiments showed that linguistically related words produced better memory performance. We also found that there were significant interactions between linguistic relation conditions and memorization on eye-movement measures, indicating that good memory for words relied on frequent and long fixations during search in the unrelated condition but to a much lesser extent in linguistically related conditions. We conclude that semantic and syntactic associations attenuate the link between overt attention allocation and subsequent memory performance, suggesting that linguistic relatedness can somewhat compensate for a relative lack of attention during word search.

Modeling Lag-2 Revisits to Understand Trade-Offs in Mixed Control of Fixation Termination During Visual Search

An important question about eye-movement behavior is when the decision is made to terminate a fixation and program the following saccade. Different approaches have found converging evidence in favor of a mixed-control account, in which there is some overlap between processing information at fixation and planning the following saccade. We examined one interesting instance of mixed control in visual search: lag-2 revisits, during which observers fixate a stimulus, move to a different stimulus, and then revisit the first stimulus on the next fixation. Results show that the probability of lag-2 revisits occurring increased with the number of target-similar stimuli, and revisits were preceded by a brief fixation on the intervening distractor stimulus. We developed the Efficient Visual Sampling (EVS) computational model to simulate our findings (fixation durations and fixation locations) and to provide insight into mixed control of fixations and the perceptual, cognitive, and motor processes that produce lag-2 revisits.

Spatial inhibition of return as a function of fixation history, task, and spatial references

In oculomotor selection, each saccade is thought to be automatically biased toward uninspected locations, inhibiting the inefficient behavior of repeatedly refixating the same objects. This automatic bias is related to inhibition of return (IOR). Although IOR seems an appealing property that increases efficiency in visual search, such a mechanism would not be efficient in other tasks. Indeed, evidence for additional, more flexible control over refixations has been provided. Here, we investigated whether task demands implicitly affect the rate of refixations. We measured the probability of refixations after series of six binary saccadic decisions under two conditions: visual search and free viewing. The rate of refixations seems influenced by two effects. One effect is related to the rate of intervening fixations, specifically, more refixations were observed with more intervening fixations. In addition, we observed an effect of task set, with fewer refixations in visual search than in free viewing. Importantly, the history-related effect was more pronounced when sufficient spatial references were provided, suggesting that this effect is dependent on spatiotopic encoding of previously fixated locations. This known history-related bias in gaze direction is not the primary influence on the refixation rate. Instead, multiple factors, such as task set and spatial references, assert strong influences as well.

The impending demise of the item in visual search

The way the cognitive system scans the visual environment for relevant information - visual search in short - has been a long-standing central topic in vision science. From its inception as a research topic, and despite a number of promising alternative perspectives, the study of visual search has been governed by the assumption that a search proceeds on the basis of individual items (whether processed in parallel or not). This has led to the additional assumptions that shallow search slopes (at most a few tens of milliseconds per item for target-present trials) are most informative about the underlying process, and that eye movements are an epiphenomenon that can be safely ignored. We argue that the evidence now overwhelmingly favours an approach that takes fixations, not individual items, as its central unit. Within fixations, items are processed in parallel, and the functional field of view determines how many fixations are needed. In this type of theoretical framework, there is a direct connection between target discrimination difficulty, fixations, and reaction time (RT) measures. It therefore promises a more fundamental understanding of visual search by offering a unified account of both eye movement and manual response behaviour across the entire range of observed search efficiency, and provides new directions for research. A high-level conceptual simulation with just one free and four fixed parameters shows the viability of this approach.

Comparing target detection errors in visual search and manually-assisted search

Subjects searched for low- or high-prevalence targets among static nonoverlapping items or items piled in heaps that could be moved using a computer mouse. We replicated the classical prevalence effect both in visual search and when unpacking items from heaps, with more target misses under low prevalence. Moreover, we replicated our previous finding that while unpacking, people often move the target item without noticing (the unpacking error) and determined that these errors also increase under low prevalence. On the basis of a comparison of item movements during the manually-assisted search and eye movements during static visual search, we suggest that low prevalence leads to broadly reduced diligence during search but that the locus of this reduced diligence depends on the nature of the task. In particular, while misses during visual search often arise from a failure to inspect all of the items, misses during manually-assisted search more often result from a failure to adequately inspect individual items. Indeed, during manually-assisted search, over 90 % of target misses occurred despite subjects having moved the target item during search.

Recurrence quantification analysis of eye movements

Recurrence quantification analysis (RQA) has been successfully used for describing dynamic systems that are too complex to be characterized adequately by standard methods in time series analysis. More recently, RQA has been used for analyzing the coordination of gaze patterns between cooperating individuals. Here, we extend RQA to the characterization of fixation sequences, and we show that the global and local temporal characteristics of fixation sequences can be captured by a small number of RQA measures that have a clear interpretation in this context. We applied RQA to the analysis of a study in which observers looked at different scenes under natural or gaze-contingent viewing conditions, and we found large differences in the RQA measures between the viewing conditions, indicating that RQA is a powerful new tool for the analysis of the temporal patterns of eye movement behavior.

A common discrete resource for visual working memory and visual search

Visual search, a dominant paradigm within attention research, requires observers to rapidly identify targets hidden among distractors. Major models of search presume that working memory (WM) provides the on-line work space for evaluating potential targets. According to this hypothesis, individuals with higher WM capacity should search more efficiently, because they should be able to apprehend a larger number of search elements at a time. Nevertheless, no compelling evidence of such a correlation has emerged, and this null result challenges a growing consensus that there is strong overlap between the neural processes that limit internal storage and those that limit external selection. Here, we provide multiple demonstrations of robust correlations between WM capacity and search efficiency, and we document a key boundary condition for observing this link. Finally, examination of a neural measure of visual selection capacity (the N2pc) demonstrates that visual search and WM storage are constrained by a common discrete resource.

Inhibition of return: Twenty years after

When responding to a suddenly appearing stimulus, we are slower and/or less accurate when the stimulus occurs at the same location of a previous event than when it appears in a new location. This phenomenon, often referred to as inhibition of return (IOR), has fostered a huge amount of research in the last 20 years. In this selective review, which introduces a Special Issue of Cognitive Neuropsychology dedicated to IOR, we discuss some of the methods used for eliciting IOR and its boundary conditions. We also address its debated relationships with orienting of attention, succinctly review findings of altered IOR in normal elderly and neuropsychiatric patients, and present results concerning its possible neural bases. We conclude with an outline of the papers collected in this issue, which offer a more in-depth treatment of behavioural, neural, and theoretical issues related to IOR.

Visual and memory search in complex environments: determinants of eye movements and search performance

Previous research on visual and memory search revealed various top down and bottom up factors influencing performance. However, utilising abstract stimuli (e.g. geometrical shapes or letters) and focussing on individual factors has often limited the applicability of research findings. Two experiments were designed to analyse which attributes of a product facilitate search in an applied environment. Participants scanned displays containing juice packages while their eye movements were recorded. The familiarity, saliency, and position of search targets were systematically varied. Experiment 1 involved a visual search task, whereas Experiment 2 focussed on memory search. The results showed that bottom up (target saliency) and top down (target familiarity) factors strongly interacted. Overt visual attention was influenced by cultural habits, purposes, and current task demands. The results provide a solid database for assessing the impact and interplay of fundamental top down and bottom up determinants of search processes in applied fields of psychology. Practitioner Summary: Our study demonstrates how a product (or a visual item in general) needs to be designed and placed to ensure that it can be found effectively and efficiently within complex environments. Corresponding product design should result in faster and more accurate visual and memory based search processes.

Guidance of visual search by memory and knowledge

To behave intelligently in the world, humans must be able to find objects efficiently within the complex environments they inhabit. A growing proportion of the literature on visual search is devoted to understanding this type of natural search. In the present chapter, I review the literature on visual search through natural scenes, focusing on the role of memory and knowledge in guiding attention to task-relevant objects.

Active inhibition and memory promote exploration and search of natural scenes

Active exploration of the visual world depends on sequential shifts of gaze that bring prioritized regions of a scene into central vision. The efficiency of this system is commonly attributed to a mechanism of "inhibition of return" (IOR) that discourages re-examination of previously-visited locations. Such a process is fundamental to computational models of attentional selection and paralleled by neurophysiological observations of inhibition of target-related activity in visuomotor areas. However, studies examining eye movements in naturalistic visual scenes appear to contradict the hypothesis that IOR promotes exploration. Instead, these reports reveal a surprisingly strong tendency to shift gaze back to the previously fixated location, suggesting that refixations might even be facilitated under natural conditions. Here we resolve this apparent contradiction, based on a probabilistic analysis of gaze patterns recorded during both free-viewing and search of naturalistic scenes. By simulating saccadic selection based on instantaneous influences alone, we show that the observed frequency of return saccades is in fact substantially less than predicted for a memoryless system, demonstrating that refixation is actively inhibited under natural viewing conditions. Furthermore, these observations reveal that gaze history significantly influences the way in which natural scenes are explored, contrary to accounts that suggest visual search has no memory.

Incidental learning speeds visual search by lowering response thresholds, not by improving efficiency: evidence from eye movements

When observers search for a target object, they incidentally learn the identities and locations of "background" objects in the same display. This learning can facilitate search performance, eliciting faster reaction times for repeated displays. Despite these findings, visual search has been successfully modeled using architectures that maintain no history of attentional deployments; they are amnesic (e.g., Guided Search Theory). In the current study, we asked two questions: 1) under what conditions does such incidental learning occur? And 2) what does viewing behavior reveal about the efficiency of attentional deployments over time? In two experiments, we tracked eye movements during repeated visual search, and we tested incidental memory for repeated nontarget objects. Across conditions, the consistency of search sets and spatial layouts were manipulated to assess their respective contributions to learning. Using viewing behavior, we contrasted three potential accounts for faster searching with experience. The results indicate that learning does not result in faster object identification or greater search efficiency. Instead, familiar search arrays appear to allow faster resolution of search decisions, whether targets are present or absent.

Visual Search in the Real World: Evidence for the Formation of Distractor Representations

Visual search in the real world often requires that we search the same environment a number of times for different targets. What is the fate of information about fixated distractor objects during these searches? Here, participants searched the same array of real objects on a tabletop twice for two different targets successively whilst wearing a head-mounted eye-tracker. We found that fixating an object when it was a distractor in the first search facilitated search for that same object when it became the target in the second search. The results suggest that the location and identity of fixated distractor objects are represented to a level that guides subsequent searches, even when this information is not required at the time of fixation.

Location and identity memory of saccade targets

While the memory of objects' identity and of their spatiotopic location may sustain transsaccadic spatial constancy, the memory of their retinotopic location may hamper it. Is it then true that saccades perturb retinotopic but not spatiotopic memory? We address this issue by assessing localization performances of the last and of the penultimate saccade target in a series of 2-6 saccades. Upon fixation, nine letter-pairs, eight black and one white, were displayed at 3° eccentricity around fixation within a 20° × 20° grey frame, and subjects were instructed to saccade to the white letter-pair; the cycle was then repeated. Identical conditions were run with the eyes maintaining fixation throughout the trial but with the grey frame moving so as to mimic its retinal displacement when the eyes moved. At the end of a trial, subjects reported the identity and/or the location of the target in either retinotopic (relative to the current fixation dot) or frame-based(1) (relative to the grey frame) coordinates. Saccades degraded target's retinotopic location memory but not its frame-based location or its identity memory. Results are compatible with the notion that spatiotopic representation takes over retinotopic representation during eye movements thereby contributing to the stability of the visual world as its retinal projection jumps on our retina from saccade to saccade.

Rapid Communication: Finding memory in search: The effect of visual working memory load on visual search

There is now substantial evidence that during visual search, previously searched distractors are stored in memory to prevent them from being reselected. Studies examining which memory resources are involved in this process have indicated that while a concurrent spatial working memory task does affect search slopes, depleting visual working memory (VWM) resources does not. In the present study, we confirm that VWM load indeed has no effect on the search slope; however, there is an increase in overall reaction times that is directly related to the number of items held in VWM. Importantly, this effect on search time increases proportionally with the memory load until the capacity of VWM is reached. Furthermore, the search task interfered with the number of items stored in VWM during the concurrent change-detection task. These findings suggest that VWM plays a role in the inhibition of previously searched distractors.

Overlapping functional anatomy for working memory and visual search

Recent behavioural findings using dual-task paradigms demonstrate the importance of both spatial and non-spatial working memory processes in inefficient visual search (Anderson et al. in Exp Psychol 55:301–312, 2008). Here, using functional magnetic resonance imaging (fMRI), we sought to determine whether brain areas recruited during visual search are also involved in working memory. Using visually matched spatial and non-spatial working memory tasks, we confirmed previous behavioural findings that show significant dual-task interference effects occur when inefficient visual search is performed concurrently with either working memory task. Furthermore, we find considerable overlap in the cortical network activated by inefficient search and both working memory tasks. Our findings suggest that the interference effects observed behaviourally may have arisen from competition for cortical processes subserved by these overlapping regions. Drawing on previous findings (Anderson et al. in Exp Brain Res 180:289–302, 2007), we propose that the most likely anatomical locus for these interference effects is the inferior and middle frontal cortex of the right hemisphere. These areas are associated with attentional selection from memory as well as manipulation of information in memory, and we propose that the visual search and working memory tasks used here compete for common processing resources underlying these mechanisms.

Why is Information Displaced from Visual Working Memory during Visual Search?

Research has shown that performing visual search while maintaining representations in visual working memory displaces up to one object's worth of information from memory. This memory displacement has previously been attributed to a nonspecific disruption of the memory representation by the mere presentation of the visual search array, and the goal of the present study was to determine whether it instead reflects the use of visual working memory in the actual search process. The first hypothesis tested was that working memory displacement occurs because observers preemptively discard about an object's worth of information from visual working memory in anticipation of performing visual search. Second, we tested the hypothesis that on target-absent trials no information is displaced from visual working memory because no target is entered into memory when search is completed. Finally, we tested whether visual working memory displacement is due to the need to select a response to the search array. The findings rule out these alternative explanations. The present study supports the hypothesis that change-detection performance is impaired when a search array appears during the retention interval due to nonspecific disruption or masking.

The mechanism underlying inhibition of saccadic return

Human observers take longer to re-direct gaze to a previously fixated location. Although there has been some exploration of the characteristics of inhibition of saccadic return (ISR), the exact mechanisms by which ISR operates are currently unknown. In the framework of accumulation models of response times, in which evidence is integrated over time to a response threshold, ISR could reflect a reduction in the rate of accumulation for saccades to return locations or an increase in the effective criterion for response. In two experiments, participants generated sequences of three saccades, in response to a peripheral or a central cue. ISR occurred across these manipulations: saccade latency was consistently increased for movements to the immediately previously fixated location. Latency distributions from individual observers were fit with a Linear Ballistic Accumulator model. ISR was best accounted for as a change in the accumulation rate. We suggest this parameter represents the overall desirability of a particular course of action, the evidence for which may be derived from a variety of sensory and non-sensory sources.

A Role for Spatial and Nonspatial Working Memory Processes in Visual Search

Searching a cluttered visual scene for a specific item of interest can take several seconds to perform if the target item is difficult to discriminate from surrounding items. Whether working memory processes are utilized to guide the path of attentional selection during such searches remains under debate. Previous studies have found evidence to support a role for spatial working memory in inefficient search, but the role of nonspatial working memory remains unclear. Here, we directly compared the role of spatial and nonspatial working memory for both an efficient and inefficient search task. In Experiment 1, we used a dual-task paradigm to investigate the effect of performing visual search within the retention interval of a spatial working memory task. Importantly, by incorporating two working memory loads (low and high) we were able to make comparisons between dual-task conditions, rather than between dual-task and single-task conditions. This design allows any interference effects observed to be attributed to changes in memory load, rather than to nonspecific effects related to “dual-task” performance. We found that the efficiency of the inefficient search task declined as spatial memory load increased, but that the efficient search task remained efficient. These results suggest that spatial memory plays an important role in inefficient but not efficient search. In Experiment 2, participants performed the same visual search tasks within the retention interval of visually matched spatial and verbal working memory tasks. Critically, we found comparable dual-task interference between inefficient search and both the spatial and nonspatial working memory tasks, indicating that inefficient search recruits working memory processes common to both domains.