Inhibition of return with rapid serial shifts of attention: implications for memory and visual search

Activated long-term memory and visual working memory during hybrid visual search: Effects on target memory search and distractor memory

In hybrid visual search, observers must maintain multiple target templates and subsequently search for any one of those targets. If the number of potential target templates exceeds visual working memory (VWM) capacity, then the target templates are assumed to be maintained in activated long-term memory (aLTM). Observers must search the array for potential targets (visual search), as well as search through memory (target memory search). Increasing the target memory set size reduces accuracy, increases search response times (RT), and increases dwell time on distractors. However, the extent of observers' memory for distractors during hybrid search is largely unknown. In the current study, the impact of hybrid search on target memory search (measured by dwell time on distractors, false alarms, and misses) and distractor memory (measured by distractor revisits and recognition memory of recently viewed distractors) was measured. Specifically, we aimed to better understand how changes in behavior during hybrid search impacts distractor memory. Increased target memory set size led to an increase in search RTs, distractor dwell times, false alarms, and target identification misses. Increasing target memory set size increased revisits to distractors, suggesting impaired distractor location memory, but had no effect on a two-alternative forced-choice (2AFC) distractor recognition memory test presented during the search trial. The results from the current study suggest a lack of interference between memory stores maintaining target template representations (aLTM) and distractor information (VWM). Loading aLTM with more target templates does not impact VWM for distracting information.

Visual search in neurodevelopmental disorders: evidence towards a continuum of impairment

Disorders with neurodevelopmental aetiology such as Attention-Deficit/Hyperactivity Disorder (ADHD), Autism Spectrum Disorder (ASD) and Schizophrenia share commonalities at many levels of investigation despite phenotypic differences. Evidence of genetic overlap has led to the concept of a continuum of neurodevelopmental impairment along which these disorders can be positioned in aetiological, pathophysiological and developmental features. This concept requires their simultaneous comparison at different levels, which has not been accomplished so far. Given that cognitive impairments are core to the pathophysiology of these disorders, we provide for the first time differentiated head-to-head comparisons in a complex cognitive function, visual search, decomposing the task with eye movement-based process analyses. N = 103 late-adolescents with schizophrenia, ADHD, ASD and healthy controls took a serial visual search task, while their eye movements were recorded. Patients with schizophrenia presented the greatest level of impairment across different phases of search, followed by patients with ADHD, who shared with patients with schizophrenia elevated intra-subject variability in the pre-search stage. ASD was the least impaired group, but similar to schizophrenia in post-search processes and to schizophrenia and ADHD in pre-search processes and fixation duration while scanning the items. Importantly, the profiles of deviancy from controls were highly correlated between all three clinical groups, in line with the continuum idea. Findings suggest the existence of one common neurodevelopmental continuum of performance for the three disorders, while quantitative differences appear in the level of impairment. Given the relevance of cognitive impairments in these three disorders, we argue in favour of overlapping pathophysiological mechanisms.

Reward Weakened Inhibition of Return (IOR) in the Near Depth Plane

In attentional orienting, researchers have proposed that reward history is a component of attentional control, as the reward value might enhance the spatial attention process to achieve more efficient goal-directed behavior and to improve target-detection performance. Although the effect of reward-induced motivation on attentional orienting has been studied in two-dimensional (2-D) space, the specific mechanisms underlying the influence of reward on inhibition of return (IOR) of attentional orienting in three-dimensional space (3-D) remain unclear. In the present study, by incorporating the Posner spatial-cueing paradigm into a virtual 3-D environment, we aimed to investigate the influence of reward on IOR in 3-D space. The results showed the following: (1) IOR size in the rewarded conditions was smaller than IOR size in the unrewarded condition in the near depth plane, resulting in an IOR difference with or without reward. (2) Reward weakened IOR in the near depth plane because the response to the uncued location was delayed, not because the response to the cued location was accelerated. The present study indicated that the different depth planes of the target location in 3-D space could influence the interaction between reward and IOR, and reward weakened IOR in the near depth plane.

Spatial Inhibition of Return Affected by Self-Prioritization Effect in Three-Dimensional Space

Spatial inhibition of return (IOR) being affected by the self-prioritization effect (SPE) in a two-dimensional plane has been well documented. However, it remains unknown how the spatial IOR interacts with the SPE in three-dimensional (3D) space. By constructing a virtual 3D environment, Posner's classically two-dimensional cue-target paradigm was applied to a 3D space. Participants first associated labels for themselves, their best friends, and strangers with geometric shapes in a shape-label matching task, then performed Experiment 1 (referential information appeared as the cue label) and Experiment 2 (referential information appeared as the target label) to investigate whether the IOR effect could be influenced by the SPE in 3D space. This study showed that when the cue was temporarily established with a self-referential shape and appeared in far space, the IOR effect was the smallest. When the target was temporarily established with a self-referential shape and appeared in near space, the IOR effect disappeared. This study suggests that the IOR effect was affected by the SPE when attention was oriented or reoriented in 3D space and that the IOR effect disappeared or decreased when affected by the SPE in 3D space.

Numerosity estimates for attended and unattended items in visual search

The goal of this research was to examine memories created for the number of items during a visual search task. Participants performed a visual search task for a target defined by a single feature (Experiment 1A), by a conjunction of features (Experiment 1B), or by a specific spatial configuration of features (Experiment 1C). On some trials following the search task, subjects were asked to recall the total number of items in the previous display. In all search types, participants underestimated the total number of items, but the severity of the underestimation varied depending on the efficiency of the search. In three follow-up studies (Experiments 2A, 2B, and 2C) using the same visual stimuli, the participants' only task was to estimate the number of items on each screen. Participants still underestimated the numerosity of the items, although the degree of underestimation was smaller than in the search tasks and did not depend on the type of visual stimuli. In Experiment 3, participants were asked to recall the number of items in a display only once. Subjects still displayed a tendency to underestimate, indicating that the underestimation effects seen in Experiments 1A-1C were not attributable to knowledge of the estimation task. The degree of underestimation depends on the efficiency of the search task, with more severe underestimation in efficient search tasks. This suggests that the lower attentional demands of very efficient searches leads to less encoding of numerosity of the distractor set.

A Microsaccadic Account of Attentional Capture and Inhibition of Return in Posner Cueing

Microsaccades exhibit systematic oscillations in direction after spatial cueing, and these oscillations correlate with facilitatory and inhibitory changes in behavioral performance in the same tasks. However, independent of cueing, facilitatory and inhibitory changes in visual sensitivity also arise pre-microsaccadically. Given such pre-microsaccadic modulation, an imperative question to ask becomes: how much of task performance in spatial cueing may be attributable to these peri-movement changes in visual sensitivity? To investigate this question, we adopted a theoretical approach. We developed a minimalist model in which: (1) microsaccades are repetitively generated using a rise-to-threshold mechanism, and (2) pre-microsaccadic target onset is associated with direction-dependent modulation of visual sensitivity, as found experimentally. We asked whether such a model alone is sufficient to account for performance dynamics in spatial cueing. Our model not only explained fine-scale microsaccade frequency and direction modulations after spatial cueing, but it also generated classic facilitatory (i.e., attentional capture) and inhibitory [i.e., inhibition of return (IOR)] effects of the cue on behavioral performance. According to the model, cues reflexively reset the oculomotor system, which unmasks oscillatory processes underlying microsaccade generation; once these oscillatory processes are unmasked, "attentional capture" and "IOR" become direct outcomes of pre-microsaccadic enhancement or suppression, respectively. Interestingly, our model predicted that facilitatory and inhibitory effects on behavior should appear as a function of target onset relative to microsaccades even without prior cues. We experimentally validated this prediction for both saccadic and manual responses. We also established a potential causal mechanism for the microsaccadic oscillatory processes hypothesized by our model. We used retinal-image stabilization to experimentally control instantaneous foveal motor error during the presentation of peripheral cues, and we found that post-cue microsaccadic oscillations were severely disrupted. This suggests that microsaccades in spatial cueing tasks reflect active oculomotor correction of foveal motor error, rather than presumed oscillatory covert attentional processes. Taken together, our results demonstrate that peri-microsaccadic changes in vision can go a long way in accounting for some classic behavioral phenomena.

A computational model of fMRI activity in the intraparietal sulcus that supports visual working memory

A computational model was developed to explain a pattern of results of fMRI activation in the intraparietal sulcus (IPS) supporting visual working memory for multiobject scenes. The model is based on the hypothesis that dendrites of excitatory neurons are major computational elements in the cortical circuit. Dendrites enable formation of a competitive queue that exhibits a gradient of activity values for nodes encoding different objects, and this pattern is stored in working memory. In the model, brain imaging data are interpreted as a consequence of blood flow arising from dendritic processing. Computer simulations showed that the model successfully simulates data showing the involvement of inferior IPS in object individuation and spatial grouping through representation of objects' locations in space, along with the involvement of superior IPS in object identification through representation of a set of objects' features. The model exhibits a capacity limit due to the limited dynamic range for nodes and the operation of lateral inhibition among them. The capacity limit is fixed in the inferior IPS regardless of the objects' complexity, due to the normalization of lateral inhibition, and variable in the superior IPS, due to the different encoding demands for simple and complex shapes. Systematic variation in the strength of self-excitation enables an understanding of the individual differences in working memory capacity. The model offers several testable predictions regarding the neural basis of visual working memory.

When do I quit? The search termination problem in visual search

In visual search tasks, observers look for targets in displays or scenes containing distracting, non-target items. Most of the research on this topic has concerned the finding of those targets. Search termination is a less thoroughly studied topic. When is it time to abandon the current search? The answer is fairly straight forward when the one and only target has been found (There are my keys.). The problem is more vexed if nothing has been found (When is it time to stop looking for a weapon at the airport checkpoint?) or when the number of targets is unknown (Have we found all the tumors?). This chapter reviews the development of ideas about quitting time in visual search and offers an outline of our current theory.

Spatial distribution of attentional inhibition is not altered in healthy aging

Inhibition of return (IOR) is a phenomenon of attentional orienting that is indexed by slower responses to targets presented at previously attended locations. The purpose of this study was to examine adult age differences in the distribution of IOR to multiple locations. In three experiments, young adults (ages 18-30 years) and older adults (ages 60-87 years) completed an IOR task that varied in the number of simultaneous onset cues (one to seven) and the number of display locations (four or eight). Analyses were conducted to explore whether IOR patterns were most consistent with limited inhibitory resources, with regional distribution of inhibition, or with vector averaging of cues. The IOR effects were most consistent with vector averaging, such that multiple cues initiated a directional gradient of inhibition centered on the average direction of the cues. The IOR patterns varied minimally with age, consistent with the conclusion that older adults and young adults distributed inhibition in a similar manner.

Incidental learning speeds visual search by lowering response thresholds, not by improving efficiency: evidence from eye movements

When observers search for a target object, they incidentally learn the identities and locations of "background" objects in the same display. This learning can facilitate search performance, eliciting faster reaction times for repeated displays. Despite these findings, visual search has been successfully modeled using architectures that maintain no history of attentional deployments; they are amnesic (e.g., Guided Search Theory). In the current study, we asked two questions: 1) under what conditions does such incidental learning occur? And 2) what does viewing behavior reveal about the efficiency of attentional deployments over time? In two experiments, we tracked eye movements during repeated visual search, and we tested incidental memory for repeated nontarget objects. Across conditions, the consistency of search sets and spatial layouts were manipulated to assess their respective contributions to learning. Using viewing behavior, we contrasted three potential accounts for faster searching with experience. The results indicate that learning does not result in faster object identification or greater search efficiency. Instead, familiar search arrays appear to allow faster resolution of search decisions, whether targets are present or absent.

Inhibitory interaction: the effects of multiple non-predictive visual cues

When the interval between a non-predictive cue and a target appearing at the same spatial location is longer than about 200 ms, target performance is typically poorer than when the cue and target appear at different locations. Recent studies have shown that this effect, known as inhibition of return (IOR), can occur at multiple cued locations, and is enhanced when multiple cues are presented at the same spatial location. However, little is known about how the magnitude of IOR at one spatial location is influenced by a subsequent or preceding cue presented at a different spatial location. We investigated this issue by presenting single or multiple cues at varying inter-cue intervals prior to target onset. Results suggest that the magnitude of IOR at a given location is influenced by the presentation of a preceding cue, but that once IOR occurs, it is unaffected by the presentation of a subsequent cue.

Novelty Is Not Always the Best Policy

We report a study that examined whether inhibition of return (IOR) is specific to visual search or a general characteristic of visual behavior. Participants were shown a series of scenes and were asked to (a) search each scene for a target, (b) memorize each scene, (c) rate how pleasant each scene was, or (d) view each scene freely. An examination of saccadic reaction times to probes provided evidence of IOR during search: Participants were slower to look at probes at previously fixated locations than to look at probes at novel locations. For the other three conditions, however, the opposite pattern of results was observed: Participants were faster to look at probes at previously fixated locations than to look at probes at novel locations, a facilitation-of-return effect that has not been reported previously. These results demonstrate that IOR is a search-specific strategy and not a general characteristic of visual attention.

Aging and temporal patterns of inhibition of return

Inhibition of return (IOR), an inhibitory component of spatial attention that is thought to bias visual search toward novel locations, is considered relatively well preserved with normal aging. We conducted two experiments to assess age-related changes in the temporal pattern of IOR. Inhibitory effects, which were strongly reflected in the performance of both younger adults (ages 18-34 years) and older adults (ages 60-79 years), diminished over a period of 5 s. The time point at which IOR began to diminish was delayed by approximately 1 s for older adults compared with younger adults; this pattern was observed on both a target detection task (Experiment 1) and a color discrimination task (Experiment 2). The finding that timing characteristics of IOR are altered by normal aging has potential implications for the manner in which inhibition aids search performance.

Memory for where, but not what, is used during visual search

Although the role of memory in visual search is debatable, most researchers agree with a limited-capacity model of memory in visual search. The authors demonstrate the role of memory by replicating previous findings showing that visual search is biased away from old items (previously examined items) and toward new items (nonexamined items). Furthermore, the authors examined the type of memory representations used to bias search by changing an item's individuating feature or location during search. Changing the individuating feature of an item did not disrupt normal search biases. However, when the location of an item changed, normal search biases were disrupted. These results suggest that memory used in visual search is based on items' locations rather than their identity.

Effect of previously fixated locations on saccade trajectory during free visual search

Recent studies have shown that the saccade trajectory often curved away from an object that was previously attended but irrelevant to the current saccade goal. We investigated whether such curved saccades occur during serial visual search, which requires sequential saccades possibly controlled by inhibition to multiple locations. The results show that the saccade trajectories were affected by at least three previous fixations. Furthermore, the effect of the previous fixations on saccade trajectories decreased exponentially with time or the number of intervening saccades. The relationship between the curved saccade trajectory and inhibition of return during serial visual search was discussed.

Rapid onset and long-term inhibition of return in the multiple cuing paradigm

Inhibition of return (IOR) refers to the finding that targets at cued locations are responded to more slowly than targets at uncued locations when a relatively long temporal interval occurs between the two events. In studies which have examined the time course of IOR (e.g., Samuel & Kat in Psychonomic Bulletin & Review, 10, 897-906, 2003), the effect is generally shown to develop at around 200 ms and dissipate at around 3,000 ms following a cue. A number of recent studies, however, have demonstrated that IOR can develop much more quickly (up to 50 ms following a cue) and last much longer (up to 13 min following a cue) in certain tasks. The present study uses the multiple cuing paradigm to determine whether IOR can be observed outside the normally reported temporal boundaries (300-3,000 ms) when attention is shifted very quickly (every 15 ms) or very slowly (every 1,500 ms) throughout the visual field. IOR was observed as quickly as 30 ms following cue onset and as long as 6,000 ms following cue onset. Implications for the role of IOR in visual search are discussed.

Object- and location-based inhibition of return in younger and older adults

When a cued object moves to new spatial coordinates, inhibition of return (IOR) with younger adults is found at the original cued location (location-based IOR) and at the current location of the object (object-based IOR). Older adults, however, show only location-based IOR. To determine whether this pattern of results represents a general age-related deficit in object-based IOR, the authors used static displays in which the placeholders (i.e., objects) were either present (location-based IOR+object-based IOR) or absent (location-based IOR only). Both age groups showed location-based IOR, but the older adults failed to show object-based IOR, consistent with age-related differences in visual pathways.

Why don't we see changes?: The role of attentional bottlenecks and limited visual memory

Seven experiments explore the role of bottlenecks in selective attention and access to visual short-term memory in the failure of observers to identify clearly visible changes in otherwise stable visual displays. Experiment One shows that observers fail to register a color change in an object even if they are cued to the location of the object by a transient at that location as the change is occurring. Experiment Two shows the same for orientation change. In Experiments Three and Four, attention is directed to specific objects prior to making changes in those objects. Observers have only a very limited memory for the status of recently attended items. Experiment Five reveals that observers have no ability to detect changes that happen after attention has been directed to an object and before attention returns to that object. In Experiment Six, attention is cued at rates that more closely resemble natural rates and Experiment Seven uses natural images. Memory capacity remains very small (<4 items).

The effect of previous trial type on inhibition of return

Inhibition of return (IOR) refers to the finding that targets appearing at previously cued locations are more slowly responded to than targets appearing at previously uncued locations when a relatively long temporal interval occurs between the cue and target. This experiment was conducted to determine whether the magnitude of IOR is influenced by the type of preceding trial (cued or uncued) and/or the location of the cue/target on the previous trial. Although no effect of cue/target location is observed, there was a marked effect of previous trial type, as IOR was greater following an uncued trial relative to a cued trial. This effect was attributable to differences in the response time as a function of previous trial type: specifically, participants were faster to respond to cued and uncued trials when the previous trial type was identical.

Inhibition of return is not a foraging facilitator in saccadic search and free viewing

The ability to search and scan the environment effectively is a prerequisite for spatial behavior. A longstanding theory proposes that inhibition of previously attended loci (Inhibition of return; IOR) serves to facilitate exploration by increasing the likelihood to inspect new areas instead of returning to locations that have been inspected before. In this eye movement study we tested whether we could find evidence in favor of this hypothesis. Here we report that IOR does occur during search and free viewing, because we found increased fixation times preceding return saccades (eye movements that return to previously fixated locations). Meanwhile we observed no influence of IOR on the search strategy. Rather than the predicted low number we found many return saccades. Therefore, IOR does not serve as a foraging facilitator in saccadic search and free viewing. We hypothesize that IOR is an intrinsic aspect of shifting attention and gaze direction and furthermore that it is not always advantageous to prevent return saccades.

Examining inhibition of return with onset and offset cues in the multiple-cuing paradigm

Three experiments examined inhibition of return (IOR) with onset and offset cues in a multiple-cuing paradigm. In the first two experiments, five sequential cues either appeared and remained present (onset cues) or disappeared and remained absent (offset cues). In the third experiment, the cues were either onset cues or on-off cues (appeared and then disappeared quickly after). With placeholders present, onset and offset cues produced similar declines in IOR from the most recently cued location (Experiment 1). In contrast, onset cues produced overall more IOR than on-off cues (Experiment 3). With placeholders absent (Experiment 2), no IOR was found for either onset or offset cues. The results suggest that even in a complex multiple-cuing paradigm, onsets and offsets are treated similarly by the attentional system. Furthermore, it appears that onset cues are easier to encode as previously searched than on-off cues, suggesting a role of working memory in IOR. Finally, when multiple locations are cued sequentially by onsets and offsets they must be marked by placeholders for inhibition to occur.