Looking back at Waldo: Oculomotor inhibition of return does not prevent return fixations

Adaptation and exogenous attention interact in the early visual cortex: A TMS study

Transcranial magnetic stimulation (TMS) to early visual cortex modulates the effect of adaptation and eliminates the effect of exogenous (involuntary) attention on contrast sensitivity. Here we investigated whether adaptation modulates exogenous attention under TMS to V1/V2. Observers performed an orientation discrimination task while attending to one of two stimuli, with or without adaptation. Following an attentional cue, two stimuli were presented in the stimulated region and its contralateral symmetric region. A response cue indicated the stimulus whose orientation observers had to discriminate. Without adaptation, in the distractor-stimulated condition, contrast sensitivity increased at the attended location and decreased at the unattended location via response gain-but these effects were eliminated in the target-stimulated condition. Critically, after adaptation, exogenous attention altered performance similarly in both distractor-stimulated and target-stimulated conditions. These results reveal that (1) adaptation and attention interact in the early visual cortex, and (2) adaptation shields exogenous attention from TMS effects.

Motor "laziness" constrains fixation selection in real-world tasks

Humans coordinate their eye, head, and body movements to gather information from a dynamic environment while maximizing reward and minimizing biomechanical and energetic costs. However, such natural behavior is not possible in traditional experiments employing head/body restraints and artificial, static stimuli. Therefore, it is unclear to what extent mechanisms of fixation selection discovered in lab studies, such as inhibition-of-return (IOR), influence everyday behavior. To address this gap, participants performed nine real-world tasks, including driving, visually searching for an item, and building a Lego set, while wearing a mobile eye tracker (169 recordings; 26.6 h). Surprisingly, in all tasks, participants most often returned to what they just viewed and saccade latencies were shorter preceding return than forward saccades, i.e., consistent with facilitation, rather than inhibition, of return. We hypothesize that conservation of eye and head motor effort ("laziness") contributes. Correspondingly, we observed center biases in fixation position and duration relative to the head's orientation. A model that generates scanpaths by randomly sampling these distributions reproduced all return phenomena we observed, including distinct 3-fixation sequences for forward versus return saccades. After controlling for orbital eccentricity, one task (building a Lego set) showed evidence for IOR. This, along with small discrepancies between model and data, indicates that the brain balances minimization of motor costs with maximization of rewards (e.g., accomplished by IOR and other mechanisms) and that the optimal balance varies according to task demands. Supporting this account, the orbital range of motion used in each task traded off lawfully with fixation duration.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Attention allocation in complementary joint action: How joint goals affect spatial orienting

When acting jointly, individuals often attend and respond to the same object or spatial location in complementary ways (e.g., when passing a mug, one person grasps its handle with a precision grip; the other receives it with a whole-hand grip). At the same time, the spatial relation between individuals' actions affects attentional orienting: one is slower to attend and respond to locations another person previously acted upon than to alternate locations ("social inhibition of return", social IOR). Achieving joint goals (e.g., passing a mug), however, often requires complementary return responses to a co-actor's previous location. This raises the question of whether attentional orienting, and hence the social IOR, is affected by the (joint) goal our actions are directed at. The present study addresses this question. Participants responded to cued locations on a computer screen, taking turns with a virtual co-actor. They pursued either an individual goal or performed complementary actions with the co-actor, in pursuit of a joint goal. Four experiments showed that the social IOR was significantly modulated when participant and co-actor pursued a joint goal. This suggests that attentional orienting is affected not only by the spatial but also by the social relation between two agents' actions. Our findings thus extend research on interpersonal perception-action effects, showing that the way another agent's perceived action shapes our own depends on whether we share a joint goal with that agent.

Working memory control predicts fixation duration in scene-viewing

When viewing scenes, observers differ in how long they linger at each fixation location and how far they move their eyes between fixations. What factors drive these differences in eye-movement behaviors? Previous work suggests individual differences in working memory capacity may influence fixation durations and saccade amplitudes. In the present study, participants (N = 98) performed two scene-viewing tasks, aesthetic judgment and memorization, while viewing 100 photographs of real-world scenes. Working memory capacity, working memory processing ability, and fluid intelligence were assessed with an operation span task, a memory updating task, and Raven's Advanced Progressive Matrices, respectively. Across participants, we found significant effects of task on both fixation durations and saccade amplitudes. At the level of each individual participant, we also found a significant relationship between memory updating task performance and participants' fixation duration distributions. However, we found no effect of fluid intelligence and no effect of working memory capacity on fixation duration or saccade amplitude distributions, inconsistent with previous findings. These results suggest that the ability to flexibly maintain and update working memory is strongly related to fixation duration behavior.

Look twice: A generalist computational model predicts return fixations across tasks and species

Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.

Between the Scenes

We constantly move our eyes to new information while inspecting a scene, but these patterns of eye movements change based on the task and goals of the observer. Inhibition of return (IOR) may facilitate visual search by reducing the likelihood of revisiting previously attended locations. However, IOR may present in any visual task, or it may be search-specific. We investigated the presence of IOR in foraging, memorization, change detection, and two versions of visual search. One version of search used a static search array that remained stable throughout the trial, but the second used a scene flickering paradigm similar to the change detection task. IOR was observed in both versions of visual search, memorization, and foraging, but not in change detection. Visual search and change detection both had temporal nonscene components, and we observed that IOR could be maintained despite the scene removal but only for search. Although IOR is maintained in scene coordinates, short disruptions to this scene are insufficient to completely remove the inhibitory tags. Finally, we compare return saccades in trials without a probe and observe fewer return saccades in tasks for which IOR was observed, providing further evidence that IOR might serve as a novelty drive.

The Spatial Leaky Competing Accumulator Model

The Leaky Competing Accumulator model (LCA) of Usher and McClelland is able to simulate the time course of perceptual decision making between an arbitrary number of stimuli. Reaction times, such as saccadic latencies, produce a typical distribution that is skewed toward longer latencies and accumulator models have shown excellent fit to these distributions. We propose a new implementation called the Spatial Leaky Competing Accumulator (SLCA), which can be used to predict the timing of subsequent fixation durations during a visual task. SLCA uses a pre-existing saliency map as input and represents accumulation neurons as a two-dimensional grid to generate predictions in visual space. The SLCA builds on several biologically motivated parameters: leakage, recurrent self-excitation, randomness and non-linearity, and we also test two implementations of lateral inhibition. A global lateral inhibition, as implemented in the original model of Usher and McClelland, is applied to all competing neurons, while a local implementation allows only inhibition of immediate neighbors. We trained and compared versions of the SLCA with both global and local lateral inhibition with use of a genetic algorithm, and compared their performance in simulating human fixation latency distribution in a foraging task. Although both implementations were able to produce a positively skewed latency distribution, only the local SLCA was able to match the human data distribution from the foraging task. Our model is discussed for its potential in models of salience and priority, and its benefits as compared to other models like the Leaky integrate and fire network.

Looking for the neural basis of memory

Memory neuroscientists often measure neural activity during task trials designed to recruit specific memory processes. Behavior is championed as crucial for deciphering brain-memory linkages but is impoverished in typical experiments that rely on summary judgments. We criticize this approach as being blind to the multiple cognitive, neural, and behavioral processes that occur rapidly within a trial to support memory. Instead, time-resolved behaviors such as eye movements occur at the speed of cognition and neural activity. We highlight successes using eye-movement tracking with in vivo electrophysiology to link rapid hippocampal oscillations to encoding and retrieval processes that interact over hundreds of milliseconds. This approach will improve research on the neural basis of memory because it pinpoints discrete moments of brain-behavior-cognition correspondence.

Does task complexity impact the neurovascular coupling response similarly between males and females?

BACKGROUND

While previous studies have demonstrated a complex visual scene search elicits a robust neurovascular coupling (NVC) response, it is unknown how the duration of visual stimuli presentation influences NVC metrics. This study examined how stimuli duration, in addition to biological sex and self-reported engagement impact NVC responses.

METHODS

Participants (n = 20, female = 10) completed four visual paradigms. Three involved simple visual shapes presented at 0.5-, 2-, and 4-s intervals in randomized orders. The fourth paradigm was a complex visual scene search ("Where's Waldo?"). Participants completed eight cycles of 20-s eyes-closed followed by 40-s eyes-open. Transcranial Doppler ultrasound indexed posterior and middle cerebral artery velocities (PCA and MCA). Participants self-reported their engagement following each task (1 [minimal] to 10 [maximal]).

RESULTS

The "Where's Waldo?" task evoked greater PCA percent increase (all p < 0.001) and area under the curve during the first 30-s of the task (all p < 0.001) compared to simple shapes. Females displayed greater absolute baseline and peak PCA and MCA velocities across all tasks (all p < 0.002). Subjective engagement displayed moderate correlation levels with PCA percent increase (Spearman ρ = 0.58) and area under the curve (Spearman ρ = 0.60) metrics in males, whereas these were weak for females (Spearman ρ = 0.43 and ρ = 0.38, respectively).

CONCLUSIONS

The complex visual paradigm "Where's Waldo?" greatly augmented the signal-to-noise ratio within the PCA aspects of the NVC response compared to simple shapes. While both sexes had similar NVC responses, task engagement was more related to NVC metrics in males compared to females. Therefore, future NVC investigations should consider task engagement when designing studies.

Crossmodal spatial distraction across the lifespan

The ability to resist distracting stimuli whilst voluntarily focusing on a task is fundamental to our everyday cognitive functioning. Here, we investigated how this ability develops, and thereafter declines, across the lifespan using a single task/experiment. Young children (5-7 years), older children (10-11 years), young adults (20-27 years), and older adults (62-86 years) were presented with complex visual scenes. Endogenous (voluntary) attention was engaged by having the participants search for a visual target presented on either the left or right side of the display. The onset of the visual scenes was preceded - at stimulus onset asynchronies (SOAs) of 50, 200, or 500 ms - by a task-irrelevant sound (an exogenous crossmodal spatial distractor) delivered either on the same or opposite side as the visual target, or simultaneously on both sides (cued, uncued, or neutral trials, respectively). Age-related differences were revealed, especially in the extreme age-groups, which showed a greater impact of crossmodal spatial distractors. Young children were highly susceptible to exogenous spatial distraction at the shortest SOA (50 ms), whereas older adults were distracted at all SOAs, showing significant exogenous capture effects during the visual search task. By contrast, older children and young adults' search performance was not significantly affected by crossmodal spatial distraction. Overall, these findings present a detailed picture of the developmental trajectory of endogenous resistance to crossmodal spatial distraction from childhood to old age and demonstrate a different efficiency in coping with distraction across the four age-groups studied.

No supplementary evidence of attention to a spatial cue when saccadic facilitation is absent

Attending a location in space facilitates responses to targets at that location when the time between cue and target is short. Certain types of exogenous cues – such as sudden peripheral onsets – have been described as reflexive and automatic. Recent studies however, have been showing many cases where exogenous cues are less automatic than previously believed and do not always result in facilitation. A lack of the behavioral facilitation, however, does not automatically necessitate a lack of underlying attention to that location. We test exogenous cueing in two experiments where facilitation is and is not likely to be observed with saccadic responses. We also test alternate measures linked to the allocation of attention such as saccadic curvature, microsaccades and pupil size. As expected, we find early facilitation as measured by saccadic reaction time when CTOAs are predictable but not when they are randomized within a block. We find no impact of the cue on microsaccade direction for either experiment, and only a slight dip in the frequency of microsaccades after the cue. We do find that change in pupil size to the cue predicts the magnitude of the validity effect, but only in the experiment where facilitation was observed. In both experiments, we observed a tendency for saccadic curvature to deviate away from the cued location and this was stronger for early CTOAs and toward vertical targets. Overall, we find that only change in pupil size is consistent with observed facilitation. Saccadic curvature is influenced by the onset of the cue, buts its direction is indicative of oculomotor inhibition whether we see RT facilitation or not. Microsaccades were not diagnostic in either experiment. Finally, we see little to no evidence of attention at the cued location in any additional measures when facilitation of saccadic responses is absent.

Temporal ambiguity of onsets in a cueing task prevents facilitation but not inhibition of return

Cueing effects, i.e., early facilitation of reaction time and inhibition of return (IOR), are well-established and robust phenomena characterizing exogenous orienting and are widely observed in experiments with a traditional Posner cueing paradigm. Krüger, MacInnes, and Hunt (2014) proposed that facilitatory effects of peripheral cues are the result of a cue-target perceptual merging due to re-entrant visual processing. To test the role and timing of these feedback mechanisms in peripheral cueing effects, we modified the traditional cueing task in Experiments 1-3 by interleaving pre- and post-cue trials at the valid and invalid location and random cue-target onset asynchrony (CTOA) ranging from -300 to +1,000 ms. Analysis of the manual reaction time distribution over CTOA showed well-pronounced IOR in the valid pre-cue condition and a small cost of perceptual merging in the post-cue condition, but no early facilitation of reaction time was observed in the pre-cue condition. In Experiment 4, we tested directly whether temporal ambiguity eliminated facilitation by restricting CTOAs to only the pre-cue time range and including a between-subject manipulation of a) random, b) mixed discrete, and c) blocked discrete CTOAs. Results obtained in the continuous and binned conditions showed no facilitation but robust IOR. We found both early facilitation and IOR in the blocked condition. Overall, the present findings show a small perceptual merging result without accompanying facilitation, suggesting different underlying mechanisms. Second, they demonstrate that early facilitation is likely to be affected by the presence or absence of temporal expectations and that the early onset of IOR might be masked by stronger facilitation in traditional cueing experiments.

A Generative Model of Cognitive State from Task and Eye Movements

The early eye tracking studies of Yarbus provided descriptive evidence that an observer's task influences patterns of eye movements, leading to the tantalizing prospect that an observer's intentions could be inferred from their saccade behavior. We investigate the predictive value of task and eye movement properties by creating a computational cognitive model of saccade selection based on instructed task and internal cognitive state using a Dynamic Bayesian Network (DBN). Understanding how humans generate saccades under different conditions and cognitive sets links recent work on salience models of low-level vision with higher level cognitive goals. This model provides a Bayesian, cognitive approach to top-down transitions in attentional set in pre-frontal areas along with vector-based saccade generation from the superior colliculus. Our approach is to begin with eye movement data that has previously been shown to differ across task. We first present an analysis of the extent to which individual saccadic features are diagnostic of an observer's task. Second, we use those features to infer an underlying cognitive state that potentially differs from the instructed task. Finally, we demonstrate how changes of cognitive state over time can be incorporated into a generative model of eye movement vectors without resorting to an external decision homunculus. Internal cognitive state frees the model from the assumption that instructed task is the only factor influencing observers' saccadic behavior. While the inclusion of hidden temporal state does not improve the classification accuracy of the model, it does allow accurate prediction of saccadic sequence results observed in search paradigms. Given the generative nature of this model, it is capable of saccadic simulation in real time. We demonstrated that the properties from its generated saccadic vectors closely match those of human observers given a particular task and cognitive state. Many current models of vision focus entirely on bottom-up salience to produce estimates of spatial "areas of interest" within a visual scene. While a few recent models do add top-down knowledge and task information, we believe our contribution is important in three key ways. First, we incorporate task as learned attentional sets that are capable of self-transition given only information available to the visual system. This matches influential theories of bias signals by (Miller and Cohen Annu Rev Neurosci 24:167-202, 2001) and implements selection of state without simply shifting the decision to an external homunculus. Second, our model is generative and capable of predicting sequence artifacts in saccade generation like those found in visual search. Third, our model generates relative saccadic vector information as opposed to absolute spatial coordinates. This matches more closely the internal saccadic representations as they are generated in the superior colliculus.

Neural field model of memory-guided search

Many organisms can remember locations they have previously visited during a search. Visual search experiments have shown exploration is guided away from these locations, reducing redundancies in the search path before finding a hidden target. We develop and analyze a two-layer neural field model that encodes positional information during a search task. A position-encoding layer sustains a bump attractor corresponding to the searching agent's current location, and search is modeled by velocity input that propagates the bump. A memory layer sustains persistent activity bounded by a wave front, whose edges expand in response to excitatory input from the position layer. Search can then be biased in response to remembered locations, influencing velocity inputs to the position layer. Asymptotic techniques are used to reduce the dynamics of our model to a low-dimensional system of equations that track the bump position and front boundary. Performance is compared for different target-finding tasks.

An extensive dataset of eye movements during viewing of complex images

We present a dataset of free-viewing eye-movement recordings that contains more than 2.7 million fixation locations from 949 observers on more than 1000 images from different categories. This dataset aggregates and harmonizes data from 23 different studies conducted at the Institute of Cognitive Science at Osnabrück University and the University Medical Center in Hamburg-Eppendorf. Trained personnel recorded all studies under standard conditions with homogeneous equipment and parameter settings. All studies allowed for free eye-movements, and differed in the age range of participants (~7-80 years), stimulus sizes, stimulus modifications (phase scrambled, spatial filtering, mirrored), and stimuli categories (natural and urban scenes, web sites, fractal, pink-noise, and ambiguous artistic figures). The size and variability of viewing behavior within this dataset presents a strong opportunity for evaluating and comparing computational models of overt attention, and furthermore, for thoroughly quantifying strategies of viewing behavior. This also makes the dataset a good starting point for investigating whether viewing strategies change in patient groups.

Revisiting the global effect and inhibition of return

Saccades toward previously cued locations have longer latencies than saccades toward other locations, a phenomenon known as inhibition of return (IOR). Watanabe (Exp Brain Res 138:330–342. doi:10.1007/s002210100709, 2001) combined IOR with the global effect (where saccade landing points fall in between neighboring objects) to investigate whether IOR can also have a spatial component. When one of two neighboring targets was cued, there was a clear bias away from the cued location. In a condition where both targets were cued, it appeared that the global effect magnitude was similar to the condition without any cues. However, as the latencies in the double cue condition were shorter compared to the no cue condition, it is still an open question whether these results are representative for IOR. Considering the double cue condition can provide valuable insight into the interaction of the mechanisms underlying the two phenomena, here, we revisit this condition in an adapted paradigm. Our paradigm does result in longer latencies for the cued locations, and we find that the magnitude of the global effect is reduced significantly. Unexpectedly, this holds even when only including saccades with the same latencies for both conditions. Thus, the increased latencies associated with IOR cannot directly explain the reduction in global effect. The global effect reduction can likely best be seen as either a result of short-term depression of exogenous visual signals or a result of IOR established at the center of gravity of cues.

Environment- and eye-centered inhibitory cueing effects are both observed after a methodological confound is eliminated

Inhibition of return (IOR), typically explored in cueing paradigms, is a performance cost associated with previously attended locations and has been suggested as a crucial attentional mechanism that biases orientation towards novelty. In their seminal IOR paper, Posner and Cohen (1984) showed that IOR is coded in spatiotopic or environment-centered coordinates. Recent studies, however, have consistently reported IOR effects in both spatiotopic and retinotopic (eye-centered) coordinates. One overlooked methodological confound of all previous studies is that the spatial gradient of IOR is not considered when selecting the baseline for estimating IOR effects. This methodological issue makes it difficult to tell if the IOR effects reported in previous studies were coded in retinotopic or spatiotopic coordinates, or in both. The present study addresses this issue with the incorporation of no-cue trials to a modified cueing paradigm in which the cue and target are always intervened by a gaze-shift. The results revealed that a) IOR is indeed coded in both spatiotopic and retinotopic coordinates, and b) the methodology of previous work may have underestimated spatiotopic and retinotopic IOR effects.

Effects of task and task-switching on temporal inhibition of return, facilitation of return, and saccadic momentum during scene viewing

During scene viewing, saccades directed toward a recently fixated location tend to be delayed relative to saccades in other directions ("delay effect"), an effect attributable to inhibition of return (IOR) and/or saccadic momentum (SM). Previous work indicates this effect may be task-specific, suggesting that gaze control parameters are task-relevant and potentially affected by task-switching. Accordingly, the present study investigated task-set control of gaze behavior using the delay effect as a measure of task performance. The delay effect was measured as the effect of relative saccade direction on preceding fixation duration. Participants were cued on each trial to perform either a search, memory, or rating task. Tasks were performed either in pure-task or mixed-task blocks. This design allowed separation of switch-cost and mixing-cost. The critical result was that expression of the delay effect at 2-back locations was reversed on switch versus repeat trials such that return was delayed in repeat trials but speeded in switch trials. This difference between repeat and switch trials suggests that gaze-relevant parameters may be represented and switched as part of a task-set. Existing and new tests for dissociating IOR and SM accounts of the delay effect converged on the conclusion that the delay at 2-back locations was due to SM, and that task-switching affects SM. Additionally, the new test simultaneously replicated noncorroborating results in the literature regarding facilitation-of-return (FOR), which confirmed its existence and showed that FOR is "reversed" SM that occurs when preceding and current saccades are both directed toward the 2-back location.

Just passing through? Inhibition of return in saccadic sequences

Responses tend to be slower to previously fixated spatial locations, an effect known as “inhibition of return” (IOR). Saccades cannot be assumed to be independent, however, and saccade sequences programmed in parallel differ from independent eye movements. We measured the speed of both saccadic and manual responses to probes appearing in previously fixated locations when those locations were fixated as part of either parallel or independent saccade sequences. Saccadic IOR was observed in independent but not parallel saccade sequences, while manual IOR was present in both parallel and independent sequence types. Saccadic IOR was also short-lived, and dissipated with delays of more than ∼1500 ms between the intermediate fixation and the probe onset. The results confirm that the characteristics of IOR depend critically on the response modality used for measuring it, with saccadic and manual responses giving rise to motor and attentional forms of IOR, respectively. Saccadic IOR is relatively short-lived and is not observed at intermediate locations of parallel saccade sequences, while attentional IOR is long-lasting and consistent for all sequence types.

In search of a reliable electrophysiological marker of oculomotor inhibition of return

Inhibition of return (IOR) operationalizes a behavioral phenomenon characterized by slower responding to cued, relative to uncued, targets. Two independent forms of IOR have been theorized: input-based IOR occurs when the oculomotor system is quiescent, while output-based IOR occurs when the oculomotor system is engaged. EEG studies forbidding eye movements have demonstrated that reductions of target-elicited P1 components are correlated with IOR magnitude, but when eye movements occur, P1 effects bear no relationship to behavior. We expand on this work by adapting the cueing paradigm and recording event-related potentials: IOR is caused by oculomotor responses to central arrows or peripheral onsets and measured by key presses to peripheral targets. Behavioral IOR is observed in both conditions, but P1 reductions are absent in the central arrow condition. By contrast, arrow and peripheral cues enhance Nd, especially over contralateral electrode sites.

Analysis of microsaccades and pupil dilation reveals a common decisional origin during visual search

During free viewing visual search, observers often refixate the same locations several times before and after target detection is reported with a button press. We analyzed the rate of microsaccades in the sequence of refixations made during visual search and found two important components. One related to the visual content of the region being fixated; fixations on targets generate more microsaccades and more microsaccades are generated for those targets that are more difficult to disambiguate. The other empathizes non-visual decisional processes; fixations containing the button press generate more microsaccades than those made on the same target but without the button press. Pupil dilation during the same refixations reveals a similar modulation. We inferred that generic sympathetic arousal mechanisms are part of the articulated complex of perceptual processes governing fixational eye movements.

Modelling eye movements in a categorical search task

We introduce a model of eye movements during categorical search, the task of finding and recognizing categorically defined targets. It extends a previous model of eye movements during search (target acquisition model, TAM) by using distances from an support vector machine classification boundary to create probability maps indicating pixel-by-pixel evidence for the target category in search images. Other additions include functionality enabling target-absent searches, and a fixation-based blurring of the search images now based on a mapping between visual and collicular space. We tested this model on images from a previously conducted variable set-size (6/13/20) present/absent search experiment where participants searched for categorically defined teddy bear targets among random category distractors. The model not only captured target-present/absent set-size effects, but also accurately predicted for all conditions the numbers of fixations made prior to search judgements. It also predicted the percentages of first eye movements during search landing on targets, a conservative measure of search guidance. Effects of set size on false negative and false positive errors were also captured, but error rates in general were overestimated. We conclude that visual features discriminating a target category from non-targets can be learned and used to guide eye movements during categorical search.

Temporal oculomotor inhibition of return and spatial facilitation of return in a visual encoding task

Oculomotor inhibition of return (O-IOR) is an increase in saccade latency prior to an eye movement to a recently fixated location compared to other locations. It has been proposed that this temporal O-IOR may have spatial consequences, facilitating foraging by inhibiting return to previously attended regions. In order to test this possibility, participants viewed arrays of objects and of words while their eye movements were recorded. Temporal O-IOR was observed, with equivalent effects for object and word arrays, indicating that temporal O-IOR is an oculomotor phenomenon independent of array content. There was no evidence for spatial inhibition of return (IOR). Instead, spatial facilitation of return was observed: participants were significantly more likely than chance to make return saccades and to re-fixate just-visited locations. Further, the likelihood of making a return saccade to an object or word was contingent on the amount of time spent viewing that object or word before leaving it. This suggests that, unlike temporal O-IOR, return probability is influenced by cognitive processing. Taken together, these results are inconsistent with the hypothesis that IOR functions as a foraging facilitator. The results also provide strong evidence for a different oculomotor bias that could serve as a foraging facilitator: saccadic momentum, a tendency to repeat the most recently executed saccade program. We suggest that models of visual attention could incorporate saccadic momentum in place of IOR.

Saccadic momentum and facilitation of return saccades contribute to an optimal foraging strategy

The interest in saccadic IOR is funneled by the hypothesis that it serves a clear functional purpose in the selection of fixation points: the facilitation of foraging. In this study, we arrive at a different interpretation of saccadic IOR. First, we find that return saccades are performed much more often than expected from the statistical properties of saccades and saccade pairs. Second, we find that fixation durations before a saccade are modulated by the relative angle of the saccade, but return saccades show no sign of an additional temporal inhibition. Thus, we do not find temporal saccadic inhibition of return. Interestingly, we find that return locations are more salient, according to empirically measured saliency (locations that are fixated by many observers) as well as stimulus dependent saliency (defined by image features), than regular fixation locations. These results and the finding that return saccades increase the match of individual trajectories with a grand total priority map evidences the return saccades being part of a fixation selection strategy that trades off exploration and exploitation.

Sometimes humans look at the same location twice. To appreciate the importance of this inconspicuous statement you have to consider that we move our eyes several billion (10⁹) times during our lives and that looking at something is a necessary condition to enable conscious visual awareness. Thus, understanding why and how we move our eyes provides a window into our mental life. Here we investigate one heavily discussed aspect of human's fixation selection strategy: whether it inhibits returning to previously fixated locations. We analyze a large data set (more than 550,000 fixations from 235 subjects) and find that, returning to previously fixated locations happens much more often than expected from the statistical properties of eye-movement trajectories. Furthermore, those locations that we return to are not ordinary – they are more salient than locations that we do not return to. Thus, the inconspicuous statement that we look at the same locations twice reveals an important aspect of our strategy to select fixation points: That we trade off exploring our environment against making sure that we have fully comprehended the relevant parts of our environment.

When is it time to move to the next raspberry bush? Foraging rules in human visual search

Animals, including humans, engage in many forms of foraging behavior in which resources are collected from the world. This paper examines human foraging in a visual search context. A real-world analog would be berry picking. The selection of individual berries is not the most interesting problem in such a task. Of more interest is when does a forager leave one patch or berry bush for the next one? Marginal Value Theorem (MVT; Charnov, 1976) predicts that observers will leave a patch when the instantaneous yield from that patch drops below the average yield from the entire "field." Experiments 1, 2, 3, and 4 show that MVT gives a good description of human behavior for roughly uniform collections of patches. Experiments 5 and 6 show strong departures from MVT when patch quality varies and when visual information is degraded.

Dissociable spatial and temporal effects of inhibition of return

Inhibition of return (IOR) refers to the relative suppression of processing at locations that have recently been attended. It is frequently explored using a spatial cueing paradigm and is characterized by slower responses to cued than to uncued locations. The current study investigates the impact of IOR on overt visual orienting involving saccadic eye movements. Using a spatial cueing paradigm, our experiments have demonstrated that at a cue-target onset asynchrony (CTOA) of 400 ms saccades to the vicinity of cued locations are not only delayed (temporal cost) but also biased away (spatial effect). Both of these effects are basically no longer present at a CTOA of 1200 ms. At a shorter 200 ms CTOA, the spatial effect becomes stronger while the temporal cost is replaced by a temporal benefit. These findings suggest that IOR has a spatial effect that is dissociable from its temporal effect. Simulations using a neural field model of the superior colliculus (SC) revealed that a theory relying on short-term depression (STD) of the input pathway can explain most, but not all, temporal and spatial effects of IOR.

Guidance of visual search by memory and knowledge

To behave intelligently in the world, humans must be able to find objects efficiently within the complex environments they inhabit. A growing proportion of the literature on visual search is devoted to understanding this type of natural search. In the present chapter, I review the literature on visual search through natural scenes, focusing on the role of memory and knowledge in guiding attention to task-relevant objects.

Active inhibition and memory promote exploration and search of natural scenes

Active exploration of the visual world depends on sequential shifts of gaze that bring prioritized regions of a scene into central vision. The efficiency of this system is commonly attributed to a mechanism of "inhibition of return" (IOR) that discourages re-examination of previously-visited locations. Such a process is fundamental to computational models of attentional selection and paralleled by neurophysiological observations of inhibition of target-related activity in visuomotor areas. However, studies examining eye movements in naturalistic visual scenes appear to contradict the hypothesis that IOR promotes exploration. Instead, these reports reveal a surprisingly strong tendency to shift gaze back to the previously fixated location, suggesting that refixations might even be facilitated under natural conditions. Here we resolve this apparent contradiction, based on a probabilistic analysis of gaze patterns recorded during both free-viewing and search of naturalistic scenes. By simulating saccadic selection based on instantaneous influences alone, we show that the observed frequency of return saccades is in fact substantially less than predicted for a memoryless system, demonstrating that refixation is actively inhibited under natural viewing conditions. Furthermore, these observations reveal that gaze history significantly influences the way in which natural scenes are explored, contrary to accounts that suggest visual search has no memory.