Saccades curve away from previously inhibited locations: evidence for the role of priming in oculomotor competition

Saccadic landing positions reveal that eye movements are affected by distractor-based retrieval

Binding theories assume that stimulus and response features are integrated into short-lasting episodes and that upon repetition of any feature the whole episode is retrieved, thereby affecting performance. Such binding theories are nowadays the standard explanation for a wide range of action control tasks and aim to explain all simple actions, without making assumptions of effector specificity. Yet, it is unclear if eye movements are affected by integration and retrieval in the same way as manual responses. We asked participants to discriminate letters framed by irrelevant shapes. In Experiment 1, participants gave their responses with eye movements. Saccade landing positions showed a spatial error pattern consistent with predictions of binding theories. Saccadic latencies were not affected. In Experiment 2 with an increased interval between prime and probe, the error pattern diminished, again congruent with predictions of binding theories presuming quickly decaying retrieval effects. Experiment 3 used the same task as in Experiment 1, but participants executed their responses with manual key presses; again, we found a binding pattern in response accuracy. We conclude that eye movements and manual responses are affected by the same integration and retrieval processes, supporting the tacit assumption of binding theories to apply to any effector.

Sounds are remapped across saccades

To achieve visual space constancy, our brain remaps eye-centered projections of visual objects across saccades. Here, we measured saccade trajectory curvature following the presentation of visual, auditory, and audiovisual distractors in a double-step saccade task to investigate if this stability mechanism also accounts for localized sounds. We found that saccade trajectories systematically curved away from the position at which either a light or a sound was presented, suggesting that both modalities are represented in eye-centered oculomotor centers. Importantly, the same effect was observed when the distractor preceded the execution of the first saccade. These results suggest that oculomotor centers keep track of visual, auditory and audiovisual objects by remapping their eye-centered representations across saccades. Furthermore, they argue for the existence of a supra-modal map which keeps track of multi-sensory object locations across our movements to create an impression of space constancy.

Choice-induced inter-trial inhibition is modulated by idiosyncratic choice-consistency

Humans constantly decide among multiple action plans. Carrying out one action usually implies that other plans are suppressed. Here we make use of inter-trial effects to determine whether suppression of non-chosen action plans is due to proactively preparing for upcoming decisions or due to retroactive influences from previous decisions. Participants received rewards for timely and accurate saccades to targets appearing left or right from fixation. Each block interleaved trials with one (single-trial) or two targets (choice-trial). Whereas single-trial rewards were always identical, rewards for the two targets in choice-trials could either be identical (unbiased) or differ (biased) within one block. We analyzed single-trial latencies as a function of idiosyncratic choice-consistency or reward-bias, the previous trial type and whether the same or the other target was selected in the preceding trial. After choice-trials, single-trial responses to the previously non-chosen target were delayed. For biased choices, inter-trial effects were strongest when choices were followed by a single-trial to the non-chosen target. In the unbiased condition, inter-trial effects increased with increasing individual consistency of choice behavior. These findings suggest that the suppression of alternative action plans is not coupled to target selection and motor execution but instead depends on top-down signals like the overall preference of one target over another.

Updating spatial working memory in a dynamic visual environment

The present review describes recent developments regarding the role of the eye movement system in representing spatial information and keeping track of locations of relevant objects. First, we discuss the active vision perspective and why eye movements are considered crucial for perception and attention. The second part focuses on the question of how the oculomotor system is used to represent spatial attentional priority, and the role of the oculomotor system in maintenance of this spatial information. Lastly, we discuss recent findings demonstrating rapid updating of information across saccadic eye movements. We argue that the eye movement system plays a key role in maintaining and rapidly updating spatial information. Furthermore, we suggest that rapid updating emerges primarily to make sure actions are minimally affected by intervening eye movements, allowing us to efficiently interact with the world around us.

Target-distractor competition cannot be resolved across a saccade

When a distractor is presented in close spatial proximity to a target, a saccade tends to land in between the two objects rather than on the target. This robust phenomenon (also referred to as the global effect) is thought to reflect unresolved competition between target and distractor. It is unclear whether this landing bias persists across saccades since a saccade displaces the retinotopic representations of target and distractor. In the present study participants made successive saccades towards two saccadic targets which were presented simultaneously with an irrelevant distractor in close proximity to the second saccade target. The second saccade was either visually-guided or memory-guided. For the memory-guided trials, the second saccade showed a landing bias towards the location of the distractor, despite the disappearance of the distractor after the first saccade. In contrast, for the visually-guided trials, the bias was corrected and the landing bias was eliminated, even for saccades with the shortest intersaccadic intervals. This suggests that the biased saccade plan was remapped across the first saccade. Therefore, we conclude that the target-distractor competition was not resolved across a saccade, but can be resolved based on visual information that is available after a saccade.

Higher order, multifeatural object encoding by the oculomotor system

Previous behavioral and physiological research has demonstrated that as the behavioral relevance of potential saccade goals increases, they elicit more competition during target selection processing as evidenced by increased saccade curvature and neural activity. However, these effects have only been demonstrated for lower order feature singletons, and it remains unclear whether more complicated featural differences between higher order objects also elicit vector modulation. Therefore, we measured human saccades curvature elicited by distractors bilaterally flanking a target during a visual search saccade task and systematically varied subsets of features shared between the two distractors and the target, referred to as objective similarity (OS). Our results demonstrate that saccades deviated away from the distractor highest in OS to the target and that there was a linear relationship between the magnitude of saccade deviation and the number of feature differences between the most similar distractor and the target. Furthermore, an analysis of curvature over the time course of the saccade demonstrated that curvature only occurred in the first 20-30 ms of the movement. Given the multifeatural complexity of the novel stimuli, these results suggest that saccadic target selection processing involves dynamically reweighting vector representations for movement planning to several possible targets based on their behavioral relevance. NEW & NOTEWORTHY We demonstrate that small featural differences between unfamiliar, higher order object representations modulate vector weights during saccadic target selection processing. Such effects have previously only been demonstrated for familiar, simple feature singletons (e.g., color) in which features characterize entire objects. The complexity and novelty of our stimuli suggest that the oculomotor system dynamically receives visual/cognitive information processed in the higher order representational networks of the cortical visual processing hierarchy and integrates this information for saccadic movement planning.

Perceptual Color Space Representations in the Oculomotor System Are Modulated by Surround Suppression and Biased Selection

The oculomotor system utilizes color extensively for planning saccades. Therefore, we examined how the oculomotor system actually encodes color and several factors that modulate these representations: attention-based surround suppression and inherent biases in selecting and encoding color categories. We measured saccade trajectories while human participants performed a memory-guided saccade task with color targets and distractors and examined whether oculomotor target selection processing was functionally related to the CIE (x,y) color space distances between color stimuli and whether there were hierarchical differences between color categories in the strength and speed of encoding potential saccade goals. We observed that saccade planning was modulated by the CIE (x,y) distances between stimuli thus demonstrating that color is encoded in perceptual color space by the oculomotor system. Furthermore, these representations were modulated by (1) cueing attention to a particular color thereby eliciting surround suppression in oculomotor color space and (2) inherent selection and encoding biases based on color category independent of cueing and perceptual discriminability. Since surround suppression emerges from recurrent feedback attenuation of sensory projections, observing oculomotor surround suppression suggested that oculomotor encoding of behavioral relevance results from integrating sensory and cognitive signals that are pre-attenuated based on task demands and that the oculomotor system therefore does not functionally contribute to this process. Second, although perceptual discriminability did partially account for oculomotor processing differences between color categories, we also observed preferential processing of the red color category across various behavioral metrics. This is consistent with numerous previous studies and could not be simply explained by perceptual discriminability. Since we utilized a memory-guided saccade task, this indicates that the biased processing of the red color category does not rely on sustained sensory input and must therefore involve cortical areas associated with the highest levels of visual processing involved in visual working memory.

Rapid updating of spatial working memory across saccades

Each time we make an eye movement, positions of objects on the retina change. In order to keep track of relevant objects their positions have to be updated. The situation becomes even more complex if the object is no longer present in the world and has to be held in memory. In the present study, we used saccadic curvature to investigate the time-course of updating a memorized location across saccades. Previous studies have shown that a memorized location competes with a saccade target for selection on the oculomotor map, which leads to saccades curving away from it. In our study participants performed a sequence of two saccades while keeping a location in memory. The trajectory of the second saccade was used to measure when the memorized location was updated after the first saccade. The results showed that the memorized location was rapidly updated with the eyes curving away from its spatial coordinates within 130 ms after the first eye movement. The time-course of updating was comparable to the updating of an exogenously attended location, and depended on how well the location was memorized.

The necessity to choose causes the effects of reward on saccade preparation

When humans have to choose between different options, they can maximize their payoff by choosing the option that yields the highest reward. Information about reward is not only used to optimize decisions but also for movement preparation to minimize reaction times to rewarded targets. Here, we show that this is especially true in contexts in which participants additionally have to choose between different options. We probed eye movement preparation by measuring saccade latencies to differently rewarded single targets (single-trial) appearing left or right from fixation. In choice-trials, both targets were displayed and participants were free to decide for one target to receive the corresponding reward. In blocks without choice-trials, single-trial latencies were not or only weakly affected by reward. With choice-trials present, the influence of reward increased with the proportion and difficulty of choices and decreased when a cue indicated that no choice will be necessary. Choices caused a delay in subsequent single-trial responses to the non-chosen option. Taken together, our results suggest that reward affects saccade preparation mainly when the outcome is uncertain and depends on the participants’ behavior, for instance when they have to choose between targets differing in reward.

The Role of the Oculomotor System in Updating Visual-Spatial Working Memory across Saccades

Visual-spatial working memory (VSWM) helps us to maintain and manipulate visual information in the absence of sensory input. It has been proposed that VSWM is an emergent property of the oculomotor system. In the present study we investigated the role of the oculomotor system in updating of spatial working memory representations across saccades. Participants had to maintain a location in memory while making a saccade to a different location. During the saccade the target was displaced, which went unnoticed by the participants. After executing the saccade, participants had to indicate the memorized location. If memory updating fully relies on cancellation driven by extraretinal oculomotor signals, the displacement should have no effect on the perceived location of the memorized stimulus. However, if postsaccadic retinal information about the location of the saccade target is used, the perceived location will be shifted according to the target displacement. As it has been suggested that maintenance of accurate spatial representations across saccades is especially important for action control, we used different ways of reporting the location held in memory; a match-to-sample task, a mouse click or by making another saccade. The results showed a small systematic target displacement bias in all response modalities. Parametric manipulation of the distance between the to-be-memorized stimulus and saccade target revealed that target displacement bias increased over time and changed its spatial profile from being initially centered on locations around the saccade target to becoming spatially global. Taken together results suggest that we neither rely exclusively on extraretinal nor on retinal information in updating working memory representations across saccades. The relative contribution of retinal signals is not fixed but depends on both the time available to integrate these signals as well as the distance between the saccade target and the remembered location.

Looking away: distractor influences on saccadic trajectory and endpoint in prosaccade and antisaccade tasks

Successful target selection often occurs concurrently with distractor inhibition. A better understanding of the former thus requires a thorough study of the competition that arises between target and distractor representations. In the present study, we explore whether the presence of a distractor influences saccade processing via interfering with visual target and/or saccade goal representations. To do this, we asked participants to make either pro- or antisaccade eye movements to a target and measured the change in their saccade trajectory and landing position (collectively referred to as deviation) in response to distractors placed near or far from the saccade goal. The use of an antisaccade paradigm may help to distinguish between stimulus- and goal-related distractor interference, as unlike with prosaccades, these two features are dissociated in space when making a goal-directed antisaccade response away from a visual target stimulus. The present results demonstrate that for both pro- and antisaccades, distractors near the saccade goal elicited the strongest competition, as indicated by greater saccade trajectory deviation and landing position error. Though distractors far from the saccade goal elicited, on average, greater deviation away in antisaccades than in prosaccades, a time-course analysis revealed a significant effect of far-from-goal distractors in prosaccades as well. Considered together, the present findings support the view that goal-related representations most strongly influence the saccade metrics tested, though stimulus-related representations may play a smaller role in determining distractor-based interference effects on saccade execution under certain circumstances. Further, the results highlight the advantage of considering temporal changes in distractor-based interference.

The Effects of Feature-Based Priming and Visual Working Memory on Oculomotor Capture

Recently, it has been demonstrated that objects held in working memory can influence rapid oculomotor selection. This has been taken as evidence that perceptual salience can be modified by active working memory representations. The goal of the present study was to examine whether these results could also be caused by feature-based priming. In two experiments, participants were asked to saccade to a target line segment of a certain orientation that was presented together with a to-be-ignored distractor. Both objects were given a task-irrelevant color that varied per trial. In a secondary task, a color had to be memorized, and that color could either match the color of the target, match the color of the distractor, or it did not match the color of any of the objects in the search task. The memory task was completed either after the search task (Experiment 1), or before it (Experiment 2). The results showed that in both experiments the memorized color biased oculomotor selection. Eye movements were more frequently drawn towards objects that matched the memorized color, irrespective of whether the memory task was completed after (Experiment 1) or before (Experiment 2) the search task. This bias was particularly prevalent in short-latency saccades. The results show that early oculomotor selection performance is not only affected by properties that are actively maintained in working memory but also by those previously memorized. Both working memory and feature priming can cause early biases in oculomotor selection.

Common Priority Map for Selection History, Reward and Emotion in the Oculomotor System

In natural scenes, many objects compete for visual selection. However, it is not always clear why certain objects win this competition. I will demonstrate that the eye movement system lives in a constant state of competition among different oculomotor programs. This competition is not limited to the competition between the current goals of the observer and salient objects in the environment but incorporates independent influences from memory, reward, and emotional systems. These involuntary and automatic biases often overcome the goal-directed selection and expose severe limits in goal-driven control. There is also a striking similarity in the way that these very different sources of bias activate the oculomotor system and compete for representation. The inputs from various information sources are integrated in the common map in the oculomotor system for the sole purpose of improving the efficiency of oculomotor selection.

Target-distractor competition in the oculomotor system is spatiotopic

In natural scenes, multiple visual stimuli compete for selection; however, each saccade displaces the stimulus representations in retinotopicaly organized visual and oculomotor maps. In the present study, we used saccade curvature to investigate whether oculomotor competition across eye movements is represented in retinotopic or spatiotopic coordinates. Participants performed a sequence of saccades and we induced oculomotor competition by briefly presenting a task-irrelevant distractor at different times during the saccade sequence. Despite the intervening saccade, the second saccade curved away from a spatial representation of the distractor that was presented before the first saccade. Furthermore, the degree of saccade curvature increased with the salience of the distractor presented before the first saccade. The results suggest that spatiotopic representations of target-distractor competition are crucial for successful interaction with objects of interest despite the intervening eye movements.