Directional Prediction by the Saccadic System

Semantically predictable input streams impede gaze-orientation to surprising locations

When available, people use prior knowledge to predict dimensions of future events such as their location and semantic features. However, few studies have examined how multi-dimensional predictions are implemented, and mechanistic accounts are absent. Using eye tracking, we evaluated whether predictions of target-location and target-category interact during the earliest stages of orientation. We presented stochastic series so that across four conditions, participants could predict either the location of the next target-image, its semantic category, both dimensions, or neither. Participants observed images in absence of any task involving their semantic content. We modeled saccade latencies using ELATER, a rise-to-threshold model that accounts for accumulation rate (AR), variance of AR over trials, and variance of decision baseline. The main findings were: 1) AR scaled with the degree of surprise associated with a target's location; 2) predictability of semantic-category hindered saccade latencies, suggesting a bottleneck in implementing joint predictions; 3) saccades to targets that satisfied semantic expectations were associated with greater AR-variance than saccades to semantically-surprising images, consistent with a richer repertoire of early evaluative processes for semantically-expected images. Predictability of target-category also impacted gaze pre-positioning prior to target presentation. The results indicate a strong interaction between foreknowledge of object location and semantics during stimulus-guided saccades, and suggest statistical regularities in an input stream can also impact anticipatory, non-stimulus-guided processes.

Modelling Prosaccade Latencies across Multiple Decision-Making Tasks

Oculomotor decision making can be investigated by a simple step task, where a person decides whether a target has jumped to the left or the right. More complex tasks include the countermanding task (look at the jumped target, except when a subsequent signal instructs you not to) and the Wheeless task (where the jumped target sometimes then quickly jumps to a new location). Different instantiations of the LATER (Linear Approach to Threshold with Ergodic Rate) model have been shown to explain the saccadic latency data arising from these tasks, despite it being almost inconceivable that completely separate decision-making mechanisms exist for each. However, these models have an identical construction with regards to predicting prosaccadic responses (all step task trials, and control trials in countermanding and Wheeless tasks, where no countermanding signal is given or when the target does not make a second jump). We measured saccadic latencies for 23 human observers each performing the three tasks, and modelled prosaccade latencies with LATER to see if model parameters were usefully preserved across tasks. We found no significant difference in reaction times and model parameters between the step and Wheeless tasks (mean 175 and 177 ms, respectively; standard deviation, SD 22 and 24 ms). In contrast, we identified prolonged latencies in the countermanding tasks (236 ms; SD 37 ms) explained by a slower rise and an elevated threshold of the decision making signal, suggesting elevated participant caution. Our findings support the idea that common machinery exists for oculomotor decision-making, which can be flexibly deployed depending upon task demands.

Choice-induced inter-trial inhibition is modulated by idiosyncratic choice-consistency

Humans constantly decide among multiple action plans. Carrying out one action usually implies that other plans are suppressed. Here we make use of inter-trial effects to determine whether suppression of non-chosen action plans is due to proactively preparing for upcoming decisions or due to retroactive influences from previous decisions. Participants received rewards for timely and accurate saccades to targets appearing left or right from fixation. Each block interleaved trials with one (single-trial) or two targets (choice-trial). Whereas single-trial rewards were always identical, rewards for the two targets in choice-trials could either be identical (unbiased) or differ (biased) within one block. We analyzed single-trial latencies as a function of idiosyncratic choice-consistency or reward-bias, the previous trial type and whether the same or the other target was selected in the preceding trial. After choice-trials, single-trial responses to the previously non-chosen target were delayed. For biased choices, inter-trial effects were strongest when choices were followed by a single-trial to the non-chosen target. In the unbiased condition, inter-trial effects increased with increasing individual consistency of choice behavior. These findings suggest that the suppression of alternative action plans is not coupled to target selection and motor execution but instead depends on top-down signals like the overall preference of one target over another.

Trajectory curvature in saccade sequences: spatiotopic influences vs. residual motor activity

When decisions drive saccadic eye movements, traces of the decision process can be inferred from the movement trajectories. For example, saccades can curve away from distractor stimuli, which was thought to reflect cortical inhibition biasing activity in the superior colliculus. Recent neurophysiological work does not support this theory, and two recent models have replaced top-down inhibition with lateral interactions in the superior colliculus or neural fatigue in the brainstem saccadic burst generator. All current models operate in retinotopic coordinates and are based on single saccade paradigms. To extend these models to sequences of saccades, we assessed whether and how saccade curvature depends on previously fixated locations and the direction of previous saccades. With a two-saccade paradigm, we first demonstrated that second saccades curved away from the initial fixation stimulus. Furthermore, by varying the time from fixation offset and the intersaccadic duration, we distinguished the extent of curvature originating from the spatiotopic representation of the previous fixation location or residual motor activity of the previous saccade. Results suggest that both factors drive curvature, and we discuss how these effects could be implemented in current models. In particular, we propose that the collicular retinotopic maps receive an excitatory spatiotopic update from the lateral interparial region.NEW & NOTEWORTHY Saccades curve away from locations of previous fixation. Varying stimulus timing demonstrates the effects of both 1) spatiotopic representation and 2) motor residual activity from previous saccades. The spatiotopic effect can be explained if current models are augmented with an excitatory top-down spatiotopic signal.

Directional interactions between current and prior saccades

One way to explore how prior sensory and motor events impact eye movements is to ask someone to look to targets located about a central point, returning gaze to the central point after each eye movement. Concerned about the contribution of this return to center movement, Anderson et al. (2008) used a sequential saccade paradigm in which participants made a continuous series of saccades to peripheral targets that appeared to the left or right of the currently fixated location in a random sequence (the next eye movement began from the last target location). Examining the effects of previous saccades (n−x) on current saccade latency (n), they found that saccadic reaction times (RT) were reduced when the direction of the current saccade matched that of a preceding saccade (e.g., two left saccades), even when the two saccades in question were separated by multiple saccades in any direction. We examined if this pattern extends to conditions in which targets appear inside continuously marked locations that provide stable visual features (i.e., target “placeholders”) and when saccades are prompted by central arrows. Participants completed 3 conditions: peripheral targets (PT; continuous, sequential saccades to peripherally presented targets) without placeholders; PT with placeholders; and centrally presented arrows (CA; left or right pointing arrows at the currently fixated location instructing participants to saccade to the left or right). We found reduced saccadic RT when the immediately preceding saccade (n−1) was in the same (vs. opposite) direction in the PT without placeholders and CA conditions. This effect varied when considering the effect of the previous 2–5 (n−x) saccades on current saccade latency (n). The effects of previous eye movements on current saccade latency may be determined by multiple, time-varying mechanisms related to sensory (i.e., retinotopic location), motor (i.e., saccade direction), and environmental (i.e., persistent visual objects) factors.

Target direction rather than position determines oculomotor expectation in repeating sequences

Saccadic latencies to targets appearing to the left and right of fixation in a repeating sequence are significantly increased when a target is presented out of sequence. Is this because the target is in the wrong position, the wrong direction, or both? To find out, we arranged for targets in a horizontal plane occasionally to appear with an unexpected eccentricity, though in the correct direction. This had no significant effect on latency, unlike what is observed when targets appeared in the unexpected direction. That subjects learnt sequences of directions rather than simply positions was further confirmed in an experiment where saccade direction was a repeating sequence, but eccentricity was randomised. Latency was elevated when a target was episodically presented in an unexpected direction. Latencies were also elevated when targets appeared in the correct hemifield but at an unexpected direction (35° polar angular displacement from the horizontal, a displacement roughly equivalent in collicular spacing to our unexpected eccentricity), although this elevation was of a smaller magnitude than when targets appeared in an unexpected direction along the horizontal. Finally, we confirmed that not all changes in the stimulus cause disruption: an unexpected change in the orientation or colour of the target did not alter latency. Our results show that in a repeating sequence, the oculomotor system is primarily concerned with predicting the direction of an upcoming eye movement rather than its position. This is consistent with models of oculomotor control developed for randomly appearing targets in which the direction and amplitude of saccades are programmed separately.

Saccadic momentum and facilitation of return saccades contribute to an optimal foraging strategy

The interest in saccadic IOR is funneled by the hypothesis that it serves a clear functional purpose in the selection of fixation points: the facilitation of foraging. In this study, we arrive at a different interpretation of saccadic IOR. First, we find that return saccades are performed much more often than expected from the statistical properties of saccades and saccade pairs. Second, we find that fixation durations before a saccade are modulated by the relative angle of the saccade, but return saccades show no sign of an additional temporal inhibition. Thus, we do not find temporal saccadic inhibition of return. Interestingly, we find that return locations are more salient, according to empirically measured saliency (locations that are fixated by many observers) as well as stimulus dependent saliency (defined by image features), than regular fixation locations. These results and the finding that return saccades increase the match of individual trajectories with a grand total priority map evidences the return saccades being part of a fixation selection strategy that trades off exploration and exploitation.

Sometimes humans look at the same location twice. To appreciate the importance of this inconspicuous statement you have to consider that we move our eyes several billion (10⁹) times during our lives and that looking at something is a necessary condition to enable conscious visual awareness. Thus, understanding why and how we move our eyes provides a window into our mental life. Here we investigate one heavily discussed aspect of human's fixation selection strategy: whether it inhibits returning to previously fixated locations. We analyze a large data set (more than 550,000 fixations from 235 subjects) and find that, returning to previously fixated locations happens much more often than expected from the statistical properties of eye-movement trajectories. Furthermore, those locations that we return to are not ordinary – they are more salient than locations that we do not return to. Thus, the inconspicuous statement that we look at the same locations twice reveals an important aspect of our strategy to select fixation points: That we trade off exploring our environment against making sure that we have fully comprehended the relevant parts of our environment.

The spatial impact of visual distractors on saccade latency

Remote transient changes in the environment, such as the onset of visual distractors, impact on the execution of target directed saccadic eye movements. Studies that have examined the latency of the saccade response have shown conflicting results. When there was an element of target selection, saccade latency increased as the distance between distractor and target increased. In contrast, when target selection is minimized by restricting the target to appear on one axis position, latency has been found to be slowest when the distractor is shown at fixation and reduces as it moves away from this position, rather than from the target. Here we report four experiments examining saccade latency as target and distractor positions are varied. We find support for both a dependence of saccade latency on distractor distance from target and from fixation: saccade latency was longer when distractor is shown close to fixation and even longer still when shown in an opposite location (180°) to the target. We suggest that this is due to inhibitory interactions between the distractor, fixation and the target interfering with fixation disengagement and target selection.

Context-gated statistical learning and its role in visual-saccadic decisions

Adaptive behavior in a nonstationary world requires humans to learn and track the statistics of the environment. We examined the mechanisms of adaptation in a nonstationary environment in the context of visual-saccadic inhibition of return (IOR). IOR is adapted to the likelihood that return locations will be refixated in the near future. We examined 2 potential learning mechanisms underlying adaptation: (a) a local tracking or priming mechanism that facilitates behavior that is consistent with recent experience and (b) a mechanism that supports retrieval of knowledge of the environmental statistics based on the contextual features of the environment. Participants generated sequences of 2 saccadic eye movements in conditions where the probability that the 2nd saccade was directed back to the previously fixated location varied from low (.17) to high (.50). In some conditions, the contingency was signaled by a contextual cue (the shape of the movement cue). Adaptation occurred in the absence of contextual signals but was more pronounced in the presence of contextual cues. Adaptation even occurred when different contingencies were randomly intermixed, showing the parallel formation of multiple associations between context and statistics. These findings are accounted for by an evidence accumulation framework in which the resting baseline of decision alternatives is adjusted on a trial-by-trial basis. This baseline tracks the subjective prior beliefs about the behavioral relevance of the different alternatives and is updated on the basis of the history of recent events and the contextual features of the current environment.

Economic value biases uncertain perceptual choices in the parietal and prefrontal cortices

An observer detecting a noisy sensory signal is biased by the costs and benefits associated with its presence or absence. When these costs and benefits are asymmetric, sensory, and economic information must be integrated to inform the final choice. However, it remains unknown how this information is combined at the neural or computational levels. To address this question, we asked healthy human observers to judge the presence or absence of a noisy sensory signal under economic conditions that favored yes responses (liberal blocks), no responses (conservative blocks), or neither response (neutral blocks). Economic information biased fast choices more than slow choices, suggesting that value and sensory information are integrated early in the decision epoch. More formal simulation analyses using an Ornstein–Uhlenbeck process demonstrated that the influence of economic information was best captured by shifting the origin of evidence accumulation toward the more valuable bound. We then used the computational model to generate trial-by-trial estimates of decision-related evidence that were based on combined sensory and economic information (the decision variable or DV), and regressed these against fMRI activity recorded whilst participants performed the task. Extrastriate visual regions responded to the level of sensory input (momentary evidence), but fMRI signals in the parietal and prefrontal cortices responded to the decision variable. These findings support recent single-neuron data suggesting that economic information biases decision-related signals in higher cortical regions.

Saccadic preparation in the frontal eye field is modulated by distinct trial history effects as revealed by magnetoencephalography

Optimizing outcomes involves rapidly and continuously adjusting behavior based on context. While most behavioral studies focus on immediate task conditions, responses to events are also influenced by recent history. We used magnetoencephalography and a saccadic paradigm to investigate the neural bases of 2 trial history effects that are well characterized in the behavioral eye movement literature: task-switching and the prior-antisaccade effect. We found that switched trials were associated with increased errors and transient increases in activity in the frontal eye field (FEF) and anterior cingulate cortex early in the preparatory period. These activity changes are consistent with active reconfiguration of the task set, a time-limited process that is triggered by the instructional cue. Following an antisaccade versus prosaccade, there was increased activity in the FEF and prefrontal cortex that persisted into the preparatory period of the subsequent trial, and saccadic latencies were prolonged. We attribute these effects to persistent inhibition of the ocular motor response system from the prior antisaccade. These findings refine our understanding of how trial history interacts with current task demands to adjust responses. Such dynamic modulations of neural activity and behavior by recent experience are at the heart of adaptive flexible behavior.

Influence of environmental statistics on inhibition of saccadic return

Initiating an eye movement is slowed if the saccade is directed to a location that has been fixated in the recent past. We show that this inhibitory effect is modulated by the temporal statistics of the environment: If a return location is likely to become behaviorally relevant, inhibition of return is absent. By fitting an accumulator model of saccadic decision-making, we show that the inhibitory effect and the sensitivity to local statistics can be dissociated in their effects on the rate of accumulation of evidence, and the threshold controlling the amount of evidence needed to generate a saccade.

The influence of motor training on human express saccade production

Express saccadic eye movements are saccades of extremely short latency. In monkey, express saccades have been shown to occur much more frequently when the monkey has been trained to make saccades in a particular direction to targets that appear in predictable locations. Such results suggest that express saccades occur in large number only under highly specific conditions, leading to the view that vector-specific training and motor preparatory processes are required to make an express saccade of a particular magnitude and direction. To evaluate this hypothesis in humans, we trained subjects to make saccades quickly to particular locations and then examined whether the frequency of express saccades depended on training and the number of possible target locations. Training significantly decreased saccade latency and increased express saccade production to both trained and untrained locations. Increasing the number of possible target locations (two vs. eight possible targets) led to only a modest increase of saccade latency. For most subjects, the probability of express saccade occurrence was much higher than that expected if vector-specific movement preparation were necessary for their production. These results suggest that vector-specific motor preparation and vector-specific saccade training are not necessary for express saccade production in humans and that increases in express saccade production are due in part to a facilitation in fixation disengagement or else a general enhancement in the ability of the saccadic system to respond to suddenly appearing visual stimuli.

The mechanism underlying inhibition of saccadic return

Human observers take longer to re-direct gaze to a previously fixated location. Although there has been some exploration of the characteristics of inhibition of saccadic return (ISR), the exact mechanisms by which ISR operates are currently unknown. In the framework of accumulation models of response times, in which evidence is integrated over time to a response threshold, ISR could reflect a reduction in the rate of accumulation for saccades to return locations or an increase in the effective criterion for response. In two experiments, participants generated sequences of three saccades, in response to a peripheral or a central cue. ISR occurred across these manipulations: saccade latency was consistently increased for movements to the immediately previously fixated location. Latency distributions from individual observers were fit with a Linear Ballistic Accumulator model. ISR was best accounted for as a change in the accumulation rate. We suggest this parameter represents the overall desirability of a particular course of action, the evidence for which may be derived from a variety of sensory and non-sensory sources.