Spatial constancy of attention across eye movements is mediated by the presence of visual objects

Different extrapolation of moving object locations in perception, smooth pursuit, and saccades

The ability to accurately perceive and track moving objects is crucial for many everyday activities. In this study, we use a "double-drift stimulus" to explore the processing of visual motion signals that underlie perception, pursuit, and saccade responses to a moving object. Participants were presented with peripheral moving apertures filled with noise that either drifted orthogonally to the aperture's direction or had no net motion. Participants were asked to saccade to and track these targets with their gaze as soon as they appeared and then to report their direction. In the trials with internal motion, the target disappeared at saccade onset so that the first 100 ms of the postsaccadic pursuit response was driven uniquely by peripheral information gathered before saccade onset. This provided independent measures of perceptual, pursuit, and saccadic responses to the double-drift stimulus on a trial-by-trial basis. Our analysis revealed systematic differences between saccadic responses, on one hand, and perceptual and pursuit responses, on the other. These differences are unlikely to be caused by differences in the processing of motion signals because both saccades and pursuits seem to rely on shared target position and velocity information. We conclude that our results are instead due to a difference in how the processing mechanisms underlying perception, pursuit, and saccades combine motor signals with target position. These findings advance our understanding of the mechanisms underlying dissociation in visual processing between perception and eye movements.

Perceptual decisions and oculomotor responses rely on temporally distinct streams of evidence

Perceptual decisions often require the integration of noisy sensory evidence over time. This process is formalized with sequential sampling models, where evidence is accumulated up to a decision threshold before a choice is made. Although intuition suggests that decision formation must precede the preparation of a motor response (i.e., the action used to communicate the choice), neurophysiological findings have suggested that these two processes might be one and the same. To test this idea, we developed a reverse-correlation protocol in which the visual stimuli that influence decisions can be distinguished from those guiding motor responses. In three experiments, we found that the temporal weighting function of oculomotor responses did not overlap with the relatively early weighting function of stimulus properties having an impact on decision formation. These results support a timeline in which perceptual decisions are formed, at least in part, prior to the preparation of a motor response.

A paradigm in which visual stimuli that influence decisions can be distinguished from those guiding motor responses demonstrates that in humans, perceptual decisions are formed, at least in part, prior to the preparation of a motor response.

Low-Level Visual Information Is Maintained across Saccades, Allowing for a Postsaccadic Handoff between Visual Areas

Experience seems continuous and detailed despite saccadic eye movements changing retinal input several times per second. There is debate whether neural signals related to updating across saccades contain information about stimulus features, or only location pointers without visual details. We investigated the time course of low-level visual information processing across saccades by decoding the spatial frequency of a stationary stimulus that changed from one visual hemifield to the other because of a horizontal saccadic eye movement. We recorded magnetoencephalography while human subjects (both sexes) monitored the orientation of a grating stimulus, making spatial frequency task irrelevant. Separate trials, in which subjects maintained fixation, were used to train a classifier, whose performance was then tested on saccade trials. Decoding performance showed that spatial frequency information of the presaccadic stimulus remained present for ∼200 ms after the saccade, transcending retinotopic specificity. Postsaccadic information ramped up rapidly after saccade offset. There was an overlap of over 100 ms during which decoding was significant from both presaccadic and postsaccadic processing areas. This suggests that the apparent richness of perception across saccades may be supported by the continuous availability of low-level information with a "soft handoff" of information during the initial processing sweep of the new fixation.SIGNIFICANCE STATEMENT Saccades create frequent discontinuities in visual input, yet perception appears stable and continuous. How is this discontinuous input processed resulting in visual stability? Previous studies have focused on presaccadic remapping. Here we examined the time course of processing of low-level visual information (spatial frequency) across saccades with magnetoencephalography. The results suggest that spatial frequency information is not predictively remapped but also is not discarded. Instead, they suggest a soft handoff over time between different visual areas, making this information continuously available across the saccade. Information about the presaccadic stimulus remains available, while the information about the postsaccadic stimulus has also become available. The simultaneous availability of both the presaccadic and postsaccadic information could enable rich and continuous perception across saccades.

Location-cued visual selection-Placeholder dots improve target identification

Visual cues help to select a target and attract attention to it. In the present study, a 50-ms exogenous cue was presented to select one of 80 tilted lines, and attention effects at various delays were measured as the time observers needed to identify this target. Like in earlier detection studies, there was a transient cuing effect; targets presented soon after the cue (delays of 50-300 ms) were identified particularly fast. This benefit was followed by a continuous decay of performance toward longer delays (measured up to 5 s), at which the necessary presentation time to identify the target was strongly increased. The decay was substantially reduced when placeholder dots were shown during the delay, at subsequent line positions. The simple presentation of a structured background in the form of random dots did not have this effect. When the presentation times for constant performance were taken to compute the presumed strength of underlying neural responses, the effect of placeholders was seen as a nearly constant addition to the cued target signals, with an additional transient peak about 100 ms after cue (and placeholders) onset.

Updating spatial working memory in a dynamic visual environment

The present review describes recent developments regarding the role of the eye movement system in representing spatial information and keeping track of locations of relevant objects. First, we discuss the active vision perspective and why eye movements are considered crucial for perception and attention. The second part focuses on the question of how the oculomotor system is used to represent spatial attentional priority, and the role of the oculomotor system in maintenance of this spatial information. Lastly, we discuss recent findings demonstrating rapid updating of information across saccadic eye movements. We argue that the eye movement system plays a key role in maintaining and rapidly updating spatial information. Furthermore, we suggest that rapid updating emerges primarily to make sure actions are minimally affected by intervening eye movements, allowing us to efficiently interact with the world around us.

Feature-based attention across saccades: Pop-out in color search is spatiotopic

Our perception of the world remains stable despite the retinal shifts that occur with each saccade. The role of spatial attention in matching pre- to postsaccadic visual information has been well established, but the role of feature-based attention remains unclear. In this study, we examined the transsaccadic processing of a color pop-out target. Participants made a saccade towards a neutral target and performed a search task on a peripheral array presented once the saccade landed. A similar array was presented just before the saccade and we analyzed what aspect of this preview benefitted the postsaccadic search task. We assessed the preview effect in the spatiotopic and retinotopic reference frames, and the potential transfer of feature selectivity across the saccade. In the first experiment, the target and distractor colors remained identical for the preview and the postsaccadic array and performance improved. The largest benefit was observed at the spatiotopic location. In the second experiment, the target and distractor colors were swapped across the saccade. All responses were slowed but the cost was least at the spatiotopic location. Our results show that the preview attracted spatial attention to the target location, which was then remapped, and suggest that previewed features, specifically colors, were transferred across the saccade. Furthermore, the preview induced a spatiotopic advantage regardless of whether the target switched color or not, suggesting that spatiotopy was established independently of feature processing. Our results support independent priming effects of features versus location and underline the role of feature-based selection in visual stability.

Remapping locations and features across saccades: a dual-spotlight theory of attentional updating

How do we maintain visual stability across eye movements? Much work has focused on how visual information is rapidly updated to maintain spatiotopic representations. However, predictive spatial remapping is only part of the story. Here I review key findings, recent debates, and open questions regarding remapping and its implications for visual attention and perception. This review focuses on two key questions: when does remapping occur, and what is the impact on feature perception? Findings are reviewed within the framework of a two-stage, or dual- spotlight, remapping process, where spatial attention must be both updated to the new location (fast, predictive stage) and withdrawn from the previous retinotopic location (slow, post-saccadic stage), with a particular focus on the link between spatial and feature information across eye movements.

Rapid updating of spatial working memory across saccades

Each time we make an eye movement, positions of objects on the retina change. In order to keep track of relevant objects their positions have to be updated. The situation becomes even more complex if the object is no longer present in the world and has to be held in memory. In the present study, we used saccadic curvature to investigate the time-course of updating a memorized location across saccades. Previous studies have shown that a memorized location competes with a saccade target for selection on the oculomotor map, which leads to saccades curving away from it. In our study participants performed a sequence of two saccades while keeping a location in memory. The trajectory of the second saccade was used to measure when the memorized location was updated after the first saccade. The results showed that the memorized location was rapidly updated with the eyes curving away from its spatial coordinates within 130 ms after the first eye movement. The time-course of updating was comparable to the updating of an exogenously attended location, and depended on how well the location was memorized.

Target Localization after Saccades and at Fixation: Nontargets both Facilitate and Bias Responses

The image on our retina changes every time we make an eye movement. To maintain visual stability after saccades, specifically to locate visual targets, we may use nontarget objects as "landmarks". In the current study, we compared how the presence of nontargets affects target localization after saccades and during sustained fixation. Participants fixated a target object, which either maintained its location on the screen (sustained-fixation trials), or displaced to trigger a saccade (saccade trials). After the target disappeared, participants reported the most recent target location with a mouse click. We found that the presence of nontargets decreased response error magnitude and variability. However, this nontarget facilitation effect was not larger for saccade trials than sustained-fixation trials, indicating that nontarget facilitation might be a general effect for target localization, rather than of particular importance to post-saccadic stability. Additionally, participants' responses were biased towards the nontarget locations, particularly when the nontarget-target relationships were preserved in relative coordinates across the saccade. This nontarget bias interacted with biases from other spatial references, e.g. eye movement paths, possibly in a way that emphasized non-redundant information. In summary, the presence of nontargets is one of several sources of reference that combine to influence (both facilitate and bias) target localization.

Selective attention within the foveola

Efficient control of attentional resources and high-acuity vision are both fundamental for survival. Shifts in visual attention are known to covertly enhance processing at locations away from the center of gaze, where visual resolution is low. It is unknown, however, whether selective spatial attention operates where the observer already looks, i.e., within the high-acuity foveola, the small, yet disproportionally important rod-free region of the retina. Using new methods for precisely controlling retinal stimulation, here we show that covert attention flexibly improves and speeds-up both detection and discrimination at loci only a fraction of a degree apart within the foveola. These findings reveal a surprisingly precise control of attention and its involvement in fine spatial vision. They show that the commonly studied covert shifts of attention away from the fovea are the expression of a global mechanism that exerts its action across the entire visual field.

Object-location binding across a saccade: A retinotopic spatial congruency bias

Despite frequent eye movements that rapidly shift the locations of objects on our retinas, our visual system creates a stable perception of the world. To do this, it must convert eye-centered (retinotopic) input to world-centered (spatiotopic) percepts. Moreover, for successful behavior we must also incorporate information about object features/identities during this updating - a fundamental challenge that remains to be understood. Here we adapted a recent behavioral paradigm, the "spatial congruency bias," to investigate object-location binding across an eye movement. In two initial baseline experiments, we showed that the spatial congruency bias was present for both gabor and face stimuli in addition to the object stimuli used in the original paradigm. Then, across three main experiments, we found the bias was preserved across an eye movement, but only in retinotopic coordinates: Subjects were more likely to perceive two stimuli as having the same features/identity when they were presented in the same retinotopic location. Strikingly, there was no evidence of location binding in the more ecologically relevant spatiotopic (world-centered) coordinates; the reference frame did not update to spatiotopic even at longer post-saccade delays, nor did it transition to spatiotopic with more complex stimuli (gabors, shapes, and faces all showed a retinotopic congruency bias). Our results suggest that object-location binding may be tied to retinotopic coordinates, and that it may need to be re-established following each eye movement rather than being automatically updated to spatiotopic coordinates.

Presaccadic motion integration between current and future retinotopic locations of attended objects

Object tracking across eye movements is thought to rely on presaccadic updating of attention between the object's current and its "remapped" location (i.e., the postsaccadic retinotopic location). We report evidence for a bifocal, presaccadic sampling between these two positions. While preparing a saccade, participants viewed four spatially separated random dot kinematograms, one of which was cued by a colored flash. They reported the direction of a coherent motion signal at the cued location while a second signal occurred simultaneously either at the cue's remapped location or at one of several control locations. Motion integration between the signals occurred only when the two motion signals were congruent and were shown at the cue and at its remapped location. This shows that the visual system integrates features between both the current and the future retinotopic locations of an attended object and that such presaccadic sampling is feature specific.

Spatial inhibition of return as a function of fixation history, task, and spatial references

In oculomotor selection, each saccade is thought to be automatically biased toward uninspected locations, inhibiting the inefficient behavior of repeatedly refixating the same objects. This automatic bias is related to inhibition of return (IOR). Although IOR seems an appealing property that increases efficiency in visual search, such a mechanism would not be efficient in other tasks. Indeed, evidence for additional, more flexible control over refixations has been provided. Here, we investigated whether task demands implicitly affect the rate of refixations. We measured the probability of refixations after series of six binary saccadic decisions under two conditions: visual search and free viewing. The rate of refixations seems influenced by two effects. One effect is related to the rate of intervening fixations, specifically, more refixations were observed with more intervening fixations. In addition, we observed an effect of task set, with fewer refixations in visual search than in free viewing. Importantly, the history-related effect was more pronounced when sufficient spatial references were provided, suggesting that this effect is dependent on spatiotopic encoding of previously fixated locations. This known history-related bias in gaze direction is not the primary influence on the refixation rate. Instead, multiple factors, such as task set and spatial references, assert strong influences as well.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron's spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Attention in Active Vision: A Perspective on Perceptual Continuity Across Saccades

Alfred L. Yarbus was among the first to demonstrate that eye movements actively serve our perceptual and cognitive goals, a crucial recognition that is at the heart of today's research on active vision. He realized that not the changes in fixation stick in memory but the changes in shifts of attention. Indeed, oculomotor control is tightly coupled to functions as fundamental as attention and memory. This tight relationship offers an intriguing perspective on transsaccadic perceptual continuity, which we experience despite the fact that saccades cause rapid shifts of the image across the retina. Here, I elaborate this perspective based on a series of psychophysical findings. First, saccade preparation shapes the visual system's priorities; it enhances visual performance and perceived stimulus intensity at the targets of the eye movement. Second, before saccades, the deployment of visual attention is updated, predictively facilitating perception at those retinal locations that will be relevant once the eyes land. Third, saccadic eye movements strongly affect the contents of visual memory, highlighting their crucial role for which parts of a scene we remember or forget. Together, these results provide insights on how attentional processes enable the visual system to cope with the retinal consequences of saccades.