Low-Level Visual Information Is Maintained across Saccades, Allowing for a Postsaccadic Handoff between Visual Areas

Presaccadic preview shapes postsaccadic processing more where perception is poor

The presaccadic preview of a peripheral target enhances the efficiency of its postsaccadic processing, termed the extrafoveal preview effect. Peripheral visual performance-and thus the quality of the preview-varies around the visual field, even at isoeccentric locations: It is better along the horizontal than vertical meridian and along the lower than upper vertical meridian. To investigate whether these polar angle asymmetries influence the preview effect, we asked human participants to preview four tilted gratings at the cardinals, until a central cue indicated which one to saccade to. During the saccade, the target orientation either remained or slightly changed (valid/invalid preview). After saccade landing, participants discriminated the orientation of the (briefly presented) second grating. Stimulus contrast was titrated with adaptive staircases to assess visual performance. Expectedly, valid previews increased participants' postsaccadic contrast sensitivity. This preview benefit, however, was inversely related to polar angle perceptual asymmetries; largest at the upper, and smallest at the horizontal meridian. This finding reveals that the visual system compensates for peripheral asymmetries when integrating information across saccades, by selectively assigning higher weights to the less-well perceived preview information. Our study supports the recent line of evidence showing that perceptual dynamics around saccades vary with eye movement direction.

Decoding Remapped Spatial Information in the Peri-Saccadic Period

It has been suggested that, prior to a saccade, visual neurons predictively respond to stimuli that will fall in their receptive fields after completion of the saccade. This saccadic remapping process is thought to compensate for the shift of the visual world across the retina caused by eye movements. To map the timing of this predictive process in the brain, we recorded neural activity using electroencephalography during a saccade task. Human participants (male and female) made saccades between two fixation points while covertly attending to oriented gratings briefly presented at various locations on the screen. Data recorded during trials in which participants maintained fixation were used to train classifiers on stimuli in different positions. Subsequently, data collected during saccade trials were used to test for the presence of remapped stimulus information at the post-saccadic retinotopic location in the peri-saccadic period, providing unique insight into when remapped information becomes available. We found that the stimulus could be decoded at the remapped location ∼180 ms post-stimulus onset, but only when the stimulus was presented 100-200 ms before saccade onset. Within this range, we found that the timing of remapping was dictated by stimulus onset rather than saccade onset. We conclude that presenting the stimulus immediately before the saccade allows for optimal integration of the corollary discharge signal with the incoming peripheral visual information, resulting in a remapping of activation to the relevant post-saccadic retinotopic neurons.

A bias in transsaccadic perception of spatial frequency changes

Visual processing differs between the foveal and peripheral visual field. These differences can lead to different appearances of objects in the periphery and the fovea, posing a challenge to perception across saccades. Differences in the appearance of visual features between the peripheral and foveal visual field may bias change discrimination across saccades. Previously it has been reported that spatial frequency (SF) appears higher in the periphery compared to the fovea (Davis et al., 1987). In this study, we investigated the visual appearance of SF before and after a saccade and the discrimination of SF changes during saccades. In addition, we tested the contributions of pre- and postsaccadic information to change discrimination performance. In the first experiment, we found no differences in the appearance of SF before and after a saccade. However, participants showed a clear bias to report SF increases. Interestingly, a 200-ms postsaccadic blank improved the precision of the responses but did not affect the bias. In the second experiment, participants showed lower thresholds for SF increases than for decreases, suggesting that the bias in the first experiment was not just a response bias. Finally, we asked participants to discriminate the SF of stimuli presented before a saccade. Thresholds in the presaccadic discrimination task were lower than in the change discrimination task, suggesting that transsaccadic change discrimination is not merely limited by presaccadic discrimination in the periphery. The change direction bias might stem from more effective masking or overwriting of the presaccadic stimulus by the postsaccadic low SF stimulus.

An early effect of the parafoveal preview on post-saccadic processing of English words

A key aspect of efficient visual processing is to use current and previous information to make predictions about what we will see next. In natural viewing, and when looking at words, there is typically an indication of forthcoming visual information from extrafoveal areas of the visual field before we make an eye movement to an object or word of interest. This "preview effect" has been studied for many years in the word reading literature and, more recently, in object perception. Here, we integrated methods from word recognition and object perception to investigate the timing of the preview on neural measures of word recognition. Through a combined use of EEG and eye-tracking, a group of multilingual participants took part in a gaze-contingent, single-shot saccade experiment in which words appeared in their parafoveal visual field. In valid preview trials, the same word was presented during the preview and after the saccade, while in the invalid condition, the saccade target was a number string that turned into a word during the saccade. As hypothesized, the valid preview greatly reduced the fixation-related evoked response. Interestingly, multivariate decoding analyses revealed much earlier preview effects than previously reported for words, and individual decoding performance correlated with participant reading scores. These results demonstrate that a parafoveal preview can influence relatively early aspects of post-saccadic word processing and help to resolve some discrepancies between the word and object literatures.

Pre-saccadic Preview Shapes Post-Saccadic Processing More Where Perception is Poor

The pre-saccadic preview of a peripheral target enhances the efficiency of its post-saccadic processing, termed the extrafoveal preview effect. Peripheral visual performance -and thus the quality of the preview- varies around the visual field, even at iso-eccentric locations: it is better along the horizontal than vertical meridian and along the lower than upper vertical meridian. To investigate whether these polar angle asymmetries influence the preview effect, we asked human participants (to preview four tilted gratings at the cardinals, until a central cue indicated to which one to saccade. During the saccade, the target orientation either remained or slightly changed (valid/invalid preview). After saccade landing, participants discriminated the orientation of the (briefly presented) second grating. Stimulus contrast was titrated with adaptive staircases to assess visual performance. Expectedly, valid previews increased participants' post-saccadic contrast sensitivity. This preview benefit, however, was inversely related to polar angle perceptual asymmetries; largest at the upper, and smallest at the horizontal meridian. This finding reveals that the visual system compensates for peripheral asymmetries when integrating information across saccades, by selectively assigning higher weights to the less-well perceived preview information. Our study supports the recent line of evidence showing that perceptual dynamics around saccades vary with eye movement direction.

Peri-Saccadic Orientation Identification Performance and Visual Neural Sensitivity Are Higher in the Upper Visual Field

Visual neural processing is distributed among a multitude of sensory and sensory-motor brain areas exhibiting varying degrees of functional specializations and spatial representational anisotropies. Such diversity raises the question of how perceptual performance is determined, at any one moment in time, during natural active visual behavior. Here, exploiting a known dichotomy between the primary visual cortex (V1) and superior colliculus (SC) in representing either the upper or lower visual fields, we asked whether peri-saccadic orientation identification performance is dominated by one or the other spatial anisotropy. Humans (48 participants, 29 females) reported the orientation of peri-saccadic upper visual field stimuli significantly better than lower visual field stimuli, unlike their performance during steady-state gaze fixation, and contrary to expected perceptual superiority in the lower visual field in the absence of saccades. Consistent with this, peri-saccadic superior colliculus visual neural responses in two male rhesus macaque monkeys were also significantly stronger in the upper visual field than in the lower visual field. Thus, peri-saccadic orientation identification performance is more in line with oculomotor, rather than visual, map spatial anisotropies.SIGNIFICANCE STATEMENT Different brain areas respond to visual stimulation, but they differ in the degrees of functional specializations and spatial anisotropies that they exhibit. For example, the superior colliculus (SC) both responds to visual stimulation, like the primary visual cortex (V1), and controls oculomotor behavior. Compared with the primary visual cortex, the superior colliculus exhibits an opposite pattern of upper/lower visual field anisotropy, being more sensitive to the upper visual field. Here, we show that human peri-saccadic orientation identification performance is better in the upper compared with the lower visual field. Consistent with this, monkey superior colliculus visual neural responses to peri-saccadic stimuli follow a similar pattern. Our results indicate that peri-saccadic perceptual performance reflects oculomotor, rather than visual, map spatial anisotropies.

Occipital and parietal cortex participate in a cortical network for transsaccadic discrimination of object shape and orientation

Saccades change eye position and interrupt vision several times per second, necessitating neural mechanisms for continuous perception of object identity, orientation, and location. Neuroimaging studies suggest that occipital and parietal cortex play complementary roles for transsaccadic perception of intrinsic versus extrinsic spatial properties, e.g., dorsomedial occipital cortex (cuneus) is sensitive to changes in spatial frequency, whereas the supramarginal gyrus (SMG) is modulated by changes in object orientation. Based on this, we hypothesized that both structures would be recruited to simultaneously monitor object identity and orientation across saccades. To test this, we merged two previous neuroimaging protocols: 21 participants viewed a 2D object and then, after sustained fixation or a saccade, judged whether the shape or orientation of the re-presented object changed. We, then, performed a bilateral region-of-interest analysis on identified cuneus and SMG sites. As hypothesized, cuneus showed both saccade and feature (i.e., object orientation vs. shape change) modulations, and right SMG showed saccade-feature interactions. Further, the cuneus activity time course correlated with several other cortical saccade/visual areas, suggesting a 'functional network' for feature discrimination. These results confirm the involvement of occipital/parietal cortex in transsaccadic vision and support complementary roles in spatial versus identity updating.

Dynamic neural reconstructions of attended object location and features using EEG

Attention allows us to select relevant and ignore irrelevant information from our complex environments. What happens when attention shifts from one item to another? To answer this question, it is critical to have tools that accurately recover neural representations of both feature and location information with high temporal resolution. In the present study, we used human electroencephalography (EEG) and machine learning to explore how neural representations of object features and locations update across dynamic shifts of attention. We demonstrate that EEG can be used to create simultaneous time courses of neural representations of attended features (time point-by-time point inverted encoding model reconstructions) and attended location (time point-by-time point decoding) during both stable periods and across dynamic shifts of attention. Each trial presented two oriented gratings that flickered at the same frequency but had different orientations; participants were cued to attend one of them and on half of trials received a shift cue midtrial. We trained models on a stable period from Hold attention trials and then reconstructed/decoded the attended orientation/location at each time point on Shift attention trials. Our results showed that both feature reconstruction and location decoding dynamically track the shift of attention and that there may be time points during the shifting of attention when 1) feature and location representations become uncoupled and 2) both the previously attended and currently attended orientations are represented with roughly equal strength. The results offer insight into our understanding of attentional shifts, and the noninvasive techniques developed in the present study lend themselves well to a wide variety of future applications.NEW & NOTEWORTHY We used human EEG and machine learning to reconstruct neural response profiles during dynamic shifts of attention. Specifically, we demonstrated that we could simultaneously read out both location and feature information from an attended item in a multistimulus display. Moreover, we examined how that readout evolves over time during the dynamic process of attentional shifts. These results provide insight into our understanding of attention, and this technique carries substantial potential for versatile extensions and applications.

Bilateral increase in MEG planar gradients prior to saccade onset

Every time we move our eyes, the retinal locations of objects change. To distinguish the changes caused by eye movements from actual external motion of the objects, the visual system is thought to anticipate the consequences of eye movements (saccades). Single neuron recordings have indeed demonstrated changes in receptive fields before saccade onset. Although some EEG studies with human participants have also demonstrated a pre-saccadic increased potential over the hemisphere that will process a stimulus after a saccade, results have been mixed. Here, we used magnetoencephalography to investigate the timing and lateralization of visually evoked planar gradients before saccade onset. We modelled the gradients from trials with both a saccade and a stimulus as the linear combination of the gradients from two conditions with either only a saccade or only a stimulus. We reasoned that any residual gradients in the condition with both a saccade and a stimulus must be uniquely linked to visually-evoked neural activity before a saccade. We observed a widespread increase in residual planar gradients. Interestingly, this increase was bilateral, showing activity both contralateral and ipsilateral to the stimulus, i.e. over the hemisphere that would process the stimulus after saccade offset. This pattern of results is consistent with predictive pre-saccadic changes involving both the current and the future receptive fields involved in processing an attended object, well before the start of the eye movement. The active, sensorimotor coupling of vision and the oculomotor system may underlie the seamless subjective experience of stable and continuous perception.

Saccades and presaccadic stimulus repetition alter cortical network topology and dynamics: evidence from EEG and graph theoretical analysis

Parietal and frontal cortex are involved in saccade generation, and their output signals modify visual signals throughout cortex. Local signals associated with these interactions are well described, but their large-scale progression and network dynamics are unknown. Here, we combined source localized electroencephalography (EEG) and graph theory analysis (GTA) to understand how saccades and presaccadic visual stimuli interactively alter cortical network dynamics in humans. Twenty-one participants viewed 1-3 vertical/horizontal grids, followed by grid with the opposite orientation just before a horizontal saccade or continued fixation. EEG signals from the presaccadic interval (or equivalent fixation period) were used for analysis. Source localization-through-time revealed a rapid frontoparietal progression of presaccadic motor signals and stimulus-motor interactions, with additional band-specific modulations in several frontoparietal regions. GTA analysis revealed a saccade-specific functional network with major hubs in inferior parietal cortex (alpha) and the frontal eye fields (beta), and major saccade-repetition interactions in left prefrontal (theta) and supramarginal gyrus (gamma). This network showed enhanced segregation, integration, synchronization, and complexity (compared with fixation), whereas stimulus repetition interactions reduced synchronization and complexity. These cortical results demonstrate a widespread influence of saccades on both regional and network dynamics, likely responsible for both the motor and perceptual aspects of saccades.

Visual Remapping

Our visual system is fundamentally retinotopic. When viewing a stable scene, each eye movement shifts object features and locations on the retina. Thus, sensory representations must be updated, or remapped, across saccades to align presaccadic and postsaccadic inputs. The earliest remapping studies focused on anticipatory, presaccadic shifts of neuronal spatial receptive fields. Over time, it has become clear that there are multiple forms of remapping and that different forms of remapping may be mediated by different neural mechanisms. This review attempts to organize the various forms of remapping into a functional taxonomy based on experimental data and ongoing debates about forward versus convergent remapping, presaccadic versus postsaccadic remapping, and spatial versus attentional remapping. We integrate findings from primate neurophysiological, human neuroimaging and behavioral, and computational modeling studies. We conclude by discussing persistent open questions related to remapping, with specific attention to binding of spatial and featural information during remapping and speculations about remapping's functional significance. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

The behavioural preview effect with faces is susceptible to statistical regularities: Evidence for predictive processing across the saccade

The world around us appears stable and continuous despite saccadic eye movements. This apparent visual stability is achieved by trans-saccadic perception leading at the behavioural level to preview effects: performance in processing a foveal stimulus is better if the stimulus remained unchanged (valid) compared to when it changed (invalid) during the saccade that brought it into focus. Trans-saccadic perception is known to predictively adapt to the statistics of the environment. Here, we asked whether the behavioural preview effect shows the same characteristics, employing a between-participants training design. Participants made saccades to faces which could change their orientation (upright/inverted) during the saccade. In addition, the post-saccadic face was slightly tilted and participants reported this tilt upon fixation. In a training phase, one group of participants conducted only invalid trials whereas another group conducted only valid trials. In a subsequent test phase with 50% valid and 50% invalid trials, we measured the preview effect. Invalid training reduced the preview effect. With a mixed-model analysis, we could show how this training effect gradually declines in the course of the test phase. These results show that the behavioural preview effect adapts to the statistics of the environment suggesting that it results from predictive processes.