Shape selectivity and remapping in dorsal stream visual area LIP

WIDESPREAD RECEPTIVE FIELD REMAPPING IN EARLY VISUAL CORTEX

Our eyes are in constant motion, yet we perceive the visual world as stable. Predictive remapping of receptive fields is thought to be one of the critical mechanisms for enforcing perceptual stability during eye movements. While receptive field remapping has been identified in several cortical areas, the spatiotemporal dynamics of remapping, and its consequences on the tuning properties of neurons, remain poorly understood. Here, we tracked remapping receptive fields in hundreds of neurons from visual Area V2 while subjects performed a cued saccade task. We found that remapping was far more widespread in Area V2 than previously reported and can be found in neurons from all recorded cortical layers and cell types. Surprisingly, neurons undergoing remapping exhibit sensitivity to two punctate locations in visual space. Furthermore, we found that feature selectivity is not only maintained during remapping but transiently increases due to untuned suppression. Taken together, these results shed light on the spatiotemporal dynamics of remapping and its ubiquitous prevalence in the early visual cortex, and force us to revise current models of perceptual stability.

Perceptual learning across saccades: Feature but not location specific

Perceptual learning is the ability to enhance perception through practice. The hallmark of perceptual learning is its specificity for the trained location and stimulus features, such as orientation. For example, training in discriminating a grating's orientation improves performance only at the trained location but not in other untrained locations. Perceptual learning has mostly been studied using stimuli presented briefly while observers maintained gaze at one location. However, in everyday life, stimuli are actively explored through eye movements, which results in successive projections of the same stimulus at different retinal locations. Here, we studied perceptual learning of orientation discrimination across saccades. Observers were trained to saccade to a peripheral grating and to discriminate its orientation change that occurred during the saccade. The results showed that training led to transsaccadic perceptual learning (TPL) and performance improvements which did not generalize to an untrained orientation. Remarkably, however, for the trained orientation, we found a complete transfer of TPL to the untrained location in the opposite hemifield suggesting high flexibility of reference frame encoding in TPL. Three control experiments in which participants were trained without saccades did not show such transfer, confirming that the location transfer was contingent upon eye movements. Moreover, performance at the trained location, but not at the untrained location, was also improved in an untrained fixation task. Our results suggest that TPL has both, a location-specific component that occurs before the eye movement and a saccade-related component that involves location generalization.

Oculomotor feature discrimination is cortically mediated

Eye movements are often directed toward stimuli with specific features. Decades of neurophysiological research has determined that this behavior is subserved by a feature-reweighting of the neural activation encoding potential eye movements. Despite the considerable body of research examining feature-based target selection, no comprehensive theoretical account of the feature-reweighting mechanism has yet been proposed. Given that such a theory is fundamental to our understanding of the nature of oculomotor processing, we propose an oculomotor feature-reweighting mechanism here. We first summarize the considerable anatomical and functional evidence suggesting that oculomotor substrates that encode potential eye movements rely on the visual cortices for feature information. Next, we highlight the results from our recent behavioral experiments demonstrating that feature information manifests in the oculomotor system in order of featural complexity, regardless of whether the feature information is task-relevant. Based on the available evidence, we propose an oculomotor feature-reweighting mechanism whereby (1) visual information is projected into the oculomotor system only after a visual representation manifests in the highest stage of the cortical visual processing hierarchy necessary to represent the relevant features and (2) these dynamically recruited cortical module(s) then perform feature discrimination via shifting neural feature representations, while also maintaining parity between the feature representations in cortical and oculomotor substrates by dynamically reweighting oculomotor vectors. Finally, we discuss how our behavioral experiments may extend to other areas in vision science and its possible clinical applications.

Occipital and parietal cortex participate in a cortical network for transsaccadic discrimination of object shape and orientation

Saccades change eye position and interrupt vision several times per second, necessitating neural mechanisms for continuous perception of object identity, orientation, and location. Neuroimaging studies suggest that occipital and parietal cortex play complementary roles for transsaccadic perception of intrinsic versus extrinsic spatial properties, e.g., dorsomedial occipital cortex (cuneus) is sensitive to changes in spatial frequency, whereas the supramarginal gyrus (SMG) is modulated by changes in object orientation. Based on this, we hypothesized that both structures would be recruited to simultaneously monitor object identity and orientation across saccades. To test this, we merged two previous neuroimaging protocols: 21 participants viewed a 2D object and then, after sustained fixation or a saccade, judged whether the shape or orientation of the re-presented object changed. We, then, performed a bilateral region-of-interest analysis on identified cuneus and SMG sites. As hypothesized, cuneus showed both saccade and feature (i.e., object orientation vs. shape change) modulations, and right SMG showed saccade-feature interactions. Further, the cuneus activity time course correlated with several other cortical saccade/visual areas, suggesting a 'functional network' for feature discrimination. These results confirm the involvement of occipital/parietal cortex in transsaccadic vision and support complementary roles in spatial versus identity updating.

Impairments to the multisensory integration brain regions during migraine chronification: correlation with the vestibular dysfunction

Objectives

Migraine is often combined with vestibular dysfunction, particularly in patients with chronic migraine (CM). However, the pathogenesis of migraine chronification leading to vestibular dysfunction is not fully understood. The current study investigated whether structural or functional impairments to the brain during migraine chronification could be associated with vestibular dysfunction development.

Methods

The eligible participants underwent clinical assessment and magnetic resonance imaging (MRI) scans. Voxel-based morphometry (VBM) determined structural impairment by evaluating alterations in gray matter volume (GMV). Functional impairment was assessed by the mean amplitude of low-frequency fluctuation (mALFF). Furthermore, the resting-state functional connectivity (rsFC) of regions possessing impairment was examined with a seed-based approach. We also analyzed the correlations between altered neuroimaging features with clinical variables and performed multiple linear regression.

Results

Eighteen CM patients, 18 episodic migraine (EM) patients, and 18 healthy controls (HCs) were included in this study. A one-way ANOVA indicated the group differences in mALFF. These were located within right supramarginal gyrus (SMG), left angular gyrus (AG), middle frontal gyrus (MFG), left middle occipital gyrus (MOG), right rolandic operculum (Rol) and left superior parietal gyrus (SPG). During rsFC analysis, the CM group had more enhanced rsFC of left SPG with left MOG than the EM and HC groups. The EM group revealed enhanced rsFC of left SPG with left AG than the CM and HC groups. In multiple linear regression, after controlling for age, body mass index (BMI) and disease duration, the rsFC of left SPG with left MOG (β = 48.896, p = 0.021) was found to predict the total Dizziness Handicap Inventory (DHI) score with an explained variance of 25.1%. Moreover, the rsFC of left SPG with left MOG (β = 1.253, p = 0.003) and right SMG (β = -1.571, p = 0.049) were significant predictors of migraine frequency, accounting for a total explained variance of 73.8%.

Conclusion

The functional impairments due to migraine chronification are primarily concentrated in the multisensory integration-related brain regions. Additionally, the rsFC of SPG with MOG can predict the frequency of migraine and the degree of vestibular dysfunction. Therefore, these neuroimaging features could be potential mechanisms and therapeutic targets for developing vestibular dysfunction in migraine.

Dynamic saccade context triggers more stable object-location binding

Our visual systems rapidly perceive and integrate information about object identities and locations. There is long-standing debate about how we achieve world-centered (spatiotopic) object representations across eye movements, with many studies reporting persistent retinotopic (eye-centered) effects even for higher-level object-location binding. But these studies are generally conducted in fairly static experimental contexts. Might spatiotopic object-location binding only emerge in more dynamic saccade contexts? In the present study, we investigated this using the Spatial Congruency Bias paradigm in healthy adults. In the static (single saccade) context, we found purely retinotopic binding, as before. However, robust spatiotopic binding emerged in the dynamic (multiple frequent saccades) context. We further isolated specific factors that modulate retinotopic and spatiotopic binding. Our results provide strong evidence that dynamic saccade context can trigger more stable object-location binding in ecologically-relevant spatiotopic coordinates, perhaps via a more flexible brain state which accommodates improved visual stability in the dynamic world.

Saccades and presaccadic stimulus repetition alter cortical network topology and dynamics: evidence from EEG and graph theoretical analysis

Parietal and frontal cortex are involved in saccade generation, and their output signals modify visual signals throughout cortex. Local signals associated with these interactions are well described, but their large-scale progression and network dynamics are unknown. Here, we combined source localized electroencephalography (EEG) and graph theory analysis (GTA) to understand how saccades and presaccadic visual stimuli interactively alter cortical network dynamics in humans. Twenty-one participants viewed 1-3 vertical/horizontal grids, followed by grid with the opposite orientation just before a horizontal saccade or continued fixation. EEG signals from the presaccadic interval (or equivalent fixation period) were used for analysis. Source localization-through-time revealed a rapid frontoparietal progression of presaccadic motor signals and stimulus-motor interactions, with additional band-specific modulations in several frontoparietal regions. GTA analysis revealed a saccade-specific functional network with major hubs in inferior parietal cortex (alpha) and the frontal eye fields (beta), and major saccade-repetition interactions in left prefrontal (theta) and supramarginal gyrus (gamma). This network showed enhanced segregation, integration, synchronization, and complexity (compared with fixation), whereas stimulus repetition interactions reduced synchronization and complexity. These cortical results demonstrate a widespread influence of saccades on both regional and network dynamics, likely responsible for both the motor and perceptual aspects of saccades.

Decision formation in parietal cortex transcends a fixed frame of reference

Neurons in the lateral intraparietal cortex represent the formation of a decision when it is linked to a specific action, such as an eye movement to a choice target. However, these neurons should be unable to represent a decision that transpires across actions that would disrupt this linkage. We investigated this limitation by simultaneously recording many neurons from two rhesus monkeys. Although intervening actions disrupt the representation by single neurons, the ensemble achieves continuity of the decision process by passing information from currently active neurons to neurons that will become active after the action. In this way, the representation of an evolving decision can be generalized across actions and transcends the frame of reference that specifies the neural response fields. The finding extends previous observations of receptive field remapping, thought to support the stability of perception across eye movements, to the continuity of a thought process, such as a decision.

The Visuospatial and Sensorimotor Functions of Posterior Parietal Cortex in Drawing Tasks: A Review

Drawing is a comprehensive skill that primarily involves visuospatial processing, eye-hand coordination, and other higher-order cognitive functions. Various drawing tasks are widely used to assess brain function. The neuropsychological basis of drawing is extremely sophisticated. Previous work has addressed the critical role of the posterior parietal cortex (PPC) in drawing, but the specific functions of the PPC in drawing remain unclear. Functional magnetic resonance imaging and electrophysiological studies found that drawing activates the PPC. Lesion-symptom mapping studies have shown an association between PPC injury and drawing deficits in patients with global and focal cerebral pathology. These findings depicted a core framework of the fronto-parietal network in drawing tasks. Here, we review neuroimaging and electrophysiological studies applying drawing paradigms and discuss the specific functions of the PPC in visuospatial and sensorimotor aspects. Ultimately, we proposed a hypothetical model based on the dorsal stream. It demonstrates the organization of a PPC-centered network for drawing and provides systematic insights into drawing for future neuropsychological research.

Visual Remapping

Our visual system is fundamentally retinotopic. When viewing a stable scene, each eye movement shifts object features and locations on the retina. Thus, sensory representations must be updated, or remapped, across saccades to align presaccadic and postsaccadic inputs. The earliest remapping studies focused on anticipatory, presaccadic shifts of neuronal spatial receptive fields. Over time, it has become clear that there are multiple forms of remapping and that different forms of remapping may be mediated by different neural mechanisms. This review attempts to organize the various forms of remapping into a functional taxonomy based on experimental data and ongoing debates about forward versus convergent remapping, presaccadic versus postsaccadic remapping, and spatial versus attentional remapping. We integrate findings from primate neurophysiological, human neuroimaging and behavioral, and computational modeling studies. We conclude by discussing persistent open questions related to remapping, with specific attention to binding of spatial and featural information during remapping and speculations about remapping's functional significance. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The functional roles of neural remapping in cortex

Remapping is a property of some cortical and subcortical neurons that update their responses around the time of an eye movement to account for the shift of stimuli on the retina due to the saccade. Physiologically, remapping is traditionally tested by briefly presenting a single stimulus around the time of the saccade and looking at the onset of the response and the locations in space to which the neuron is responsive. Here we suggest that a better way to understand the functional role of remapping is to look at the time at which the neural signal emerges when saccades are made across a stable scene. Based on data obtained using this approach, we suggest that remapping in the lateral intraparietal area is sufficient to play a role in maintaining visual stability across saccades, whereas in the frontal eye field, remapped activity carries information that affects future saccadic choices and, in a separate subset of neurons, is used to maintain a map of locations in the scene that have been previously fixated.

Occipital cortex is modulated by transsaccadic changes in spatial frequency: an fMRI study

Previous neuroimaging studies have shown that inferior parietal and ventral occipital cortex are involved in the transsaccadic processing of visual object orientation. Here, we investigated whether the same areas are also involved in transsaccadic processing of a different feature, namely, spatial frequency. We employed a functional magnetic resonance imaging paradigm where participants briefly viewed a grating stimulus with a specific spatial frequency that later reappeared with the same or different frequency, after a saccade or continuous fixation. First, using a whole-brain Saccade > Fixation contrast, we localized two frontal (left precentral sulcus and right medial superior frontal gyrus), four parietal (bilateral superior parietal lobule and precuneus), and four occipital (bilateral cuneus and lingual gyri) regions. Whereas the frontoparietal sites showed task specificity, the occipital sites were also modulated in a saccade control task. Only occipital cortex showed transsaccadic feature modulations, with significant repetition enhancement in right cuneus. These observations (parietal task specificity, occipital enhancement, right lateralization) are consistent with previous transsaccadic studies. However, the specific regions differed (ventrolateral for orientation, dorsomedial for spatial frequency). Overall, this study supports a general role for occipital and parietal cortex in transsaccadic vision, with a specific role for cuneus in spatial frequency processing.

Low-Level Visual Information Is Maintained across Saccades, Allowing for a Postsaccadic Handoff between Visual Areas

Experience seems continuous and detailed despite saccadic eye movements changing retinal input several times per second. There is debate whether neural signals related to updating across saccades contain information about stimulus features, or only location pointers without visual details. We investigated the time course of low-level visual information processing across saccades by decoding the spatial frequency of a stationary stimulus that changed from one visual hemifield to the other because of a horizontal saccadic eye movement. We recorded magnetoencephalography while human subjects (both sexes) monitored the orientation of a grating stimulus, making spatial frequency task irrelevant. Separate trials, in which subjects maintained fixation, were used to train a classifier, whose performance was then tested on saccade trials. Decoding performance showed that spatial frequency information of the presaccadic stimulus remained present for ∼200 ms after the saccade, transcending retinotopic specificity. Postsaccadic information ramped up rapidly after saccade offset. There was an overlap of over 100 ms during which decoding was significant from both presaccadic and postsaccadic processing areas. This suggests that the apparent richness of perception across saccades may be supported by the continuous availability of low-level information with a "soft handoff" of information during the initial processing sweep of the new fixation.SIGNIFICANCE STATEMENT Saccades create frequent discontinuities in visual input, yet perception appears stable and continuous. How is this discontinuous input processed resulting in visual stability? Previous studies have focused on presaccadic remapping. Here we examined the time course of processing of low-level visual information (spatial frequency) across saccades with magnetoencephalography. The results suggest that spatial frequency information is not predictively remapped but also is not discarded. Instead, they suggest a soft handoff over time between different visual areas, making this information continuously available across the saccade. Information about the presaccadic stimulus remains available, while the information about the postsaccadic stimulus has also become available. The simultaneous availability of both the presaccadic and postsaccadic information could enable rich and continuous perception across saccades.

Stimulus blanking reveals contrast-dependent transsaccadic feature transfer

Across saccadic eye movements, the visual system receives two successive static images corresponding to the pre- and the postsaccadic projections of the visual field on the retina. The existence of a mechanism integrating the content of these images is today still a matter of debate. Here, we studied the transfer of a visual feature across saccades using a blanking paradigm. Participants moved their eyes to a peripheral grating and discriminated a change in its orientation occurring during the eye movement. The grating was either constantly on the screen or briefly blanked during and after the saccade. Moreover, it either was of the same luminance as the background (i.e., isoluminant) or anisoluminant with respect to it. We found that for anisoluminant gratings, the orientation discrimination across saccades was improved when a blank followed the onset of the eye movement. Such effect was however abolished with isoluminant gratings. Additionally, performance was also improved when an anisoluminant grating presented before the saccade was followed by an isoluminant one. These results demonstrate that a detailed representation of the presaccadic image was transferred across saccades allowing participants to perform better on the transsaccadic orientation task. While such a transfer of visual orientation across saccade is masked in real-life anisoluminant conditions, the use of a blank and of an isoluminant postsaccadic grating allowed to reveal its existence.

Parietal Cortex Integrates Saccade and Object Orientation Signals to Update Grasp Plans

Coordinated reach-to-grasp movements are often accompanied by rapid eye movements (saccades) that displace the desired object image relative to the retina. Parietal cortex compensates for this by updating reach goals relative to current gaze direction, but its role in the integration of oculomotor and visual orientation signals for updating grasp plans is unknown. Based on a recent perceptual experiment, we hypothesized that inferior parietal cortex (specifically supramarginal gyrus [SMG]) integrates saccade and visual signals to update grasp plans in additional intraparietal/superior parietal regions. To test this hypothesis in humans (7 females, 6 males), we used a functional magnetic resonance paradigm, where saccades sometimes interrupted grasp preparation toward a briefly presented object that later reappeared (with the same/different orientation) just before movement. Right SMG and several parietal grasp regions, namely, left anterior intraparietal sulcus and bilateral superior parietal lobule, met our criteria for transsaccadic orientation integration: they showed task-dependent saccade modulations and, during grasp execution, they were specifically sensitive to changes in object orientation that followed saccades. Finally, SMG showed enhanced functional connectivity with both prefrontal saccade regions (consistent with oculomotor input) and anterior intraparietal sulcus/superior parietal lobule (consistent with sensorimotor output). These results support the general role of parietal cortex for the integration of visuospatial perturbations, and provide specific cortical modules for the integration of oculomotor and visual signals for grasp updating.SIGNIFICANCE STATEMENT How does the brain simultaneously compensate for both external and internally driven changes in visual input? For example, how do we grasp an unstable object while eye movements are simultaneously changing its retinal location? Here, we used fMRI to identify a group of inferior parietal (supramarginal gyrus) and superior parietal (intraparietal and superior parietal) regions that show saccade-specific modulations during unexpected changes in object/grasp orientation, and functional connectivity with frontal cortex saccade centers. This provides a network, complementary to the reach goal updater, that integrates visuospatial updating into grasp plans, and may help to explain some of the more complex symptoms associated with parietal damage, such as constructional ataxia.

Intra-saccadic displacement sensitivity after a lesion to the posterior parietal cortex

Visual perception is introspectively stable and continuous across eye movements. It has been hypothesized that displacements in retinal input caused by eye movements can be dissociated from displacements in the external world using extra-retinal information, such as a corollary discharge from the oculomotor system. The extra-retinal information can inform the visual system about an upcoming eye movement and accompanying displacements in retinal input. The parietal cortex has been hypothesized to be critically involved in integrating retinal and extra-retinal information. Two tasks have been widely used to assess the quality of this integration: double-step saccades and intra-saccadic displacements. Double-step saccades performed by patients with parietal cortex lesions seemed to show hypometric second saccades. However, recently idea has been refuted by demonstrating that patients with very similar lesions were able to perform the double step saccades, albeit taking multiple saccades to reach the saccade target. So, it seems that extra-retinal information is still available for saccade execution after a lesion to the parietal lobe. Here, we investigated whether extra-retinal signals are also available for perceptual judgements in nine patients with strokes affecting the posterior parietal cortex. We assessed perceptual continuity with the intra-saccadic displacement task. We exploited the increased sensitivity when a small temporal blank is introduced after saccade offset (blank effect). The blank effect is thought to reflect the availability of extra-retinal signals for perceptual judgements. Although patients exhibited a relative difference to control subjects, they still demonstrated the blank effect. The data suggest that a lesion to the posterior parietal cortex (PPC) alters the processing of extra-retinal signals but does not abolish their influence altogether.

Salience Models: A Computational Cognitive Neuroscience Review

The seminal model by Laurent Itti and Cristoph Koch demonstrated that we can compute the entire flow of visual processing from input to resulting fixations. Despite many replications and follow-ups, few have matched the impact of the original model-so what made this model so groundbreaking? We have selected five key contributions that distinguish the original salience model by Itti and Koch; namely, its contribution to our theoretical, neural, and computational understanding of visual processing, as well as the spatial and temporal predictions for fixation distributions. During the last 20 years, advances in the field have brought up various techniques and approaches to salience modelling, many of which tried to improve or add to the initial Itti and Koch model. One of the most recent trends has been to adopt the computational power of deep learning neural networks; however, this has also shifted their primary focus to spatial classification. We present a review of recent approaches to modelling salience, starting from direct variations of the Itti and Koch salience model to sophisticated deep-learning architectures, and discuss the models from the point of view of their contribution to computational cognitive neuroscience.

Transsaccadic integration benefits are not limited to the saccade target

Across saccades, humans can integrate the low-resolution presaccadic information of an upcoming saccade target with the high-resolution postsaccadic information. There is converging evidence to suggest that transsaccadic integration occurs at the saccade target. However, given divergent evidence on the spatial specificity of related mechanisms such as attention, visual working memory, and remapping, it is unclear whether integration is also possible at locations other than the saccade target. We tested the spatial profile of transsaccadic integration, by testing perceptual performance at six locations around the saccade target and between the saccade target and initial fixation. Results show that integration benefits do not differ between the saccade target and surrounding locations. Transsaccadic integration benefits are not specific to the saccade target and can occur at other locations when they are behaviorally relevant, although there is a trend for worse performance for the location above initial fixation compared with those in the direction of the saccade. This suggests that transsaccadic integration may be a more general mechanism used to reconcile task-relevant pre- and postsaccadic information at attended locations other than the saccade target.

NEW & NOTEWORTHY This study shows that integration of pre- and postsaccadic information across saccades is not restricted to the saccade target. We found performance benefits of transsaccadic integration at attended locations other than the saccade target, and these benefits did not differ from those found at the saccade target. This suggests that transsaccadic integration may be a more general mechanism used to reconcile pre- and postsaccadic information at task-relevant locations.

Representation of shape, space, and attention in monkey cortex

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both 'what' and 'where' cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.

An Integrative Framework for Sensory, Motor, and Cognitive Functions of the Posterior Parietal Cortex

Throughout the history of modern neuroscience, the parietal cortex has been associated with a wide array of sensory, motor, and cognitive functions. The use of non-human primates as a model organism has been instrumental in our current understanding of how areas in the posterior parietal cortex (PPC) modulate our perception and influence our behavior. In this Perspective, we highlight a series of influential studies over the last five decades examining the role of the PPC in visual perception and motor planning. We also integrate long-standing views of PPC functions with more recent evidence to propose a more general model framework to explain integrative sensory, motor, and cognitive functions of the PPC.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

Remapping locations and features across saccades: a dual-spotlight theory of attentional updating

How do we maintain visual stability across eye movements? Much work has focused on how visual information is rapidly updated to maintain spatiotopic representations. However, predictive spatial remapping is only part of the story. Here I review key findings, recent debates, and open questions regarding remapping and its implications for visual attention and perception. This review focuses on two key questions: when does remapping occur, and what is the impact on feature perception? Findings are reviewed within the framework of a two-stage, or dual- spotlight, remapping process, where spatial attention must be both updated to the new location (fast, predictive stage) and withdrawn from the previous retinotopic location (slow, post-saccadic stage), with a particular focus on the link between spatial and feature information across eye movements.

Responses of Neurons in Lateral Intraparietal Area Depend on Stimulus-Associated Reward During Binocular Flash Suppression

Discovering neural correlates of subjective perception and dissociating them from sensory input has fascinated neuroscientists for a long time. Bistable and multistable perception phenomena have exhibited great experimental potential to address this question. Here, we performed electrophysiological recordings from single neurons in lateral intraparietal area (LIP) of rhesus macaques during stimulus and perceptual transitions induced by binocular flash suppression (BFS). LIP neurons demonstrated transient bursts of activity after stimulus presentation and stimulus or perceptual switches but only a minority of cells demonstrated stimulus and perceptual selectivity. To enhance LIP neural selectivity, we performed a second experiment in which the competing stimuli were associated with asymmetric rewards. We found that transient and sustained activities substantially increased while the proportion of stimulus selective neurons remained approximately the same, albeit with increased selectivity magnitude. In addition, we observed mild increases in the proportion of perceptually selective neurons which also showed increase magnitude of selectivity. Importantly, the increased selectivity of cells after the reward manipulation was not directly reflecting the reward size per se but an enhancement in stimulus differentiation. Based on our results, we conjecture that LIP contributes to perceptual transitions and serves a modulatory role in perceptual selection taking into account the stimulus behavioral value.

Feature integration is unaffected by saccade landing point, even when saccades land outside of the range of regular oculomotor variance

The experience of our visual surroundings appears continuous, contradicting the erratic nature of visual processing due to saccades. A possible way the visual system can construct a continuous experience is by integrating presaccadic and postsaccadic visual input. However, saccades rarely land exactly at the intended location. Feature integration would therefore need to be robust against variations in saccade execution to facilitate visual continuity. In the current study, observers reported a feature (color) of the saccade target, which occasionally changed slightly during the saccade. In transsaccadic change-trials, observers reported a mixture of the pre- and postsaccadic color, indicating transsaccadic feature integration. Saccade landing distance was not a significant predictor of the reported color. Next, to investigate the influence of more extreme deviations of saccade landing point on color reports, we used a global effect paradigm in a second experiment. In global effect trials, a distractor appeared together with the saccade target, causing most saccades to land in between the saccade target and the distractor. Strikingly, even when saccades land further away (up to 4°) from the saccade target than one would expect under single target conditions, there was no effect of saccade landing point on the reported color. We reason that saccade landing point does not affect feature integration, due to dissociation between the intended saccade target and the actual saccade landing point. Transsaccadic feature integration seems to be a mechanism that is dependent on visual spatial attention, and, as a result, is robust against variance in saccade landing point.

Coding of spatial attention priorities and object features in the macaque lateral intraparietal cortex

Primate posterior parietal cortex (PPC) is known to be involved in controlling spatial attention. Neurons in one part of the PPC, the lateral intraparietal area (LIP), show enhanced responses to objects at attended locations. Although many are selective for object features, such as the orientation of a visual stimulus, it is not clear how LIP circuits integrate feature-selective information when providing attentional feedback about behaviorally relevant locations to the visual cortex. We studied the relationship between object feature and spatial attention properties of LIP cells in two macaques by measuring the cells' orientation selectivity and the degree of attentional enhancement while performing a delayed match-to-sample task. Monkeys had to match both the location and orientation of two visual gratings presented separately in time. We found a wide range in orientation selectivity and degree of attentional enhancement among LIP neurons. However, cells with significant attentional enhancement had much less orientation selectivity in their response than cells which showed no significant modulation by attention. Additionally, orientation-selective cells showed working memory activity for their preferred orientation, whereas cells showing attentional enhancement also synchronized with local neuronal activity. These results are consistent with models of selective attention incorporating two stages, where an initial feature-selective process guides a second stage of focal spatial attention. We suggest that LIP contributes to both stages, where the first stage involves orientation-selective LIP cells that support working memory of the relevant feature, and the second stage involves attention-enhanced LIP cells that synchronize to provide feedback on spatial priorities.

Object-location binding across a saccade: A retinotopic spatial congruency bias

Despite frequent eye movements that rapidly shift the locations of objects on our retinas, our visual system creates a stable perception of the world. To do this, it must convert eye-centered (retinotopic) input to world-centered (spatiotopic) percepts. Moreover, for successful behavior we must also incorporate information about object features/identities during this updating - a fundamental challenge that remains to be understood. Here we adapted a recent behavioral paradigm, the "spatial congruency bias," to investigate object-location binding across an eye movement. In two initial baseline experiments, we showed that the spatial congruency bias was present for both gabor and face stimuli in addition to the object stimuli used in the original paradigm. Then, across three main experiments, we found the bias was preserved across an eye movement, but only in retinotopic coordinates: Subjects were more likely to perceive two stimuli as having the same features/identity when they were presented in the same retinotopic location. Strikingly, there was no evidence of location binding in the more ecologically relevant spatiotopic (world-centered) coordinates; the reference frame did not update to spatiotopic even at longer post-saccade delays, nor did it transition to spatiotopic with more complex stimuli (gabors, shapes, and faces all showed a retinotopic congruency bias). Our results suggest that object-location binding may be tied to retinotopic coordinates, and that it may need to be re-established following each eye movement rather than being automatically updated to spatiotopic coordinates.

Two subdivisions of macaque LIP process visual-oculomotor information differently

Although the cerebral cortex is thought to be composed of functionally distinct areas, the actual parcellation of area and assignment of function are still highly controversial. An example is the much-studied lateral intraparietal cortex (LIP). Despite the general agreement that LIP plays an important role in visual-oculomotor transformation, it remains unclear whether the area is primary sensory- or motor-related (the attention-intention debate). Although LIP has been considered as a functionally unitary area, its dorsal (LIPd) and ventral (LIPv) parts differ in local morphology and long-distance connectivity. In particular, LIPv has much stronger connections with two oculomotor centers, the frontal eye field and the deep layers of the superior colliculus, than does LIPd. Such anatomical distinctions imply that compared with LIPd, LIPv might be more involved in oculomotor processing. We tested this hypothesis physiologically with a memory saccade task and a gap saccade task. We found that LIP neurons with persistent memory activities in memory saccade are primarily provoked either by visual stimulation (vision-related) or by both visual and saccadic events (vision-saccade-related) in gap saccade. The distribution changes from predominantly vision-related to predominantly vision-saccade-related as the recording depth increases along the dorsal-ventral dimension. Consistently, the simultaneously recorded local field potential also changes from visual evoked to saccade evoked. Finally, local injection of muscimol (GABA agonist) in LIPv, but not in LIPd, dramatically decreases the proportion of express saccades. With these results, we conclude that LIPd and LIPv are more involved in visual and visual-saccadic processing, respectively.

Circuits for presaccadic visual remapping

Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF's local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about the reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.

Presaccadic motion integration between current and future retinotopic locations of attended objects

Object tracking across eye movements is thought to rely on presaccadic updating of attention between the object's current and its "remapped" location (i.e., the postsaccadic retinotopic location). We report evidence for a bifocal, presaccadic sampling between these two positions. While preparing a saccade, participants viewed four spatially separated random dot kinematograms, one of which was cued by a colored flash. They reported the direction of a coherent motion signal at the cued location while a second signal occurred simultaneously either at the cue's remapped location or at one of several control locations. Motion integration between the signals occurred only when the two motion signals were congruent and were shown at the cue and at its remapped location. This shows that the visual system integrates features between both the current and the future retinotopic locations of an attended object and that such presaccadic sampling is feature specific.

Trans-saccadic interactions in human parietal and occipital cortex during the retention and comparison of object orientation

The cortical sites for the trans-saccadic storage and integration of visual object features are unknown. Here, we used a variant of fMRI-Adaptation where subjects fixated to the left or right of a briefly presented visual grating, maintained fixation or saccaded to the opposite side, then judged whether a re-presented grating had the same or different orientation. fMRI analysis revealed trans-saccadic interactions (different > same orientation) in a visual field-insensitive cluster within right supramarginal gyrus. This cluster was located at the anterolateral pole of the parietal eye field (identified in a localizer task). We also observed gaze centered, field-specific interactions (same > different orientation) in an extrastriate cluster overlapping with putative 'V4'. Based on these data and our literature review, we conclude that these supramarginal and extrastriate areas are involved in the retention, spatial updating, and evaluation of object orientation information across saccades.

No Evidence for Automatic Remapping of Stimulus Features or Location Found with fMRI

The input to our visual system shifts every time we move our eyes. To maintain a stable percept of the world, visual representations must be updated with each saccade. Near the time of a saccade, neurons in several visual areas become sensitive to the regions of visual space that their receptive fields occupy after the saccade. This process, known as remapping, transfers information from one set of neurons to another, and may provide a mechanism for visual stability. However, it is not clear whether remapping transfers information about stimulus features in addition to information about stimulus location. To investigate this issue, we recorded blood-oxygen-level dependent (BOLD) functional magnetic resonance imaging (fMRI) responses while human subjects viewed images of faces and houses (two visual categories with many feature differences). Immediately after some image presentations, subjects made a saccade that moved the previously stimulated location to the opposite side of the visual field. We then used a combination of univariate analyses and multivariate pattern analyses to test whether information about stimulus location and stimulus features were remapped to the ipsilateral hemisphere after the saccades. We found no reliable indication of stimulus feature remapping in any region. However, we also found no reliable indication of stimulus location remapping, despite the fact that our paradigm was highly similar to previous fMRI studies of remapping. The absence of location remapping in our study precludes strong conclusions regarding feature remapping. However, these results also suggest that measurement of location remapping with fMRI depends strongly on the details of the experimental paradigm used. We highlight differences in our approach from the original fMRI studies of remapping, discuss potential reasons for the failure to generalize prior location remapping results, and suggest directions for future research.

Saccadic remapping of object-selective information

Saccadic remapping, a presaccadic increase in neural activity when a saccade is about to bring an object into a neuron's receptive field, may be crucial for our perception of a stable world. Studies of perception and saccadic remapping, like ours, focus on the presaccadic acquisition of information from the saccade target, with no direct reference to underlying physiology. While information is known to be acquired prior to a saccade, it is unclear whether object-selective or feature-specific information is remapped. To test this, we performed a series of psychophysical experiments in which we presented a peripheral, nonfoveated face as a presaccadic target. The target face disappeared at saccade onset. After making a saccade to the location of the peripheral target face (which was no longer visible), subjects misperceived the expression of a subsequent, foveally presented neutral face as being repelled away from the peripheral presaccadic face target. This effect was similar to a sequential shape contrast or negative aftereffect but required a saccade, because covert attention was not sufficient to generate the illusion. Additional experiments further revealed that inverting the faces disrupted the illusion, suggesting that presaccadic remapping is object-selective and not based on low-level features. Our results demonstrate that saccadic remapping can be an object-selective process, spatially tuned to the target of the saccade and distinct from covert attention in the absence of a saccade.

An Attention-Sensitive Memory Trace in Macaque MT Following Saccadic Eye Movements

We experience a visually stable world despite frequent retinal image displacements induced by eye, head, and body movements. The neural mechanisms underlying this remain unclear. One mechanism that may contribute is transsaccadic remapping, in which the responses of some neurons in various attentional, oculomotor, and visual brain areas appear to anticipate the consequences of saccades. The functional role of transsaccadic remapping is actively debated, and many of its key properties remain unknown. Here, recording from two monkeys trained to make a saccade while directing attention to one of two spatial locations, we show that neurons in the middle temporal area (MT), a key locus in the motion-processing pathway of humans and macaques, show a form of transsaccadic remapping called a memory trace. The memory trace in MT neurons is enhanced by the allocation of top-down spatial attention. Our data provide the first demonstration, to our knowledge, of the influence of top-down attention on the memory trace anywhere in the brain. We find evidence only for a small and transient effect of motion direction on the memory trace (and in only one of two monkeys), arguing against a role for MT in the theoretically critical yet empirically contentious phenomenon of spatiotopic feature-comparison and adaptation transfer across saccades. Our data support the hypothesis that transsaccadic remapping represents the shift of attentional pointers in a retinotopic map, so that relevant locations can be tracked and rapidly processed across saccades. Our results resolve important issues concerning the perisaccadic representation of visual stimuli in the dorsal stream and demonstrate a significant role for top-down attention in modulating this representation.

How does the brain keep track of specific attended features after eye movements? A new study of the macaque brain implicates the middle temporal (MT) area in the remapping of attentional pointers across saccades.

Humans experience a visually stable world despite the fact that eye, head, and body movements cause frequent shifts of the image on the retina. Humans and monkeys are also able to keep track of visual stimuli across such movements. One mechanism that may contribute to these abilities is “transsaccadic remapping,” in which the responses of some neurons in various attentional, oculomotor, and visual brain areas appear to anticipate the consequences of saccades. A current hypothesis proposes that the brain maintains “attentional pointers” to the locations of relevant stimuli and that, via transsaccadic remapping, it rapidly relocates these pointers to compensate for intervening eye movements. Whether stimulus features are also remapped across saccades (along with their location) remains unclear. Here, we show the presence of transsaccadic remapping in a macaque monkey brain area critical for visual motion processing, the middle temporal area (MT). This remapped response is stronger for an attended stimulus. We find only weak evidence for motion-direction information in the remapped response. These results support the attentional pointer hypothesis and demonstrate for the first time, to our knowledge, the impact of top-down attention on transsaccadic remapping in the brain.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron's spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Two distinct types of remapping in primate cortical area V4

Visual neurons typically receive information from a limited portion of the retina, and such receptive fields are a key organizing principle for much of visual cortex. At the same time, there is strong evidence that receptive fields transiently shift around the time of saccades. The nature of the shift is controversial: Previous studies have found shifts consistent with a role for perceptual constancy; other studies suggest a role in the allocation of spatial attention. Here we present evidence that both the previously documented functions exist in individual neurons in primate cortical area V4. Remapping associated with perceptual constancy occurs for saccades in all directions, while attentional shifts mainly occur for neurons with receptive fields in the same hemifield as the saccade end point. The latter are relatively sluggish and can be observed even during saccade planning. Overall these results suggest a complex interplay of visual and extraretinal influences during the execution of saccades.

Visual receptive fields are known to change positions around the time of a saccade, but the nature of this remapping is unclear. Here Neupane and colleagues show that neurons in area V4 of the visual cortex exhibit two types of remapping, one consistent with a role in maintaining perceptual stability, and a second that seems to reflect shifts of attention.

Remapping, Spatial Stability, and Temporal Continuity: From the Pre-Saccadic to Postsaccadic Representation of Visual Space in LIP

As our eyes move, we have a strong percept that the world is stable in space and time; however, the signals in cortex coming from the retina change with each eye movement. It is not known how this changing input produces the visual percept we experience, although the predictive remapping of receptive fields has been described as a likely candidate. To explain how remapping accounts for perceptual stability, we examined responses of neurons in the lateral intraparietal area while animals performed a visual foraging task. When a stimulus was brought into the response field of a neuron that exhibited remapping, the onset of the postsaccadic representation occurred shortly after the saccade ends. Whenever a stimulus was taken out of the response field, the presaccadic representation abruptly ended shortly after the eyes stopped moving. In the 38% (20/52) of neurons that exhibited remapping, there was no more than 30 ms between the end of the presaccadic representation and the start of the postsaccadic representation and, in some neurons, and the population as a whole, it was continuous. We conclude by describing how this seamless shift from a presaccadic to postsaccadic representation could contribute to spatial stability and temporal continuity.

Object Affordance Modulates Visual Responses in the Macaque Medial Posterior Parietal Cortex

Area V6A is a visuomotor area of the dorsomedial visual stream that contains cells modulated by object observation and by grip formation. As different objects have different shapes but also evoke different grips, the response selectivity during object presentation could reflect either the coding of object geometry or object affordances. To clarify this point, we here investigate neural responses of V6A cells when monkeys observed two objects with similar visual features but different contextual information, such as the evoked grip type. We demonstrate that many V6A cells respond to the visual presentation of objects and about 30% of them by the object affordance. Given that area V6A is an early stage in the visuomotor processes underlying grasping, these data suggest that V6A may participate in the computation of object affordances. These results add some elements in the recent literature about the role of the dorsal visual stream areas in object representation and contribute in elucidating the neural correlates of the extraction of action-relevant information from general object properties, in agreement with recent neuroimaging studies on humans showing that vision of graspable objects activates action coding in the dorsomedial visual steam.

The effects of TMS over dorsolateral prefrontal cortex on trans-saccadic memory of multiple objects

Humans typically make several rapid eye movements (saccades) per second. It is thought that visual working memory can retain and spatially integrate three to four objects or features across each saccade but little is known about this neural mechanism. Previously we showed that transcranial magnetic stimulation (TMS) to the posterior parietal cortex and frontal eye fields degrade trans-saccadic memory of multiple object features (Prime, Vesia, & Crawford, 2008, Journal of Neuroscience, 28(27), 6938-6949; Prime, Vesia, & Crawford, 2010, Cerebral Cortex, 20(4), 759-772.). Here, we used a similar protocol to investigate whether dorsolateral prefrontal cortex (DLPFC), an area involved in spatial working memory, is also involved in trans-saccadic memory. Subjects were required to report changes in stimulus orientation with (saccade task) or without (fixation task) an eye movement in the intervening memory interval. We applied single-pulse TMS to left and right DLPFC during the memory delay, timed at three intervals to arrive approximately 100 ms before, 100 ms after, or at saccade onset. In the fixation task, left DLPFC TMS produced inconsistent results, whereas right DLPFC TMS disrupted performance at all three intervals (significantly for presaccadic TMS). In contrast, in the saccade task, TMS consistently facilitated performance (significantly for left DLPFC/perisaccadic TMS and right DLPFC/postsaccadic TMS) suggesting a dis-inhibition of trans-saccadic processing. These results are consistent with a neural circuit of trans-saccadic memory that overlaps and interacts with, but is partially separate from the circuit for visual working memory during sustained fixation.