Visual Remapping

Widespread receptive field remapping in early primate visual cortex

Predictive remapping of receptive fields (RFs) is thought to be one of the critical mechanisms for enforcing perceptual stability during eye movements. While RF remapping has been observed in several cortical areas, its role in early visual cortex and its consequences on the tuning properties of neurons have been poorly understood. Here, we track remapping RFs in hundreds of neurons from visual area V2 while subjects perform a cued saccade task. We find that remapping is widespread in area V2 across neurons from all recorded cortical layers and cell types. Furthermore, our results suggest that remapping RFs not only maintain but also transiently enhance their feature selectivity due to untuned suppression. Taken together, these findings shed light on the dynamics and prevalence of remapping in the early visual cortex, forcing us to revise current models of perceptual stability during saccadic eye movements.

Compression of time in double-step saccades

Temporal intervals appear compressed at the time of saccades. Here, I asked if saccadic compression of time is related to motor planning or to saccade execution. To dissociate saccade motor planning from its execution, I used the double-step paradigm, in which subjects have to perform two horizontal saccades successively. At various times around the saccade sequence, I presented two large horizontal bars, which marked an interval lasting 100 ms. After 700 ms, a second temporal interval was presented, varying in duration across trials. Subjects were required to judge which interval appeared shorter. I found that during the first saccades in the double-step paradigm, temporal intervals were compressed. Maximum temporal compression coincided with saccade onset. Around the time of the second saccade, I found temporal compression as well, however, the time of maximum compression preceded saccade onset by about 70 ms. I compared the magnitude and time of temporal compression between double-step saccades and amplitude-matched single saccades, which I measured separately. Although I found no difference in time compression magnitude, the time when maximum compression occurred differed significantly. I conclude that the temporal shift of time compression in double-step saccades demonstrates the influence of saccade motor planning on time perception.NEW & NOTEWORTHY Visually defined temporal intervals appear compressed at the time of saccades. Here, I tested time perception during double-step saccades dissociating saccade planning from execution. Although around the time of the first saccade, peak compression was found at saccade onset, compression around the time of the second saccade peaked 70 ms before saccade onset. The results suggest that saccade motor planning influences time perception.

Feature-selective responses in macaque visual cortex follow eye movements during natural vision

In natural vision, primates actively move their eyes several times per second via saccades. It remains unclear whether, during this active looking, visual neurons exhibit classical retinotopic properties, anticipate gaze shifts or mirror the stable quality of perception, especially in complex natural scenes. Here, we let 13 monkeys freely view thousands of natural images across 4.6 million fixations, recorded 883 h of neuronal responses in six areas spanning primary visual to anterior inferior temporal cortex and analyzed spatial, temporal and featural selectivity in these responses. Face neurons tracked their receptive field contents, indicated by category-selective responses. Self-consistency analysis showed that general feature-selective responses also followed eye movements and remained gaze-dependent over seconds of viewing the same image. Computational models of feature-selective responses located retinotopic receptive fields during free viewing. We found limited evidence for feature-selective predictive remapping and no viewing-history integration. Thus, ventral visual neurons represent the world in a predominantly eye-centered reference frame during natural vision.

WIDESPREAD RECEPTIVE FIELD REMAPPING IN EARLY VISUAL CORTEX

Our eyes are in constant motion, yet we perceive the visual world as stable. Predictive remapping of receptive fields is thought to be one of the critical mechanisms for enforcing perceptual stability during eye movements. While receptive field remapping has been identified in several cortical areas, the spatiotemporal dynamics of remapping, and its consequences on the tuning properties of neurons, remain poorly understood. Here, we tracked remapping receptive fields in hundreds of neurons from visual Area V2 while subjects performed a cued saccade task. We found that remapping was far more widespread in Area V2 than previously reported and can be found in neurons from all recorded cortical layers and cell types. Surprisingly, neurons undergoing remapping exhibit sensitivity to two punctate locations in visual space. Furthermore, we found that feature selectivity is not only maintained during remapping but transiently increases due to untuned suppression. Taken together, these results shed light on the spatiotemporal dynamics of remapping and its ubiquitous prevalence in the early visual cortex, and force us to revise current models of perceptual stability.

Grouping cells in primate visual cortex

Our perception of how objects are laid out in visual scenes is remarkably stable, despite rapid shifts in the patterns of light that fall on the retina with each saccade. One mechanism that may help establish perceptual stability is border ownership assignment. Studies in macaque area V2 have identified border ownership neurons that signal which side of a border belongs to a foreground surface. This signal persists for hundreds of milliseconds after border ownership has been rendered ambiguous by deleting the stimulus features that distinguish foreground from background. Remarkably, this signal survives eye movements: border ownership neurons also exhibit border ownership signals de novo when an eye movement places the newly ambiguous border within their receptive field. The grouping cell hypothesis proposes the existence of hypothetical grouping cells in a downstream brain area. These cells would compute persistent proto-object representations and therefore have the properties to endow cells in upstream brain areas with selectivity for border ownership. Such grouping cells have been predicted to show a centripetal and persistent pattern of preferred side of ownership for a border placed parallel to the perimeter of their classical receptive field, and such a centripetal ownership preference pattern should also occur de novo in these same cells if an ambiguous border lands in their receptive field after a saccade. It is unknown if grouping cells exist. Here we used laminar multielectrodes in area V4 - the main source of feedback to V2 - of behaving macaques to determine whether such grouping cells exist. Consistent with the model prediction we find a substantial population of neurons with these properties, in all laminar compartments, and they exhibit a response latency that is short enough to act as the source that endows neurons in V2 with selectivity for border ownership. While grouping cell activity provides information about the location of foreground surfaces, these neurons are, counterintuitively, not as strongly tuned for luminance contrast polarity, a feature of those surfaces, as are border ownership cells. Our data suggest a division of labor in which these newly discovered grouping cells provide spatiotemporal continuity of segmented surfaces whereas border ownership cells link this location information with surface features such as luminance contrast.

A presaccadic perceptual impairment at the postsaccadic location of the blindspot

Saccadic eye movements are preceded by profound changes in visual perception. These changes have been linked to the phenomenon of 'forward remapping', in which cells begin to respond to stimuli that appear in their post-saccadic receptive field before the eye has moved. Few studies have examined the perceptual consequences of remapping of areas of impaired sensory acuity, such as the blindspot. Understanding the perceptual consequences of remapping of scotomas may produce important insights into why some neurovisual deficits, such as hemianopia are so intractable for rehabilitation. The current study took advantage of a naturally occurring scotoma in healthy participants (the blindspot) to examine pre-saccadic perception at the upcoming location of the blindspot. Participants viewed stimuli monocularly and were required to make stimulus-driven vertical eye-movements. At a variable latency between the onset of saccade target (ST) and saccade execution a discrimination target (DT) was presented at one of 4 possible locations; within the blindspot, contralateral to the blindspot, in post-saccadic location of the blindspot and contralateral to the post-saccadic location of the blindspot. There was a significant perceptual impairment at the post-saccadic location of the blindspot relative to the contralateral post-saccadic location of the blindspot and the post-saccadic location of the blindspot in a no-saccade control condition. These data are consistent with the idea that the visual system includes a representation of the blindspot which is remapped prior to saccade onset.

Occipital and parietal cortex participate in a cortical network for transsaccadic discrimination of object shape and orientation

Saccades change eye position and interrupt vision several times per second, necessitating neural mechanisms for continuous perception of object identity, orientation, and location. Neuroimaging studies suggest that occipital and parietal cortex play complementary roles for transsaccadic perception of intrinsic versus extrinsic spatial properties, e.g., dorsomedial occipital cortex (cuneus) is sensitive to changes in spatial frequency, whereas the supramarginal gyrus (SMG) is modulated by changes in object orientation. Based on this, we hypothesized that both structures would be recruited to simultaneously monitor object identity and orientation across saccades. To test this, we merged two previous neuroimaging protocols: 21 participants viewed a 2D object and then, after sustained fixation or a saccade, judged whether the shape or orientation of the re-presented object changed. We, then, performed a bilateral region-of-interest analysis on identified cuneus and SMG sites. As hypothesized, cuneus showed both saccade and feature (i.e., object orientation vs. shape change) modulations, and right SMG showed saccade-feature interactions. Further, the cuneus activity time course correlated with several other cortical saccade/visual areas, suggesting a 'functional network' for feature discrimination. These results confirm the involvement of occipital/parietal cortex in transsaccadic vision and support complementary roles in spatial versus identity updating.

Dynamic saccade context triggers more stable object-location binding

Our visual systems rapidly perceive and integrate information about object identities and locations. There is long-standing debate about how we achieve world-centered (spatiotopic) object representations across eye movements, with many studies reporting persistent retinotopic (eye-centered) effects even for higher-level object-location binding. But these studies are generally conducted in fairly static experimental contexts. Might spatiotopic object-location binding only emerge in more dynamic saccade contexts? In the present study, we investigated this using the Spatial Congruency Bias paradigm in healthy adults. In the static (single saccade) context, we found purely retinotopic binding, as before. However, robust spatiotopic binding emerged in the dynamic (multiple frequent saccades) context. We further isolated specific factors that modulate retinotopic and spatiotopic binding. Our results provide strong evidence that dynamic saccade context can trigger more stable object-location binding in ecologically-relevant spatiotopic coordinates, perhaps via a more flexible brain state which accommodates improved visual stability in the dynamic world.

Decision formation in parietal cortex transcends a fixed frame of reference

Neurons in the lateral intraparietal cortex represent the formation of a decision when it is linked to a specific action, such as an eye movement to a choice target. However, these neurons should be unable to represent a decision that transpires across actions that would disrupt this linkage. We investigated this limitation by simultaneously recording many neurons from two rhesus monkeys. Although intervening actions disrupt the representation by single neurons, the ensemble achieves continuity of the decision process by passing information from currently active neurons to neurons that will become active after the action. In this way, the representation of an evolving decision can be generalized across actions and transcends the frame of reference that specifies the neural response fields. The finding extends previous observations of receptive field remapping, thought to support the stability of perception across eye movements, to the continuity of a thought process, such as a decision.

Spatiotopic and retinotopic memory in the context of natural images

Neural responses throughout the visual cortex encode stimulus location in a retinotopic (i.e., eye-centered) reference frame, and memory for stimulus position is most precise in retinal coordinates. Yet visual perception is spatiotopic: objects are perceived as stationary, even though eye movements cause frequent displacement of their location on the retina. Previous studies found that, after a single saccade, memory of retinotopic locations is more accurate than memory of spatiotopic locations. However, it is not known whether various aspects of natural viewing affect the retinotopic reference frame advantage. We found that the retinotopic advantage may in part depend on a retinal afterimage, which can be effectively nullified through backwards masking. Moreover, in the presence of natural scenes, spatiotopic memory is more accurate than retinotopic memory, but only when subjects are provided sufficient time to process the scene before the eye movement. Our results demonstrate that retinotopic memory is not always more accurate than spatiotopic memory and that the fidelity of memory traces in both reference frames are sensitive to the presence of contextual cues.