Landmarks facilitate visual space constancy across saccades and during fixation

A bias in transsaccadic perception of spatial frequency changes

Visual processing differs between the foveal and peripheral visual field. These differences can lead to different appearances of objects in the periphery and the fovea, posing a challenge to perception across saccades. Differences in the appearance of visual features between the peripheral and foveal visual field may bias change discrimination across saccades. Previously it has been reported that spatial frequency (SF) appears higher in the periphery compared to the fovea (Davis et al., 1987). In this study, we investigated the visual appearance of SF before and after a saccade and the discrimination of SF changes during saccades. In addition, we tested the contributions of pre- and postsaccadic information to change discrimination performance. In the first experiment, we found no differences in the appearance of SF before and after a saccade. However, participants showed a clear bias to report SF increases. Interestingly, a 200-ms postsaccadic blank improved the precision of the responses but did not affect the bias. In the second experiment, participants showed lower thresholds for SF increases than for decreases, suggesting that the bias in the first experiment was not just a response bias. Finally, we asked participants to discriminate the SF of stimuli presented before a saccade. Thresholds in the presaccadic discrimination task were lower than in the change discrimination task, suggesting that transsaccadic change discrimination is not merely limited by presaccadic discrimination in the periphery. The change direction bias might stem from more effective masking or overwriting of the presaccadic stimulus by the postsaccadic low SF stimulus.

The role of color in transsaccadic object correspondence

With each saccade, visual information is disrupted, and the visual system is tasked with establishing object correspondence between the presaccadic and postsaccadic representations of the saccade target. There is substantial evidence that the visual system consults spatiotemporal continuity when determining object correspondence across saccades. The evidence for surface feature continuity, however, is mixed. Surface features that are integral to the saccade target object's identity (e.g., shape and contrast polarity) are informative of object continuity, but features that may only imply the state of the object (e.g., orientation) are ignored. The present study tested whether color information is consulted to determine transsaccadic object continuity. We used two variations of the intrasaccadic target displacement task. In Experiments 1 and 2, participants reported the direction of the target displacement. In Experiments 3 and 4, they instead reported whether they detected any target movement. In all experiments, we manipulated the saccade target's continuity by removing it briefly (i.e., blanking) and by changing its color. We found that large color changes can disrupt stability and increase sensitivity to displacements for both direction and movement reports, although not as strongly as long blank durations (250 ms). Interestingly, even smaller color changes, but not blanking, reduced response biases. These results indicate that disrupting surface feature continuity may impact the process of transsaccadic object correspondence more strongly than spatiotemporal disruptions by both increasing the sensitivity and decreasing the response bias.

Paradoxical stabilization of relative position in moving frames

To capture where things are and what they are doing, the visual system may extract the position and motion of each object relative to its surrounding frame of reference [K. Duncker, Routledge and Kegan Paul, London 161-172 (1929) and G. Johansson, Acta Psychol (Amst.) 7, 25-79 (1950)]. Here we report a particularly powerful example where a paradoxical stabilization is produced by a moving frame. We first take a frame that moves left and right and we flash its right edge before, and its left edge after, the frame's motion. For all frame displacements tested, the two edges are perceived as stabilized, with the left edge on the left and right edge on the right, separated by the frame's width as if the frame were not moving. This stabilization is paradoxical because the motion of the frame itself remains visible, albeit much reduced. A second experiment demonstrated that unlike other motion-induced position shifts (e.g., flash lag, flash grab, flash drag, or Fröhlich), the illusory shift here is independent of speed and is set instead by the distance of the frame's travel. In this experiment, two probes are flashed inside the frame at the same physical location before and after the frame moves. Despite being physically superimposed, the probes are perceived widely separated, again as if they were seen in the frame's coordinates and the frame were stationary. This paradoxical stabilization suggests a link to visual stability across eye movements where the displacement of the entire visual scene may act as a frame to stabilize the perception of relative locations.

Achieving visual stability during smooth pursuit eye movements: Directional and confidence judgements favor a recalibration model

During smooth pursuit eye movements, the visual system is faced with the task of telling apart reafferent retinal motion from motion in the world. While an efference copy signal can be used to predict the amount of reafference to subtract from the image, an image-based adaptive mechanism can ensure the continued accuracy of this computation. Indeed, repeatedly exposing observers to background motion with a fixed direction relative to that of the target that is pursued leads to a shift in their point of subjective stationarity (PSS). We asked whether the effect of exposure reflects adaptation to motion contingent on pursuit direction, recalibration of a reference signal or both. A recalibration account predicts a shift in reference signal (i.e. predicted reafference), resulting in a shift of PSS, but no change in sensitivity. Results show that both directional judgements and confidence judgements about them favor a recalibration account, whereby there is an adaptive shift in the reference signal caused by the prevailing retinal motion during pursuit. We also found that the recalibration effect is specific to the exposed visual hemifield.

Stronger saccadic suppression of displacement and blanking effect in children

Humans do not notice small displacements to objects that occur during saccades, termed saccadic suppression of displacement (SSD), and this effect is reduced when a blank is introduced between the pre- and postsaccadic stimulus (Bridgeman, Hendry, & Stark, 1975; Deubel, Schneider, & Bridgeman, 1996). While these effects have been studied extensively in adults, it is unclear how these phenomena are characterized in children. A potentially related mechanism, saccadic suppression of contrast sensitivity-a prerequisite to achieve a stable percept-is stronger for children (Bruno, Brambati, Perani, & Morrone, 2006). However, the evidence for how transsaccadic stimulus displacements may be suppressed or integrated is mixed. While they can integrate basic visual feature information from an early age, they cannot integrate multisensory information (Gori, Viva, Sandini, & Burr, 2008; Nardini, Jones, Bedford, & Braddick, 2008), suggesting a failure in the ability to integrate more complex sensory information. We tested children 7 to 12 years old and adults 19 to 23 years old on their ability to perceive intrasaccadic stimulus displacements, with and without a postsaccadic blank. Results showed that children had stronger SSD than adults and a larger blanking effect. Children also had larger undershoots and more variability in their initial saccade endpoints, indicating greater intrinsic uncertainty, and they were faster in executing corrective saccades to account for these errors. Together, these results suggest that children may have a greater internal expectation or prediction of saccade error than adults; thus, the stronger SSD in children may be due to higher intrinsic uncertainty in target localization or saccade execution.

Perisaccadic remapping: What? How? Why?

About 25 years ago, the discovery of receptive field (RF) remapping in the parietal cortex of nonhuman primates revealed that visual RFs, widely assumed to have a fixed retinotopic organization, can change position before every saccade. Measuring such changes can be deceptively difficult. As a result, studies that followed have generated a fascinating but somewhat confusing picture of the phenomenon. In this review, we describe how observations of RF remapping depend on the spatial and temporal sampling of visual RFs and saccade directions. Further, we summarize some of the theories of how remapping might occur in neural circuitry. Finally, based on neurophysiological and psychophysical observations, we discuss the ways in which remapping information might facilitate computations in downstream brain areas.

Eye and head movements and vestibulo-ocular reflex in the context of indirect, referent control of motor actions

Conventional explanations of the vestibulo-ocular reflex (VOR) and eye and head movements are revisited by considering two alternative frameworks addressing the question of how the brain controls motor actions. Traditionally, biomechanical and/or computational frameworks reflect the views of several prominent scholars of the past, including Helmholtz and von Holst, who assumed that the brain directly specifies the desired motor outcome and uses efference copy to influence perception. However, empirical studies resulting in the theory of referent control of action and perception (an extension of the equilibrium-point hypothesis) revealed that direct specification of motor outcome is inconsistent with nonlinear properties of motoneurons and with the physical principle that the brain can control motor actions only indirectly, by changing or maintaining the values of neurophysiological parameters that influence, but can remain independent of, biomechanical variables. Some parameters are used to shift the origin (referent) points of spatial frames of reference (FRs) or system of coordinates in which motor actions emerge without being predetermined. Parameters are adjusted until the emergent motor actions meet the task demands. Several physiological parameters and spatial FRs have been identified, supporting the notion of indirect, referent control of movements. Instead of integration of velocity-dependent signals, position-dimensional referent signals underlying head motion can likely be transmitted to motoneurons of extraocular muscles. This would produce compensatory eye movement preventing shifts in gaze during head rotation, even after bilateral destruction of the labyrinths. The referent control framework symbolizes a shift in the paradigm for the understanding of VOR and eye and head movement production.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

Updating spatial working memory in a dynamic visual environment

The present review describes recent developments regarding the role of the eye movement system in representing spatial information and keeping track of locations of relevant objects. First, we discuss the active vision perspective and why eye movements are considered crucial for perception and attention. The second part focuses on the question of how the oculomotor system is used to represent spatial attentional priority, and the role of the oculomotor system in maintenance of this spatial information. Lastly, we discuss recent findings demonstrating rapid updating of information across saccadic eye movements. We argue that the eye movement system plays a key role in maintaining and rapidly updating spatial information. Furthermore, we suggest that rapid updating emerges primarily to make sure actions are minimally affected by intervening eye movements, allowing us to efficiently interact with the world around us.

Calibration of peripheral perception of shape with and without saccadic eye movements

The cortical representations of a visual object differ radically across saccades. Several studies claim that the visual system adapts the peripheral percept to better match the subsequent foveal view. Recently, Herwig, Weiß, and Schneider (2015, Annals of the New York Academy of Sciences, 1339(1), 97-105) found that the perception of shape demonstrates a saccade-dependent learning effect. Here, we ask whether this learning actually requires saccades. We replicated Herwig et al.'s (2015) study and introduced a fixation condition. In a learning phase, participants were exposed to objects whose shape systematically changed during a saccade, or during a displacement from peripheral to foveal vision (without a saccade). In a subsequent test, objects were perceived as less (more) curved if they previously changed from more circular (triangular) in the periphery to more triangular (circular) in the fovea. Importantly, this pattern was seen both with and without saccades. We then tested whether a variable delay between the presentations of the peripheral and foveal objects would affect their association-hypothetically weakening it at longer delays. Again, we found that shape judgments depended on the changes experienced during the learning phase and that they were similar in both the saccade and fixation conditions. Surprisingly, they were not affected by the delay between the peripheral and foveal presentations over the range we tested. These results suggest that a general associative process, independent of saccade execution, contributes to the perception of shape across viewpoints.

Independent working memory resources for egocentric and allocentric spatial information

Visuospatial working memory enables us to maintain access to visual information for processing even when a stimulus is no longer present, due to occlusion, our own movements, or transience of the stimulus. Here we show that, when localizing remembered stimuli, the precision of spatial recall does not rely solely on memory for individual stimuli, but additionally depends on the relative distances between stimuli and visual landmarks in the surroundings. Across three separate experiments, we consistently observed a spatially selective improvement in the precision of recall for items located near a persistent landmark. While the results did not require that the landmark be visible throughout the memory delay period, it was essential that it was visible both during encoding and response. We present a simple model that can accurately capture human performance by considering relative (allocentric) spatial information as an independent localization estimate which degrades with distance and is optimally integrated with egocentric spatial information. Critically, allocentric information was encoded without cost to egocentric estimation, demonstrating independent storage of the two sources of information. Finally, when egocentric and allocentric estimates were put in conflict, the model successfully predicted the resulting localization errors. We suggest that the relative distance between stimuli represents an additional, independent spatial cue for memory recall. This cue information is likely to be critical for spatial localization in natural settings which contain an abundance of visual landmarks.

Implicit and explicit learning in reactive and voluntary saccade adaptation

Saccades can either be elicited automatically by salient peripheral stimuli or can additionally depend on explicit cognitive goals. Similarly, it is thought that motor adaptation is driven by the combination of a more automatic, implicit process and a more explicit, cognitive process. However, the degree to which such implicit and explicit learning contribute to the adaptation of more reactive and voluntary saccades remains elusive. To study this question, we employed a global saccadic adaptation paradigm with both increasing and decreasing saccade amplitudes. We assessed the resulting adaptation using a dual state model of motor adaptation. This model decomposes learning into a fast and slow process, which are thought to constitute explicit and implicit learning, respectively. Our results show that adaptation of reactive saccades is equally driven by fast and slow learning, while fast learning is nearly absent when adapting voluntary (i.e. scanning) saccades. This pattern of results was present both when saccade gain was increased or decreased. Our results suggest that the increased cognitive demands associated with voluntary compared to reactive saccade planning interfere specifically with explicit learning.

What is a cognitive map? Unravelling its mystery using robots

Despite years of research into cognitive mapping, the process remains controversial and little understood. A computational theory of cognitive mapping is needed, but developing it is difficult due to the lack of a clear interpretation of the empirical findings. For example, without knowing what a cognitive map is or how landmarks are defined, how does one develop a computational theory for it? We thus face the conundrum of trying to develop a theory without knowing what is computed. In this paper, we overcome the conundrum by abandoning the idea that the process begins by integrating successive views to form a global map of the environment experienced. Instead, we argue that cognitive mapping begins by remembering views as local maps and we empower a mobile robot with the process and study its behaviour as it acquires its "cognitive map". Our results show that what is computed initially could be described as a "route" map and from it, some form of a "survey map" can be computed. The latter, as it turns out, bears much of the characteristics of a cognitive map. Based on our findings, we discuss what a cognitive map is, how cognitive mapping evolves and why such a process also supports the perception of a stable world.

Target Localization after Saccades and at Fixation: Nontargets both Facilitate and Bias Responses

The image on our retina changes every time we make an eye movement. To maintain visual stability after saccades, specifically to locate visual targets, we may use nontarget objects as "landmarks". In the current study, we compared how the presence of nontargets affects target localization after saccades and during sustained fixation. Participants fixated a target object, which either maintained its location on the screen (sustained-fixation trials), or displaced to trigger a saccade (saccade trials). After the target disappeared, participants reported the most recent target location with a mouse click. We found that the presence of nontargets decreased response error magnitude and variability. However, this nontarget facilitation effect was not larger for saccade trials than sustained-fixation trials, indicating that nontarget facilitation might be a general effect for target localization, rather than of particular importance to post-saccadic stability. Additionally, participants' responses were biased towards the nontarget locations, particularly when the nontarget-target relationships were preserved in relative coordinates across the saccade. This nontarget bias interacted with biases from other spatial references, e.g. eye movement paths, possibly in a way that emphasized non-redundant information. In summary, the presence of nontargets is one of several sources of reference that combine to influence (both facilitate and bias) target localization.

Oculomotor Prediction: A Window into the Psychotic Mind

Psychosis - an impaired contact with reality - is a hallmark of schizophrenia. Many psychotic symptoms are associated with disruptions in agency - the sense that 'I' cause my actions. A failure to predict sensory consequences of one's own actions may underlie agency disturbances. Such predictions rely on corollary discharge (CD) signals, 'copies' of movement commands sent to sensory regions prior to action execution. Here, we make a case that the oculomotor system is a promising model for understanding CD in psychosis, building on advances in our understanding of the behavioral and neurophysiological correlates of CD associated with eye movements. In this opinion article, we provide an overview of recent evidence for disturbed oculomotor CD in schizophrenia, potentially linking bizarre and disturbing psychotic experiences with basic physiological processes.

The Role of the Oculomotor System in Updating Visual-Spatial Working Memory across Saccades

Visual-spatial working memory (VSWM) helps us to maintain and manipulate visual information in the absence of sensory input. It has been proposed that VSWM is an emergent property of the oculomotor system. In the present study we investigated the role of the oculomotor system in updating of spatial working memory representations across saccades. Participants had to maintain a location in memory while making a saccade to a different location. During the saccade the target was displaced, which went unnoticed by the participants. After executing the saccade, participants had to indicate the memorized location. If memory updating fully relies on cancellation driven by extraretinal oculomotor signals, the displacement should have no effect on the perceived location of the memorized stimulus. However, if postsaccadic retinal information about the location of the saccade target is used, the perceived location will be shifted according to the target displacement. As it has been suggested that maintenance of accurate spatial representations across saccades is especially important for action control, we used different ways of reporting the location held in memory; a match-to-sample task, a mouse click or by making another saccade. The results showed a small systematic target displacement bias in all response modalities. Parametric manipulation of the distance between the to-be-memorized stimulus and saccade target revealed that target displacement bias increased over time and changed its spatial profile from being initially centered on locations around the saccade target to becoming spatially global. Taken together results suggest that we neither rely exclusively on extraretinal nor on retinal information in updating working memory representations across saccades. The relative contribution of retinal signals is not fixed but depends on both the time available to integrate these signals as well as the distance between the saccade target and the remembered location.

Active sensing without efference copy: referent control of perception

Although action and perception are different behaviors, they are likely to be interrelated, as implied by the notions of perception-action coupling and active sensing. Traditionally, it has been assumed that the nervous system directly preprograms motor commands required for actions and uses a copy of them called efference copy (EC) to also influence our senses. This review offers a critical analysis of the EC concept by identifying its limitations. An alternative to the EC concept is based on the experimentally confirmed notion that sensory signals from receptors are perceived relative to referent signals specified by the brain. These referents also underlie the control of motor actions by predetermining where, in the spatial domain, muscles can work without preprogramming how they should work in terms of motor commands or EC. This approach helps solve several problems of action and explain several sensory experiences, including position sense and the sense that the world remains stationary despite changes in its retinal image during eye or body motion (visual space constancy). The phantom limb phenomenon and other kinesthetic illusions are also explained within this framework.

No Evidence for Automatic Remapping of Stimulus Features or Location Found with fMRI

The input to our visual system shifts every time we move our eyes. To maintain a stable percept of the world, visual representations must be updated with each saccade. Near the time of a saccade, neurons in several visual areas become sensitive to the regions of visual space that their receptive fields occupy after the saccade. This process, known as remapping, transfers information from one set of neurons to another, and may provide a mechanism for visual stability. However, it is not clear whether remapping transfers information about stimulus features in addition to information about stimulus location. To investigate this issue, we recorded blood-oxygen-level dependent (BOLD) functional magnetic resonance imaging (fMRI) responses while human subjects viewed images of faces and houses (two visual categories with many feature differences). Immediately after some image presentations, subjects made a saccade that moved the previously stimulated location to the opposite side of the visual field. We then used a combination of univariate analyses and multivariate pattern analyses to test whether information about stimulus location and stimulus features were remapped to the ipsilateral hemisphere after the saccades. We found no reliable indication of stimulus feature remapping in any region. However, we also found no reliable indication of stimulus location remapping, despite the fact that our paradigm was highly similar to previous fMRI studies of remapping. The absence of location remapping in our study precludes strong conclusions regarding feature remapping. However, these results also suggest that measurement of location remapping with fMRI depends strongly on the details of the experimental paradigm used. We highlight differences in our approach from the original fMRI studies of remapping, discuss potential reasons for the failure to generalize prior location remapping results, and suggest directions for future research.

Detection of Stimulus Displacements Across Saccades is Capacity-Limited and Biased in Favor of the Saccade Target

Retinal image displacements caused by saccadic eye movements are generally unnoticed. Recent theories have proposed that perceptual stability across saccades depends on a local evaluation process centered on the saccade target object rather than on remapping and evaluating the positions of all objects in a display. In three experiments, we examined whether objects other than the saccade target also influence perceptual stability by measuring displacement detection thresholds across saccades for saccade targets and a variable number of non-saccade objects. We found that the positions of multiple objects are maintained across saccades, but with variable precision, with the saccade target object having priority in the perception of displacement, most likely because it is the focus of attention before the saccade and resides near the fovea after the saccade. The perception of displacement of objects that are not the saccade target is affected by acuity limitations, attentional limitations, and limitations on memory capacity. Unlike previous studies that have found that a postsaccadic blank improves the detection of displacement direction across saccades, we found that postsaccadic blanking hurt the detection of displacement per se by increasing false alarms. Overall, our results are consistent with the hypothesis that visual working memory underlies the perception of stability across saccades.

The visual component to saccadic compression

Visual objects presented around the time of saccadic eye movements are strongly mislocalized towards the saccadic target, a phenomenon known as "saccadic compression." Here we show that perisaccadic compression is modulated by the presence of a visual saccadic target. When subjects saccaded to the center of the screen with no visible target, perisaccadic localization was more veridical than when tested with a target. Presenting a saccadic target sometime before saccade initiation was sufficient to induce mislocalization. When we systematically varied the onset of the saccade target, we found that it had to be presented around 100 ms before saccade execution to cause strong mislocalization: saccadic targets presented after this time caused progressively less mislocalization. When subjects made a saccade to screen center with a reference object placed at various positions, mislocalization was focused towards the position of the reference object. The results suggest that saccadic compression is a signature of a mechanism attempting to match objects seen before the saccade with those seen after.

Integration of egocentric and allocentric information during memory-guided reaching to images of a natural environment

When interacting with our environment we generally make use of egocentric and allocentric object information by coding object positions relative to the observer or relative to the environment, respectively. Bayesian theories suggest that the brain integrates both sources of information optimally for perception and action. However, experimental evidence for egocentric and allocentric integration is sparse and has only been studied using abstract stimuli lacking ecological relevance. Here, we investigated the use of egocentric and allocentric information during memory-guided reaching to images of naturalistic scenes. Participants encoded a breakfast scene containing six objects on a table (local objects) and three objects in the environment (global objects). After a 2 s delay, a visual test scene reappeared for 1 s in which 1 local object was missing (= target) and of the remaining, 1, 3 or 5 local objects or one of the global objects were shifted to the left or to the right. The offset of the test scene prompted participants to reach to the target as precisely as possible. Only local objects served as potential reach targets and thus were task-relevant. When shifting objects we predicted accurate reaching if participants only used egocentric coding of object position and systematic shifts of reach endpoints if allocentric information were used for movement planning. We found that reaching movements were largely affected by allocentric shifts showing an increase in endpoint errors in the direction of object shifts with the number of local objects shifted. No effect occurred when one local or one global object was shifted. Our findings suggest that allocentric cues are indeed used by the brain for memory-guided reaching towards targets in naturalistic visual scenes. Moreover, the integration of egocentric and allocentric object information seems to depend on the extent of changes in the scene.

Action-perception coupling in kinesthesia: a new approach

According to recent findings, intentional motor actions are controlled by resetting the referent position, R, at which neuromuscular elements, including reflexes, begin to act. It is suggested that somatosensory afferents inform the brain about the deviation (P) of body segments from the centrally set referent position. To perceive the actual position (Q) of body segments and form the position sense (PS), the central and afferent signals are combined: Q=R+P. In previous studies, the R has been shown to remain invariant during involuntary changes in the wrist position elicited by sudden unloading of muscles, suggesting that only the afferent component is responsible for the PS during this reflex. In contrast, the central PS component, R, is predominantly responsible for PS during intentional motion in isotonic conditions. We tested the hypothesis that the R and P are interchangeable PS components such that involuntary changes in wrist position elicited by the unloading reflex can easily be reproduced by making intentional changes in wrist position in isotonic conditions, in the absence of vision. The PS rule also suggests that PS is independent of sense of effort, which was tested by asking subjects to reproduce elbow joint angles under different constant loads. We also tested the hypothesis that tendon vibration may elicit motion that may not be perceived by subjects (no-motion illusion). These hypotheses were confirmed in three experiments. It is concluded that the R and P are additive components of PS and that, contrary to the conventional view, PS is independent of the sense of effort or efference copy. The PS rule also explains kinesthetic illusions and the phantom limb phenomenon. This study advances the understanding of action-perception coupling in kinesthesia.

Saccade adaptation as a model of flexible and general motor learning

The rapid point-to-point movements of the eyes called saccades are the most commonly made movement by humans, yet differ from nearly every other type of motor output in that they are completed too quickly to be adjusted during their execution by visual feedback. Saccadic accuracy remains quite high over a lifetime despite inevitable changes to the physical structures controlling the eyes, indicating that the oculomotor system actively monitors and adjusts motor commands to achieve consistent behavioral production. Indeed, it seems that beyond the ability to compensate for slow, age-related bodily changes, saccades can be modified following traumatic injury or pathology that affects their production, or in response to more short-term systematic alterations to post-saccadic visual feedback in a laboratory setting. These forms of plasticity rely on the visual detection of accuracy errors by a unified set of mechanisms that support the process known as saccade adaptation. Saccade adaptation has been mostly studied as a phenomenon in its own right, outside of motor learning in general. Here, we highlight the commonalities between eye and arm movement adaptation by reviewing the literature across these fields wherever there are compelling overlapping theories or data. Recent exciting findings are challenging previous interpretations of the underlying mechanisms of saccade adaptation with the incorporation of concepts including prediction, reinforcement and contextual learning. We review the emerging ideas and evidence with particular emphasis on the important contributions made by Josh Wallman in this sphere over the past 15 years.

Allocation of attention across saccades

Whenever the eyes move, spatial attention must keep track of the locations of targets as they shift on the retina. This study investigated transsaccadic updating of visual attention to cued targets. While observers prepared a saccade, we flashed an irrelevant, but salient, color cue in their visual periphery and measured the allocation of spatial attention before and after the saccade using a tilt discrimination task. We found that just before the saccade, attention was allocated to the cue's future retinal location, its predictively "remapped" location. Attention was sustained at the cue's location in the world across the saccade, despite the change of retinal position whereas it decayed quickly at the retinal location of the cue, after the eye landed. By extinguishing the color cue across the saccade, we further demonstrate that the visual system relies only on predictive allocation of spatial attention, as the presence of the cue after the saccade did not substantially affect attentional allocation. These behavioral results support and extend physiological evidence showing predictive activation of visual neurons when an attended stimulus will fall in their receptive field after a saccade. Our results show that tracking of spatial locations across saccades is a plausible consequence of physiological remapping.

Transient spatiotopic integration across saccadic eye movements mediates visual stability

Eye movements pose major problems to the visual system, because each new saccade changes the mapping of external objects on the retina. It is known that stimuli briefly presented around the time of saccades are systematically mislocalized, whereas continuously visible objects are perceived as spatially stable even when they undergo large transsaccadic displacements. In this study we investigated the relationship between these two phenomena and measured how human subjects perceive the position of pairs of bars briefly displayed around the time of large horizontal saccades. We show that they interact strongly, with the perisaccadic bar being drawn toward the other, dramatically altering the pattern of perisaccadic mislocalization. The interaction field extends over a wide range (200 ms and 20°) and is oriented along the retinotopic trajectory of the saccade-induced motion, suggesting a mechanism that integrates pre- and postsaccadic stimuli at different retinal locations but similar external positions. We show how transient changes in spatial integration mechanisms, which are consistent with the present psychophysical results and with the properties of "remapping cells" reported in the literature, can create transient craniotopy by merging the distinct retinal images of the pre- and postsaccadic fixations to signal a single stable object.

Apparent motion from outside the visual field, retinotopic cortices may register extra-retinal positions

Observers made a saccade between two fixation markers while a probe was flashed sequentially at two locations on a side screen. The first probe was presented in the far periphery just within the observer's visual field. This target was extinguished and the observers made a large saccade away from the probe, which would have left it far outside the visual field if it had still been present. The second probe was then presented, displaced from the first in the same direction as the eye movement and by about the same distance as the saccade step. Because both eyes and probes shifted by similar amounts, there was little or no shift between the first and second probe positions on the retina. Nevertheless, subjects reported seeing motion corresponding to the spatial displacement not the retinal displacement. When the second probe was presented, the effective location of the first probe lay outside the visual field demonstrating that apparent motion can be seen from a location outside the visual field to a second location inside the visual field. Recent physiological results suggest that target locations are “remapped” on retinotopic representations to correct for the effects of eye movements. Our results suggest that the representations on which this remapping occurs include locations that fall beyond the limits of the retina.

Corticospinal control strategies underlying voluntary and involuntary wrist movements

The difference between voluntary and involuntary motor actions has been recognized since ancient times, but the nature of this difference remains unclear. We compared corticospinal influences at wrist positions established before and after voluntary motion with those established before and after involuntary motion elicited by sudden removal of a load (the unloading reflex). To minimize the effect of motoneuronal excitability on the evaluation of corticospinal influences, motor potentials from transcranial magnetic stimulation of the wrist motor cortex area were evoked during an EMG silent period produced by brief muscle shortening. The motoneuronal excitability was thus equalized at different wrist positions. Results showed that the unloading reflex was generated in the presence of a corticospinal drive, rather than autonomously by the spinal cord. Although the tonic EMG levels were substantially different, the corticospinal influences remained the same at the pre- and post-unloading wrist positions. These influences however changed when subjects voluntarily moved the wrist to another position. Previous studies showed that the corticospinal system sets the referent position (R) at which neuromuscular posture-stabilizing mechanisms begin to act. In self-initiated actions, the corticospinal system shifts the R to relay these mechanisms to a new posture, thus converting them from mechanisms resisting to those assisting motion. This solves the classical posture-movement problem. In contrast, by maintaining the R value constant, the corticospinal system relies on these posture-stabilizing mechanisms to allow involuntary responses to occur after unloading. Thus, central control strategies underlying the two types of motor actions are fundamentally different.

Context dependence of receptive field remapping in superior colliculus

Our perception of the positions of objects in our surroundings is surprisingly unaffected by movements of the eyes, head, and body. This suggests that the brain has a mechanism for maintaining perceptual stability, based either on the spatial relationships among visible objects or internal copies of its own motor commands. Strong evidence for the latter mechanism comes from the remapping of visual receptive fields that occurs around the time of a saccade. Remapping occurs when a single neuron responds to visual stimuli placed presaccadically in the spatial location that will be occupied by its receptive field after the completion of a saccade. Although evidence for remapping has been found in many brain areas, relatively little is known about how it interacts with sensory context. This interaction is important for understanding perceptual stability more generally, as the brain may rely on extraretinal signals or visual signals to different degrees in different contexts. Here, we have studied the interaction between visual stimulation and remapping by recording from single neurons in the superior colliculus of the macaque monkey, using several different visual stimulus conditions. We find that remapping responses are highly sensitive to low-level visual signals, with the overall luminance of the visual background exerting a particularly powerful influence. Specifically, although remapping was fairly common in complete darkness, such responses were usually decreased or abolished in the presence of modest background illumination. Thus the brain might make use of a strategy that emphasizes visual landmarks over extraretinal signals whenever the former are available.

Computational models of spatial updating in peri-saccadic perception

Perceptual phenomena that occur around the time of a saccade, such as peri-saccadic mislocalization or saccadic suppression of displacement, have often been linked to mechanisms of spatial stability. These phenomena are usually regarded as errors in processes of trans-saccadic spatial transformations and they provide important tools to study these processes. However, a true understanding of the underlying brain processes that participate in the preparation for a saccade and in the transfer of information across it requires a closer, more quantitative approach that links different perceptual phenomena with each other and with the functional requirements of ensuring spatial stability. We review a number of computational models of peri-saccadic spatial perception that provide steps in that direction. Although most models are concerned with only specific phenomena, some generalization and interconnection between them can be obtained from a comparison. Our analysis shows how different perceptual effects can coherently be brought together and linked back to neuronal mechanisms on the way to explaining vision across saccades.

Visual attention and stability

In the present review, we address the relationship between attention and visual stability. Even though with each eye, head and body movement the retinal image changes dramatically, we perceive the world as stable and are able to perform visually guided actions. However, visual stability is not as complete as introspection would lead us to believe. We attend to only a few items at a time and stability is maintained only for those items. There appear to be two distinct mechanisms underlying visual stability. The first is a passive mechanism: the visual system assumes the world to be stable, unless there is a clear discrepancy between the pre- and post-saccadic image of the region surrounding the saccade target. This is related to the pre-saccadic shift of attention, which allows for an accurate preview of the saccade target. The second is an active mechanism: information about attended objects is remapped within retinotopic maps to compensate for eye movements. The locus of attention itself, which is also characterized by localized retinotopic activity, is remapped as well. We conclude that visual attention is crucial in our perception of a stable world.