Neural mechanisms of saccadic suppression

Visual perception: saccadic omission--suppression or temporal masking?

Although we don't perceive visual stimuli during saccadic eye movements, new evidence shows that our brains do process these stimuli and they can influence our subsequent visual perception.

Modulation of visual signals in macaque MT and MST neurons during pursuit eye movement

Retinal image motion is produced with each eye movement, yet we usually do not perceive this self-produced "reafferent" motion, nor are motion judgments much impaired when the eyes move. To understand the neural mechanisms involved in processing reafferent motion and distinguishing it from the motion of objects in the world, we studied the visual responses of single cells in middle temporal (MT) and medial superior temporal (MST) areas during steady fixation and smooth-pursuit eye movements in awake, behaving macaques. We measured neuronal responses to random-dot patterns moving at different speeds in a stimulus window that moved with the pursuit target and the eyes. This allowed us to control retinal image motion at all eye velocities. We found the expected high proportion of cells selective for the direction of visual motion. Pursuit tracking changed both response amplitude and preferred retinal speed for some cells. The changes in preferred speed were on average weakly but systematically related to the speed of pursuit for area MST cells, as would be expected if the shifts in speed selectivity were compensating for reafferent input. In area MT, speed tuning did not change systematically during pursuit. Many cells in both areas also changed response amplitude during pursuit; the most common form of modulation was response suppression when pursuit was opposite in direction to the cell's preferred direction. These results suggest that some cells in area MST encode retinal image motion veridically during eye movements, whereas others in both MT and MST contribute to the suppression of visual responses to reafferent motion.

Cortical contributions to saccadic suppression

The stability of visual perception is partly maintained by saccadic suppression: the selective reduction of visual sensitivity that accompanies rapid eye movements. The neural mechanisms responsible for this reduced perisaccadic visibility remain unknown, but the Lateral Geniculate Nucleus (LGN) has been proposed as a likely site. Our data show, however, that the saccadic suppression of a target flashed in the right visual hemifield increased with an increase in background luminance in the left visual hemifield. Because each LGN only receives retinal input from a single hemifield, this hemifield interaction cannot be explained solely on the basis of neural mechanisms operating in the LGN. Instead, this suggests that saccadic suppression must involve processing in higher level cortical areas that have access to a considerable part of the ipsilateral hemifield.

The geometry of perisaccadic visual perception

Our ability to explore our surroundings requires a combination of high-resolution vision and frequent rotations of the visual axis toward objects of interest. Such gaze shifts are themselves a source of powerful retinal stimulation, and so the visual system appears to have evolved mechanisms to maintain perceptual stability during movements of the eyes in space. The mechanisms underlying this perceptual stability can be probed in the laboratory by briefly presenting a stimulus around the time of a saccadic eye movement and asking subjects to report its position. Under such conditions, there is a systematic misperception of the probes toward the saccade end point. This perisaccadic compression of visual space has been the subject of much research, but few studies have attempted to relate it to specific brain mechanisms. Here, we show that the magnitude of perceptual compression for a wide variety of probe stimuli and saccade amplitudes is quantitatively predicted by a simple heuristic model based on the geometry of retinotopic representations in the primate brain. Specifically, we propose that perisaccadic compression is determined by the distance between the probe and saccade end point on a map that has a logarithmic representation of visual space, similar to those found in numerous cortical and subcortical visual structures. Under this assumption, the psychophysical data on perisaccadic compression can be appreciated intuitively by imagining that, around the time of a saccade, the brain confounds nearby oculomotor and sensory signals while attempting to localize the position of objects in visual space.

The effect of blinks and saccadic eye movements on visual reaction times

Vision is suppressed during blinks and saccadic eye movements. We hypothesized that visual reaction times (RTs) in a vigilance test would be significantly increased when a blink or a saccade happened to coincide with the stimulus onset. Thirty healthy volunteers each performed a visual RT test for 15 min while their eye and eyelid movements were monitored by a system of infrared reflectance oculography. RTs increased significantly, many by more than 200 msec, when a blink occurred between 75 msec before and up to 150 msec after the stimulus onset. A similar result was observed with saccades that started 75 to 150 msec after the stimulus. Vision or attention was evidently inhibited before each blink and for longer than the saccades lasted. We suggest that visual suppression is involved in this process, which could explain some of the normal variability in RTs over periods of seconds that has not been adequately explained before.

The relationship between saccadic suppression and perceptual stability

Introspection makes it clear that we do not see the visual motion generated by our saccadic eye movements. We refer to the lack of awareness of the motion across the retina that is generated by a saccade as saccadic omission [1]: the visual stimulus generated by the saccade is omitted from our subjective awareness. In the laboratory, saccadic omission is often studied by investigating saccadic suppression, the reduction in visual sensitivity before and during a saccade (see Ross et al. [2] and Wurtz [3] for reviews). We investigated whether perceptual stability requires that a mechanism like saccadic suppression removes perisaccadic stimuli from visual processing to prevent their presumed harmful effect on perceptual stability [4, 5]. Our results show that a stimulus that undergoes saccadic omission can nevertheless generate a shape contrast illusion. This illusion can be generated when the inducer and test stimulus are separated in space and is therefore thought to be generated at a later stage of visual processing [6]. This shows that perceptual stability is attained without removing stimuli from processing and suggests a conceptually new view of perceptual stability in which perisaccadic stimuli are processed by the early visual system, but these signals are prevented from reaching awareness at a later stage of processing.

(Micro)Saccades, corollary activity and cortical oscillations

In natural vision, attention and eye movements are linked. Furthermore, eye movements structure the inflow of information into the visual system. Saccades, where little vision occurs, alternate with fixations, when most vision occurs. A mechanism must be in place to maximize information intake during fixations. Oscillatory synchrony has been proposed as a mechanism for rapid and reliable communication of signals, subserving cognitive functions such as attention and object identification. We propose that saccade-related corollary activity has a crucial role in anticipatory preparation of visual centers, which interacts with ongoing oscillation, favoring the processing of postfixational signals. During prolonged fixations, microsaccades could be generated to exploit this mechanism. Studying this interplay between the sensory and the motor system will provide novel insight into the dynamics of natural vision.

Improved visual sensitivity during smooth pursuit eye movements: Temporal and spatial characteristics

Recently, we showed that contrast sensitivity for color and high–spatial frequency luminance stimuli is enhanced during smooth pursuit eye movements (Schütz et al., 2008). In this study, we investigated the enhancement over a wide range of temporal and spatial frequencies. In Experiment 1, we measured the temporal impulse response function (TIRF) for colored stimuli. The TIRF for pursuit and fixation differed mostly with respect to the gain but not with respect to the natural temporal frequency. Hence, the sensitivity enhancement seems to be rather independent of the temporal frequency of the stimuli. In Experiment 2, we measured the spatial contrast sensitivity function for luminance-defined Gabor patches with spatial frequencies ranging from 0.2 to 7 cpd. We found a sensitivity improvement during pursuit for spatial frequencies above 2–3 cpd. Between 0.5 and 3 cpd, sensitivity was impaired by smooth pursuit eye movements, but no consistent difference was observed below 0.5 cpd. The results of both experiments are consistent with an increased contrast gain of the parvocellular retinogeniculate pathway.

Optimal sensorimotor control in eye movement sequences

Fast and accurate motor behavior requires combining noisy and delayed sensory information with knowledge of self-generated body motion; much evidence indicates that humans do this in a near-optimal manner during arm movements. However, it is unclear whether this principle applies to eye movements. We measured the relative contributions of visual sensory feedback and the motor efference copy (and/or proprioceptive feedback) when humans perform two saccades in rapid succession, the first saccade to a visual target and the second to a memorized target. Unbeknownst to the subject, we introduced an artificial motor error by randomly "jumping" the visual target during the first saccade. The correction of the memory-guided saccade allowed us to measure the relative contributions of visual feedback and efferent copy (and/or proprioceptive feedback) to motor-plan updating. In a control experiment, we extinguished the target during the saccade rather than changing its location to measure the relative contribution of motor noise and target localization error to saccade variability without any visual feedback. The motor noise contribution increased with saccade amplitude, but remained <30% of the total variability. Subjects adjusted the gain of their visual feedback for different saccade amplitudes as a function of its reliability. Even during trials where subjects performed a corrective saccade to compensate for the target-jump, the correction by the visual feedback, while stronger, remained far below 100%. In all conditions, an optimal controller predicted the visual feedback gain well, suggesting that humans combine optimally their efferent copy and sensory feedback when performing eye movements.

Direction and contrast tuning of macaque MSTd neurons during saccades

Saccades are rapid eye movements that change the direction of gaze, although the full-field image motion associated with these movements is rarely perceived. The attenuation of visual perception during saccades is referred to as saccadic suppression. The mechanisms that produce saccadic suppression are not well understood. We recorded from neurons in the dorsal medial superior temporal area (MSTd) of alert macaque monkeys and compared the neural responses produced by the retinal slip associated with saccades (active motion) to responses evoked by identical motion presented during fixation (passive motion). We provide evidence for a neural correlate of saccadic suppression and expand on two contentious results from previous studies. First, we confirm the finding that some neurons in MSTd reverse their preferred direction during saccades. We quantify this effect by calculating changes in direction tuning index for a large cell population. Second, it has been noted that neural activity associated with saccades can arrive in the parietal cortex <or=30 ms earlier than activity produced by similar visual stimulation during fixation. This led to the question of whether the saccade-related responses were visual in origin or were motor signals arising from saccade-planning areas of the brain. By comparing the responses to saccades made over textured backgrounds of different contrasts, we provide strong evidence that saccade-related responses were visual in origin. Refinements of the possible models of saccadic suppression are discussed.

Throwing a glance at the neural code: rapid information transmission in the visual system

Our visual system can operate at fascinating speeds. Psychophysical experiments teach us that the processing of complex natural images and visual object recognition require a mere split second. Even in everyday life, our gaze seldom rests for long on any particular spot of the visual scene before a sudden movement of the eyes or the head shifts it to a new location. These observations challenge our understanding of how neurons in the visual system of the brain represent, process, and transmit the relevant visual information quickly enough. This article argues that the speed of visual processing provides an adjuvant framework for studying the neural code in the visual system. In the retina, which constitutes the first stage of visual processing, recent experiments have highlighted response features that allow for particularly rapid information transmission. This sets the stage for discussing some of the fundamental questions in the research of neural coding. How do downstream brain regions read out signals from the retina and combine them with intrinsic signals that accompany eye movements? And, how do the neural response features ultimately affect perception and behavior?

Saccadic modulation of neural responses: possible roles in saccadic suppression, enhancement, and time compression

Humans use saccadic eye movements to make frequent gaze changes, yet the associated full-field image motion is not perceived. The theory of saccadic suppression has been proposed to account for this phenomenon, but it is not clear whether suppression originates from a retinal signal at saccade onset or from the brain before saccade onset. Perceptually, visual sensitivity is reduced before saccades and enhanced afterward. Over the same time period, the perception of time is compressed and even inverted. We explore the origins and neural basis of these effects by recording from neurons in the dorsal medial superior temporal area (MSTd) of alert macaque monkeys. Neuronal responses to flashed presentations of a textured pattern presented at random times relative to saccades exhibit a stereotypical pattern of modulation. Response amplitudes are strongly suppressed for flashes presented up to 90 ms before saccades. Immediately after the suppression, there is a period of 200-450 ms in which flashes generate enhanced response amplitudes. Our results show that (1) MSTd is not directly suppressed, rather suppression is inherited from earlier visual areas; (2) early suppression of the visual system must be of extra-retinal origin; (3) postsaccadic enhancement of neural activity occurs in MSTd; and (4) the enhanced responses have reduced latencies. As a whole, these observations reveal response properties that could account for perceptual observations relating to presaccadic suppression, postsaccadic enhancement and time compression.

Brain circuits for the internal monitoring of movements

Each movement we make activates our own sensory receptors, thus causing a problem for the brain: the spurious, movement-related sensations must be discriminated from the sensory inputs that really matter, those representing our environment. Here we consider circuits for solving this problem in the primate brain. Such circuits convey a copy of each motor command, known as a corollary discharge (CD), to brain regions that use sensory input. In the visual system, CD signals may help to produce a stable visual percept from the jumpy images resulting from our rapid eye movements. A candidate pathway for providing CD for vision ascends from the superior colliculus to the frontal cortex in the primate brain. This circuit conveys warning signals about impending eye movements that are used for planning subsequent movements and analyzing the visual world. Identifying this circuit has provided a model for studying CD in other primate sensory systems and may lead to a better understanding of motor and mental disorders.

Neuronal mechanisms of visual stability

Human vision is stable and continuous in spite of the incessant interruptions produced by saccadic eye movements. These rapid eye movements serve vision by directing the high resolution fovea rapidly from one part of the visual scene to another. They should detract from vision because they generate two major problems: displacement of the retinal image with each saccade and blurring of the image during the saccade. This review considers the substantial advances in understanding the neuronal mechanisms underlying this visual stability derived primarily from neuronal recording and inactivation studies in the monkey, an excellent model for systems in the human brain. For the first problem, saccadic displacement, two neuronal candidates are salient. First are the neurons in frontal and parietal cortex with shifting receptive fields that provide anticipatory activity with each saccade and are driven by a corollary discharge. These could provide the mechanism for a retinotopic hypothesis of visual stability and possibly for a transsaccadic memory hypothesis, The second neuronal mechanism is provided by neurons whose visual response is modulated by eye position (gain field neurons) or are largely independent of eye position (real position neurons), and these neurons could provide the basis for a spatiotopic hypothesis. For the second problem, saccadic suppression, visual masking and corollary discharge are well established mechanisms, and possible neuronal correlates have been identified for each.

Corollary discharge across the animal kingdom

Our movements can hinder our ability to sense the world. Movements can induce sensory input (for example, when you hit something) that is indistinguishable from the input that is caused by external agents (for example, when something hits you). It is critical for nervous systems to be able to differentiate between these two scenarios. A ubiquitous strategy is to route copies of movement commands to sensory structures. These signals, which are referred to as corollary discharge (CD), influence sensory processing in myriad ways. Here we review the CD circuits that have been uncovered by neurophysiological studies and suggest a functional taxonomic classification of CD across the animal kingdom. This broad understanding of CD circuits lays the groundwork for more challenging studies that combine neurophysiology and psychophysics to probe the role of CD in perception.

Differential detection of global luminance and contrast changes across saccades and flickers during active scene perception

How sensitive are viewers to changes in global image properties across saccades during active real-world scene perception? This question was investigated by globally increasing and/or decreasing luminance or contrast in photographs of real-world scenes across saccadic eye movements or during matched brief interruptions in a flicker paradigm. The results from two experiments demonstrated very poor sensitivity to global image changes in both the saccade-contingent and flicker paradigms, suggesting that the specific values of basic sensory properties do not contribute to the perception of stability across saccades during complex scene perception. In addition, overall sensitivity was significantly worse in the saccade-contingent change paradigm than the flicker paradigm, suggesting that the flicker paradigm is an imperfect simulation of transsaccadic vision.

Corollary discharge circuits for saccadic modulation of the pigeon visual system

A saccadic eye movement causes a variety of transient perceptual sequelae that might be the results of corollary discharge. Here we describe the neural circuits for saccadic corollary discharge that modulates activity throughout the pigeon visual system. Saccades in pigeons caused inhibition that was mediated by corollary discharge followed by enhancement of firing activity in the telencephalic hyperpallium, visual thalamus and pretectal nucleus lentiformis mesencephali (nLM) with opposite responses in the accessory optic nucleus (nBOR). Inactivation of thalamic neurons eliminated saccadic responses in telencephalic neurons, and inactivation of both the nLM and the nBOR abolished saccadic responses in thalamic neurons. Saccade-related omnipause neurons in the brainstem raphe complex inhibited the nBOR and excited the nLM, whereas inactivation of raphe neurons eliminated saccadic responses in both optokinetic and thalamic neurons. It seems that saccadic responses in telencephalic neurons are generated by corollary discharge signals from brainstem neurons that are transmitted through optokinetic and thalamic neurons. These signals might have important roles in visual perception.

A computational neuroanatomy for motor control

The study of patients to infer normal brain function has a long tradition in neurology and psychology. More recently, the motor system has been subject to quantitative and computational characterization. The purpose of this review is to argue that the lesion approach and theoretical motor control can mutually inform each other. Specifically, one may identify distinct motor control processes from computational models and map them onto specific deficits in patients. Here we review some of the impairments in motor control, motor learning and higher-order motor control in patients with lesions of the corticospinal tract, the cerebellum, parietal cortex, the basal ganglia, and the medial temporal lobe. We attempt to explain some of these impairments in terms of computational ideas such as state estimation, optimization, prediction, cost, and reward. We suggest that a function of the cerebellum is system identification: to build internal models that predict sensory outcome of motor commands and correct motor commands through internal feedback. A function of the parietal cortex is state estimation: to integrate the predicted proprioceptive and visual outcomes with sensory feedback to form a belief about how the commands affected the states of the body and the environment. A function of basal ganglia is related to optimal control: learning costs and rewards associated with sensory states and estimating the "cost-to-go" during execution of a motor task. Finally, functions of the primary and the premotor cortices are related to implementing the optimal control policy by transforming beliefs about proprioceptive and visual states, respectively, into motor commands.

Macaque V1 activity during natural vision: effects of natural scenes and saccades

In the present study, we examined the way that scene complexity and saccades combine to sculpt the temporal response patterns of V1 neurons. To bridge the gap between conventional and free viewing experiments, we compared responses of neurons across four paradigms ranging from less to more natural. An optimal bar stimulus was either flashed into a receptive field (RF) or brought into it via saccade and was embedded in either a natural scene or a uniform gray background. Responses to a flashed bar tended to be higher with a uniform rather than natural background. The most novel result reported here is that responses evoked by stimuli brought into the RF via saccades were enhanced compared with the same stimuli flashed during steady fixation. No single factor appears to account entirely for this surprising effect, but there were small contributions from fixational saccades and residual activity carried over from the previous fixation. We also found a negative correlation with cells' response "history" in that a larger response on one fixation was associated with a lower response on the subsequent fixation. The effects of the natural background and saccades exhibited a significant nonlinear interaction with the suppressive effects of the natural background less for stimuli entering RFs with saccades. Together, these results suggest that even responses to standard optimal stimuli are difficult to predict under conditions similar to natural vision, and further demonstrate the importance of naturalistic experimental paradigms to the study of visual processing in V1.

Noise-improved signal detection in cat primary visual cortex via a well-balanced stochastic resonance-like procedure

Adding noise to a weak signal can paradoxically improve signal detection, a process called 'stochastic resonance' (SR). In the visual system, noise might be introduced by the image jitter resulting from high-frequency eye movements, like eye microtremor and microsaccades. To test whether this kind of noise might be beneficial or detrimental for cortical signal detection, we performed single-unit recordings from area 17 of anaesthetized cats while jittering the visual stimulus in a frequency and amplitude range resembling the possible range of eye movements. We used weak, sub- and peri-threshold visual stimuli, on top of which we superimposed noise with variable jitter amplitude. In accordance with the typical SR effect, we found that small noise levels actually increased the signal-to-noise ratio (SNR) of previously weak cortical visual responses, while originally strong responses were little affected or even reduced. Above a certain noise level, the SNR dropped a little, but not as a result of increased background activity - as would be proposed by SR theory - but because of a lowered response to signal and noise. Therefore, it seems that the ascending visual pathway optimally utilizes signal detection improvement by a SR-like process, while at the same time preventing spurious noise-induced activity and keeping the SNR sufficiently high.

Contrast sensitivity during the initiation of smooth pursuit eye movements

Eye movements challenge the perception of a stable world by inducing retinal image displacement. During saccadic eye movements visual stability is accompanied by a remapping of visual receptive fields, a compression of visual space and perceptual suppression. Here we explore whether a similar suppression changes the perception of briefly presented low contrast targets during the initiation of smooth pursuit eye movements. In a 2AFC design we investigated the contrast sensitivity for threshold-level stimuli during the initiation of smooth pursuit and during saccades. Pursuit was elicited by horizontal step-ramp and ramp stimuli. At any time from 200 ms before to 500 ms after pursuit stimulus onset, a blurred 0.3 deg wide horizontal line with low contrast just above detection threshold appeared for 10 ms either 2 deg above or below the pursuit trajectory. Observers had to pursue the moving stimulus and to indicate whether the target line appeared above or below the pursuit trajectory. In contrast to perceptual suppression effects during saccades, no pronounced suppression was found at pursuit onset for step-ramp motion. When pursuit was elicited by a ramp stimulus, pursuit initiation was accompanied by catch-up saccades, which caused saccadic suppression. Additionally, contrast sensitivity was attenuated at the time of pursuit or saccade stimulus onset. This attenuation might be due to an attentional deficit, because the stimulus required the focus of attention during the programming of the following eye movement.

Behavioral context-dependent modulation of descending statocyst pathways during free walking, as revealed by optical telemetry in crayfish

Crustacean posture control is based on a complex interaction between the statocyst input and other sensory inputs as well as the animal's behavioral context. We examined the effects of behavioral condition on the activity of descending statocyst pathways using an optical telemetry system that allowed underwater recording of neuronal signals from freely behaving crayfish. A functionally identified statocyst-driven interneuron that directionally responded to body tilting without a footboard and to tilting of the footboard was found to show complicated responses depending upon the ongoing behavior of the animal when it freely walked around in water on the aquarium floor. The spike firing frequency of the interneuron increased significantly during walking. When the animal stood or walked on the tilted floor, the interneuron activity represented the tilt angle and direction if the abdomen was actively flexed, but not if it was extended. Two other statocyst-driven descending interneurons were found to be affected differently by the animal's behavioral condition: the spike activity of one interneuron increased during walking, but its directional response on the tilted floor was completely absent during abdominal posture movements, whereas that of another interneuron was enhanced during abdominal extension only, representing the tilt angle and direction. The results obtained in this study provide the first experimental demonstration that crustacean postural control under natural conditions is dependent on very fine aspects of the animal's locomotor behavioral context, suggesting far more complex control mechanisms than those expected from the experimental data obtained in isolated and fixed animals.

Natural vision reveals regional specialization to local motion and to contrast-invariant, global flow in the human brain

Visual changes in feature movies, like in real-live, can be partitioned into global flow due to self/camera motion, local/differential flow due to object motion, and residuals, for example, due to illumination changes. We correlated these measures with brain responses of human volunteers viewing movies in an fMRI scanner. Early visual areas responded only to residual changes, thus lacking responses to equally large motion-induced changes, consistent with predictive coding. Motion activated V5+ (MT+), V3A, medial posterior parietal cortex (mPPC) and, weakly, lateral occipital cortex (LOC). V5+ responded to local/differential motion and depended on visual contrast, whereas mPPC responded to global flow spanning the whole visual field and was contrast independent. mPPC thus codes for flow compatible with unbiased heading estimation in natural scenes and for the comparison of visual flow with nonretinal, multimodal motion cues in it or downstream. mPPC was functionally connected to anterior portions of V5+, whereas laterally neighboring putative homologue of lateral intraparietal area (LIP) connected with frontal eye fields. Our results demonstrate a progression of selectivity from local and contrast-dependent motion processing in V5+ toward global and contrast-independent motion processing in mPPC. The function, connectivity, and anatomical neighborhood of mPPC imply several parallels to monkey ventral intraparietal area (VIP).

Transient cortical excitation at the onset of visual fixation

Primates actively examine the visual world by rapidly shifting gaze (fixation) over the elements in a scene. Despite this fact, we typically study vision by presenting stimuli with gaze held constant. To better understand the dynamics of natural vision, we examined how the onset of visual fixation affects ongoing neuronal activity in the absence of visual stimulation. We used multiunit activity and current source density measurements to index neuronal firing patterns and underlying synaptic processes in macaque V1. Initial averaging of neural activity synchronized to the onset of fixation suggested that a brief period of cortical excitation follows each fixation. Subsequent single-trial analyses revealed that 1) neuronal oscillation phase transits from random to a highly organized state just after the fixation onset, 2) this phase concentration is accompanied by increased spectral power in several frequency bands, and 3) visual response amplitude is enhanced at the specific oscillatory phase associated with fixation. We hypothesize that nonvisual inputs are used by the brain to increase cortical excitability at fixation onset, thus "priming" the system for new visual inputs generated at fixation. Despite remaining mechanistic questions, it appears that analysis of fixation-related responses may be useful in studying natural vision.

MST Neurons Code for Visual Motion in Space Independent of Pursuit Eye Movements

When a person tracks a small moving object, the visual images in the background of the visual scene move across his/her retina. It, however, is possible to estimate the actual motion of the images despite the eye-movement-induced motion. To understand the neural mechanism that reconstructs a stable visual world independent of eye movements, we explored areas MT (middle temporal) and MST (medial superior temporal) in the monkey cortex, both of which are known to be essential for visual motion analysis. We recorded the responses of neurons to a moving textured image that appeared briefly on the screen while the monkeys were performing smooth pursuit or stationary fixation tasks. Although neurons in both areas exhibited significant responses to the motion of the textured image with directional selectivity, the responses of MST neurons were mostly correlated with the motion of the image on the screen independent of pursuit eye movement, whereas the responses of MT neurons were mostly correlated with the motion of the image on the retina. Thus these MST neurons were more likely than MT neurons to distinguish between external and self-induced motion. The results are consistent with the idea that MST neurons code for visual motion in the external world while compensating for the counter-rotation of retinal images due to pursuit eye movements.

Identity of a pathway for saccadic suppression

Neurons in the superficial gray layer (SGS) of the superior colliculus receive visual input and excite intermediate layer (SGI) neurons that play a critical role in initiating rapid orienting movements of the eyes, called saccades. In the present study, two types of experiments demonstrate that a population of SGI neurons gives rise to a reciprocal pathway that inhibits neurons in SGS. First, in GAD67-GFP knockin mice, GABAergic SGI neurons that expressed GFP fluorescence were injected with the tracer biocytin to reveal their axonal projections. Axons arising from GFP-positive neurons in SGI terminated densely in SGS. Next, SGI neurons in rats and mice were stimulated by using the photolysis of caged glutamate, and in vitro whole-cell patch-clamp recordings were used to measure the responses evoked in SGS cells. Large, synaptically mediated outward currents were evoked in SGS neurons. These currents were blocked by gabazine, confirming that they were GABA(A) receptor-mediated inhibitory postsynaptic currents. This inhibitory pathway from SGI transiently suppresses visual activity in SGS, which in turn could have multiple effects. These effects could include reduction of perceptual blurring during saccades as well as prevention of eye movements that might be spuriously triggered by the sweep of the visual field across the retina.

Saccade-related information in the superior temporal motion complex: quantitative functional mapping in the monkey

Although the role of the motion complex [cortical areas middle temporal (V5/MT), medial superior temporal (MST), and fundus of the superior temporal (FST)] in visual motion and smooth-pursuit eye movements is well understood, little is known about its involvement in rapid eye movements (saccades). To address this issue, we used the quantitative 14C-deoxyglucose method to obtain functional maps of the cerebral cortex lying in the superior temporal sulcus of rhesus monkeys executing saccades to visual targets and saccades to memorized targets in complete darkness. Fixational effects were observed in MT-foveal, FST, the anterior part of V4-transitional (V4t), and temporal-occipital areas. Saccades to memorized targets activated areas V5/MT, MST, and V4t, which were also activated for saccades to visual targets. Regions activated in the light and in the dark overlapped extensively. In addition, saccades to visual targets activated areas FST and the intermediate part of the polysensory temporal-parietal-occipital area. Cortical activity related to visually guided saccades could be explained, at least in part, by visual motion. Because only oculomotor signals can account for the equally robust activations induced by memory saccades in complete darkness, we suggest that areas V5/MT, MST, and V4t receive and/or process saccade-related oculomotor information.

Saccadic adaptation in children

Saccades are fast-orienting eye movements. Saccadic adaptation, a form of motor learning, is a corrective change in the amplitude of saccades in response to error. The aim of the study was to ascertain whether saccadic adaptation occurs in typically developing children. We recorded saccades with an infrared eye tracker in 39 children, aged 8 to 19 years, at baseline to 12-degree horizontal target steps and after an adaptive task. During the adaptive task, a saccadic hypometric error was induced. This task consisted of 200 12-degree target steps that stepped backward 3 degrees during the initial saccade and without the participants' awareness. The initial saccade triggered the back-step. This paradigm required a corrective reduction of the amplitude of the initial saccades in response to the induced error. Saccadic adaptation was achieved in 26 participants, whose mean saccadic amplitudes decreased by 13% (P < .05). Saccadic adaptation was not influenced by age. We conclude that children as young as 8 years old have established functions of the neural circuits responsible for the motor learning required for saccadic adaptation.

Perception: Transient Disruptions to Neural Space–Time

How vision operates efficiently in the face of continuous shifts of gaze remains poorly understood. Recent studies show that saccades cause dramatic, but transient, changes in the spatial and also temporal tuning of cells in many visual areas, which may underly the perceptual compression of space and time, and serve to counteract the effects of the saccades and maintain visual stability.

Encoding of Naturalistic Optic Flow by a Population of Blowfly Motion-Sensitive Neurons

In sensory systems information is encoded by the activity of populations of neurons. To analyze the coding properties of neuronal populations sensory stimuli have usually been used that were much simpler than those encountered in real life. It has been possible only recently to stimulate visual interneurons of the blowfly with naturalistic visual stimuli reconstructed from eye movements measured during free flight. Therefore we now investigate with naturalistic optic flow the coding properties of a small neuronal population of identified visual interneurons in the blowfly, the so-called VS and HS neurons. These neurons are motion sensitive and directionally selective and are assumed to extract information about the animal's self-motion from optic flow. We could show that neuronal responses of VS and HS neurons are mainly shaped by the characteristic dynamical properties of the fly's saccadic flight and gaze strategy. Individual neurons encode information about both the rotational and the translational components of the animal's self-motion. Thus the information carried by individual neurons is ambiguous. The ambiguities can be reduced by considering neuronal population activity. The joint responses of different subpopulations of VS and HS neurons can provide unambiguous information about the three rotational and the three translational components of the animal's self-motion and also, indirectly, about the three-dimensional layout of the environment.

Retinally-generated saccadic suppression of a locust looming-detector neuron: investigations using a robot locust

A fundamental task performed by many visual systems is to distinguish apparent motion caused by eye movements from real motion occurring within the environment. During saccadic eye movements, this task is achieved by inhibitory signals of central and retinal origin that suppress the output of motion-detecting neurons. To investigate the retinally-generated component of this suppression, we used a computational model of a locust looming-detecting pathway that experiences saccadic suppression. This model received input from the camera of a mobile robot that performed simple saccade-like movements, allowing the model's response to simplified real stimuli to be tested. Retinally-generated saccadic suppression resulted from two inhibitory mechanisms within the looming-detector's input architecture. One mechanism fed inhibition forward through the network, inhibiting the looming-detector's initial response to movement. The second spread inhibition laterally within the network, suppressing the looming-detector's maintained response to movement. These mechanisms prevent a looming-detector model response to whole-field visual stimuli. In the locust, this mechanism of saccadic suppression may operate in addition to centrally-generated suppression. Because lateral inhibition is a common feature of early visual processing in many organisms, we discuss whether the mechanism of retinally-generated saccadic suppression found in the locust looming-detector model may also operate in these species.

Saccadic suppression of retinotopically localized blood oxygen level-dependent responses in human primary visual area V1

Saccadic eye movements are responsible for bringing relevant parts of the visual field onto the fovea for detailed analysis. Because the retina is physiologically unable to deliver sharp images at very high transsaccadic speeds, the visual system minimizes the repercussion of the blurry images we would otherwise perceive during transsaccadic vision by reducing general visual sensitivity and increasing the detection threshold for visual stimuli. Ruling out a pure retinal origin, the effects of saccadic suppression can be already observed some 75 ms before the onset of a saccadic eye movement and are maximal at the onset of motion. The perception of a briefly presented stimulus immediately before the onset of any retinal motion is thus impaired despite the fact that this stimulus is projected onto the stationary retina and is, therefore, physically identical to that presented when no saccadic programming is in course. In this functional magnetic resonance imaging event-related study, we flashed Gabor patches at different times before the onset of a horizontal saccade and measured blood oxygen level-dependent responses at their encoding regions in primary visual cortex (V1) while subjects judged the relative orientation of the stimuli. Closely matching the significant reduction in behavioral performance, the amplitude of the responses in V1 consistently decreased as the stimuli were presented closer to the saccadic onset. These results demonstrate that the neural processes underlying saccade programming transiently modulate cortical responses to briefly presented visual stimuli in areas as early a V1, providing additional evidence for the existence of an active saccadic suppression mechanism in humans.

Retinal ganglion cell coding in simulated active vision

The image on the retina is almost never static. Eye, head, and body movements, and externally generated motion create rapid and continual changes in the retinal image (“active vision”). Virtually all vision in animals such as primates, which make saccades as often as 3–4 times/s, is based on information that must be derived from the first few hundred milliseconds after sudden, global changes in the retinal image. These changes may be accompanied by large changes in area mean luminance, as well as higher order image contrast statistics. This study investigated how retinal ganglion cell responses, whose response properties have been typically studied and defined in a stable stimulus regime, are affected by sudden changes in mean luminance that are characteristic of active vision. Specifically, the steady-state responses of retinal ganglion cells to static or moving square-wave grating stimuli were recorded in an isolated, superfused rabbit eyecup preparation and compared to responses after saccade-like changes in luminance. The manner of coding after luminance changes was different for different ganglion cell classes; both suppression and enhancement of responses to patterns following luminance changes were found. Brisk-transient Off cells unambiguously signaled the darkening of the overall image, but were also modulated by the subsequently appearing grating stimulus. Several types of On-center cell behavior were observed, ranging from strong suppression of the subsequent response by luminance changes, to strong enhancement. Overall, most ganglion cells distinguished static patterns after a luminance change via differences in their spike discharges nearly as well as before, although there were clear asymmetries between the On and Off pathways. Changes in mean luminance in some ganglion cells, such as On–Off directionally selective ganglion cells, could create large phase shifts in the response to patterned, moving stimuli, although these stimuli were still detected immediately after luminance changes. The results of this study show that the image dynamics of active vision may be a fundamental challenge for the visual system because of strong effects on retinal ganglion cell function. However, rapid extraction of unambiguous information after luminance changes appears to be encoded in differences in the spike discharges in different retinal ganglion cell classes. Asymmetries among ganglion cell classes in sensitivity to luminance changes may provide a basis by which some provide the “context” for interpreting the firing of others.

Development of saccadic suppression in children

We measured saccadic suppression in adolescent children and young adults using spatially curtailed low spatial frequency stimuli. For both groups, sensitivity for color-modulated stimuli was unchanged during saccades. Sensitivity for luminance-modulated stimuli was greatly reduced during saccades in both groups but far more for adolescents than for young adults. Adults' suppression was on average a factor of about 3, whereas that for the adolescent group was closer to a factor of 10. The specificity of the suppression to luminance-modulated stimuli excludes generic explanations such as task difficulty and attention. We suggest that the enhanced suppression in adolescents results from the immaturity of the ocular-motor system at that age.

Perceptual issues in the use of head-mounted visual displays

OBJECTIVE

We provide a review and analysis of much of the published literature on visual perception issues that impact the design and use of head-mounted displays (HMDs).

BACKGROUND

Unlike the previous literature on HMDs, this review draws heavily from the basic vision literature in order to help provide insight for future design solutions for HMDs.

METHOD

Included in this review are articles and books found cited in other works as well as articles and books obtained from an Internet search.

RESULTS

Issues discussed include the effect of brightness and contrast on depth of field, dark focus, dark vergence, and perceptual constancy; the effect of accommodation-vergence synergy on perceptual constancy, eyestrain, and discomfort; the relationship of field of view to the functioning of different visual pathways and the types of visual-motor tasks mediated by them; the relationship of binocular input to visual suppression; and the importance of head movements, head tracking, and display update lag.

CONCLUSION

This paper offers a set of recommendations for the design and use of HMDs.

APPLICATION

Consideration of the basic vision literature will provide insight for future design solutions for HMDs.

Vision: a brake on the speed of sight

We move our eyes more often than our heart beats. Our brain seems to cope effortlessly with the consequences of these rapid visual alterations, but a new study shows that similar scene changes in the absence of eye movements delay the speed of information processing. So are there costs in constantly shifting our focus of gaze?

Structure and function of visual area MT

The small visual area known as MT or V5 has played a major role in our understanding of the primate cerebral cortex. This area has been historically important in the concept of cortical processing streams and the idea that different visual areas constitute highly specialized representations of visual information. MT has also proven to be a fertile culture dish--full of direction- and disparity-selective neurons--exploited by many labs to study the neural circuits underlying computations of motion and depth and to examine the relationship between neural activity and perception. Here we attempt a synthetic overview of the rich literature on MT with the goal of answering the question, What does MT do?

Post-saccadic updating of visual space in the posterior parietal cortex in humans

Updating of visual space takes place in the posterior parietal cortex to guarantee spatial constancy across eye movements. However, the timing of updating with respect to saccadic eye movements remains a matter of debate. In the present study, event-related potentials (ERPs) were recorded in 15 volunteers during a saccadic double-step task to elucidate the time course of the updating process. In the experimental condition updating of visual space was required, because both saccade targets had already disappeared before the first saccade was executed. A similar task without updating requirements served as control condition. ERP analysis revealed a significantly larger slow positive wave in the retino-spatial dissonance condition compared to the control condition, starting between 150 and 200 ms after first saccade onset. Source analysis showed an asymmetry with respect to the direction of the first saccade. Whereas the source was restricted to the right PPC in trials with leftward first saccades, left and right PPC were involved in rightward trials. The results of the present study suggest that updating of visual space in a saccadic double-step task occurs not earlier than 150 ms after the onset of the first saccade. We conclude that extraretinal information about the first saccade is integrated with motor information about the second saccade in the inter-saccade interval.

Background changes delay the perceptual availability of form information

In natural visual situations, unlike most psychophysical experiments, when a new stimulus appears in a portion of the visual field, the surrounding background changes simultaneously. In recordings from macaque V1, we found that a visual stimulus presented simultaneously with a background change evokes a response that is qualitatively different from the response to the same stimulus flashed on a static background. With the changing background, information about stimulus orientation and contrast is significantly delayed compared with the static-background situation. Our physiological results make several predictions that we test in the present paper with human psychophysical experiments. In a backward masking paradigm, a bar stimulus was either flashed onto a static background or presented simultaneously with a change in background luminance or pattern. Subjects discriminated bar orientation or detected that the scene changed before the mask. To achieve an equivalent contrast threshold for orientation discrimination, a longer stimulus-mask onset asynchrony (SOA) was needed in the changing than in the static-background condition; to match the orientation discrimination performance in the static and changing-background conditions at a fixed SOA, a higher bar contrast was needed when the background changed. Moreover, in the changing-background condition, a longer SOA was needed to discriminate bar orientation than to detect the scene change. These results suggest that orientation information is available more slowly when the background changes; orientation information is available earlier as stimulus contrast increases. The psychophysical findings are consistent with our physiological predictions. Compared with the common technique of flashing stimuli onto a static background, the changing-background paradigm may be more similar to natural vision in which saccades bring new stimuli and backgrounds into the visual field.

Distribution of Activity Across the Monkey Cerebral Cortical Surface, Thalamus and Midbrain during Rapid, Visually Guided Saccades

To examine the distribution of visual and oculomotor activity across the macaque brain, we performed functional magnetic resonance imaging (fMRI) on awake, behaving monkeys trained to perform visually guided saccades. Two subjects alternated between periods of making saccades and central fixations while blood oxygen level dependent (BOLD) images were collected [3 T, (1.5 mm)3 spatial resolution]. BOLD activations from each of four cerebral hemispheres were projected onto the subjects' cortical surfaces and aligned to a surface-based atlas for comparison across hemispheres and subjects. This surface-based analysis revealed patterns of visuo-oculomotor activity across much of the cerebral cortex, including activations in the posterior parietal cortex, superior temporal cortex and frontal lobe. For each cortical domain, we show the anatomical position and extent of visuo-oculomotor activity, including evidence that the dorsolateral frontal activation, which includes the frontal eye field (on the anterior bank of the arcuate sulcus), extends anteriorly into posterior principal sulcus (area 46) and posteriorly into part of dorsal premotor cortex (area 6). Our results also suggest that subcortical BOLD activity in the pulvinar thalamus may be lateralized during voluntary eye movements. These findings provide new neuroanatomical information as to the complex neural substrates that underlie even simple goal-directed behaviors.

The primate working memory networks

Working memory has long been associated with the prefrontal cortex, since damage to this brain area can critically impair the ability to maintain and update mnemonic information. Anatomical and physiological evidence suggests, however, that the prefrontal cortex is part of a broader network of interconnected brain areas involved in working memory. These include the parietal and temporal association areas of the cerebral cortex, cingulate and limbic areas, and subcortical structures such as the mediodorsal thalamus and the basal ganglia. Neurophysiological studies in primates confirm the involvement of areas beyond the frontal lobe and illustrate that working memory involves parallel, distributed neuronal networks. In this article, we review the current understanding of the anatomical organization of networks mediating working memory and the neural correlates of memory manifested in each of their nodes. The neural mechanisms of memory maintenance and the integrative role of the prefrontal cortex are also discussed.

Rapid processing of retinal slip during saccades in macaque area MT

The primate middle temporal area (MT) is involved in the analysis and perception of visual motion, which is generated actively by eye and body movements and passively when objects move. We studied the responses of single cells in area MT of awake macaques, comparing the direction tuning and latencies of responses evoked by wide-field texture motion during fixation (passive viewing) and during rewarded, target-directed saccades and non-rewarded, spontaneous saccades over the same stationary texture (active viewing). We found that MT neurons have similar motion sensitivity and direction-selectivity for retinal slip associated with active and passive motion. No cells showed reversals in direction tuning between the active and passive viewing conditions. However, mean latencies were significantly different for saccade-evoked responses (30 ms) and stimulus-evoked responses (67 ms). Our results demonstrate that neurons in area MT retain their direction-selectivity and display reduced processing times during saccades. This rapid, accurate processing of peri-saccadic motion may facilitate post-saccadic ocular following reflexes or corrective saccades.

The role of the human thalamus in processing corollary discharge

Corollary discharge signals play an important role in monitoring self-generated movements to guarantee spatial constancy. Recent work in macaques suggests that the thalamus conveys corollary discharge information of upcoming saccades passing from the superior colliculus to the frontal eye field. The present study aimed to investigate the involvement of the thalamus in humans by assessing the effect of thalamic lesions on the processing of corollary discharge information. Thirteen patients with selective thalamic lesions and 13 healthy age-matched control subjects performed a saccadic double-step task in which retino-spatial dissonance was induced, i.e. the retinal vector of the second target and the movement vector of the second saccade were different. Thus, the subjects could not rely on retinal information alone, but had to use corollary discharge information to correctly perform the second saccade. The amplitudes of first and second saccades were significantly smaller in patients than in controls. Five thalamic lesion patients showed unilateral deficits in using corollary discharge information, as revealed by asymmetries compared with the other patients and controls. Three patients with lateral thalamic lesions including the ventrolateral nucleus (VL) were impaired contralaterally to the side of damage and one patient with a lesion in the mediodorsal thalamus (MD) was impaired ipsilaterally to the lesion. The largest asymmetry was found in a patient with a bilateral thalamic lesion. The results provide evidence for a thalamic involvement in the processing of corollary discharge information in humans, with a potential role of both the VL and MD nuclei.

Saccades Differentially Modulate Human LGN and V1 Responses in the Presence and Absence of Visual Stimulation

Saccades occur several times each second in normal human vision. The visual image moves across the retina at high velocity during a saccade, yet no blurring of the visual scene is perceived . Active suppression of visual input may account for this perceptual continuity, but the neural mechanisms underlying such saccadic suppression remain unclear. We used functional MRI to specifically examine responses in the lateral geniculate nucleus (LGN) and primary visual cortex (V1) during saccades. Activity in both V1 and LGN was strongly modulated by saccades. Furthermore, this modulation depended on whether visual stimulation was present or absent. In complete darkness, saccades led to reliable signal increases in V1 and LGN, whereas in the presence of visual stimulation, saccades led to suppression of visually evoked responses. These findings represent unequivocal evidence for saccadic suppression in human LGN and retinotopically defined V1 and are consistent with the earliest site of saccadic suppression lying at or before V1.

Contrast dependency of sacadic compression and suppression

In the occurrence of a saccadic eye movement vision becomes suppressed. Supra-threshold visual stimuli that are briefly presented at that time become perceptually compressed towards the saccade target (saccadic compression) and shifted in saccade direction (saccadic shift). We show that the strength of saccadic compression, like the strength of saccadic suppression, varies with stimulus contrast. Low contrast stimuli lead to stronger compression than high contrast stimuli. The similarity of contrast dependence and time course suggests that saccadic compression is related to saccadic suppression. Because the saccadic shift did not depend on contrast we suggest that shift and compression are different effects.

A model of active visual search with object-based attention guiding scan paths

When a monkey searches for a colour and orientation feature conjunction target, the scan path is guided to target coloured locations in preference to locations containing the target orientation [Vision Res. 38 (1998b) 1805]. An active vision model, using biased competition, is able to replicate this behaviour. As object-based attention develops in extrastriate cortex, featural information is passed to posterior parietal cortex (LIP), enabling it to represent behaviourally relevant locations [J. Neurophysiol. 76 (1996) 2841] and guide the scan path. Attention evolves from an early spatial effect to being object-based later in the response of the model neurons, as has been observed in monkey single cell recordings. This is the first model to reproduce these effects with temporal precision and is reported here at the systems level allowing the replication of psychophysical scan paths.

On the fight between excitation and inhibition: location is everything

In the intact brain, neurons are constantly subjected to both excitatory and inhibitory inputs to their dendritic trees. Although it is accepted that the overall response of a neuron--its train of output spikes--depends on the balance of excitation and inhibition, we continue to lack specific knowledge of the rules that govern how excitatory and inhibitory inputs interact in space and time within the confines of individual neurons. In a recent paper, Liu starts by providing evidence that the relative locations and numbers of excitatory and inhibitory synapses are tightly regulated in cultured neurons from the hippocampus. This is consistent with findings in other labs that suggest neurons work hard, and in a variety of different ways, to maintain their inputs in proper balance and their outputs within appropriate ranges. On this backdrop, Liu's most important finding of a functional nature is that inhibition appears to act quite locally; that is, an inhibitory synapse effectively opposes an excitatory synapse only when it is very close by within the same dendritic branch (Fig. 1). This finding provides further support for the view--anticipated by neural theorists more than 20 years ago--that the brain's principal neurons contain a potentially large number of separate computational subunits.