The visual input to the retina during natural head-free fixation

Consequences of eye movements for spatial selectivity

What determines spatial tuning in the visual system? Standard views rely on the assumption that spatial information is directly inherited from the relative position of photoreceptors and shaped by neuronal connectivity.1,2 However, human eyes are always in motion during fixation,3,4,5,6 so retinal neurons receive temporal modulations that depend on the interaction of the spatial structure of the stimulus with eye movements. It has long been hypothesized that these modulations might contribute to spatial encoding,7,8,9,10,11,12 a proposal supported by several recent observations.13,14,15,16 A fundamental, yet untested, consequence of this encoding strategy is that spatial tuning is not hard-wired in the visual system but critically depends on how the fixational motion of the eye shapes the temporal structure of the signals impinging onto the retina. Here we used high-resolution techniques for eye-tracking17 and gaze-contingent display control18 to quantitatively test this distinctive prediction. We examined how contrast sensitivity, a hallmark of spatial vision, is influenced by fixational motion, both during normal active fixation and when the spatiotemporal stimulus on the retina is altered to mimic changes in fixational control. We showed that visual sensitivity closely follows the strength of the luminance modulations delivered within a narrow temporal bandwidth, so changes in fixational motion have opposite visual effects at low and high spatial frequencies. By identifying a key role for oculomotor activity in spatial selectivity, these findings have important implications for the perceptual consequences of abnormal eye movements, the sources of perceptual variability, and the function of oculomotor control.

An eye for detail: Eye movements and attention at the foveal scale

Human vision relies on a tiny region of the retina, the 1-deg foveola, to achieve high spatial resolution. Foveal vision is of paramount importance in daily activities, yet its study is challenging, as eye movements incessantly displace stimuli across this region. Here I will review work that, building on recent advances in eye-tracking and gaze-contingent display, examines how attention and eye movements operate at the foveal level. This research highlights how exploration of fine spatial detail unfolds following visuomotor strategies reminiscent of those occurring at larger scales. It shows that, together with highly precise control of attention, this motor activity is linked to non-homogenous processing within the foveola and selectively modulates sensitivity both in space and time. Overall, the picture emerges of a highly dynamic foveal perception in which fine spatial vision, rather than simply being the result of placing a stimulus at the center of gaze, is the result of a finely tuned and orchestrated synergy of motor, cognitive, and attentional processes.

Cognitive influences on fixational eye movements

We perceive the world based on visual information acquired via oculomotor control,1 an activity intertwined with ongoing cognitive processes.2,3,4 Cognitive influences have been primarily studied in the context of macroscopic movements, like saccades and smooth pursuits. However, our eyes are never still, even during periods of fixation. One of the fixational eye movements, ocular drifts, shifts the stimulus over hundreds of receptors on the retina, a motion that has been argued to enhance the processing of spatial detail by translating spatial into temporal information.5 Despite their apparent randomness, ocular drifts are under neural control.6,7,8 However little is known about the control of drift beyond the brainstem circuitry of the vestibulo-ocular reflex.9,10 Here, we investigated the cognitive control of ocular drifts with a letter discrimination task. The experiment was designed to reveal open-loop effects, i.e., cognitive oculomotor control driven by specific prior knowledge of the task, independent of incoming sensory information. Open-loop influences were isolated by randomly presenting pure noise fields (no letters) while subjects engaged in discriminating specific letter pairs. Our results show open-loop control of drift direction in human observers.

Inferring visual space from ultra-fine extra-retinal knowledge of gaze position

It has long been debated how humans resolve fine details and perceive a stable visual world despite the incessant fixational motion of their eyes. Current theories assume these processes to rely solely on the visual input to the retina, without contributions from motor and/or proprioceptive sources. Here we show that contrary to this widespread assumption, the visual system has access to high-resolution extra-retinal knowledge of fixational eye motion and uses it to deduce spatial relations. Building on recent advances in gaze-contingent display control, we created a spatial discrimination task in which the stimulus configuration was entirely determined by oculomotor activity. Our results show that humans correctly infer geometrical relations in the absence of spatial information on the retina and accurately combine high-resolution extraretinal monitoring of gaze displacement with retinal signals. These findings reveal a sensory-motor strategy for encoding space, in which fine oculomotor knowledge is used to interpret the fixational input to the retina.

Can Microsaccades Be Used for Biometrics?

Human eyes are in constant motion. Even when we fix our gaze on a certain point, our eyes continue to move. When looking at a point, scientists have distinguished three different fixational eye movements (FEM)-microsaccades, drift and tremor. The main goal of this paper is to investigate one of these FEMs-microsaccades-as a source of information for biometric analysis. The paper argues why microsaccades are preferred for biometric analysis over the other two fixational eye movements. The process of microsaccades' extraction is described. Thirteen parameters are defined for microsaccade analysis, and their derivation is given. A gradient algorithm was used to solve the biometric problem. An assessment of the weights of the different pairs of parameters in solving the biometric task was made.

Tracker/Camera Calibration for Accurate Automatic Gaze Annotation of Images and Videos

Modern appearance-based gaze tracking algorithms require vast amounts of training data, with images of a viewer annotated with "ground truth" gaze direction. The standard approach to obtain gaze annotations is to ask subjects to fixate at specific known locations, then use a head model to determine the location of "origin of gaze". We propose using an IR gaze tracker to generate gaze annotations in natural settings that do not require the fixation of target points. This requires prior geometric calibration of the IR gaze tracker with the camera, such that the data produced by the IR tracker can be expressed in the camera's reference frame. This contribution introduces a simple tracker/camera calibration procedure based on the PnP algorithm and demonstrates its use to obtain a full characterization of gaze direction that can be used for ground truth annotation.

Microsaccades, Drifts, Hopf Bundle and Neurogeometry

The first part of the paper contains a short review of the image processing in early vision is static, when the eyes and the stimulus are stable, and in dynamics, when the eyes participate in fixation eye movements. In the second part, we give an interpretation of Donders’ and Listing’s law in terms of the Hopf fibration of the 3-sphere over the 2-sphere. In particular, it is shown that the configuration space of the eye ball (when the head is fixed) is the 2-dimensional hemisphere SL+, called Listing hemisphere, and saccades are described as geodesic segments of SL+ with respect to the standard round metric. We study fixation eye movements (drift and microsaccades) in terms of this model and discuss the role of fixation eye movements in vision. A model of fixation eye movements is proposed that gives an explanation of presaccadic shift of receptive fields.

Oculo-retinal dynamics can explain the perception of minimal recognizable configurations

Natural vision is a dynamic and continuous process. Under natural conditions, visual object recognition typically involves continuous interactions between ocular motion and visual contrasts, resulting in dynamic retinal activations. In order to identify the dynamic variables that participate in this process and are relevant for image recognition, we used a set of images that are just above and below the human recognition threshold and whose recognition typically requires >2 s of viewing. We recorded eye movements of participants while attempting to recognize these images within trials lasting 3 s. We then assessed the activation dynamics of retinal ganglion cells resulting from ocular dynamics using a computational model. We found that while the saccadic rate was similar between recognized and unrecognized trials, the fixational ocular speed was significantly larger for unrecognized trials. Interestingly, however, retinal activation level was significantly lower during these unrecognized trials. We used retinal activation patterns and oculomotor parameters of each fixation to train a binary classifier, classifying recognized from unrecognized trials. Only retinal activation patterns could predict recognition, reaching 80% correct classifications on the fourth fixation (on average, ∼2.5 s from trial onset). We thus conclude that the information that is relevant for visual perception is embedded in the dynamic interactions between the oculomotor sequence and the image. Hence, our results suggest that ocular dynamics play an important role in recognition and that understanding the dynamics of retinal activation is crucial for understanding natural vision.

Head jitter enhances three-dimensional motion perception

Motion perception is a critical function of the visual system. In a three-dimensional environment, multiple sensory cues carry information about an object's motion trajectory. Previous work has quantified the contribution of binocular motion cues, such as interocular velocity differences and changing disparities over time, as well as monocular motion cues, such as size and density changes. However, even when these cues are presented in concert, observers will systematically misreport the direction of motion-in-depth. Although in the majority of laboratory experiments head position is held fixed using a chin or head rest, an observer's head position is subject to involuntary small movements under real-world viewing conditions. Here, we considered the potential impact of such “head jitter” on motion-in-depth perception. We presented visual stimuli in a head-mounted virtual reality device that facilitated low latency head tracking and asked observers to judge 3D object motion. We found performance improved when we updated the visual display consistent with the small changes in head position. When we disrupted or delayed head movement–contingent updating of the visual display, the proportion of motion-in-depth misreports again increased, reflected in both a reduction in sensitivity and an increase in bias. Our findings identify a critical function of head jitter in visual motion perception, which has been obscured in most (head-fixed and non-head jitter contingent) laboratory experiments.

A quantitative description of macaque ganglion cell responses to natural scenes: the interplay of time and space

KEY POINTS

Responses to natural scenes are the business of the retina. We find primate ganglion cell responses to such scenes consistent with those to simpler stimuli. A biophysical model confirmed this and predicted ganglion cell responses with close to retinal reliability. Primate ganglion cell responses to natural scenes were driven by temporal variations in colour and luminance over the receptive field centre caused by eye movements, and little influenced by interaction of centre and surround with structure in the scene. We discuss implications in the context of efficient coding of the visual environment. Much information in a higher spatiotemporal frequency band is concentrated in the magnocellular pathway.

ABSTRACT

Responses of visual neurons to natural scenes provide a link between classical descriptions of receptive field structure and visual perception of the natural environment. A natural scene video with a movement pattern resembling that of primate eye movements was used to evoke responses from macaque ganglion cells. Cell responses were well described through known properties of cell receptive fields. Different analyses converge to show that responses primarily derive from the temporal pattern of stimulation derived from eye movements, rather than spatial receptive field structure beyond centre size and position. This was confirmed using a model that predicted ganglion cell responses close to retinal reliability, with only a small contribution of the surround relative to the centre. We also found that the spatiotemporal spectrum of the stimulus is modified in ganglion cell responses, and this can reduce redundancy in the retinal signal. This is more pronounced in the magnocellular pathway, which is much better suited to transmit the detailed structure of natural scenes than the parvocellular pathway. Whitening is less important for chromatic channels. Taken together, this shows how a complex interplay across space, time and spectral content sculpts ganglion cell responses.

Accuracy and precision of small saccades

Despite recent advances on the mechanisms and purposes of fine oculomotor behavior, a rigorous assessment of the precision and accuracy of the smallest saccades is still lacking. Yet knowledge of how effectively these movements shift gaze is necessary for understanding their functions and is helpful in further elucidating their motor underpinnings. Using a combination of high-resolution eye-tracking and gaze-contingent control, here we examined the accuracy and precision of saccades aimed toward targets ranging from [Formula: see text] to [Formula: see text] eccentricity. We show that even small saccades of just 14-[Formula: see text] are very effective in centering the stimulus on the retina. Furthermore, we show that for a target at any given eccentricity, the probability of eliciting a saccade depends on its efficacy in reducing the foveal offset. The pattern of results reported here is consistent with current knowledge on the motor mechanisms of microsaccade production.

Spatiotemporal Content of Saccade Transients

Humans use rapid gaze shifts, known as saccades, to explore visual scenes. These movements yield abrupt luminance changes on the retina, which elicit robust neural discharges at fixation onsets. Yet little is known about the spatial content of saccade transients. Here, we show that saccades redistribute spatial information within the temporal range of retinal sensitivity following two distinct regimes: saccade modulations counterbalance (whiten) the spectral density of natural scenes at low spatial frequencies and follow the external power distribution at higher frequencies. This redistribution is a consequence of saccade dynamics, particularly the speed/amplitude/duration relation known as the main sequence. It resembles the redistribution resulting from inter-saccadic eye drifts, revealing a continuum in the modulations given by different eye movements, with oculomotor transitions primarily acting by regulating the bandwidth of whitening. Our findings suggest important computational roles for saccade transients in the establishment of spatial representations and lead to testable predictions about their consequences for visual functions and encoding mechanisms.

Optimized Signal Flow through Photoreceptors Supports the High-Acuity Vision of Primates

The fovea is a neural specialization that endows humans and other primates with the sharpest vision among mammals. This performance originates in the foveal cones, which are extremely narrow and long to form a high-resolution pixel array. Puzzlingly, this form is predicted to impede electrical conduction to an extent that appears incompatible with vision. We observe the opposite: signal flow through even the longest cones (0.4-mm axons) is essentially lossless. Unlike in most neurons, amplification and impulse generation by voltage-gated channels are dispensable. Rather, sparse channel activity preserves intracellular current, which flows as if unobstructed by organelles. Despite these optimizations, signaling would degrade if cones were lengthier. Because cellular packing requires that cone elongation accompanies foveal expansion, this degradation helps explain why the fovea is a constant, miniscule size despite multiplicative changes in eye size through evolution. These observations reveal how biophysical mechanisms tailor form-function relationships for primate behavioral performance.

High-acuity vision from retinal image motion

A mathematical model and a possible neural mechanism are proposed to account for how fixational drift motion in the retina confers a benefit for the discrimination of high-acuity targets. We show that by simultaneously estimating object shape and eye motion, neurons in visual cortex can compute a higher quality representation of an object by averaging out non-uniformities in the retinal sampling lattice. The model proposes that this is accomplished by two separate populations of cortical neurons - one providing a representation of object shape and another representing eye position or motion - which are coupled through specific multiplicative connections. Combined with recent experimental findings, our model suggests that the visual system may utilize principles not unlike those used in computational imaging for achieving "super-resolution" via camera motion.

Microsaccades in Applied Environments: Real-World Applications of Fixational Eye Movement Measurements

Across a wide variety of research environments, the recording of microsaccades and other fixational eye movements has provided insight and solutions into practical problems. Here we review the literature on fixational eye movements-especially microsaccades-in applied and ecologically-valid scenarios. Recent technical advances allow noninvasive fixational eye movement recordings in real-world contexts, while observers perform a variety of tasks. Thus, fixational eye movement measures have been obtained in a host of real-world scenarios, such as in connection with driver fatigue, vestibular sensory deprivation in astronauts, and elite athletic training, among others. Here we present the state of the art in the practical applications of fixational eye movement research, examine its potential future uses, and discuss the benefits of including microsaccade measures in existing eye movement detection technologies. Current evidence supports the inclusion of fixational eye movement measures in real-world contexts, as part of the development of new or improved oculomotor assessment tools. The real-world applications of fixational eye movement measurements will only grow larger and wider as affordable high-speed and high-spatial resolution eye trackers become increasingly prevalent.

Temporal Coding of Visual Space

Establishing a representation of space is a major goal of sensory systems. Spatial information, however, is not always explicit in the incoming sensory signals. In most modalities it needs to be actively extracted from cues embedded in the temporal flow of receptor activation. Vision, on the other hand, starts with a sophisticated optical imaging system that explicitly preserves spatial information on the retina. This may lead to the assumption that vision is predominantly a spatial process: all that is needed is to transmit the retinal image to the cortex, like uploading a digital photograph, to establish a spatial map of the world. However, this deceptively simple analogy is inconsistent with theoretical models and experiments that study visual processing in the context of normal motor behavior. We argue here that, as with other senses, vision relies heavily on temporal strategies and temporal neural codes to extract and represent spatial information.

Predictive Smooth Pursuit Eye Movements

Smooth pursuit eye movements maintain the line of sight on smoothly moving targets. Although often studied as a response to sensory motion, pursuit anticipates changes in motion trajectories, thus reducing harmful consequences due to sensorimotor processing delays. Evidence for predictive pursuit includes (a) anticipatory smooth eye movements (ASEM) in the direction of expected future target motion that can be evoked by perceptual cues or by memory for recent motion, (b) pursuit during periods of target occlusion, and (c) improved accuracy of pursuit with self-generated or biologically realistic target motions. Predictive pursuit has been linked to neural activity in the frontal cortex and in sensory motion areas. As behavioral and neural evidence for predictive pursuit grows and statistically based models augment or replace linear systems approaches, pursuit is being regarded less as a reaction to immediate sensory motion and more as a predictive response, with retinal motion serving as one of a number of contributing cues.

Predictions as a window into learning: Anticipatory fixation offsets carry more information about environmental statistics than reactive stimulus-responses

A core question underlying neurobiological and computational models of behavior is how individuals learn environmental statistics and use them to make predictions. Most investigations of this issue have relied on reactive paradigms, in which inferences about predictive processes are derived by modeling responses to stimuli that vary in likelihood. Here we deployed a novel anticipatory oculomotor metric to determine how input statistics impact anticipatory behavior that is decoupled from target-driven-response. We implemented transition constraints between target locations, so that the probability of a target being presented on the same side as the previous trial was 70% in one condition (pret70) and 30% in the other (pret30). Rather than focus on responses to targets, we studied subtle endogenous anticipatory fixation offsets (AFOs) measured while participants fixated the screen center, awaiting a target. These AFOs were small (<0.4° from center on average), but strongly tracked global-level statistics. Speaking to learning dynamics, trial-by-trial fluctuations in AFO were well-described by a learning model, which identified a lower learning rate in pret70 than pret30, corroborating prior suggestions that pret70 is subjectively treated as more regular. Most importantly, direct comparisons with saccade latencies revealed that AFOs: (a) reflected similar temporal integration windows, (b) carried more information about the statistical context than did saccade latencies, and (c) accounted for most of the information that saccade latencies also contained about inputs statistics. Our work demonstrates how strictly predictive processes reflect learning dynamics, and presents a new direction for studying learning and prediction.

Contrast sensitivity reveals an oculomotor strategy for temporally encoding space

The contrast sensitivity function (CSF), how sensitivity varies with the frequency of the stimulus, is a fundamental assessment of visual performance. The CSF is generally assumed to be determined by low-level sensory processes. However, the spatial sensitivities of neurons in the early visual pathways, as measured in experiments with immobilized eyes, diverge from psychophysical CSF measurements in primates. Under natural viewing conditions, as in typical psychophysical measurements, humans continually move their eyes even when looking at a fixed point. Here, we show that the resulting transformation of the spatial scene into temporal modulations on the retina constitutes a processing stage that reconciles human CSF and the response characteristics of retinal ganglion cells under a broad range of conditions. Our findings suggest a fundamental integration between perception and action: eye movements work synergistically with the spatio-temporal sensitivities of retinal neurons to encode spatial information.

A Riemannian Geometry Theory of Three-Dimensional Binocular Visual Perception

We present a Riemannian geometry theory to examine the systematically warped geometry of perceived visual space attributable to the size-distance relationship of retinal images associated with the optics of the human eye. Starting with the notion of a vector field of retinal image features over cortical hypercolumns endowed with a metric compatible with that size-distance relationship, we use Riemannian geometry to construct a place-encoded theory of spatial representation within the human visual system. The theory draws on the concepts of geodesic spray fields, covariant derivatives, geodesics, Christoffel symbols, curvature tensors, vector bundles and fibre bundles to produce a neurally-feasible geometric theory of visuospatial memory. The characteristics of perceived 3D visual space are examined by means of a series of simulations around the egocentre. Perceptions of size and shape are elucidated by the geometry as are the removal of occlusions and the generation of 3D images of objects. Predictions of the theory are compared with experimental observations in the literature. We hold that the variety of reported geometries is accounted for by cognitive perturbations of the invariant physically-determined geometry derived here. When combined with previous description of the Riemannian geometry of human movement this work promises to account for the non-linear dynamical invertible visual-proprioceptive maps and selection of task-compatible movement synergies required for the planning and execution of visuomotor tasks.

Perspective: Can eye movements contribute to emmetropization?

During development, the eye tunes its size to its optics so that distant objects are in focus, a state known as emmetropia. Although multiple factors contribute to this process, a strong influence appears to be exerted by the visual input signals entering the eye. Much research has been dedicated to the possible roles of specific features of the retinal image, such as the magnitude of blur. However, in humans and other species, the input to the retina is not an image, but a spatiotemporal flow of luminance. Small eye movements occur incessantly during natural fixation, continually transforming the spatial scene into temporal modulations on the retina. An emerging body of evidence suggests that this space-time reformatting is crucial to many aspects of visual processing, including sensitivity to fine spatial detail. The resulting temporal modulations depend not only on ocular dynamics, but also on the optics and shape of the eye, and the spatial statistics of the visual scene. Here we examine the characteristics of these signals and suggest that they may play a role in emmetropization. A direct consequence of this viewpoint is that abnormal oculomotor behavior may contribute to the development of myopia and hyperopia.

Monocular microsaccades: Do they really occur?

Small saccades, known as microsaccades, occur frequently during fixation. Several recent studies have argued that a considerable fraction of these movements are present in the traces from one eye only. This claim contrasts with the findings of older reports, which concluded that microsaccades, like larger saccades, are virtually always binocular events. Here we examined the characteristics of small saccades by means of two of the most established high-resolution eye-tracking techniques available. A binocular Dual Purkinje Image eye-tracker was used to record eye movements while observers fixated, with their head immobilized, on markers displayed on a monitor. A specially designed eye-coil system was used to measure eye movements during normal head-free viewing, while subjects fixated on markers at various distances. Monocular microsaccades were virtually absent in both datasets. In the head-fixed data, not a single monocular microsaccade was observed. In the head-free data, only one event appeared to be monocular out of more than a thousand saccades. Monocular microsaccades do not seem to occur during normal head-free or head-immobilized fixation.

Systematic misperceptions of 3-D motion explained by Bayesian inference

People make surprising but reliable perceptual errors. Here, we provide a unified explanation for systematic errors in the perception of three-dimensional (3-D) motion. To do so, we characterized the binocular retinal motion signals produced by objects moving through arbitrary locations in 3-D. Next, we developed a Bayesian model, treating 3-D motion perception as optimal inference given sensory noise in the measurement of retinal motion. The model predicts a set of systematic perceptual errors, which depend on stimulus distance, contrast, and eccentricity. We then used a virtual-reality headset as well as a standard 3-D desktop stereoscopic display to test these predictions in a series of perceptual experiments. As predicted, we found evidence that errors in 3-D motion perception depend on the contrast, viewing distance, and eccentricity of a stimulus. These errors include a lateral bias in perceived motion direction and a surprising tendency to misreport approaching motion as receding and vice versa. In sum, we present a Bayesian model that provides a parsimonious account for a range of systematic misperceptions of motion in naturalistic environments.

Neuronal control of fixation and fixational eye movements

Ocular fixation is a dynamic process that is actively controlled by many of the same brain structures involved in the control of eye movements, including the superior colliculus, cerebellum and reticular formation. In this article, we review several aspects of this active control. First, the decision to move the eyes not only depends on target-related signals from the peripheral visual field, but also on signals from the currently fixated target at the fovea, and involves mechanisms that are shared between saccades and smooth pursuit. Second, eye position during fixation is actively controlled and depends on bilateral activity in the superior colliculi and medio-posterior cerebellum; disruption of activity in these circuits causes systematic deviations in eye position during both fixation and smooth pursuit eye movements. Third, the eyes are not completely still during fixation but make continuous miniature movements, including ocular drift and microsaccades, which are controlled by the same neuronal mechanisms that generate larger saccades. Finally, fixational eye movements have large effects on visual perception. Ocular drift transforms the visual input in ways that increase spatial acuity; microsaccades not only improve vision by relocating the fovea but also cause momentary changes in vision analogous to those caused by larger saccades.

This article is part of the themed issue ‘Movement suppression: brain mechanisms for stopping and stillness’.

Use of cues in virtual reality depends on visual feedback

3D motion perception is of central importance to daily life. However, when tested in laboratory settings, sensitivity to 3D motion signals is found to be poor, leading to the view that heuristics and prior assumptions are critical for 3D motion perception. Here we explore an alternative: sensitivity to 3D motion signals is context-dependent and must be learned based on explicit visual feedback in novel environments. The need for action-contingent visual feedback is well-established in the developmental literature. For example, young kittens that are passively moved through an environment, but unable to move through it themselves, fail to develop accurate depth perception. We find that these principles also obtain in adult human perception. Observers that do not experience visual consequences of their actions fail to develop accurate 3D motion perception in a virtual reality environment, even after prolonged exposure. By contrast, observers that experience the consequences of their actions improve performance based on available sensory cues to 3D motion. Specifically, we find that observers learn to exploit the small motion parallax cues provided by head jitter. Our findings advance understanding of human 3D motion processing and form a foundation for future study of perception in virtual and natural 3D environments.

Selective attention within the foveola

Efficient control of attentional resources and high-acuity vision are both fundamental for survival. Shifts in visual attention are known to covertly enhance processing at locations away from the center of gaze, where visual resolution is low. It is unknown, however, whether selective spatial attention operates where the observer already looks, i.e., within the high-acuity foveola, the small, yet disproportionally important rod-free region of the retina. Using new methods for precisely controlling retinal stimulation, here we show that covert attention flexibly improves and speeds-up both detection and discrimination at loci only a fraction of a degree apart within the foveola. These findings reveal a surprisingly precise control of attention and its involvement in fine spatial vision. They show that the commonly studied covert shifts of attention away from the fovea are the expression of a global mechanism that exerts its action across the entire visual field.

Consequences of the Oculomotor Cycle for the Dynamics of Perception

Much evidence indicates that humans and other species process large-scale visual information before fine spatial detail. Neurophysiological data obtained with paralyzed eyes suggest that this coarse-to-fine sequence results from spatiotemporal filtering by neurons in the early visual pathway. However, the eyes are normally never stationary: rapid gaze shifts (saccades) incessantly alternate with slow fixational movements. To investigate the consequences of this oculomotor cycle on the dynamics of perception, we combined spectral analysis of visual input signals, neural modeling, and gaze-contingent control of retinal stimulation in humans. We show that the saccade/fixation cycle reformats the flow impinging on the retina in a way that initiates coarse-to-fine processing at each fixation. This finding reveals that the visual system uses oculomotor-induced temporal modulations to sequentially encode different spatial components and suggests that, rather than initiating coarse-to-fine processing, spatiotemporal coupling in the early visual pathway builds on the information dynamics of the oculomotor cycle.

Eye movements in the wild: Oculomotor control, gaze behavior & frames of reference

Understanding the brain's capacity to encode complex visual information from a scene and to transform it into a coherent perception of 3D space and into well-coordinated motor commands are among the outstanding questions in the study of integrative brain function. Eye movement methodologies have allowed us to begin addressing these questions in increasingly naturalistic tasks, where eye and body movements are ubiquitous and, therefore, the applicability of most traditional neuroscience methods restricted. This review explores foundational issues in (1) how oculomotor and motor control in lab experiments extrapolates into more complex settings and (2) how real-world gaze behavior in turn decomposes into more elementary eye movement patterns. We review the received typology of oculomotor patterns in laboratory tasks, and how they map onto naturalistic gaze behavior (or not). We discuss the multiple coordinate systems needed to represent visual gaze strategies, how the choice of reference frame affects the description of eye movements, and the related but conceptually distinct issue of coordinate transformations between internal representations within the brain.

Eye movements between saccades: Measuring ocular drift and tremor

Intersaccadic periods of fixation are characterized by incessant retinal motion due to small eye movements. While these movements are often disregarded as noise, the temporal modulations they introduce to retinal receptors are significant. However, analysis of these input modulations is challenging because the intersaccadic eye motion is close to the resolution limits of most eyetrackers, including widespread pupil-based video systems. Here, we analyzed in depth the limits of two high-precision eyetrackers, the Dual-Purkinje Image and the scleral search coil, and compared the intersaccadic eye movements of humans to those of a non-human primate. By means of a model eye we determined that the resolution of both techniques is sufficient to reliably measure intersaccadic ocular activity up to approximately 80Hz. Our results show that the characteristics of ocular drift are remarkably similar in the two species; a clear deviation from a scale-invariant spectrum occurs in the range between 50 and 100Hz, generally attributed to ocular tremor, leading to intersaccadic retinal speeds as high as 1.5deg/s. The amplitude of this deviation differs on the two axes of motion. In addition to our experimental observations, we suggest basic guidelines to evaluate the performance of eyetrackers and to optimize experimental conditions for the measurement of ocular drift and tremor.

Microsaccades enable efficient synchrony-based coding in the retina: a simulation study

It is now reasonably well established that microsaccades (MS) enhance visual perception, although the underlying neuronal mechanisms are unclear. Here, using numerical simulations, we show that MSs enable efficient synchrony-based coding among the primate retinal ganglion cells (RGC). First, using a jerking contrast edge as stimulus, we demonstrate a qualitative change in the RGC responses: synchronous firing, with a precision in the 10 ms range, only occurs at high speed and high contrast. MSs appear to be sufficiently fast to be able reach the synchronous regime. Conversely, the other kinds of fixational eye movements known as tremor and drift both hardly synchronize RGCs because of a too weak amplitude and a too slow speed respectively. Then, under natural image stimulation, we find that each MS causes certain RGCs to fire synchronously, namely those whose receptive fields contain contrast edges after the MS. The emitted synchronous spike volley thus rapidly transmits the most salient edges of the stimulus, which often constitute the most crucial information. We demonstrate that the readout could be done rapidly by simple coincidence-detector neurons without knowledge of the MS landing time, and that the required connectivity could emerge spontaneously with spike timing-dependent plasticity.

Are the visual transients from microsaccades helpful? Measuring the influences of small saccades on contrast sensitivity

Like all saccades, microsaccades cause both spatial and temporal changes in the input to the retina. In space, recent studies have shown that these small shifts precisely relocate a narrow (smaller than the foveola) high-acuity retinal locus on the stimulus. However, it has long been questioned whether the temporal modulations resulting from microsaccades are also beneficial for vision. To address this question, we combined spectral analysis of the visual input to the retina with measurements of contrast sensitivity in humans. Estimation of how different types of eye movements redistribute the power of an otherwise stationary stimulus shows that small saccades contribute more temporal power than ocular drift in the low-frequency range, suggesting a specific role for these movements in the encoding of low spatial frequencies. However, an influence on contrast sensitivity was only found for saccades with amplitudes larger than 30'. Contrast thresholds remained highly similar in the presence and absence of smaller saccades. Furthermore, saccades of all amplitudes, including microsaccades, were strongly suppressed during exposure to the stimulus. These findings do not support an important function of the visual transients caused by microsaccades.

Head-Eye Coordination at a Microscopic Scale

Humans explore static visual scenes by alternating rapid eye movements (saccades) with periods of slow and incessant eye drifts [1-3]. These drifts are commonly believed to be the consequence of physiological limits in maintaining steady gaze, resulting in Brownian-like trajectories [4-7], which are almost independent in the two eyes [8-10]. However, because of the technical difficulty of recording minute eye movements, most knowledge on ocular drift comes from artificial laboratory conditions, in which the head of the observer is strictly immobilized. Little is known about eye drift during natural head-free fixation, when microscopic head movements are also continually present [11-13]. We have recently observed that the power spectrum of the visual input to the retina during ocular drift is largely unaffected by fixational head movements [14]. Here we elucidate the mechanism responsible for this invariance. We show that, contrary to common assumption, ocular drift does not move the eyes randomly, but compensates for microscopic head movements, thereby yielding highly correlated movements in the two eyes. This compensatory behavior is extremely fast, persists with one eye patched, and results in image motion trajectories that are only partially correlated on the two retinas. These findings challenge established views of how humans acquire visual information. They show that ocular drift is precisely controlled, as long speculated [15], and imply the existence of neural mechanisms that integrate minute multimodal signals.

Control and Functions of Fixational Eye Movements

Humans and other species explore a visual scene by rapidly shifting their gaze 2-3 times every second. Although the eyes may appear immobile in the brief intervals in between saccades, microscopic (fixational) eye movements are always present, even when attending to a single point. These movements occur during the very periods in which visual information is acquired and processed and their functions have long been debated. Recent technical advances in controlling retinal stimulation during normal oculomotor activity have shed new light on the visual contributions of fixational eye movements and their degree of control. The emerging body of evidence, reviewed in this article, indicates that fixational eye movements are important components of the strategy by which the visual system processes fine spatial details, enabling both precise positioning of the stimulus on the retina and encoding of spatial information into the joint space-time domain.

Neuroscience: Tiny Eye Movements Link Vision and Attention

Eye movements are essential to human vision. A new study shows that the tiny eye movements we make while holding our gaze on a point of interest are associated with brief, attention-like changes in the sensitivity of visual neurons.

The unsteady eye: an information-processing stage, not a bug

How is space represented in the visual system? At first glance, the answer to this fundamental question appears straightforward: spatial information is directly encoded in the locations of neurons within maps. This concept has long dominated visual neuroscience, leading to mainstream theories of how neurons encode information. However, an accumulation of evidence indicates that this purely spatial view is incomplete and that, even for static images, the representation is fundamentally spatiotemporal. The evidence for this new understanding centers on recent experimental findings concerning the functional role of fixational eye movements, the tiny movements humans and other species continually perform, even when attending to a single point. We review some of these findings and discuss their functional implications.

A compact field guide to the study of microsaccades: Challenges and functions

Following a period of quiescence at the end of last century, the study of microsaccades has now regained strong impetus and broad attention within the vision research community. This wave of interest, partly fueled by the advent of user-friendly high-resolution eyetrackers, has attracted researchers and led to novel ideas. Old hypothesis have been revisited and new ones formulated. This article is designed to serve as a practical guide for researchers in the field. Because of the history of the field and the difficulty of measuring very small eye movements, the study of microsaccades presents peculiar challenges. Here, we summarize some of the main challenges and describe methods for assessing and improving the quality of the recordings. Furthermore, we examine how these experimental challenges have influenced analysis of the visual functions of microsaccades and critically review current evidence on three long-debated proposals: the maintenance of fixation, the prevention of visual fading, and the exploration of fine spatial detail.

Characteristics of fixational eye movements in amblyopia: Limitations on fixation stability and acuity?

Persons with amblyopia, especially those with strabismus, are known to exhibit abnormal fixational eye movements. In this paper, we compared six characteristics of fixational eye movements among normal control eyes (n=16), the non-amblyopic fellow eyes and the amblyopic eyes of anisometropic (n=14) and strabismic amblyopes (n=14). These characteristics include the frequency, magnitude of landing errors, amplitude and speed of microsaccades, and the amplitude and speed of slow drifts. Fixational eye movements were recorded using retinal imaging while observers monocularly fixated a 1° cross. Eye position data were recovered using a cross-correlation procedure. We found that in general, the characteristics of fixational eye movements are not significantly different between the fellow eyes of amblyopes and controls, and that the strabismic amblyopic eyes are always different from the other groups. Next, we determined the primary factors that limit fixation stability and visual acuity in amblyopic eyes by examining the relative importance of the different oculomotor characteristics, adding acuity (for fixation stability) or fixation stability (for acuity), and the type of amblyopia, as predictive factors in a multiple linear regression model. We show for the first time that the error magnitude of microsaccades, acuity, amplitude and frequency of microsaccades are primary factors limiting fixation stability; while the error magnitude, fixation stability, amplitude of drifts and amplitude of microsaccades are the primary factors limiting acuity. A mediation analysis showed that the effects of error magnitude and amplitude of microsaccades on acuity could be explained, at least in part, by their effects on fixation stability.