Fixational eye movements, natural image statistics, and fine spatial vision

Consequences of eye movements for spatial selectivity

What determines spatial tuning in the visual system? Standard views rely on the assumption that spatial information is directly inherited from the relative position of photoreceptors and shaped by neuronal connectivity.1,2 However, human eyes are always in motion during fixation,3,4,5,6 so retinal neurons receive temporal modulations that depend on the interaction of the spatial structure of the stimulus with eye movements. It has long been hypothesized that these modulations might contribute to spatial encoding,7,8,9,10,11,12 a proposal supported by several recent observations.13,14,15,16 A fundamental, yet untested, consequence of this encoding strategy is that spatial tuning is not hard-wired in the visual system but critically depends on how the fixational motion of the eye shapes the temporal structure of the signals impinging onto the retina. Here we used high-resolution techniques for eye-tracking17 and gaze-contingent display control18 to quantitatively test this distinctive prediction. We examined how contrast sensitivity, a hallmark of spatial vision, is influenced by fixational motion, both during normal active fixation and when the spatiotemporal stimulus on the retina is altered to mimic changes in fixational control. We showed that visual sensitivity closely follows the strength of the luminance modulations delivered within a narrow temporal bandwidth, so changes in fixational motion have opposite visual effects at low and high spatial frequencies. By identifying a key role for oculomotor activity in spatial selectivity, these findings have important implications for the perceptual consequences of abnormal eye movements, the sources of perceptual variability, and the function of oculomotor control.

Expectation violations produce error signals in mouse V1

Repeated exposure to visual sequences changes the form of evoked activity in the primary visual cortex (V1). Predictive coding theory provides a potential explanation for this, namely that plasticity shapes cortical circuits to encode spatiotemporal predictions and that subsequent responses are modulated by the degree to which actual inputs match these expectations. Here we use a recently developed statistical modeling technique called Model-Based Targeted Dimensionality Reduction (MbTDR) to study visually evoked dynamics in mouse V1 in the context of an experimental paradigm called "sequence learning." We report that evoked spiking activity changed significantly with training, in a manner generally consistent with the predictive coding framework. Neural responses to expected stimuli were suppressed in a late window (100-150 ms) after stimulus onset following training, whereas responses to novel stimuli were not. Substituting a novel stimulus for a familiar one led to increases in firing that persisted for at least 300 ms. Omitting predictable stimuli in trained animals also led to increased firing at the expected time of stimulus onset. Finally, we show that spiking data can be used to accurately decode time within the sequence. Our findings are consistent with the idea that plasticity in early visual circuits is involved in coding spatiotemporal information.

Perceptual filling-in dispels the veridicality problem of conscious perception research

Conscious perceptual experiences are expected to correlate with content-specific brain activity. A veridicality problem arises when attempting to disentangle unconscious and conscious brain processes if conscious perceptual contents accurately match the physical nature of the stimulus. We argue that perceptual filling-in, a phenomenon whereby visual information inaccurately spreads across visual space, is a promising approach to circumvent the veridicality problem. Filling-in generates non-veridical although unambiguous percepts dissociated from stimulus input. In particular, the radial uniformity illusion induces a filling-in experience between a central disk and the surrounding periphery. We discuss how this illusion facilitates both the detection of neurophysiological responses and subjective phenomenological monitoring. We report behavioral effects from a large-sample (n = 200) psychophysics study and examine key stimulus parameters that drive the conscious filling-in experience. We propose that these data underpin future hypothesis-driven studies of filling-in to further delineate the neural mechanisms of conscious perception.

Fixational eye movements following concussion

The purpose of this study was to evaluate fixational eye movements (FEMs) with high spatial and temporal resolution following concussion, where oculomotor symptoms and impairments are common. Concussion diagnosis was determined using current consensus guidelines. A retinal eye-tracking device, the tracking scanning laser ophthalmoscope (TSLO), was used to measure FEMs in adolescents and young adults following a concussion and in an unaffected control population. FEMs were quantified in two fixational paradigms: (1) when fixating on the center, or (2) when fixating on the corner of the TSLO imaging raster. Fixational saccade amplitude in recent concussion patients (≤ 21 days) was significantly greater, on average, in the concussion group (mean = 1.03°; SD = 0.36°) compared with the controls (mean = 0.82°; SD = 0.31°), when fixating on the center of the imaging raster (t = 2.87, df = 82, p = 0.005). These fixational saccades followed the main sequence and therefore also had greater peak velocity (t = 2.86, df = 82, p = 0.006) and peak acceleration (t = 2.80, df = 82, p = 0.006). These metrics significantly differentiated concussed from controls (AUC = 0.67-0.68, minimum p = 0.005). No group differences were seen for the drift metrics in either task or for any of the FEMs metrics in the corner-of-raster fixation task. Fixational saccade amplitudes were significantly different in the concussion group, but only when fixating on the center of the raster. This task specificity suggests that task optimization may improve differentiation and warrants further study. FEMs measured in the acute-to-subacute period of concussion recovery may provide a quick (<3 minutes), objective, sensitive, and accurate ocular dysfunction assessment. Future work should assess the impact of age, mechanism of injury, and post-concussion recovery on FEM alterations following concussion.

Fixational eye movements in passive versus active sustained fixation tasks

Human fixational eye movements are so small and precise that high-speed, accurate tools are needed to fully reveal their properties and functional roles. Where the fixated image lands on the retina and how it moves for different levels of visually demanding tasks is the subject of the current study. An Adaptive Optics Scanning Laser Ophthalmoscope (AOSLO) was used to image, track and present a variety of fixation targets (Maltese cross, disk, concentric circles, Vernier and tumbling-E letter) to healthy subjects. During these different passive (static) or active (discriminating) tasks under natural eye motion, the landing position of the target on the retina was tracked in space and time over the retinal image directly with high spatial (<1 arcmin) and temporal (960 Hz) resolution. We computed both the eye motion and the exact trajectory of the fixated target's motion over the retina. We confirmed that compared to passive tasks, active tasks elicited a partial inhibition of microsaccades, leading to longer drift periods compensated by larger corrective saccades. Consequently, the overall fixation stability during active tasks was on average 57% larger than during passive tasks. The preferred retinal locus of fixation was the same for each task and did not coincide with the location of the peak cone density.

A low-cost, high-performance video-based binocular eye tracker for psychophysical research

We describe a high-performance, pupil-based binocular eye tracker that approaches the performance of a well-established commercial system, but at a fraction of the cost. The eye tracker is built from standard hardware components, and its software (written in Visual C++) can be easily implemented. Because of its fast and simple linear calibration scheme, the eye tracker performs best in the central 10 degrees of the visual field. The eye tracker possesses a number of useful features: (1) automated calibration simultaneously in both eyes while subjects fixate four fixation points sequentially on a computer screen, (2) automated realtime continuous analysis of measurement noise, (3) automated blink detection, (4) and realtime analysis of pupil centration artifacts. This last feature is critical because it is known that pupil diameter changes can be erroneously registered by pupil-based trackers as a change in eye position. We evaluated the performance of our system against that of a wellestablished commercial system using simultaneous measurements in 10 participants. We propose our low-cost eye tracker as a promising resource for studies of binocular eye movements.

Optimized Signal Flow through Photoreceptors Supports the High-Acuity Vision of Primates

The fovea is a neural specialization that endows humans and other primates with the sharpest vision among mammals. This performance originates in the foveal cones, which are extremely narrow and long to form a high-resolution pixel array. Puzzlingly, this form is predicted to impede electrical conduction to an extent that appears incompatible with vision. We observe the opposite: signal flow through even the longest cones (0.4-mm axons) is essentially lossless. Unlike in most neurons, amplification and impulse generation by voltage-gated channels are dispensable. Rather, sparse channel activity preserves intracellular current, which flows as if unobstructed by organelles. Despite these optimizations, signaling would degrade if cones were lengthier. Because cellular packing requires that cone elongation accompanies foveal expansion, this degradation helps explain why the fovea is a constant, miniscule size despite multiplicative changes in eye size through evolution. These observations reveal how biophysical mechanisms tailor form-function relationships for primate behavioral performance.

Temporal Coding of Visual Space

Establishing a representation of space is a major goal of sensory systems. Spatial information, however, is not always explicit in the incoming sensory signals. In most modalities it needs to be actively extracted from cues embedded in the temporal flow of receptor activation. Vision, on the other hand, starts with a sophisticated optical imaging system that explicitly preserves spatial information on the retina. This may lead to the assumption that vision is predominantly a spatial process: all that is needed is to transmit the retinal image to the cortex, like uploading a digital photograph, to establish a spatial map of the world. However, this deceptively simple analogy is inconsistent with theoretical models and experiments that study visual processing in the context of normal motor behavior. We argue here that, as with other senses, vision relies heavily on temporal strategies and temporal neural codes to extract and represent spatial information.

Neural scaling laws for an uncertain world

Autonomous neural systems must efficiently process information in a wide range of novel environments which may have very different statistical properties. We consider the problem of how to optimally distribute receptors along a 1-dimensional continuum consistent with the following design principles. First, neural representations of the world should obey a neural uncertainty principle-making as few assumptions as possible about the statistical structure of the world. Second, neural representations should convey, as much as possible, equivalent information about environments with different statistics. The results of these arguments resemble the structure of the visual system and provide a natural explanation of the behavioral Weber-Fechner law, a foundational result in psychology. Because the derivation is extremely general, this suggests that similar scaling relationships should be observed not only in sensory continua, but also in neural representations of "cognitive" 1-dimensional quantities such as time or numerosity. (PsycINFO Database Record

Fixational saccades are more disconjugate in adults than in children

PURPOSE

Fixational eye movements are of particular interest for three reasons. They are critical for preventing visual fading and enhancing visual perception; their disconjugacy allows scanning in three dimensions, and their neural correlates span through the cortico-striatal, striato-collicular and brainstem networks. Fixational eye movements are altered in various pediatric ophthalmologic and neurologic disorders. The goal of this study was to compare the dynamics of fixational eye movements in normal children and adults.

METHODS

We measured the fixational saccades and inter-saccadic drifts in eye positions using infrared video-oculography in children and adults. We assessed the frequency, amplitude, main-sequence, and disconjugacy of fixational saccades as well as the intra-saccadic drift velocity and variance between these two groups.

RESULTS

We found a similar frequency but an increase in the amplitude of fixational saccades in children compared to adults. We also found that the fixational saccades were more conjugate in children than in adults. The inter-saccadic drifts were comparable between the two groups.

DISCUSSION

This study provides normative values of dynamics of fixational eye movement in children and adults. The greater disconjugacy of fixational saccades in adults suggests the existence of neural mechanisms that can independently regulate the movements of two eyes. The differences between adult and pediatric populations could be due to completion of the development of binocularly independent regulation of fixational saccades nearing adulthood. The alternate possibility is that the increased disconjugacy between the two eyes may represent a deficiency in the eye movement performance as a function of increasing age.

Modulation of microsaccades by spatial frequency during object categorization

The organization of visual processing into a coarse-to-fine information processing based on the spatial frequency properties of the input forms an important facet of the object recognition process. During visual object categorization tasks, microsaccades occur frequently. One potential functional role of these eye movements is to resolve high spatial frequency information. To assess this hypothesis, we examined the rate, amplitude and speed of microsaccades in an object categorization task in which participants viewed object and non-object images and classified them as showing either natural objects, man-made objects or non-objects. Images were presented unfiltered (broadband; BB) or filtered to contain only low (LSF) or high spatial frequency (HSF) information. This allowed us to examine whether microsaccades were modulated independently by the presence of a high-level feature - the presence of an object - and by low-level stimulus characteristics - spatial frequency. We found a bimodal distribution of saccades based on their amplitude, with a split between smaller and larger microsaccades at 0.4° of visual angle. The rate of larger saccades (⩾0.4°) was higher for objects than non-objects, and higher for objects with high spatial frequency content (HSF and BB objects) than for LSF objects. No effects were observed for smaller microsaccades (<0.4°). This is consistent with a role for larger microsaccades in resolving HSF information for object identification, and previous evidence that more microsaccades are directed towards informative image regions.

Multifocal Electroretinography in the Presence of Temporal and Spatial Correlations and Eye Movements

Releasing patients from the fixation task, and permitting them to view natural stimuli such as movies, would provide increased comfort, and potentially additional signs of retinal function, when recording multifocal electroretinograms (mfERGs). Techniques must be developed to handle the difficulties that arise from these alternative stimulation strategies. Multifocal stimuli were presented to volunteer human subjects with and without fixation. Retinocentric analyses were performed to deal with shifts of the stimulus across the retina in the presence of eye movements. Artificial scotomas that moved with the eyes to simulate local retinal defects were presented to assess whether such defects might be detectable in the presence of eye movements. Temporal and spatial correlations in the stimulus can be discounted, permitting retinal kernels to be measured in response to natural stimuli. Responses to temporally natural stimuli tend to have slightly stronger amplitudes because of the presence of low temporal frequencies in these stimuli. The effects of eye movement artifacts can be reduced, permitting similar kernels to be obtained in the absence and presence of eye movements. Convergence to stable kernels took slightly longer in the presence of temporal correlations or eye movements. Artificial scotomas can be localized with these methods. It may be possible to perform multifocal ERG recordings in the clinic using more flexible, natural techniques. However, work is needed to achieve results comparable to those routinely obtained with conventional methods.

Are the visual transients from microsaccades helpful? Measuring the influences of small saccades on contrast sensitivity

Like all saccades, microsaccades cause both spatial and temporal changes in the input to the retina. In space, recent studies have shown that these small shifts precisely relocate a narrow (smaller than the foveola) high-acuity retinal locus on the stimulus. However, it has long been questioned whether the temporal modulations resulting from microsaccades are also beneficial for vision. To address this question, we combined spectral analysis of the visual input to the retina with measurements of contrast sensitivity in humans. Estimation of how different types of eye movements redistribute the power of an otherwise stationary stimulus shows that small saccades contribute more temporal power than ocular drift in the low-frequency range, suggesting a specific role for these movements in the encoding of low spatial frequencies. However, an influence on contrast sensitivity was only found for saccades with amplitudes larger than 30'. Contrast thresholds remained highly similar in the presence and absence of smaller saccades. Furthermore, saccades of all amplitudes, including microsaccades, were strongly suppressed during exposure to the stimulus. These findings do not support an important function of the visual transients caused by microsaccades.

Head-Eye Coordination at a Microscopic Scale

Humans explore static visual scenes by alternating rapid eye movements (saccades) with periods of slow and incessant eye drifts [1-3]. These drifts are commonly believed to be the consequence of physiological limits in maintaining steady gaze, resulting in Brownian-like trajectories [4-7], which are almost independent in the two eyes [8-10]. However, because of the technical difficulty of recording minute eye movements, most knowledge on ocular drift comes from artificial laboratory conditions, in which the head of the observer is strictly immobilized. Little is known about eye drift during natural head-free fixation, when microscopic head movements are also continually present [11-13]. We have recently observed that the power spectrum of the visual input to the retina during ocular drift is largely unaffected by fixational head movements [14]. Here we elucidate the mechanism responsible for this invariance. We show that, contrary to common assumption, ocular drift does not move the eyes randomly, but compensates for microscopic head movements, thereby yielding highly correlated movements in the two eyes. This compensatory behavior is extremely fast, persists with one eye patched, and results in image motion trajectories that are only partially correlated on the two retinas. These findings challenge established views of how humans acquire visual information. They show that ocular drift is precisely controlled, as long speculated [15], and imply the existence of neural mechanisms that integrate minute multimodal signals.

1-D Vision: Encoding of Eye Movements by Simple Receptive Fields

Eye movements (eyeM) are an essential component of visual perception. They allow the sampling and scanning of stationary scenes at various spatial scales, primarily at the scene level, via saccades, and at the local level, via fixational eyeM. Given the constant motion of visual images on the retina, a crucial factor in resolving spatial ambiguities related to the external scene is the exact trajectory of eyeM. We show here that the trajectory of eyeM can be encoded at high resolution by simple retinal receptive fields of the symmetrical type. We also show that such encoding can account for motion illusions such as the Ouchi illusion. In addition, encoding of motion projections along horizontal and vertical symmetrical simple retinal receptive fields entails a kind of Cartesian decomposition of the 2-D image into two 1-D projections.

Control and Functions of Fixational Eye Movements

Humans and other species explore a visual scene by rapidly shifting their gaze 2-3 times every second. Although the eyes may appear immobile in the brief intervals in between saccades, microscopic (fixational) eye movements are always present, even when attending to a single point. These movements occur during the very periods in which visual information is acquired and processed and their functions have long been debated. Recent technical advances in controlling retinal stimulation during normal oculomotor activity have shed new light on the visual contributions of fixational eye movements and their degree of control. The emerging body of evidence, reviewed in this article, indicates that fixational eye movements are important components of the strategy by which the visual system processes fine spatial details, enabling both precise positioning of the stimulus on the retina and encoding of spatial information into the joint space-time domain.

Primary visual cortex represents the difference between past and present

The visual system is confronted with rapidly changing stimuli in everyday life. It is not well understood how information in such a stream of input is updated within the brain. We performed voltage-sensitive dye imaging across the primary visual cortex (V1) to capture responses to sequences of natural scene contours. We presented vertically and horizontally filtered natural images, and their superpositions, at 10 or 33 Hz. At low frequency, the encoding was found to represent not the currently presented images, but differences in orientation between consecutive images. This was in sharp contrast to more rapid sequences for which we found an ongoing representation of current input, consistent with earlier studies. Our finding that for slower image sequences, V1 does no longer report actual features but represents their relative difference in time counteracts the view that the first cortical processing stage must always transfer complete information. Instead, we show its capacities for change detection with a new emphasis on the role of automatic computation evolving in the 100-ms range, inevitably affecting information transmission further downstream.

The unsteady eye: an information-processing stage, not a bug

How is space represented in the visual system? At first glance, the answer to this fundamental question appears straightforward: spatial information is directly encoded in the locations of neurons within maps. This concept has long dominated visual neuroscience, leading to mainstream theories of how neurons encode information. However, an accumulation of evidence indicates that this purely spatial view is incomplete and that, even for static images, the representation is fundamentally spatiotemporal. The evidence for this new understanding centers on recent experimental findings concerning the functional role of fixational eye movements, the tiny movements humans and other species continually perform, even when attending to a single point. We review some of these findings and discuss their functional implications.

Decorrelation of retinal response to natural scenes by fixational eye movements

Under natural viewing conditions the input to the retina is a complex spatiotemporal signal that depends on both the scene and the way the observer moves. It is commonly assumed that the retina processes this input signal efficiently by taking into account the statistics of the natural world. It has recently been argued that incessant microscopic eye movements contribute to this process by decorrelating the input to the retina. Here we tested this theory by measuring the responses of the salamander retina to stimuli replicating the natural input signals experienced by the retina in the presence and absence of fixational eye movements. Contrary to the predictions of classic theories of efficient encoding that do not take behavior into account, we show that the response characteristics of retinal ganglion cells are not sufficient in themselves to disrupt the broad correlations of natural scenes. Specifically, retinal ganglion cells exhibited strong and extensive spatial correlations in the absence of fixational eye movements. However, the levels of correlation in the neural responses dropped in the presence of fixational eye movements, resulting in effective decorrelation of the channels streaming information to the brain. These observations confirm the predictions that microscopic eye movements act to reduce correlations in retinal responses and contribute to visual information processing.

The visual input to the retina during natural head-free fixation

Head and eye movements incessantly modulate the luminance signals impinging onto the retina during natural intersaccadic fixation. Yet, little is known about how these fixational movements influence the statistics of retinal stimulation. Here, we provide the first detailed characterization of the visual input to the human retina during normal head-free fixation. We used high-resolution recordings of head and eye movements in a natural viewing task to examine how they jointly transform spatial information into temporal modulations. In agreement with previous studies, we report that both the head and the eyes move considerably during fixation. However, we show that fixational head and eye movements mostly compensate for each other, yielding a spatiotemporal redistribution of the input power to the retina similar to that previously observed under head immobilization. The resulting retinal image motion counterbalances the spectral distribution of natural scenes, giving temporal modulations that are equalized in power over a broad range of spatial frequencies. These findings support the proposal that "ocular drift," the smooth fixational motion of the eye, is under motor control, and indicate that the spatiotemporal reformatting caused by fixational behavior is an important computational element in the encoding of visual information.

Eye movements and the neural basis of context effects on visual sensitivity

The effects of context on visual sensitivity are well established (e.g., sensitivity to luminance flicker is substantially higher on mean-gray surrounds than on white or black surrounds). The neural mechanisms generating context effects, however, remain unresolved. In the absence of direct tests, some theories invoke enhancement of edges by lateral inhibition, whereas others rely on transients caused by miniature eye movements that maintain fixation. We first replicated the luminance results on human observers and found unexpectedly that sensitivity to red-green flicker is also affected by surround color, being substantially higher on mean-gray surrounds than on red or green surrounds. To identify the neural bases of both context effects, we used in vivo electrophysiological recordings of primate magnocellular and parvocellular ganglion cell responses to luminance and red-green modulations, respectively. To test neuronal sensitivity to stationary edge contrast, neuronal responses were measured at various distances from the modulation edge against various surrounds. We found no evidence of enhanced responses to stationary edges on any surrounds, ruling out lateral inhibition-type explanations. To simulate the effects of eye movements, target patches were abruptly displaced while measuring responses. Abruptly displaced edges evoked vigorous transient responses that were selective for modulation-phase on mean-gray surrounds, but were phase-invariant on other surrounds. Eye movements could thus enhance detection of flicker on mean-gray surrounds, and neurometric analyses supported a primary role for eye movements in enhancing sensitivity. In addition, the transformation of spatial edges to transient neuronal responses by eye movements provides the signals for detecting luminance and color edges in natural scenes.

Active vision: adapting how to look

A new study has found that artificial occlusion of central vision leads to rapid emergence, and long-term maintenance of a new preferred retinal locus of fixation. These findings have important implications for the understanding of visual and oculomotor plasticity as well as for the development of rehabilitation techniques.

Microscopic eye movements compensate for nonhomogeneous vision within the fovea

Humans rely on the fovea, the small region of the retina where receptors are most densely packed, for seeing fine spatial detail. Outside the fovea, it is well established that a variety of visual functions progressively decline with eccentricity. In contrast, little is known about how vision varies within the central fovea, as incessant microscopic eye movements prevent isolation of adjacent foveal locations. Using a new method for restricting visual stimulation to a selected retinal region, we examined the discrimination of fine patterns at different eccentricities within the foveola. We show that high-acuity judgments are impaired when stimuli are presented just a few arcminutes away from the preferred retinal locus of fixation. Furthermore, we show that this dependence on eccentricity is normally counterbalanced by the occurrence of precisely directed microsaccades, which bring the preferred fixation locus onto the stimulus. Thus, contrary to common assumptions, vision is not uniform within the foveola, but targeted microscopic eye movements compensate for this lack of homogeneity. Our results reveal that microsaccades, like larger saccades, enable examination of the stimulus at a finer level of detail and suggest that a reduced precision in oculomotor control may be responsible for the visual acuity impairments observed in various disorders.

Seeing via Miniature Eye Movements: A Dynamic Hypothesis for Vision

During natural viewing, the eyes are never still. Even during fixation, miniature movements of the eyes move the retinal image across tens of foveal photoreceptors. Most theories of vision implicitly assume that the visual system ignores these movements and somehow overcomes the resulting smearing. However, evidence has accumulated to indicate that fixational eye movements cannot be ignored by the visual system if fine spatial details are to be resolved. We argue that the only way the visual system can achieve its high resolution given its fixational movements is by seeing via these movements. Seeing via eye movements also eliminates the instability of the image, which would be induced by them otherwise. Here we present a hypothesis for vision, in which coarse details are spatially encoded in gaze-related coordinates, and fine spatial details are temporally encoded in relative retinal coordinates. The temporal encoding presented here achieves its highest resolution by encoding along the elongated axes of simple-cell receptive fields and not across these axes as suggested by spatial models of vision. According to our hypothesis, fine details of shape are encoded by inter-receptor temporal phases, texture by instantaneous intra-burst rates of individual receptors, and motion by inter-burst temporal frequencies. We further describe the ability of the visual system to readout the encoded information and recode it internally. We show how reading out of retinal signals can be facilitated by neuronal phase-locked loops (NPLLs), which lock to the retinal jitter; this locking enables recoding of motion information and temporal framing of shape and texture processing. A possible implementation of this locking-and-recoding process by specific thalamocortical loops is suggested. Overall it is suggested that high-acuity vision is based primarily on temporal mechanisms of the sort presented here and low-acuity vision is based primarily on spatial mechanisms.

Motion parallax from microscopic head movements during visual fixation

Under normal viewing conditions, adjustments in body posture and involuntary head movements continually shift the eyes in space. Like all translations, these movements may yield depth information in the form of motion parallax, the differential motion on the retina of objects at different distances from the observer. However, studies on depth perception rarely consider the possible contribution of this cue, as the resulting changes in viewpoint appear too small to be of perceptual significance. Here, we quantified the parallax present during fixation in normally standing observers. We measured the trajectories followed by the eyes in space by means of a high-resolution head-tracking system and used an optical model of the eye to reconstruct the stimulus on the observer's retina. We show that, within several meters from the observer, relatively small changes in depth yield changes in the velocity of the retinal stimulus that are well above perceivable thresholds. Furthermore, relative velocities are little influenced by fixation distance, target eccentricity, and the precise oculomotor strategy followed by the observer to maintain fixation. These results demonstrate that the parallax available during normal head-free fixation is a reliable source of depth information, which the visual system may use in a variety of tasks.

Multiplexing in the primate motion pathway

This article begins by reviewing recent work on 3D motion processing in the primate visual system. Some of these results suggest that 3D motion signals may be processed in the same circuitry already known to compute 2D motion signals. Such "multiplexing" has implications for the study of visual cortical circuits and neural signals. A more explicit appreciation of multiplexing--and the computations required for demultiplexing--may enrich the study of the visual system by emphasizing the importance of a structured and balanced "encoding/decoding" framework. In addition to providing a fresh perspective on how successive stages of visual processing might be approached, multiplexing also raises caveats about the value of "neural correlates" for understanding neural computation.

Encoding visual information in retinal ganglion cells with prosthetic stimulation

Retinal prostheses aim to restore functional vision to those blinded by outer retinal diseases using electric stimulation of surviving retinal neurons. The ability to replicate the spatiotemporal pattern of ganglion cell spike trains present under normal viewing conditions is presumably an important factor for restoring high-quality vision. In order to replicate such activity with a retinal prosthesis, it is important to consider both how visual information is encoded in ganglion cell spike trains, and how retinal neurons respond to electric stimulation. The goal of the current review is to bring together these two concepts in order to guide the development of more effective stimulation strategies. We review the experiments to date that have studied how retinal neurons respond to electric stimulation and discuss these findings in the context of known retinal signaling strategies. The results from such in vitro studies reveal the advantages and disadvantages of activating the ganglion cell directly with the electric stimulus (direct activation) as compared to activation of neurons that are presynaptic to the ganglion cell (indirect activation). While direct activation allows high temporal but low spatial resolution, indirect activation yields improved spatial resolution but poor temporal resolution. Finally, we use knowledge gained from in vitro experiments to infer the patterns of elicited activity in ongoing human trials, providing insights into some of the factors limiting the quality of prosthetic vision.

Multiple components of ganglion cell desensitization in response to prosthetic stimulation

Retinal prostheses aim to restore functional vision to those blinded by outer retinal diseases using electric stimulation of surviving neurons. Previous work indicates that repetitive stimulation with stimuli that activate the synaptic network reduces the sensitivity of retinal neurons to further stimulation. Such desensitization may contribute to the fading of visual percepts over time reported by human subjects. Here, we show that desensitization may be more complex than previously considered. We recorded spike trains from rabbit retinal ganglion cells and found that desensitization persists in the presence of inhibitory blockers (strychnine and picrotoxin), indicating amacrine cell inhibition is not solely responsible for reducing sensitivity in response to electric stimulation. The threshold for direct activation of the ganglion cell changes little during the simultaneous desensitization of the synaptically mediated response, indicating that desensitization likely occurs upstream of the spike generator. In addition to rapid desensitization acting over hundreds of milliseconds (τ = 176.4 ± 8.8 ms), we report the presence of slow acting desensitization with a time course of seconds (τ = 14.0 ± 1.1 s). The time courses of the two components of desensitization that we found are similar to the two phases of brightness fading seen in human subjects. This suggests that the reduction in ganglion cell firing due to desensitization may be responsible for the fading of visual percepts over time in response to prosthetic stimulation.

Independent encoding of grating motion across stationary feature maps in primary visual cortex visualized with voltage-sensitive dye imaging

In early visual cortex different stimulus parameters are represented in overlaid feature maps. Such functioning was extensively explored by the use of drifting gratings characterized by orientation, spatial-temporal frequency, and direction of motion. However surprisingly, the direct cortical visuotopic drift of the gratings' stripy pattern has never been detected simultaneously to these stationary feature maps. It therefore remains to be demonstrated how physical signals of grating motion across the cortex are represented independently of other parametric maps and thus, how multi-dimensional input is processed independently to enable effective read-out further downstream. Taking advantage of the high spatial and temporal resolution of voltage-sensitive dye imaging, we here show the real-time encoding of position and orientation. By decomposing the cortical responses to drifting gratings we visualize the typical emergence of stationary orientation maps in which specific domains exhibited highest amplitudes. Simultaneously to these patchy maps, we demonstrate coherently propagating waves of activity that precisely matched the actual movement of the gratings in space and time, most dominantly for spatial frequencies lower than the preferred range. Thus, the primary visual cortex multiplexes information about retinotopic motion by additional temporal modulation of stationary orientation signals. These signals may be used to variably extract coarse-grained object motion and form information at higher visual processing stages.

Exploring the function of neural oscillations in early sensory systems

Neuronal oscillations appear throughout the nervous system, in structures as diverse as the cerebral cortex, hippocampus, subcortical nuclei and sense organs. Whether neural rhythms contribute to normal function, are merely epiphenomena, or even interfere with physiological processing are topics of vigorous debate. Sensory pathways are ideal for investigation of oscillatory activity because their inputs can be defined. Thus, we will focus on sensory systems as we ask how neural oscillations arise and how they might encode information about the stimulus. We will highlight recent work in the early visual pathway that shows how oscillations can multiplex different types of signals to increase the amount of information that spike trains encode and transmit. Last, we will describe oscillation-based models of visual processing and explore how they might guide further research.

Microsaccades: small steps on a long way

Contrary to common wisdom, fixations are a dynamically rich behavior, composed of continual, miniature eye movements, of which microsaccades are the most salient component. Over the last few years, interest in these small movements has risen dramatically, driven by both neurophysiological and psychophysical results and by advances in techniques, analysis, and modeling of eye movements. The field has a long history but a significant portion of the earlier work has gone missing in the current literature, in part, as a result of the collapse of the field in the 1980s that followed a series of discouraging results. The present review compiles 60 years of work demonstrating the unique contribution of microsaccades to visual and oculomotor function. Specifically, the review covers the contribution of microsaccades to (1) the control of fixation position, (2) the reduction of perceptual fading and the continuity of perception, (3) the generation of synchronized visual transients, (4) visual acuity, (5) scanning of small spatial regions, (6) shifts of spatial attention, (7) resolving perceptual ambiguities in the face of multistable perception, as well as several other functions. The accumulated evidence demonstrates that microsaccades serve both perceptual and oculomotor goals and although in some cases their contribution is neither necessary nor unique, microsaccades are a malleable tool conveniently employed by the visual system.