Macaque V1 representations in natural and reduced visual contexts: spatial and temporal properties and influence of saccadic eye movements

Seeing on the fly: Physiological and behavioral evidence show that space-to-space representation and processing enable fast and efficient performance by the visual system

When we view the world, our eyes saccade quickly between points of interest. Even when fixating a target our eyes are not completely at rest but execute small fixational eye movements (FEMs). That vision is not blurred despite this ever-present jitter has seemingly motivated an increasingly popular theory denying the reliance of the visual system on pure spatial processing in favor of a space-to-time mechanism generated by the eye drifting across the image. Accordingly, FEMs are not detrimental but rather essential to good visibility. However, the space-to-time theory is incompatible with physiological data showing that all information is conveyed by the short neural volleys generated when the eyes land on a target, and with our faithful perception of briefly displayed objects, during which time FEMs have no effect. Another difficulty in rejecting the idea of image representation by the locations and nature of responding cells in favor of a timecode, is that somewhere, somehow, this code must be decoded into a parallel spatial one when reaching perception. Thus, in addition to the implausibility of generating meaningful responses during retinal drift, the space-to-time hypothesis calls for replacing efficient point-to-point parallel transmission with a cumbersome, delayed, space-to-time-to-space process. A novel physiological framework is presented here wherein the ability of the visual system to quickly process information is mediated by the short, powerful neural volleys generated by the landing saccades. These volleys are necessary and sufficient for normal perception without FEMs contribution. This mechanism enables our excellent perception of brief stimuli and explains that visibility is not blurred by FEMs because they do not generate useful information.

Correlations between Visual Temporal Resolution and Individual Alpha Peak Frequency: Evidence that Internal and Measurement Noise Drive Null Findings

The brain organizes the continuous flow of sensory input by parsing it into discrete events. In the case of two flashes separated by a brief ISI, for example, perception may be of a single flash or two distinct flashes, depending on the ISI but also on the speed of processing. A number of studies have reported evidence that participants with a higher EEG peak alpha frequency are able to detect the presence of two flashes separated by short intervals, whereas those with slower alpha report only one flash. Other studies have not found this correlation. We investigated potential factors that might mask the relationship between individual alpha frequency and visual perception. We recorded resting-state EEG from a large sample of participants (n = 50) and measured the temporal resolution of visual perception with the two-flash fusion task. We found that individual alpha frequency over posterior channels predicted the two-flash fusion threshold, in line with previous studies, but this correlation was significant only when taking into account the steepness of the psychophysical curve of the two-flash task. Participants with a relatively shallow psychophysical curve, likely reflecting high sensory and/or decision noise, failed to show this relationship. These findings replicate previous reports of a correlation between alpha frequency and visual temporal resolution, while also suggesting that an explanation of two-flash fusion performance that neglects the role of internal noise might be insufficient to account for all individual differences.

Perceptual enhancement and suppression correlate with V1 neural activity during active sensing

Perception in multiple sensory modalities is an active process that involves exploratory behaviors. In humans and other primates, vision results from sensory sampling guided by saccadic eye movements. Saccades are known to modulate visual perception, and a corollary discharge signal associated with saccades appears to establish a sense of visual stability. Neural recordings have shown that saccades also modulate activity widely across the brain. To investigate the neural basis of saccadic effects on perception, simultaneous recordings from multiple neurons in area V1 were made as animals performed a contrast detection task. Perceptual and neural measures were compared when the animal made real saccades that brought a stimulus into V1 receptive fields and when simulated saccades were made (identical retinal stimulation but no eye movement). When real saccades were made and low spatial frequency stimuli were presented, we observed a reduction in both perceptual sensitivity and neural activity compared with simulated saccades; conversely, with higher spatial frequency stimuli, saccades increased visual sensitivity and neural activity. The performance of neural decoders, which used the activity of the population of simultaneously recorded neurons, showed saccade effects on sensitivity that mirrored the frequency-dependent perceptual changes, suggesting that the V1 population activity could support the perceptual effects. A minority of V1 neurons had significant choice probabilities, and the saccades decreased both average choice probability and pairwise noise correlations. Taken together, the findings suggest that a signal related to saccadic eye movements alters V1 spiking to increase the independence of spiking neurons and bias the system toward processing higher spatial frequencies, presumably to enhance object recognition. The effects of saccades on visual perception and noise correlations appear to parallel effects observed in other sensory modalities, suggesting a general principle of active sensory processing.

Latency shortening with enhanced sparseness and responsiveness in V1 during active visual sensing

In natural vision, neuronal responses to visual stimuli occur due to self-initiated eye movements. Here, we compare single-unit activity in the primary visual cortex (V1) of non-human primates to flashed natural scenes (passive vision condition) to when they freely explore the images by self-initiated eye movements (active vision condition). Active vision enhances the number of neurons responding, and the response latencies become shorter and less variable across neurons. The increased responsiveness and shortened latency during active vision were not explained by increased visual contrast. While the neuronal activities in all layers of V1 show enhanced responsiveness and shortened latency, a significant increase in lifetime sparseness during active vision is observed only in the supragranular layer. These findings demonstrate that the neuronal responses become more distinct in active vision than passive vision, interpreted as consequences of top-down predictive mechanisms.

Fast and nonuniform dynamics of perisaccadic vision in the central fovea

Humans shift their gaze more frequently than their heart beats. These rapid eye movements (saccades) enable high visual acuity by redirecting the tiny high-resolution region of the retina (the foveola). But in doing so, they abruptly sweep the image across receptors, raising questions on how the visual system achieves stable percepts. It is well established that visual sensitivity is transiently attenuated during saccades. However, little is known about the time course of foveal vision despite its disproportionate importance, as technical challenges have so far prevented study of how saccades affect the foveola. Here we show that saccades modulate this region in a nonuniform manner, providing stronger and faster changes at its very center, a locus with higher sensitivity.

Humans use rapid eye movements (saccades) to inspect stimuli with the foveola, the region of the retina where receptors are most densely packed. It is well established that visual sensitivity is generally attenuated during these movements, a phenomenon known as saccadic suppression. This effect is commonly studied with large, often peripheral, stimuli presented during instructed saccades. However, little is known about how saccades modulate the foveola and how the resulting dynamics unfold during natural visual exploration. Here we measured the foveal dynamics of saccadic suppression in a naturalistic high-acuity task, a task designed after primates’ social grooming, which—like most explorations of fine patterns—primarily elicits minute saccades (microsaccades). Leveraging on recent advances in gaze-contingent display control, we were able to systematically map the perisaccadic time course of sensitivity across the foveola. We show that contrast sensitivity is not uniform across this region and that both the extent and dynamics of saccadic suppression vary within the foveola. Suppression is stronger and faster in the most central portion, where sensitivity is generally higher and selectively rebounds at the onset of a new fixation. These results shed light on the modulations experienced by foveal vision during the saccade-fixation cycle and explain some of the benefits of microsaccades.

The application of noninvasive, restraint-free eye-tracking methods for use with nonhuman primates

Over the past 50 years there has been a strong interest in applying eye-tracking techniques to study a myriad of questions related to human and nonhuman primate psychological processes. Eye movements and fixations can provide qualitative and quantitative insights into cognitive processes of nonverbal populations such as nonhuman primates, clarifying the evolutionary, physiological, and representational underpinnings of human cognition. While early attempts at nonhuman primate eye tracking were relatively crude, later, more sophisticated and sensitive techniques required invasive protocols and the use of restraint. In the past decade, technology has advanced to a point where noninvasive eye-tracking techniques, developed for use with human participants, can be applied for use with nonhuman primates in a restraint-free manner. Here we review the corpus of recent studies (N=32) that take such an approach. Despite the growing interest in eye-tracking research, there is still little consensus on "best practices," both in terms of deploying test protocols or reporting methods and results. Therefore, we look to advances made in the field of developmental psychology, as well as our own collective experiences using eye trackers with nonhuman primates, to highlight key elements that researchers should consider when designing noninvasive restraint-free eye-tracking research protocols for use with nonhuman primates. Beyond promoting best practices for research protocols, we also outline an ideal approach for reporting such research and highlight future directions for the field.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Decline of Orientation and Direction Sensitivity in the Aging Population

While the aging population is growing, our knowledge regarding age-related deterioration of visual perception remains limited. In the present study, we investigated the effects of aging on orientation and direction sensitivity in a healthy population using a weighted up–down adaptive method to improve the efficiency and reliability of the task. A total of 57 healthy participants aged 22–72 years were included and divided into old and young groups. Raw experimental data were processed using a psychometric method to determine the differences between the two groups. In the orientation task, the threshold of the discrimination angle and bias (i.e., the difference between the perceived midpoint from the logistic function and the reference point) was increased, while the lapsing rate (i.e., 1—the maximum logistic function) did not significantly change in the old group compared with the young group. In the motion direction task, the threshold, bias, and lapsing rate were significantly increased in the old group compared with the young group. These results suggest that the decreased ability of old participants in discrimination of stimulus orientation and motion direction could be related to the impaired function of visual cortex.

Spatiotemporal Content of Saccade Transients

Humans use rapid gaze shifts, known as saccades, to explore visual scenes. These movements yield abrupt luminance changes on the retina, which elicit robust neural discharges at fixation onsets. Yet little is known about the spatial content of saccade transients. Here, we show that saccades redistribute spatial information within the temporal range of retinal sensitivity following two distinct regimes: saccade modulations counterbalance (whiten) the spectral density of natural scenes at low spatial frequencies and follow the external power distribution at higher frequencies. This redistribution is a consequence of saccade dynamics, particularly the speed/amplitude/duration relation known as the main sequence. It resembles the redistribution resulting from inter-saccadic eye drifts, revealing a continuum in the modulations given by different eye movements, with oculomotor transitions primarily acting by regulating the bandwidth of whitening. Our findings suggest important computational roles for saccade transients in the establishment of spatial representations and lead to testable predictions about their consequences for visual functions and encoding mechanisms.

Saccade-based termination responses in macaque V1 and visual perception

Neurons in visual areas of the brain are generally characterized by the increase in firing rate that occurs when a stimulus is flashed on in the receptive field (RF). However, neurons also increase their firing rate when a stimulus is turned off. These "termination responses" or "after-discharges" that occur with flashed stimuli have been observed in area V1 and they may be important for vision as stimulus terminations have been shown to influence visual perception. The goal of the present study was to determine the strength of termination responses in the more natural situation in which eye movements move a stimulus out of an RF. We find that termination responses do occur in macaque V1 when termination results from a saccadic eye movement, but they are smaller in amplitude compared to flashed-off stimuli. Furthermore, there are termination responses even in the absence of visual stimulation. These findings demonstrate that termination responses are a component of naturalistic vision. They appear to be based on both visual and nonvisual signals in visual cortex. We speculate that the weakening of termination responses might be a neural correlate of saccadic suppression, the loss of perceptual sensitivity around the time of saccades.

Sensitivity to image recurrence across eye-movement-like image transitions through local serial inhibition in the retina

Standard models of stimulus encoding in the retina postulate that image presentations activate neurons according to the increase of preferred contrast inside the receptive field. During natural vision, however, images do not arrive in isolation, but follow each other rapidly, separated by sudden gaze shifts. We here report that, contrary to standard models, specific ganglion cells in mouse retina are suppressed after a rapid image transition by changes in visual patterns across the transition, but respond with a distinct spike burst when the same pattern reappears. This sensitivity to image recurrence depends on opposing effects of glycinergic and GABAergic inhibition and can be explained by a circuit of local serial inhibition. Rapid image transitions thus trigger a mode of operation that differs from the processing of simpler stimuli and allows the retina to tag particular image parts or to detect transition types that lead to recurring stimulus patterns.

DOI:http://dx.doi.org/10.7554/eLife.22431.001

Contrast sensitivity, V1 neural activity, and natural vision

Contrast sensitivity is fundamental to natural visual processing and an important tool for characterizing both visual function and clinical disorders. We simultaneously measured contrast sensitivity and neural contrast response functions and compared measurements in common laboratory conditions with naturalistic conditions. In typical experiments, a subject holds fixation and a stimulus is flashed on, whereas in natural vision, saccades bring stimuli into view. Motivated by our previous V1 findings, we tested the hypothesis that perceptual contrast sensitivity is lower in natural vision and that this effect is associated with corresponding changes in V1 activity. We found that contrast sensitivity and V1 activity are correlated and that the relationship is similar in laboratory and naturalistic paradigms. However, in the more natural situation, contrast sensitivity is reduced up to 25% compared with that in a standard fixation paradigm, particularly at lower spatial frequencies, and this effect correlates with significant reductions in V1 responses. Our data suggest that these reductions in natural vision result from fast adaptation on one fixation that lowers the response on a subsequent fixation. This is the first demonstration of rapid, natural-image adaptation that carries across saccades, a process that appears to constantly influence visual sensitivity in natural vision.

NEW & NOTEWORTHY

Visual sensitivity and activity in brain area V1 were studied in a paradigm that included saccadic eye movements and natural visual input. V1 responses and contrast sensitivity were significantly reduced compared with results in common laboratory paradigms. The parallel neural and perceptual effects of eye movements and stimulus complexity appear to be due to a form of rapid adaptation that carries across saccades.

Saccadic modulation of stimulus processing in primary visual cortex

Saccadic eye movements play a central role in primate vision. Yet, relatively little is known about their effects on the neural processing of visual inputs. Here we examine this question in primary visual cortex (V1) using receptive-field-based models, combined with an experimental design that leaves the retinal stimulus unaffected by saccades. This approach allows us to analyse V1 stimulus processing during saccades with unprecedented detail, revealing robust perisaccadic modulation. In particular, saccades produce biphasic firing rate changes that are composed of divisive gain suppression followed by an additive rate increase. Microsaccades produce similar, though smaller, modulations. We furthermore demonstrate that this modulation is likely inherited from the LGN, and is driven largely by extra-retinal signals. These results establish a foundation for integrating saccades into existing models of visual cortical stimulus processing, and highlight the importance of studying visual neuron function in the context of eye movements.

Primates acquire visual information through rapid saccadic eye movements, although little is known about their effects on neural processing of visual inputs. Here the authors demonstrate that saccades produce modulations of visual cortical processing that likely originate in the thalamus.

Sensitivity to gaze-contingent contrast increments in naturalistic movies: An exploratory report and model comparison

Sensitivity to luminance contrast is a prerequisite for all but the simplest visual systems. To examine contrast increment detection performance in a way that approximates the natural environmental input of the human visual system, we presented contrast increments gaze-contingently within naturalistic video freely viewed by observers. A band-limited contrast increment was applied to a local region of the video relative to the observer's current gaze point, and the observer made a forced-choice response to the location of the target (≈25,000 trials across five observers). We present exploratory analyses showing that performance improved as a function of the magnitude of the increment and depended on the direction of eye movements relative to the target location, the timing of eye movements relative to target presentation, and the spatiotemporal image structure at the target location. Contrast discrimination performance can be modeled by assuming that the underlying contrast response is an accelerating nonlinearity (arising from a nonlinear transducer or gain control). We implemented one such model and examined the posterior over model parameters, estimated using Markov-chain Monte Carlo methods. The parameters were poorly constrained by our data; parameters constrained using strong priors taken from previous research showed poor cross-validated prediction performance. Atheoretical logistic regression models were better constrained and provided similar prediction performance to the nonlinear transducer model. Finally, we explored the properties of an extended logistic regression that incorporates both eye movement and image content features. Models of contrast transduction may be better constrained by incorporating data from both artificial and natural contrast perception settings.

Eye movements reset visual perception

Human vision uses saccadic eye movements to rapidly shift the sensitive foveal portion of our retina to objects of interest. For vision to function properly amidst these ballistic eye movements, a mechanism is needed to extract discrete percepts on each fixation from the continuous stream of neural activity that spans fixations. The speed of visual parsing is crucial because human behaviors ranging from reading to driving to sports rely on rapid visual analysis. We find that a brain signal associated with moving the eyes appears to play a role in resetting visual analysis on each fixation, a process that may aid in parsing the neural signal. We quantified the degree to which the perception of tilt is influenced by the tilt of a stimulus on a preceding fixation. Two key conditions were compared, one in which a saccade moved the eyes from one stimulus to the next and a second simulated saccade condition in which the stimuli moved in the same manner but the subjects did not move their eyes. We find that there is a brief period of time at the start of each fixation during which the tilt of the previous stimulus influences perception (in a direction opposite to the tilt aftereffect)--perception is not instantaneously reset when a fixation starts. Importantly, the results show that this perceptual bias is much greater, with nearly identical visual input, when saccades are simulated. This finding suggests that, in real-saccade conditions, some signal related to the eye movement may be involved in the reset phenomenon. While proprioceptive information from the extraocular muscles is conceivably a factor, the fast speed of the effect we observe suggests that a more likely mechanism is a corollary discharge signal associated with eye movement.