Visual perception and saccadic eye movements

Microsaccadic suppression of peripheral perceptual detection performance as a function of foveated visual image appearance

Microsaccades are known to be associated with a deficit in perceptual detection performance for brief probe flashes presented in their temporal vicinity. However, it is still not clear how such a deficit might depend on the visual environment across which microsaccades are generated. Here, and motivated by studies demonstrating an interaction between visual background image appearance and perceptual suppression strength associated with large saccades, we probed peripheral perceptual detection performance of human subjects while they generated microsaccades over three different visual backgrounds. Subjects fixated near the center of a low spatial frequency grating, a high spatial frequency grating, or a small white fixation spot over an otherwise gray background. When a computer process detected a microsaccade, it presented a brief peripheral probe flash at one of four locations (over a uniform gray background) and at different times. After collecting full psychometric curves, we found that both perceptual detection thresholds and slopes of psychometric curves were impaired for peripheral flashes in the immediate temporal vicinity of microsaccades, and they recovered with later flash times. Importantly, the threshold elevations, but not the psychometric slope reductions, were stronger for the white fixation spot than for either of the two gratings. Thus, like with larger saccades, microsaccadic suppression strength can show a certain degree of image dependence. However, unlike with larger saccades, stronger microsaccadic suppression did not occur with low spatial frequency textures. This observation might reflect the different spatiotemporal retinal transients associated with the small microsaccades in our study versus larger saccades.

Neural substrates for saccadic modulation of visual representations in mouse superior colliculus

How do sensory systems account for stimuli generated by natural behavior? We addressed this question by examining how an ethologically relevant class of saccades modulates visual representations in the mouse superior colliculus (SC), a key region for sensorimotor integration. We quantified saccadic modulation by recording SC responses to visual probes presented at stochastic saccade-probe latencies. Saccades significantly impacted population representations of the probes, with early enhancement that began prior to saccades and pronounced suppression for several hundred milliseconds following saccades, independent of units' visual response properties or directional tuning. To determine the cause of saccadic modulation, we presented fictive saccades that simulated the visual experience during saccades without motor output. Some units exhibited similar modulation by fictive and real saccades, suggesting a sensory-driven origin of saccadic modulation, while others had dissimilar modulation, indicating a motor contribution. These findings advance our understanding of the neural basis of natural visual coding.

The execution of saccadic eye movements suppresses visual processing of both color and luminance in the early visual cortex of humans

Our eyes execute rapid, directional movements known as saccades, occurring several times per second, to focus on objects of interest in our environment. During these movements, visual sensitivity is temporarily reduced. Despite numerous studies on this topic, the underlying mechanism remains elusive, including a lingering debate on whether saccadic suppression affects the parvocellular visual pathway. To address this issue, we conducted a study employing steady-state visual evoked potentials (SSVEPs) elicited by chromatic and luminance stimuli while observers performed saccadic eye movements. We also employed an innovative analysis pipeline to enhance the signal-to-noise ratio, yielding superior results compared to the previous method. Our findings revealed a clear suppression effect on SSVEP signals during saccades compared to fixation periods. Notably, this suppression effect was comparable for both chromatic and luminance stimuli. We went further to measure the suppression effect across various contrast levels, which enabled us to model SSVEP responses with contrast response functions. The results suggest that saccades primarily reduce response gain without significantly affecting contrast gain and that this reduction applies uniformly to both chromatic and luminance pathways. In summary, our study provides robust evidence that saccades similarly suppress visual processing in both the parvocellular and magnocellular pathways within the human early visual cortex, as indicated by SSVEP responses. The observation that saccadic eye movements impact response gain rather than contrast gain implies that they influence visual processing through a multiplicative mechanism.NEW & NOTEWORTHY The present study demonstrates that saccadic eye movements reduce the processing of both luminance and chromatic stimuli in the early visual cortex of humans. By modeling the contrast response function, the study further shows that saccades affect visual processing by reducing the response gain rather than altering the contrast gain, suggesting that a multiplicative mechanism of visual attenuation affects both parvocellular and magnocellular pathways.

Contralateral delay activity and alpha lateralization reflect retinotopic and screen-centered reference frames in visual memory

The visual system represents objects in a lateralized manner, with contralateral cortical hemispheres responsible for left and right visual hemifields. This organization extends to visual short-term memory (VSTM), as evidenced by electrophysiological indices of VSTM maintenance: contralateral delay activity (CDA) and alpha-band lateralization. However, it remains unclear if VSTM represents object locations in gaze-centered (retinotopic) or screen-centered (spatiotopic) coordinates, especially after eye movements. In two experiments, participants encoded the colors of target objects and made a lateral saccade during the maintenance interval, thereby shifting the object's location on the retina. A non-lateralized probe stimulus was then presented at the new fixation for a change detection task. The CDA maintained lateralization towards the target's original retinotopic location, unaffected by subsequent saccades, and did not invert polarity even when a saccade brought that location into the opposite hemifield. We also found conventional alpha lateralization towards the target's location before a saccade. After a saccade, however, alpha was lateralized towards the screen center regardless of the target's original location, even in a control condition without any memory requirements. This suggests that post-saccadic alpha-band lateralization reflects attentional processes unrelated to memory, while pre- and post-saccade CDA reflect VSTM maintenance in a retinotopic reference frame.

A dynamic sequence of visual processing initiated by gaze shifts

Animals move their head and eyes as they explore the visual scene. Neural correlates of these movements have been found in rodent primary visual cortex (V1), but their sources and computational roles are unclear. We addressed this by combining head and eye movement measurements with neural recordings in freely moving mice. V1 neurons responded primarily to gaze shifts, where head movements are accompanied by saccadic eye movements, rather than to head movements where compensatory eye movements stabilize gaze. A variety of activity patterns followed gaze shifts and together these formed a temporal sequence that was absent in darkness. Gaze-shift responses resembled those evoked by sequentially flashed stimuli, suggesting a large component corresponds to onset of new visual input. Notably, neurons responded in a sequence that matches their spatial frequency bias, consistent with coarse-to-fine processing. Recordings in freely gazing marmosets revealed a similar sequence following saccades, also aligned to spatial frequency preference. Our results demonstrate that active vision in both mice and marmosets consists of a dynamic temporal sequence of neural activity associated with visual sampling.

Rapid eye movements associated with REM sleep is involved in consolidation of visuospatial learning in rats

Rapid eye movement (REM) sleep plays a significant role in visuospatial learning and memory consolidation; however, its mechanism of action is unknown. Rapid eye movements (REMs), a characteristic active feature of REM sleep, is a potential correlate of neural processing for visual memory consolidation. The superior colliculus (SC) plays a central role in oculomotor control and spatial localization of objects in the visual field. We proposed that local reversible inactivation of the SC during post-learning sessions might interfere with REMs and negatively impact REM sleep associated consolidation of the visuospatial learnt task. Under gaseous anesthesia, bilateral cannulae aiming SC and electrodes for recording electrophysiological signals to classify sleep-waking were implanted. Following standard protocol, all rats were subjected to Morris water maze (MWM) training for 5 consecutive days followed by probe trial. After MWM training, on all except the probe test days, the rat SC were bilaterally infused with either vehicle (control, Group 1), Lidocaine hydrochloride a local anesthetic (Lox 2%, Group 2), or muscimol (Mus, GABA agonist, Group 3) and sleep-wakefulness recorded after day 1, 4, and post-probe learning sessions. Post-learning, compared to vehicle, Mus treated group significantly decreased REMs, phasic REM sleep, percent time spent in REM sleep and REM sleep frequency/hr. Also, during probe test, the escape latency was significantly increased, and the percentage time spent in the platform quadrant were significantly decreased in both, Mus and Lox 2% treated rats, while the number of platform location crossings was decreased in Mus treated group. The results showed that Lox 2% and Mus into SC reduced consolidation of visuospatial learning. The findings support our contention that SC mediated activation of REMs exerts a positive influence in processing and consolidation of visual learning during REM sleep. The findings explain the role of REMs during REM sleep in visual memory consolidation.

Evaluation of Mental Rotation Ability in Patients with Unilateral Benign Paroxysmal Positional Vertigo

BACKGROUND

Our study aims to determine whether there are differences in mental rotation abilities between unilateral benign paroxysmal positional vertigo patients and healthy controls using object-based mental rotation tasks.

METHODS

Our study included 17 unilateral posterior canal benign paroxysmal positional vertigo patients and 20 healthy adults. Spontaneous nystagmus test, saccade test, and dynamic positional tests with videonystagmography and object-based mental rotation test with 2-dimensional images of cubes rotated at certain angles in 3-dimensional space were performed on the participants. The mental rotation test response time and the number of correct answers were compared between patients and controls. We also evaluated whether there was a relationship between saccade test parameters and mental rotation test parameters in our study.

RESULTS

No significant relationship was found between benign paroxysmal positional vertigo patients and controls on any of the dependent measures (P -gt; .05). When we evaluated the relationship between saccadic latency and accuracy and mental rotation test response time and number of correct answers in benign paroxysmal positional vertigo patients, no significant relationship was found (P -gt; .05).

CONCLUSION

Our findings show that unilateral, posterior canal benign paroxysmal positional vertigo does not affect object-based mental rotation performance. In our study, no correlation was found between saccadic function and mental rotation ability in unilateral benign paroxysmal positional vertigo patients.

Visual, delay, and oculomotor timing and tuning in macaque dorsal pulvinar during instructed and free choice memory saccades

Causal perturbations suggest that primate dorsal pulvinar plays a crucial role in target selection and saccade planning, though its basic neuronal properties remain unclear. Some functional aspects of dorsal pulvinar and interconnected frontoparietal areas-e.g. ipsilesional choice bias after inactivation-are similar. But it is unknown if dorsal pulvinar shares oculomotor properties of cortical circuitry, in particular delay and choice-related activity. We investigated such properties in macaque dorsal pulvinar during instructed and free-choice memory saccades. Most recorded units showed visual (12%), saccade-related (30%), or both types of responses (22%). Visual responses were primarily contralateral; diverse saccade-related responses were predominantly post-saccadic with a weak contralateral bias. Memory delay and pre-saccadic enhancement was infrequent (11-9%)-instead, activity was often suppressed during saccade planning (25%) and further during execution (15%). Surprisingly, only few units exhibited classical visuomotor patterns combining cue and continuous delay activity or pre-saccadic ramping; moreover, most spatially-selective neurons did not encode the upcoming decision during free-choice delay. Thus, in absence of a visible goal, the dorsal pulvinar has a limited role in prospective saccade planning, with patterns partially complementing its frontoparietal partners. Conversely, prevalent visual and post-saccadic responses imply its participation in integrating spatial goals with processing across saccades.

Development of a Magnetoresistive-Based Wearable Eye-Tracking System for Oculomotor Assessment in Neurological and Otoneurological Research-Preliminary In Vivo Tests

Over the past 20 years, several eye-tracking technologies have been developed. This article aims to present a new type of eye tracker capable of producing detailed information on eye and head movements using an array of magnetoresistive detectors fixed on the patient's head and a small magnet inserted into a contact lens, adapted to the curvature of the cornea of the subject. The software used for data analysis can combine or compare eye and head movements and can represent them as 2D or 3D images. Preliminary data involve an initial patient who was asked to perform several tasks to establish the accuracy, reliability, and tolerance of the magnetic eye tracker and software. The tasks included assessment of saccadic eye movements and pursuit, "drawing" alphabetic shapes or letters, and reading. Finally, a Head Impulse Test (HIT) was performed to estimate the VOR gain, comparing the standard deviation established via vHIT with that established via this magnetic eye tracker (mHIT). This prototypical device is minimally invasive, lightweight, relatively cheap, and tolerable, with a high degree of reliability and precision. All these characteristics could lead to the future use of the magnetic eye tracker in neurological and otoneurological fields.

The perceptual consequences and neurophysiology of eye blinks

A hand passing in front of a camera produces a large and obvious disruption of a video. Yet the closure of the eyelid during a blink, which lasts for hundreds of milliseconds and occurs thousands of times per day, typically goes unnoticed. What are the neural mechanisms that mediate our uninterrupted visual experience despite frequent occlusion of the eyes? Here, we review the existing literature on the neurophysiology, perceptual consequences, and behavioral dynamics of blinks. We begin by detailing the kinematics of the eyelid that define a blink. We next discuss the ways in which blinks alter visual function by occluding the pupil, decreasing visual sensitivity, and moving the eyes. Then, to anchor our understanding, we review the similarities between blinks and other actions that lead to reductions in visual sensitivity, such as saccadic eye movements. The similarity between these two actions has led to suggestions that they share a common neural substrate. We consider the extent of overlap in their neural circuits and go on to explain how recent findings regarding saccade suppression cast doubt on the strong version of the shared mechanism hypothesis. We also evaluate alternative explanations of how blink-related processes modulate neural activity to maintain visual stability: a reverberating corticothalamic loop to maintain information in the face of lid closure; and a suppression of visual transients related to lid closure. Next, we survey the many areas throughout the brain that contribute to the execution of, regulation of, or response to blinks. Regardless of the underlying mechanisms, blinks drastically attenuate our visual abilities, yet these perturbations fail to reach awareness. We conclude by outlining opportunities for future work to better understand how the brain maintains visual perception in the face of eye blinks. Future work will likely benefit from incorporating theories of perceptual stability, neurophysiology, and novel behavior paradigms to address issues central to our understanding of natural visual behavior and for the clinical rehabilitation of active vision.

Semantic novelty modulates neural responses to visual change across the human brain

Our continuous visual experience in daily life is dominated by change. Previous research has focused on visual change due to stimulus motion, eye movements or unfolding events, but not their combined impact across the brain, or their interactions with semantic novelty. We investigate the neural responses to these sources of novelty during film viewing. We analyzed intracranial recordings in humans across 6328 electrodes from 23 individuals. Responses associated with saccades and film cuts were dominant across the entire brain. Film cuts at semantic event boundaries were particularly effective in the temporal and medial temporal lobe. Saccades to visual targets with high visual novelty were also associated with strong neural responses. Specific locations in higher-order association areas showed selectivity to either high or low-novelty saccades. We conclude that neural activity associated with film cuts and eye movements is widespread across the brain and is modulated by semantic novelty.

Bilateral increase in MEG planar gradients prior to saccade onset

Every time we move our eyes, the retinal locations of objects change. To distinguish the changes caused by eye movements from actual external motion of the objects, the visual system is thought to anticipate the consequences of eye movements (saccades). Single neuron recordings have indeed demonstrated changes in receptive fields before saccade onset. Although some EEG studies with human participants have also demonstrated a pre-saccadic increased potential over the hemisphere that will process a stimulus after a saccade, results have been mixed. Here, we used magnetoencephalography to investigate the timing and lateralization of visually evoked planar gradients before saccade onset. We modelled the gradients from trials with both a saccade and a stimulus as the linear combination of the gradients from two conditions with either only a saccade or only a stimulus. We reasoned that any residual gradients in the condition with both a saccade and a stimulus must be uniquely linked to visually-evoked neural activity before a saccade. We observed a widespread increase in residual planar gradients. Interestingly, this increase was bilateral, showing activity both contralateral and ipsilateral to the stimulus, i.e. over the hemisphere that would process the stimulus after saccade offset. This pattern of results is consistent with predictive pre-saccadic changes involving both the current and the future receptive fields involved in processing an attended object, well before the start of the eye movement. The active, sensorimotor coupling of vision and the oculomotor system may underlie the seamless subjective experience of stable and continuous perception.

Eye movement characteristics in a mental rotation task presented in virtual reality

Introduction

Eye-tracking technology provides a reliable and cost-effective approach to characterize mental representation according to specific patterns. Mental rotation tasks, referring to the mental representation and transformation of visual information, have been widely used to examine visuospatial ability. In these tasks, participants visually perceive three-dimensional (3D) objects and mentally rotate them until they identify whether the paired objects are identical or mirrored. In most studies, 3D objects are presented using two-dimensional (2D) images on a computer screen. Currently, visual neuroscience tends to investigate visual behavior responding to naturalistic stimuli rather than image stimuli. Virtual reality (VR) is an emerging technology used to provide naturalistic stimuli, allowing the investigation of behavioral features in an immersive environment similar to the real world. However, mental rotation tasks using 3D objects in immersive VR have been rarely reported.

Methods

Here, we designed a VR mental rotation task using 3D stimuli presented in a head-mounted display (HMD). An eye tracker incorporated into the HMD was used to examine eye movement characteristics during the task synchronically. The stimuli were virtual paired objects oriented at specific angular disparities (0, 60, 120, and 180°). We recruited thirty-three participants who were required to determine whether the paired 3D objects were identical or mirrored.

Results

Behavioral results demonstrated that the response times when comparing mirrored objects were longer than identical objects. Eye-movement results showed that the percent fixation time, the number of within-object fixations, and the number of saccades for the mirrored objects were significantly lower than that for the identical objects, providing further explanations for the behavioral results.

Discussion

In the present work, we examined behavioral and eye movement characteristics during a VR mental rotation task using 3D stimuli. Significant differences were observed in response times and eye movement metrics between identical and mirrored objects. The eye movement data provided further explanation for the behavioral results in the VR mental rotation task.

Targeting displacement as an indicator of binocular fixation in normal Chinese adults

Purpose

The stability of fixation is crucial for the development of visual function. In this study, we quantify the deviation of visual target during fixational and saccadic tasks using eye-tracking technology, reflecting the control ability and characteristics of fixational displacement among healthy adults in a convenient method.

Methods

One hundred healthy participants aged between 18 and 55 years were recruited in the study. All participants underwent a complete ophthalmic assessment. The eye positions in the fixational and saccadic tasks were documented and analyzed by the Tobii eye-tracking system. Participants were grouped by age and gender. Targeting displacement (TD), defined as the average displacement between visual target and the mean of fixation points corresponding to that stimuli, was used to quantitatively observe fixational displacement in the horizontal and vertical directions.

Result

There was a strong reproducibility of TD as an indicator of fixation (ICC 0.812 to 0.891, p < 0.001). The TD in fixational task was significantly smaller than that of the saccadic task (3.884 ± 0.525 vs. 4.484 ± 0.509, p < 0.001) among normal people. Moreover, the difference of TD in the horizontal and vertical meridians was related to the nature of the task: In the fixational task, the TD in horizontal was smaller than that in the vertical (p < 0.001), whereas the TD in horizontal was larger than that in vertical in the saccadic task (p = 0.003). In the different age and gender groups: There was no significant difference between different gender and age groups in fixational task. However, during the saccadic task, males had smaller TD in the vertical direction than females (4.061 ± 0.495 vs. 4.404 ± 0.484, p = 0.002), and the average TD increased with age, mainly in the vertical direction (all p < 0.05). The fixation stability decreased significantly in the group over 50-years-old.

Conclusion

By reporting the fixational displacement of different genders and ages in fixational and saccadic tasks, as well as different longitude lines among normal people, our study might provide an objective, quantitative and convenient reference index for the evaluation of fixation stability in visual impairment diseases and aging phenomenon of visual function.

Inferring visual space from ultra-fine extra-retinal knowledge of gaze position

It has long been debated how humans resolve fine details and perceive a stable visual world despite the incessant fixational motion of their eyes. Current theories assume these processes to rely solely on the visual input to the retina, without contributions from motor and/or proprioceptive sources. Here we show that contrary to this widespread assumption, the visual system has access to high-resolution extra-retinal knowledge of fixational eye motion and uses it to deduce spatial relations. Building on recent advances in gaze-contingent display control, we created a spatial discrimination task in which the stimulus configuration was entirely determined by oculomotor activity. Our results show that humans correctly infer geometrical relations in the absence of spatial information on the retina and accurately combine high-resolution extraretinal monitoring of gaze displacement with retinal signals. These findings reveal a sensory-motor strategy for encoding space, in which fine oculomotor knowledge is used to interpret the fixational input to the retina.

Vision: Neuronal mechanisms enabling stable perception

Eye movements cause rapid motion of the retinal image, potentially confusable with external motion. A recent study shows that neurons in mouse primary visual cortex distinguish self-generated from external motion by combining sensory input with saccade-related signals from the thalamic pulvinar nucleus.

Analysis and novel methods for capture of normative eye-tracking data in 2.5-month old infants

Development of attention systems is essential for both cognitive and social behavior maturation. Visual behavior has been used to assess development of these attention systems. Yet, given its importance, there is a notable lack of literature detailing successful methods and procedures for using eye-tracking in early infancy to assess oculomotor and attention dynamics. Here we show that eye-tracking technology can be used to automatically record and assess visual behavior in infants as young as 2.5 months, and present normative data describing fixation and saccade behavior at this age. Features of oculomotor dynamics were analyzed from 2.5-month old infants who viewed videos depicting live action, cartoons, geometric shapes, social and non-social scenes. Of the 54 infants enrolled, 50 infants successfully completed the eye-tracking task and high-quality data was collected for 32 of those infants. We demonstrate that modifications specifically tailored for the infant population allowed for consistent tracking of pupil and corneal reflection and minimal data loss. Additionally, we found consistent fixation and saccade behaviors across the entire six-minute duration of the videos, indicating that this is a feasible task for 2.5-month old infants. Moreover, normative oculomotor metrics for a free-viewing task in 2.5-month old infants are documented for the first time as a result of this high-quality data collection.

Perceptual hearing sensitivity during vocal production

Vocalization, such as speaking, inevitably generates sensory feedback that can cause self-generated masking. However, perceptual hearing sensitivity during vocal production is poorly understood. Using an adaptive psychophysical method, we measured the perceptual hearing sensitivity of an echolocating bat, Hipposideros pratti, in a passive listening (PL) task to detect pure tones, an active listening (AL) task to detect pure tones triggered by its vocalization, and a phantom echo task. We found that hanging H. pratti had the best hearing sensitivity of approximately 0 dB sound pressure level (SPL) in the PL task but much lower hearing sensitivity (nearly 40 dB worse) in the echo task. In the AL task, all bats gradually increased call frequency by 0.8–1.1 kHz, which improved their hearing sensitivity by 25–29 dB. This study underscores the need for studying the sensory capability of subjects engaged in active behaviors.

•

Vocal production strongly affects the perceptual hearing sensitivity of bats

•

Forward masking explains the reduced hearing sensitivity during vocalization

•

Long-term vocal plasticity enables bats to overcome self-generated auditory masking

Perceptual enhancement and suppression correlate with V1 neural activity during active sensing

Perception in multiple sensory modalities is an active process that involves exploratory behaviors. In humans and other primates, vision results from sensory sampling guided by saccadic eye movements. Saccades are known to modulate visual perception, and a corollary discharge signal associated with saccades appears to establish a sense of visual stability. Neural recordings have shown that saccades also modulate activity widely across the brain. To investigate the neural basis of saccadic effects on perception, simultaneous recordings from multiple neurons in area V1 were made as animals performed a contrast detection task. Perceptual and neural measures were compared when the animal made real saccades that brought a stimulus into V1 receptive fields and when simulated saccades were made (identical retinal stimulation but no eye movement). When real saccades were made and low spatial frequency stimuli were presented, we observed a reduction in both perceptual sensitivity and neural activity compared with simulated saccades; conversely, with higher spatial frequency stimuli, saccades increased visual sensitivity and neural activity. The performance of neural decoders, which used the activity of the population of simultaneously recorded neurons, showed saccade effects on sensitivity that mirrored the frequency-dependent perceptual changes, suggesting that the V1 population activity could support the perceptual effects. A minority of V1 neurons had significant choice probabilities, and the saccades decreased both average choice probability and pairwise noise correlations. Taken together, the findings suggest that a signal related to saccadic eye movements alters V1 spiking to increase the independence of spiking neurons and bias the system toward processing higher spatial frequencies, presumably to enhance object recognition. The effects of saccades on visual perception and noise correlations appear to parallel effects observed in other sensory modalities, suggesting a general principle of active sensory processing.

Gaze and social functioning associations in autism spectrum disorder: A systematic review and meta-analysis

Autism spectrum disorder (ASD) is characterized by significant social functioning impairments, including (but not limited to) emotion recognition, mentalizing, and joint attention. Despite extensive investigation into the correlates of social functioning in ASD, only recently has there been focus on the role of low‐level sensory input, particularly visual processing. Extensive gaze deficits have been described in ASD, from basic saccadic function through to social attention and the processing of complex biological motion. Given that social functioning often relies on accurately processing visual information, inefficient visual processing may contribute to the emergence and sustainment of social functioning difficulties in ASD. To explore the association between measures of gaze and social functioning in ASD, a systematic review and meta‐analysis was conducted. A total of 95 studies were identified from a search of CINAHL Plus, Embase, OVID Medline, and psycINFO databases in July 2021. Findings support associations between increased gaze to the face/head and eye regions with improved social functioning and reduced autism symptom severity. However, gaze allocation to the mouth appears dependent on social and emotional content of scenes and the cognitive profile of participants. This review supports the investigation of gaze variables as potential biomarkers of ASD, although future longitudinal studies are required to investigate the developmental progression of this relationship and to explore the influence of heterogeneity in ASD clinical characteristics.

Instant Reality: Gaze-Contingent Perceptual Optimization for 3D Virtual Reality Streaming

Media streaming, with an edge-cloud setting, has been adopted for a variety of applications such as entertainment, visualization, and design. Unlike video/audio streaming where the content is usually consumed passively, virtual reality applications require 3D assets stored on the edge to facilitate frequent edge-side interactions such as object manipulation and viewpoint movement. Compared to audio and video streaming, 3D asset streaming often requires larger data sizes and yet lower latency to ensure sufficient rendering quality, resolution, and latency for perceptual comfort. Thus, streaming 3D assets faces remarkably additional than streaming audios/videos, and existing solutions often suffer from long loading time or limited quality. To address this challenge, we propose a perceptually-optimized progressive 3D streaming method for spatial quality and temporal consistency in immersive interactions. On the cloud-side, our main idea is to estimate perceptual importance in 2D image space based on user gaze behaviors, including where they are looking and how their eyes move. The estimated importance is then mapped to 3D object space for scheduling the streaming priorities for edge-side rendering. Since this computational pipeline could be heavy, we also develop a simple neural network to accelerate the cloud-side scheduling process. We evaluate our method via subjective studies and objective analysis under varying network conditions (from 3G to 5G) and edge devices (HMD and traditional displays), and demonstrate better visual quality and temporal consistency than alternative solutions.

Gaze Strategies in Driving-An Ecological Approach

Human performance in natural environments is deeply impressive, and still much beyond current AI. Experimental techniques, such as eye tracking, may be useful to understand the cognitive basis of this performance, and "the human advantage." Driving is domain where these techniques may deployed, in tasks ranging from rigorously controlled laboratory settings through high-fidelity simulations to naturalistic experiments in the wild. This research has revealed robust patterns that can be reliably identified and replicated in the field and reproduced in the lab. The purpose of this review is to cover the basics of what is known about these gaze behaviors, and some of their implications for understanding visually guided steering. The phenomena reviewed will be of interest to those working on any domain where visual guidance and control with similar task demands is involved (e.g., many sports). The paper is intended to be accessible to the non-specialist, without oversimplifying the complexity of real-world visual behavior. The literature reviewed will provide an information base useful for researchers working on oculomotor behaviors and physiology in the lab who wish to extend their research into more naturalistic locomotor tasks, or researchers in more applied fields (sports, transportation) who wish to bring aspects of the real-world ecology under experimental scrutiny. Part of a Research Topic on Gaze Strategies in Closed Self-paced tasks, this aspect of the driving task is discussed. It is in particular emphasized why it is important to carefully separate the visual strategies driving (quite closed and self-paced) from visual behaviors relevant to other forms of driver behavior (an open-ended menagerie of behaviors). There is always a balance to strike between ecological complexity and experimental control. One way to reconcile these demands is to look for natural, real-world tasks and behavior that are rich enough to be interesting yet sufficiently constrained and well-understood to be replicated in simulators and the lab. This ecological approach to driving as a model behavior and the way the connection between "lab" and "real world" can be spanned in this research is of interest to anyone keen to develop more ecologically representative designs for studying human gaze behavior.

Motor-related signals support localization invariance for stable visual perception

Our ability to perceive a stable visual world in the presence of continuous movements of the body, head, and eyes has puzzled researchers in the neuroscience field for a long time. We reformulated this problem in the context of hierarchical convolutional neural networks (CNNs)-whose architectures have been inspired by the hierarchical signal processing of the mammalian visual system-and examined perceptual stability as an optimization process that identifies image-defining features for accurate image classification in the presence of movements. Movement signals, multiplexed with visual inputs along overlapping convolutional layers, aided classification invariance of shifted images by making the classification faster to learn and more robust relative to input noise. Classification invariance was reflected in activity manifolds associated with image categories emerging in late CNN layers and with network units acquiring movement-associated activity modulations as observed experimentally during saccadic eye movements. Our findings provide a computational framework that unifies a multitude of biological observations on perceptual stability under optimality principles for image classification in artificial neural networks.

A sensory memory to preserve visual representations across eye movements

Saccadic eye movements (saccades) disrupt the continuous flow of visual information, yet our perception of the visual world remains uninterrupted. Here we assess the representation of the visual scene across saccades from single-trial spike trains of extrastriate visual areas, using a combined electrophysiology and statistical modeling approach. Using a model-based decoder we generate a high temporal resolution readout of visual information, and identify the specific changes in neurons' spatiotemporal sensitivity that underly an integrated perisaccadic representation of visual space. Our results show that by maintaining a memory of the visual scene, extrastriate neurons produce an uninterrupted representation of the visual world. Extrastriate neurons exhibit a late response enhancement close to the time of saccade onset, which preserves the latest pre-saccadic information until the post-saccadic flow of retinal information resumes. These results show how our brain exploits available information to maintain a representation of the scene while visual inputs are disrupted.

Neural correlates of intra-saccadic motion perception

Retinal motion of the visual scene is not consciously perceived during ocular saccades in normal everyday conditions. It has been suggested that extra-retinal signals actively suppress intra-saccadic motion perception to preserve stable perception of the visual world. However, using stimuli optimized to preferentially activate the M-pathway, Castet and Masson (2000) demonstrated that motion can be perceived during a saccade. Based on this psychophysical paradigm, we used electroencephalography and eye-tracking recordings to investigate the neural correlates related to the conscious perception of intra-saccadic motion. We demonstrated the effective involvement during saccades of the cortical areas V1-V2 and MT-V5, which convey motion information along the M-pathway. We also showed that individual motion perception was related to retinal temporal frequency.

Rapid visual adaptation persists across saccades

Neurons in the visual cortex quickly adapt to constant input, which should lead to perceptual fading within few tens of milliseconds. However, perceptual fading is rarely observed in everyday perception, possibly because eye movements refresh retinal input. Recently, it has been suggested that amplitudes of large saccadic eye movements are scaled to maximally decorrelate presaccadic and postsaccadic inputs and thus to annul perceptual fading. However, this argument builds on the assumption that adaptation within naturally brief fixation durations is strong enough to survive any visually disruptive saccade and affect perception. We tested this assumption by measuring the effect of luminance adaptation on postsaccadic contrast perception. We found that postsaccadic contrast perception was affected by presaccadic luminance adaptation during brief periods of fixation. This adaptation effect emerges within 100 milliseconds and persists over seconds. These results indicate that adaptation during natural fixation periods can affect perception even after visually disruptive saccades.

During natural viewing, neural processing of visual targets continues throughout saccades

Relatively little is known about visual processing during free-viewing visual search in realistic dynamic environments. Free-viewing is characterized by frequent saccades. During saccades, visual processing is thought to be suppressed, yet we know that the presaccadic visual content can modulate postsaccadic processing. To better understand these processes in a realistic setting, we study here saccades and neural responses elicited by the appearance of visual targets in a realistic virtual environment. While subjects were being driven through a 3D virtual town, they were asked to discriminate between targets that appear on the road. Using a system identification approach, we separated overlapping and correlated activity evoked by visual targets, saccades, and button presses. We found that the presence of a target enhances early occipital as well as late frontocentral saccade-related responses. The earlier potential, shortly after 125 ms post-saccade onset, was enhanced for targets that appeared in the peripheral vision as compared to the central vision, suggesting that fast peripheral processing initiated before saccade onset. The later potential, at 195 ms post-saccade onset, was strongly modulated by the visibility of the target. Together these results suggest that, during natural viewing, neural processing of the presaccadic visual stimulus continues throughout the saccade, apparently unencumbered by saccadic suppression.

The effect of uncorrected ametropia on ocular torsion induced by changes in fixation

Background and Objective

Ocular torsion, the eye movements to rotating around the line of sight, has not been well investigated regarding the influence of refractive errors. The purpose of this study was to investigate the effect of uncorrected ametropia on ocular torsion induced by fixation distances.

Methods

Seventy-two subjects were classified according to the type of their refractive error, and ocular torsion of the uncorrected eye was compared based on changes induced by different fixation distances. Ocular torsion was measured using a slit-lamp biomicroscope equipped with an ophthalmic camera and a half-silvered mirror.

Results

In all groups, excyclotorsion values increased as the fixation distance decreased, but the myopia and astigmatism groups had larger amounts of ocular torsion than the emmetropia group. In addition, as the amount of uncorrected myopia and astigmatism increased, the amount of ocular torsion increased.

Conclusion

Since the amount of ocular torsion caused by a change to a shorter fixation distance was larger when the refractive error was uncorrected, we suggest that ametropia should be fully corrected in patients frequently exposed to ocular torsion due to changes in fixation distance.

N-methyl d-aspartate receptor hypofunction reduces visual contextual integration

Visual cognition is finely tuned to the elements in a scene but also relies on contextual integration to improve visual detection and discrimination. This integration is impaired in patients with schizophrenia. Studying impairments in contextual integration may lead to biomarkers of schizophrenia, tools to monitor disease progression, and, in animal models, insight into the underlying neural deficits. We developed a nonhuman primate model to test the hypothesis that hypofunction of the N-methyl d-aspartate receptor (NMDAR) impairs contextual integration. Two male rhesus macaques (Macaca mulatta) were trained to indicate which of two patterns on the screen had the highest contrast. One of these patterns appeared in isolation, and the other was surrounded by a high-contrast pattern. In humans, this high-contrast context is known to lead to an underestimation of contrast. This so-called Chubb illusion is thought to result from surround suppression, a key contextual integration mechanism. To test the involvement of NMDAR in this process, we compared animals' perceptual bias with and without intramuscular injections of a subanesthetic dose of the NMDAR antagonist ketamine. In the absence of ketamine, the animals reported a Chubb illusion - matching reports in healthy humans. Hence, monkeys - just like humans - perform visual contextual integration. This reaffirms the importance of nonhuman primates to help understand visual cognition. Injection of ketamine significantly reduced the strength of the illusion and thus impaired contextual integration. This supports the hypothesis that NMDAR hypofunction plays a causal role in specific behavioral impairments observed in schizophrenia.

Age effects on saccadic suppression of luminance and color

Saccadic eye movements modulate visual perception: they initiate and terminate high acuity vision at a certain location in space, but before and during their execution visual contrast sensitivity is strongly attenuated for 100 to 200 ms. Transient perisaccadic perceptual distortions are assumed to be an important mechanism to maintain visual stability. Little is known about age effects on saccadic suppression, even though for healthy adults other major age-related changes are well documented, like a decrease of visual contrast sensitivity for intermediate and high spatial frequencies or an increase of saccade latencies. Here, we tested saccadic suppression of luminance and isoluminant chromatic flashes in 100 participants from eight to 78 years. To estimate the effect of saccadic suppression on contrast sensitivity, we used a two-alternative forced choice (2AFC) design and an adaptive staircase procedure to modulate the luminance or chromatic contrast of a flashed detection target during fixation and 15 ms after saccade onset. The target was a single horizontal luminance or chromatic line flashed 2° above or below the fixation or saccade target. Compared to fixation, average perisaccadic contrast sensitivity decreased significantly by 66% for luminance and by 36% for color. A significant correlation was found for the strength of saccadic suppression of luminance and color. However, a small age effect was found only for the strength of saccadic suppression of luminance, which increased from 64% to 70% from young to old age. We conclude that saccadic suppression for luminance and color is present in most participants independent of their age and that mechanisms of suppression stay relatively stable during healthy aging.

Expert camouflage-breakers can accurately localize search targets

Camouflage-breaking is a special case of visual search where an object of interest, or target, can be hard to distinguish from the background even when in plain view. We have previously shown that naive, non-professional subjects can be trained using a deep learning paradigm to accurately perform a camouflage-breaking task in which they report whether or not a given camouflage scene contains a target. But it remains unclear whether such expert subjects can actually detect the target in this task, or just vaguely sense that the two classes of images are somehow different, without being able to find the target per se. Here, we show that when subjects break camouflage, they can also localize the camouflaged target accurately, even though they had received no specific training in localizing the target. The localization was significantly accurate when the subjects viewed the scene as briefly as 50 ms, but more so when the subjects were able to freely view the scenes. The accuracy and precision of target localization by expert subjects in the camouflage-breaking task were statistically indistinguishable from the accuracy and precision of target localization by naive subjects during a conventional visual search where the target ‘pops out’, i.e., is readily visible to the untrained eye. Together, these results indicate that when expert camouflage-breakers detect a camouflaged target, they can also localize it accurately.

Perisaccadic encoding of temporal information in macaque area V4

The accurate processing of temporal information is of critical importance in everyday life. Yet, psychophysical studies in humans have shown that the perception of time is distorted around saccadic eye movements. The neural correlates of this misperception are still poorly understood. Behavioral and neural evidence suggest that it is tightly linked to other known perisaccadic modulations of visual perception. To further our understanding of how temporal processing is affected by saccades, we studied the representations of brief visual time intervals during fixation and saccades in area V4 of two awake macaques. We presented random sequences of vertical bar stimuli and extracted neural responses to double-pulse stimulation at varying interstimulus intervals. Our results show that temporal information about very brief intervals of as brief as 20 ms is reliably represented in the multiunit activity in area V4. Response latencies were not systematically modulated by the saccade. However, a general increase in perisaccadic activity altered the ratio of response amplitudes within stimulus pairs compared with fixation. In line with previous studies showing that the perception of brief time intervals is partly based on response levels, this may be seen as a possible correlate of the perisaccadic misperception of time.NEW & NOTEWORTHY We investigated for the first time how temporal information on very brief timescales is represented in area V4 around the time of saccadic eye movements. Overall, the responses showed an unexpectedly precise representation of time intervals. Our finding of a perisaccadic modulation of relative response amplitudes introduces a new possible correlate of saccade-related perceptual distortions of time.

Blink- and saccade-related suppression effects in early visual areas of the human brain: Intracranial EEG investigations during natural viewing conditions

Blinks and saccades, both ubiquitous in natural viewing conditions, cause rapid changes of visual inputs that are hardly consciously perceived. The neural dynamics in early visual areas of the human brain underlying this remarkable visual stability are still incompletely understood. We used electrocorticography (ECoG) from electrodes directly implanted on the human early visual areas V1, V2, V3d/v, V4d/v and the fusiform gyrus to investigate blink- and saccade-related neuronal suppression effects during non-experimental, free viewing conditions. We found a characteristic, biphasic, broadband gamma power decrease-increase pattern in all investigated visual areas. During saccades, a decrease in gamma power clearly preceded eye movement onset, at least in V1. This may indicate that cortical information processing is actively suppressed in human early visual areas before and during saccades, which then possibly mediates perceptual visual suppression. The following eye movement offset-related increase in gamma power may indicate the recovery of visual perception and the resumption of visual processing.

Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Spatiotemporal Content of Saccade Transients

Humans use rapid gaze shifts, known as saccades, to explore visual scenes. These movements yield abrupt luminance changes on the retina, which elicit robust neural discharges at fixation onsets. Yet little is known about the spatial content of saccade transients. Here, we show that saccades redistribute spatial information within the temporal range of retinal sensitivity following two distinct regimes: saccade modulations counterbalance (whiten) the spectral density of natural scenes at low spatial frequencies and follow the external power distribution at higher frequencies. This redistribution is a consequence of saccade dynamics, particularly the speed/amplitude/duration relation known as the main sequence. It resembles the redistribution resulting from inter-saccadic eye drifts, revealing a continuum in the modulations given by different eye movements, with oculomotor transitions primarily acting by regulating the bandwidth of whitening. Our findings suggest important computational roles for saccade transients in the establishment of spatial representations and lead to testable predictions about their consequences for visual functions and encoding mechanisms.

Dynamic Modulation of Cortical Excitability during Visual Active Sensing

Visual physiology is traditionally investigated by presenting stimuli with gaze held constant. However, during active viewing of a scene, information is actively acquired using systematic patterns of fixations and saccades. Prior studies suggest that during such active viewing, both nonretinal, saccade-related signals and “extra-classical” receptive field inputs modulate visual processing. This study used a set of active viewing tasks that allowed us to compare visual responses with and without direct foveal input, thus isolating the contextual eye movement-related influences. Studying nonhuman primates, we find strong contextual modulation in primary visual cortex (V1): excitability and response amplification immediately after fixation onset, transiting to suppression leading up to the next saccade. Time-frequency decomposition suggests that this amplification and suppression cycle stems from a phase reset of ongoing neuronal oscillatory activity. The impact of saccade-related contextual modulation on stimulus processing makes active visual sensing fundamentally different from the more passive processes investigated in traditional paradigms.

By isolating contextual eye movement-related influences during active vision, Barczak et al. show that eye movements affect excitability in V1 such that responses are amplified immediately after fixation onset and suppressed as the next saccade approaches. This amplification and suppression cycle stems from a phase reset of ambient oscillatory activity in V1.

Humans use Optokinetic Eye Movements to Track Waypoints for Steering

It is well-established how visual stimuli and self-motion in laboratory conditions reliably elicit retinal-image-stabilizing compensatory eye movements (CEM). Their organization and roles in natural-task gaze strategies is much less understood: are CEM applied in active sampling of visual information in human locomotion in the wild? If so, how? And what are the implications for guidance? Here, we directly compare gaze behavior in the real world (driving a car) and a fixed base simulation steering task. A strong and quantifiable correspondence between self-rotation and CEM counter-rotation is found across a range of speeds. This gaze behavior is "optokinetic", i.e. optic flow is a sufficient stimulus to spontaneously elicit it in naïve subjects and vestibular stimulation or stereopsis are not critical. Theoretically, the observed nystagmus behavior is consistent with tracking waypoints on the future path, and predicted by waypoint models of locomotor control - but inconsistent with travel point models, such as the popular tangent point model.

Deep learning can be used to train naïve, nonprofessional observers to detect diagnostic visual patterns of certain cancers in mammograms: a proof-of-principle study

The scientific, clinical, and pedagogical significance of devising methodologies to train nonprofessional subjects to recognize diagnostic visual patterns in medical images has been broadly recognized. However, systematic approaches to doing so remain poorly established. Using mammography as an exemplar case, we use a series of experiments to demonstrate that deep learning (DL) techniques can, in principle, be used to train naïve subjects to reliably detect certain diagnostic visual patterns of cancer in medical images. In the main experiment, subjects were required to learn to detect statistical visual patterns diagnostic of cancer in mammograms using only the mammograms and feedback provided following the subjects' response. We found not only that the subjects learned to perform the task at statistically significant levels, but also that their eye movements related to image scrutiny changed in a learning-dependent fashion. Two additional, smaller exploratory experiments suggested that allowing subjects to re-examine the mammogram in light of various items of diagnostic information may help further improve DL of the diagnostic patterns. Finally, a fourth small, exploratory experiment suggested that the image information learned was similar across subjects. Together, these results prove the principle that DL methodologies can be used to train nonprofessional subjects to reliably perform those aspects of medical image perception tasks that depend on visual pattern recognition expertise.

Short-Term Attractive Tilt Aftereffects Predicted by a Recurrent Network Model of Primary Visual Cortex

Adaptation is a multi-faceted phenomenon that is of interest in terms of both its function and its potential to reveal underlying neural processing. Many behavioral studies have shown that after exposure to an oriented adapter the perceived orientation of a subsequent test is repulsed away from the orientation of the adapter. This is the well-known Tilt Aftereffect (TAE). Recently, we showed that the dynamics of recurrently connected networks may contribute substantially to the neural changes induced by adaptation, especially on short time scales. Here we extended the network model and made the novel behavioral prediction that the TAE should be attractive, not repulsive, on a time scale of a few 100 ms. Our experiments, using a novel adaptation protocol that specifically targeted adaptation on a short time scale, confirmed this prediction. These results support our hypothesis that recurrent network dynamics may contribute to short-term adaptation. More broadly, they show that understanding the neural processing of visual inputs that change on the time scale of a typical fixation requires a detailed analysis of not only the intrinsic properties of neurons, but also the slow and complex dynamics that emerge from their recurrent connectivity. We argue that this is but one example of how even simple recurrent networks can underlie surprisingly complex information processing, and are involved in rudimentary forms of memory, spatio-temporal integration, and signal amplification.

Temporal Coding of Visual Space

Establishing a representation of space is a major goal of sensory systems. Spatial information, however, is not always explicit in the incoming sensory signals. In most modalities it needs to be actively extracted from cues embedded in the temporal flow of receptor activation. Vision, on the other hand, starts with a sophisticated optical imaging system that explicitly preserves spatial information on the retina. This may lead to the assumption that vision is predominantly a spatial process: all that is needed is to transmit the retinal image to the cortex, like uploading a digital photograph, to establish a spatial map of the world. However, this deceptively simple analogy is inconsistent with theoretical models and experiments that study visual processing in the context of normal motor behavior. We argue here that, as with other senses, vision relies heavily on temporal strategies and temporal neural codes to extract and represent spatial information.

Predicting Perceptual Decisions Using Visual Cortical Population Responses and Choice History

Our understanding of the neural basis of perceptual decision making has been built in part on relating co-fluctuations of single neuron responses to perceptual decisions on a trial-by-trial basis. The strength of this relationship is often compared across neurons or brain areas, recorded in different sessions, animals, or variants of a task. We sought to extend our understanding of perceptual decision making in three ways. First, we measured neuronal activity simultaneously in early [primary visual cortex (V1)] and midlevel (V4) visual cortex while macaque monkeys performed a fine orientation discrimination perceptual task. This allowed a direct comparison of choice signals in these two areas, including their dynamics. Second, we asked how our ability to predict animals' decisions would be improved by considering small simultaneously-recorded neuronal populations rather than individual units. Finally, we asked whether predictions would be improved by taking into account the animals' choice and reward histories, which can strongly influence decision making. We found that responses of individual V4 neurons were weakly predictive of decisions, but only in a brief epoch between stimulus offset and the indication of choice. In V1, few neurons showed significant decision-related activity. Analysis of neuronal population responses revealed robust choice-related information in V4 and substantially weaker signals in V1. Including choice- and reward-history information improved performance further, particularly when the recorded populations contained little decision-related information. Our work shows the power of using neuronal populations and decision history when relating neuronal responses to the perceptual decisions they are thought to underlie.SIGNIFICANCE STATEMENT Decades of research has provided a rich description of how visual information is represented in the visual cortex. Yet how cortical responses relate to visual perception remains poorly understood. Here we relate fluctuations in small neuronal population responses, recorded simultaneously in primary visual cortex (V1) and area V4 of monkeys, to perceptual reports in an orientation discrimination task. Choice-related signals were robust in V4, particularly late in the behavioral trial, but not in V1. Models that include both neuronal responses and choice-history information were able to predict a substantial portion of decisions. Our work shows the power of integrating information across neurons and including decision history in relating neuronal responses to perceptual decisions.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

A Stable Visual World in Primate Primary Visual Cortex

Humans and other primates rely on eye movements to explore visual scenes and to track moving objects. As a result, the image that is projected onto the retina-and propagated throughout the visual cortical hierarchy-is almost constantly changing and makes little sense without taking into account the momentary direction of gaze. How is this achieved in the visual system? Here, we show that in primary visual cortex (V1), the earliest stage of cortical vision, neural representations carry an embedded "eye tracker" that signals the direction of gaze associated with each image. Using chronically implanted multi-electrode arrays, we recorded the activity of neurons in area V1 of macaque monkeys during tasks requiring fast (exploratory) and slow (pursuit) eye movements. Neurons were stimulated with flickering, full-field luminance noise at all times. As in previous studies, we observed neurons that were sensitive to gaze direction during fixation, despite comparable stimulation of their receptive fields. We trained a decoder to translate neural activity into metric estimates of gaze direction. This decoded signal tracked the eye accurately not only during fixation but also during fast and slow eye movements. After a fast eye movement, the eye-position signal arrived in V1 at approximately the same time at which the new visual information arrived from the retina. Using simulations, we show that this V1 eye-position signal could be used to take into account the sensory consequences of eye movements and map the fleeting positions of objects on the retina onto their stable position in the world.

Saccadic suppression measured by steady-state visual evoked potentials

Visual sensitivity is severely impaired during the execution of saccadic eye movements. This phenomenon has been extensively characterized in human psychophysics and nonhuman primate single-neuron studies, but a physiological characterization in humans is less established. Here, we used a method based on steady-state visually evoked potential (SSVEP), an oscillatory brain response to periodic visual stimulation, to examine how saccades affect visual sensitivity. Observers made horizontal saccades back and forth, while horizontal black-and-white gratings flickered at 5-30 Hz in the background. We analyzed EEG epochs with a length of 0.3 s either centered at saccade onset (saccade epochs) or centered at fixations half a second before the saccade (fixation epochs). Compared with fixation epochs, saccade epochs showed a broadband power increase, which most likely resulted from saccade-related EEG activity. The execution of saccades, however, led to an average reduction of 57% in the SSVEP amplitude at the stimulation frequency. This result provides additional evidence for an active saccadic suppression in the early visual cortex in humans. Compared with previous functional MRI and EEG studies, an advantage of this approach lies in its capability to trace the temporal dynamics of neural activity throughout the time course of a saccade. In contrast to previous electrophysiological studies in nonhuman primates, we did not find any evidence for postsaccadic enhancement, even though simulation results show that our method would have been able to detect it. We conclude that SSVEP is a useful technique to investigate the neural correlates of visual perception during saccadic eye movements in humans. NEW & NOTEWORTHY We make fast ballistic saccadic eye movements a few times every second. At the time of saccades, visual sensitivity is severely impaired. The present study uses steady-state visually evoked potentials to reveal a neural correlate of the fine temporal dynamics of these modulations at the time of saccades in humans. We observed a strong reduction (57%) of visually driven neural activity associated with saccades but did not find any evidence for postsaccadic enhancement.

Eye Movement Compensation and Spatial Updating in Visual Prosthetics: Mechanisms, Limitations and Future Directions

Despite appearing automatic and effortless, perceiving the visual world is a highly complex process that depends on intact visual and oculomotor function. Understanding the mechanisms underlying spatial updating (i.e., gaze contingency) represents an important, yet unresolved issue in the fields of visual perception and cognitive neuroscience. Many questions regarding the processes involved in updating visual information as a function of the movements of the eyes are still open for research. Beyond its importance for basic research, gaze contingency represents a challenge for visual prosthetics as well. While most artificial vision studies acknowledge its importance in providing accurate visual percepts to the blind implanted patients, the majority of the current devices do not compensate for gaze position. To-date, artificial percepts to the blind population have been provided either by intraocular light-sensing circuitry or by using external cameras. While the former commonly accounts for gaze shifts, the latter requires the use of eye-tracking or similar technology in order to deliver percepts based on gaze position. Inspired by the need to overcome the hurdle of gaze contingency in artificial vision, we aim to provide a thorough overview of the research addressing the neural underpinnings of eye compensation, as well as its relevance in visual prosthetics. The present review outlines what is currently known about the mechanisms underlying spatial updating and reviews the attempts of current visual prosthetic devices to overcome the hurdle of gaze contingency. We discuss the limitations of the current devices and highlight the need to use eye-tracking methodology in order to introduce gaze-contingent information to visual prosthetics.

A single retinal circuit model for multiple computations

Vision is dependent on extracting intricate features of the visual information from the outside world, and complex visual computations begin to take place as soon as at the retinal level. In multiple studies on salamander retinas, the responses of a subtype of retinal ganglion cells, i.e., fast/biphasic-OFF ganglion cells, have been shown to be able to realize multiple functions, such as the segregation of a moving object from its background, motion anticipation, and rapid encoding of the spatial features of a new visual scene. For each of these visual functions, modeling approaches using extended linear-nonlinear cascade models suggest specific preceding retinal circuitries merging onto fast/biphasic-OFF ganglion cells. However, whether multiple visual functions can be accommodated together in a certain retinal circuitry and how specific mechanisms for each visual function interact with each other have not been investigated. Here, we propose a physiologically consistent, detailed computational model of the retinal circuit based on the spatiotemporal dynamics and connections of each class of retinal neurons to implement object motion sensitivity, motion anticipation, and rapid coding in the same circuit. Simulations suggest that multiple computations can be accommodated together, thereby implying that the fast/biphasic-OFF ganglion cell has potential to output a train of spikes carrying multiple pieces of information on distinct features of the visual stimuli.

Attenuation of visual evoked responses to hand and saccade-initiated flashes

Sensory attenuation refers to reduced brain responses to self-initiated sensations relative to those produced by the external world. It is a low-level process that may be linked to higher-level cognitive tasks such as reality monitoring. The phenomenon is often explained by prediction error mechanisms of universal applicability to sensory modality; however, it is most widely reported for auditory stimuli resulting from self-initiated hand movements. The present series of event-related potential (ERP) experiments explored the generalizability of sensory attenuation to the visual domain by exposing participants to flashes initiated by either their own button press or volitional saccade and comparing these conditions to identical, computer-initiated stimuli. The key results showed that the largest reduction of anterior visual N1 amplitude occurred for saccade-initiated flashes, while button press-initiated flashes evoked an intermediary response between the saccade-initiated and externally initiated conditions. This indicates that sensory attenuation occurs for visual stimuli and suggests that the degree of electrophysiological attenuation may relate to the causal likelihood of pairings between the type of motor action and the modality of its sensory response.