Correspondence of presaccadic activity in the monkey primary visual cortex with saccadic eye movements

Decision-related activity and movement selection in primate visual cortex

Fluctuations in the activity of sensory neurons often predict perceptual decisions. This connection can be quantified with a metric called choice probability (CP), and there is a longstanding debate about whether CP reflects a causal influence on decisions or an echo of decision-making activity elsewhere in the brain. Here, we show that CP can reflect a third variable, namely, the movement used to indicate the decision. In a standard visual motion discrimination task, neurons in the middle temporal (MT) area of primate cortex responded more strongly during trials that involved a saccade toward their receptive fields. This variability accounted for much of the CP observed across the neuronal population, and it arose through training. Moreover, pharmacological inactivation of MT biased behavioral responses away from the corresponding visual field locations. These results demonstrate that training on a task with fixed sensorimotor contingencies introduces movement-related activity in sensory brain regions and that this plasticity can shape the neural circuitry of perceptual decision-making.

Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Activity in primate visual cortex is minimally driven by spontaneous movements

Organisms process sensory information in the context of their own moving bodies, an idea referred to as embodiment. This idea is important for developmental neuroscience, robotics and systems neuroscience. The mechanisms supporting embodiment are unknown, but a manifestation could be the observation in mice of brain-wide neuromodulation, including in the primary visual cortex, driven by task-irrelevant spontaneous body movements. We tested this hypothesis in macaque monkeys (Macaca mulatta), a primate model for human vision, by simultaneously recording visual cortex activity and facial and body movements. We also sought a direct comparison using an analogous approach to those used in mouse studies. Here we found that activity in the primate visual cortex (V1, V2 and V3/V3A) was associated with the animals' own movements, but this modulation was largely explained by the impact of the movements on the retinal image, that is, by changes in visual input. These results indicate that visual cortex in primates is minimally driven by spontaneous movements and may reflect species-specific sensorimotor strategies.

Presaccadic attention sharpens visual acuity

Visual perception is limited by spatial resolution, the ability to discriminate fine details. Spatial resolution not only declines with eccentricity but also differs for polar angle locations around the visual field, also known as 'performance fields'. To compensate for poor peripheral resolution, we make rapid eye movements-saccades-to bring peripheral objects into high-acuity foveal vision. Already before saccade onset, visual attention shifts to the saccade target location and prioritizes visual processing. This presaccadic shift of attention improves performance in many visual tasks, but whether it changes resolution is unknown. Here, we investigated whether presaccadic attention sharpens peripheral spatial resolution; and if so, whether such effect interacts with performance fields asymmetries. We measured acuity thresholds in an orientation discrimination task during fixation and saccade preparation around the visual field. The results revealed that presaccadic attention sharpens acuity, which can facilitate a smooth transition from peripheral to foveal representation. This acuity enhancement is similar across the four cardinal locations; thus, the typically robust effect of presaccadic attention does not change polar angle differences in resolution.

To look or not to look: dissociating presaccadic and covert spatial attention

Attention is a central neural process that enables selective and efficient processing of visual information. Individuals can attend to specific visual information either overtly, by making an eye movement to an object of interest, or covertly, without moving their eyes. We review behavioral, neuropsychological, neurophysiological, and computational evidence of presaccadic attentional modulations that occur while preparing saccadic eye movements, and highlight their differences from those of covert spatial endogenous (voluntary) and exogenous (involuntary) attention. We discuss recent studies and experimental procedures on how these different types of attention impact visual performance, alter appearance, differentially modulate the featural representation of basic visual dimensions (orientation and spatial frequency), engage different neural computations, and recruit partially distinct neural substrates. We conclude that presaccadic attention and covert attention are dissociable.

Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Rice Straw and Flax Fiber Particleboards as a Product of Agricultural Waste: An Evaluation of Technical Properties

Construction materials have a direct impact on the environment, on people, and their health. In addition, building insulation plays a decisive role in terms of energy consumption of buildings and regarding CO2-emissions over their whole life cycle. In order to achieve a holistic concept for green building worldwide, it is necessary to develop ecological insulating materials and to scientifically examine them in terms of their technical properties, as done with particleboards from agricultural waste presented in this article. This study aims to characterize the properties’ tensile and compressive strength, modulus of rupture (MOR), and elasticity (MOE) and thermal conductivity of particleboards affected by parameters, such as waste type (rice straw or flax shives), particleboard density, resin type, and content, as well as the use of treated rice straw. Particleboards made from flax shives had superior properties compared to the rice straw particles. The mechanical properties of the boards increase with an increasing resin content, except for the MOR and MOE, which decrease with an increasing resin content, and reach their peak value at a resin content of 10%.

Predicting Perceptual Decisions Using Visual Cortical Population Responses and Choice History

Our understanding of the neural basis of perceptual decision making has been built in part on relating co-fluctuations of single neuron responses to perceptual decisions on a trial-by-trial basis. The strength of this relationship is often compared across neurons or brain areas, recorded in different sessions, animals, or variants of a task. We sought to extend our understanding of perceptual decision making in three ways. First, we measured neuronal activity simultaneously in early [primary visual cortex (V1)] and midlevel (V4) visual cortex while macaque monkeys performed a fine orientation discrimination perceptual task. This allowed a direct comparison of choice signals in these two areas, including their dynamics. Second, we asked how our ability to predict animals' decisions would be improved by considering small simultaneously-recorded neuronal populations rather than individual units. Finally, we asked whether predictions would be improved by taking into account the animals' choice and reward histories, which can strongly influence decision making. We found that responses of individual V4 neurons were weakly predictive of decisions, but only in a brief epoch between stimulus offset and the indication of choice. In V1, few neurons showed significant decision-related activity. Analysis of neuronal population responses revealed robust choice-related information in V4 and substantially weaker signals in V1. Including choice- and reward-history information improved performance further, particularly when the recorded populations contained little decision-related information. Our work shows the power of using neuronal populations and decision history when relating neuronal responses to the perceptual decisions they are thought to underlie.SIGNIFICANCE STATEMENT Decades of research has provided a rich description of how visual information is represented in the visual cortex. Yet how cortical responses relate to visual perception remains poorly understood. Here we relate fluctuations in small neuronal population responses, recorded simultaneously in primary visual cortex (V1) and area V4 of monkeys, to perceptual reports in an orientation discrimination task. Choice-related signals were robust in V4, particularly late in the behavioral trial, but not in V1. Models that include both neuronal responses and choice-history information were able to predict a substantial portion of decisions. Our work shows the power of integrating information across neurons and including decision history in relating neuronal responses to perceptual decisions.

Presaccadic attention improves or impairs performance by enhancing sensitivity to higher spatial frequencies

Right before we move our eyes, visual performance and neural responses for the saccade target are enhanced. This effect, presaccadic attention, is considered to prioritize the saccade target and to enhance behavioral performance for the saccade target. Recent evidence has shown that presaccadic attention modulates the processing of feature information. Hitherto, it remains unknown whether presaccadic modulations on feature information are flexible, to improve performance for the task at hand, or automatic, so that they alter the featural representation similarly regardless of the task. Using a masking procedure, here we report that presaccadic attention can either improve or impair performance depending on the spatial frequency content of the visual input. These counterintuitive modulations were significant at a time window right before saccade onset. Furthermore, merely deploying covert attention within the same temporal interval without preparing a saccade did not affect performance. This study reveals that presaccadic attention not only prioritizes the saccade target, but also automatically modifies its featural representation.

Transsacadic Information and Corollary Discharge in Local Field Potentials of Macaque V1

Approximately three times per second, human visual perception is interrupted by a saccadic eye movement. In addition to taking the eyes to a new location, several lines of evidence suggest that the saccades play multiple roles in visual perception. Indeed, it may be crucial that visual processing is informed about movements of the eyes in order to analyze visual input distinctly and efficiently on each fixation and preserve stable visual perception of the world across saccades. A variety of studies has demonstrated that activity in multiple brain areas is modulated by saccades. The hypothesis tested here is that these signals carry significant information that could be used in visual processing. To test this hypothesis, local field potentials (LFPs) were simultaneously recorded from multiple electrodes in macaque primary visual cortex (V1); support vector machines (SVMs) were used to classify the peri-saccadic LFPs. We find that LFPs in area V1 carry information that can be used to distinguish neural activity associated with fixations from saccades, precisely estimate the onset time of fixations, and reliably infer the directions of saccades. This information may be used by the brain in processes including visual stability, saccadic suppression, receptive field (RF) remapping, fixation amplification, and trans-saccadic visual perception.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Pre-saccadic perception: Separate time courses for enhancement and spatial pooling at the saccade target

We interact with complex scenes using eye movements to select targets of interest. Studies have shown that the future target of a saccadic eye movement is processed differently by the visual system. A number of effects have been reported, including a benefit for perceptual performance at the target (“enhancement”), reduced influences of backward masking (“un-masking”), reduced crowding (“un-crowding”) and spatial compression towards the saccade target. We investigated the time course of these effects by measuring orientation discrimination for targets that were spatially crowded or temporally masked. In four experiments, we varied the target-flanker distance, the presence of forward/backward masks, the orientation of the flankers and whether participants made a saccade. Masking and randomizing flanker orientation reduced performance in both fixation and saccade trials. We found a small improvement in performance on saccade trials, compared to fixation trials, with a time course that was consistent with a general enhancement at the saccade target. In addition, a decrement in performance (reporting the average flanker orientation, rather than the target) was found in the time bins nearest saccade onset when random oriented flankers were used, consistent with spatial pooling around the saccade target. We did not find strong evidence for un-crowding. Overall, our pattern of results was consistent with both an early, general enhancement at the saccade target and a later, peri-saccadic compression/pooling towards the saccade target.

Saccade Preparation Reshapes Sensory Tuning

Human observers make large rapid eye movements-saccades-to bring behaviorally relevant information into the fovea, where spatial resolution is high. In some visual tasks [1-4], performance at the location of a saccade target improves before the eyes move. Although these findings provide evidence that extra-retinal signals evoked by saccades can enhance visual perception, it remains unknown whether and how presaccadic modulations change the processing of feature information and thus modulate visual representations. To answer this question, one must go beyond the use of methods that only probe performance accuracy (d') in different tasks. Here, using a psychophysical reverse correlation approach [5-8], we investigated how saccade preparation influences the processing of orientation and spatial frequency-two building blocks of early vision. We found that saccade preparation selectively enhanced the gain of high spatial frequency information and narrowed orientation tuning at the upcoming saccade landing position. These modulations were time locked to saccade onset, peaking right before the eyes moved (-50-0 ms). Moreover, merely deploying covert attention within the same temporal interval without preparing a saccade did not alter performance. The observed presaccadic tuning changes may correspond to the presaccadic enhancement [9-11] and receptive field shifts reported in neurophysiological studies [12-14]. Saccade preparation may support transaccadic integration by reshaping the representation of the saccade target to be more fovea-like just before the eyes move. The presaccadic modulations on spatial frequency and orientation processing illustrate a strong perception-action coupling by revealing that the visual system dynamically reshapes feature selectivity contingent upon eye movements.

An Eye in the Palm of Your Hand: Alterations in Visual Processing Near the Hand, a Mini-Review

Feedback within the oculomotor system improves visual processing at eye movement end points, also termed a visual grasp. We do not just view the world around us however, we also reach out and grab things with our hands. A growing body of literature suggests that visual processing in near-hand space is altered. The control systems for moving either the eyes or the hands rely on parallel networks of fronto-parietal regions, which have feedback connections to visual areas. Since the oculomotor system effects on visual processing occur through feedback, both through the motor plan and the motor efference copy, a parallel system where reaching and/or grasping motor-related activity also affects visual processing is likely. Areas in the posterior parietal cortex, for example, receive proprioceptive and visual information used to guide actions, as well as motor efference signals. This trio of information channels is all that would be necessary to produce spatial allocation of reach-related visual attention. We review evidence from behavioral and neurophysiological studies that support the hypothesis that feedback from the reaching and/or grasping motor control networks affects visual processing while noting ways in which it differs from that seen within the oculomotor system. We also suggest that object affordances may represent the neural mechanism through which certain object features are selected for preferential processing when stimuli are near the hand. Finally, we summarize the two effector-based feedback systems and discuss how having separate but parallel effector systems allows for efficient decoupling of eye and hand movements.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron's spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Two stages of programming eye gaze shifts in 3-D space

Accurate saccadic and vergence eye movements towards selected visual targets are fundamental to perceive the 3-D environment. Despite this importance, shifts in eye gaze are not always perfect given that they are frequently followed by small corrective eye movements. The oculomotor system receives distinct information from various visual cues that may cause incongruity in the planning of a gaze shift. To test this idea, we analyzed eye movements in humans performing a saccade task in a 3-D setting. We show that saccades and vergence movements towards peripheral targets are guided by monocular (perceptual) cues. Approximately 200ms after the start of fixation at the perceived target, a fixational saccade corrected the eye positions to the physical target location. Our findings suggest that shifts in eye gaze occur in two phases; a large eye movement toward the perceived target location followed by a corrective saccade that directs the eyes to the physical target location.

A role of eye vergence in covert attention

Covert spatial attention produces biases in perceptual and neural responses in the absence of overt orienting movements. The neural mechanism that gives rise to these effects is poorly understood. Here we report the relation between fixational eye movements, namely eye vergence, and covert attention. Visual stimuli modulate the angle of eye vergence as a function of their ability to capture attention. This illustrates the relation between eye vergence and bottom-up attention. In visual and auditory cue/no-cue paradigms, the angle of vergence is greater in the cue condition than in the no-cue condition. This shows a top-down attention component. In conclusion, observations reveal a close link between covert attention and modulation in eye vergence during eye fixation. Our study suggests a basis for the use of eye vergence as a tool for measuring attention and may provide new insights into attention and perceptual disorders.

Rapid simultaneous enhancement of visual sensitivity and perceived contrast during saccade preparation

Humans and other animals with foveate vision make saccadic eye movements to prioritize the visual analysis of behaviorally relevant information. Even before movement onset, visual processing is selectively enhanced at the target of a saccade, presumably gated by brain areas controlling eye movements. Here we assess concurrent changes in visual performance and perceived contrast before saccades, and show that saccade preparation enhances perception rapidly, altering early visual processing in a manner akin to increasing the physical contrast of the visual input. Observers compared orientation and contrast of a test stimulus, appearing briefly before a saccade, to a standard stimulus, presented previously during a fixation period. We found simultaneous progressive enhancement in both orientation discrimination performance and perceived contrast as time approached saccade onset. These effects were robust as early as 60 ms after the eye movement was cued, much faster than the voluntary deployment of covert attention (without eye movements), which takes ∼300 ms. Our results link the dynamics of saccade preparation, visual performance, and subjective experience and show that upcoming eye movements alter visual processing by increasing the signal strength.

The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.

Noise destroys feedback enhanced figure-ground segmentation but not feedforward figure-ground segmentation

Figure-ground (FG) segmentation is the separation of visual information into background and foreground objects. In the visual cortex, FG responses are observed in the late stimulus response period, when neurons fire in tonic mode, and are accompanied by a switch in cortical state. When such a switch does not occur, FG segmentation fails. Currently, it is not known what happens in the brain on such occasions. A biologically plausible feedforward spiking neuron model was previously devised that performed FG segmentation successfully. After incorporating feedback the FG signal was enhanced, which was accompanied by a change in spiking regime. In a feedforward model neurons respond in a bursting mode whereas in the feedback model neurons fired in tonic mode. It is known that bursts can overcome noise, while tonic firing appears to be much more sensitive to noise. In the present study, we try to elucidate how the presence of noise can impair FG segmentation, and to what extent the feedforward and feedback pathways can overcome noise. We show that noise specifically destroys the feedback enhanced FG segmentation and leaves the feedforward FG segmentation largely intact. Our results predict that noise produces failure in FG perception.

Eye movements and attention: the role of pre-saccadic shifts of attention in perception, memory and the control of saccades

Saccadic eye movements and perceptual attention work in a coordinated fashion to allow selection of the objects, features or regions with the greatest momentary need for limited visual processing resources. This study investigates perceptual characteristics of pre-saccadic shifts of attention during a sequence of saccades using the visual manipulations employed to study mechanisms of attention during maintained fixation. The first part of this paper reviews studies of the connections between saccades and attention, and their significance for both saccadic control and perception. The second part presents three experiments that examine the effects of pre-saccadic shifts of attention on vision during sequences of saccades. Perceptual enhancements at the saccadic goal location relative to non-goal locations were found across a range of stimulus contrasts, with either perceptual discrimination or detection tasks, with either single or multiple perceptual targets, and regardless of the presence of external noise. The results show that the preparation of saccades can evoke a variety of attentional effects, including attentionally-mediated changes in the strength of perceptual representations, selection of targets for encoding in visual memory, exclusion of external noise, or changes in the levels of internal visual noise. The visual changes evoked by saccadic planning make it possible for the visual system to effectively use saccadic eye movements to explore the visual environment.

Spikes, BOLD, attention, and awareness: a comparison of electrophysiological and fMRI signals in V1

Early fMRI studies comparing results from fMRI and electrophysiological experiments support the notion that the blood oxygen level-dependent (BOLD) signal reliably follows the spiking activity of an underlying neuronal population averaged across a small region in space and a brief period in time. However, more recent studies focusing on higher level cognitive factors such as attention and visual awareness report striking discrepancies between the fMRI response in humans and electrophysiological signals in macaque early visual areas. Four hypotheses are discussed that can explain the discrepancies between the two methods: (1) the BOLD signal follows local field potential (LFP) signals closer than spikes, and only the LFP is modulated by top-down factors, (2) the BOLD signal is reflecting electrophysiological signals that are occurring later in time due to feedback delay, (3) the BOLD signal is more sensitive than traditional electrophysiological methods due to massive pooling by the hemodynamic coupling process, and finally (4) there is no real discrepancy, and instead, weak but reliable effects on firing rates may be obscured by differences in experimental design and interpretation of results across methods.

Lateralized Frontal Eye Field Activity Precedes Occipital Activity Shortly before Saccades: Evidence for Cortico-cortical Feedback as a Mechanism Underlying Covert Attention Shifts

When an eye movement is prepared, attention is shifted toward the saccade end-goal. This coupling of eye movements and spatial attention is thought to be mediated by cortical connections between the FEFs and the visual cortex. Here, we present evidence for the existence of these connections. A visual discrimination task was performed while recording the EEG. Discrimination performance was significantly improved when the discrimination target and the saccade target matched. EEG results show that frontal activity precedes occipital activity contralateral to saccade direction when the saccade is prepared but not yet executed; these effects were absent in fixation conditions. This is consistent with the idea that the FEF exerts a direct modulatory influence on the visual cortex and enhances perception at the saccade end-goal.

Trial-to-trial variability of spike response of V1 and saccadic response time

Single neurons in the primary visual cortex (V1) show variability in spike activity in response to an identical visual stimulus. In the current study, we examined the behavioral significance of the variability in spike activity of V1 neurons for visually guided saccades. We recorded single-cell activity from V1 of monkeys trained to detect and make saccades toward visual targets of varying contrast and analyzed trial-to-trial covariation between the onset time or firing rate of neural response and saccadic response time (RT). Neural latency (NL, the time of the first spike of neural response) was correlated with RT, whereas firing rate (FR) was not. When FR was computed with respect to target onset ignoring NL, a "false" correlation between FR and RT emerged. Multiple regression and partial correlation analyses on NL and FR for predictability of RT variability, as well as a simulation with artificial Poisson spike trains, supported the conclusion that the correlation between FR with respect to target onset and RT was mediated by a correlation between NL and RT, emphasizing the role of trial-to-trial variability of NL for extracting RT-related signals. We attempted to examine laminar differences in RT-related activity. Neurons recorded in the superficial layers tended to show a higher sensitivity to stimulus contrast and a lower correlation with RT compared with those in the lower layers, suggesting a sensory-to-motor transformation within V1 that follows the order of known anatomical connections. These results demonstrate that the trial-to-trial variability of neural response in V1 propagates to the stage of saccade execution, resulting in trial-to-trial variability of RT of a visually guided saccade.

Neural and temporal dynamics underlying visual selection for action

The present study investigated the selection for action hypothesis, according to which a subject's action intention to perform a movement influences the way in which visual information is being processed. Subjects were instructed in separate blocks either to grasp or to point to a three-dimensional target-object and event-related potentials were recorded relative to stimulus onset. It was found that grasping compared with pointing resulted in a stronger N1 component and a subsequent selection negativity, which were localized to the lateral occipital complex. These effects suggest that the intention to grasp influences the processing of action-relevant features in ventral stream areas already at an early stage (e.g., enhanced processing of object orientation for grasping). These findings provide new insight in the neural and temporal dynamics underlying perception-action coupling and provide neural evidence for a selection for action principle in early human visual processing.

Mapping of contextual modulation in the population response of primary visual cortex

We review the evidence of long-range contextual modulation in V1. Populations of neurons in V1 are activated by a wide variety of stimuli outside of their classical receptive fields (RF), well beyond their surround region. These effects generally involve extra-RF features with an orientation component. The population mapping of orientation preferences to the upper layers of V1 is well understood, as far as the classical RF properties are concerned, and involves organization into pinwheel-like structures. We introduce a novel hypothesis regarding the organization of V1's contextual response. We show that RF and extra-RF orientation preferences are mapped in related ways. Orientation pinwheels are the foci of both types of features. The mapping of contextual features onto the orientation pinwheel has a form that recapitulates the organization of the visual field: an iso-orientation patch within the pinwheel also responds to extra-RF stimuli of the same orientation. We hypothesize that the same form of mapping applies to other stimulus properties that are mapped out in V1, such as colour and contrast selectivity. A specific consequence is that fovea-like properties will be mapped in a systematic way to orientation pinwheels. We review the evidence that cytochrome oxidase blobs comprise the foci of this contextual remapping for colour and low contrasts. Neurodynamics and motion in the visual field are argued to play an important role in the shaping and maintenance of this type of mapping in V1.

fMRI-guided TMS on cortical eye fields: the frontal but not intraparietal eye fields regulate the coupling between visuospatial attention and eye movements

It is well known that parts of a visual scene are prioritized for visual processing, depending on the current situation. How the CNS moves this focus of attention across the visual image is largely unknown, although there is substantial evidence that preparation of an action is a key factor. Our results support the view that direct corticocortical feedback connections from frontal oculomotor areas to the visual cortex are responsible for the coupling between eye movements and shifts of visuospatial attention. Functional magnetic resonance imaging (fMRI)-guided transcranial magnetic stimulation (TMS) was applied to the frontal eye fields (FEFs) and intraparietal sulcus (IPS). A single pulse was delivered 60, 30, or 0 ms before a discrimination target was presented at, or next to, the target of a saccade in preparation. Results showed that the known enhancement of discrimination performance specific to locations to which eye movements are being prepared was enhanced by early TMS on the FEF contralateral to eye movement direction, whereas TMS on the IPS resulted in a general performance increase. The current findings indicate that the FEF affects selective visual processing within the visual cortex itself through direct feedback projections.

A microsaccadic rhythm modulates gamma-band synchronization and behavior

Rhythms occur both in neuronal activity and in behavior. Behavioral rhythms abound at frequencies at or below 10 Hz. Neuronal rhythms cover a very wide frequency range, and the phase of neuronal low-frequency rhythms often rhythmically modulates the strength of higher-frequency rhythms, particularly of gamma-band synchronization (GBS). Here, we study stimulus-induced GBS in awake monkey areas V1 and V4 in relation to a specific form of spontaneous behavior, namely microsaccades (MSs), small fixational eye movements. We found that MSs occur rhythmically at a frequency of approximately 3.3 Hz. The rhythmic MSs were predicted by the phase of the 3.3 Hz rhythm in V1 and V4 local field potentials. In turn, the MSs modulated both visually induced GBS and the speed of visually triggered behavioral responses. Fast/slow responses were preceded by a specific temporal pattern of MSs. These MS patterns induced perturbations in GBS that in turn explained variability in behavioral response speed. We hypothesize that the 3.3 Hz rhythm structures the sampling and exploration of the environment through building and breaking neuronal ensembles synchronized in the gamma-frequency band to process sensory stimuli.

Spectral properties of induced and evoked gamma oscillations in human early visual cortex to moving and stationary stimuli

In two experiments, magnetoencephalography (MEG) was used to investigate the effects of motion on gamma oscillations in human early visual cortex. When presented centrally, but not peripherally, stationary and moving gratings elicited several evoked and induced response components in early visual cortex. Time-frequency analysis revealed two nonphase locked gamma power increases-an initial, rapidly adapting response and one sustained throughout stimulus presentation and varying in frequency across observers from 28 to 64 Hz. Stimulus motion raised the sustained gamma oscillation frequency by a mean of approximately 10 Hz. The largest motion-induced frequency increases were in those observers with the lowest gamma response frequencies for stationary stimuli, suggesting a possible saturation mechanism. Moderate gamma amplitude increases to moving versus stationary stimuli were also observed but were not correlated with the magnitude of the frequency increase. At the same site in visual cortex, sustained alpha/beta power reductions and an onset evoked response were observed, but these effects did not change significantly with the presence of motion and did not correlate with the magnitude of gamma power changes. These findings suggest that early visual areas encode moving and stationary percepts via activity at higher and lower gamma frequencies, respectively.

Saccades to a remembered location elicit spatially specific activation in human retinotopic visual cortex

The possible impact upon human visual cortex from saccades to remembered target locations was investigated using functional magnetic resonance imaging (fMRI). A specific location in the upper-right or upper-left visual quadrant served as the saccadic target. After a delay of 2,400 msec, an auditory signal indicated whether to execute a saccade to that location (go trial) or to cancel the saccade and remain centrally fixated (no-go). Group fMRI analysis revealed activation specific to the remembered target location for executed saccades, in the contralateral lingual gyrus. No-go trials produced similar, albeit significantly reduced, effects. Individual retinotopic mapping confirmed that on go trials, quadrant-specific activations arose in those parts of ventral V1, V2, and V3 that coded the target location for the saccade, whereas on no-go trials, only the corresponding parts of V2 and V3 were significantly activated. These results indicate that a spatial-motor saccadic task (i.e., making an eye movement to a remembered location) is sufficient to activate retinotopic visual cortex spatially corresponding to the target location, and that this activation is also present (though reduced) when no saccade is executed. We discuss the implications of finding that saccades to remembered locations can affect early visual cortex, not just those structures conventionally associated with eye movements, in relation to recent ideas about attention, spatial working memory, and the notion that recently activated representations can be "refreshed" when needed.

Neuronal mechanisms of visual stability

Human vision is stable and continuous in spite of the incessant interruptions produced by saccadic eye movements. These rapid eye movements serve vision by directing the high resolution fovea rapidly from one part of the visual scene to another. They should detract from vision because they generate two major problems: displacement of the retinal image with each saccade and blurring of the image during the saccade. This review considers the substantial advances in understanding the neuronal mechanisms underlying this visual stability derived primarily from neuronal recording and inactivation studies in the monkey, an excellent model for systems in the human brain. For the first problem, saccadic displacement, two neuronal candidates are salient. First are the neurons in frontal and parietal cortex with shifting receptive fields that provide anticipatory activity with each saccade and are driven by a corollary discharge. These could provide the mechanism for a retinotopic hypothesis of visual stability and possibly for a transsaccadic memory hypothesis, The second neuronal mechanism is provided by neurons whose visual response is modulated by eye position (gain field neurons) or are largely independent of eye position (real position neurons), and these neurons could provide the basis for a spatiotopic hypothesis. For the second problem, saccadic suppression, visual masking and corollary discharge are well established mechanisms, and possible neuronal correlates have been identified for each.

Changes in orientation discrimination at the time of saccadic eye movements

Perceptual performance has been known to change around the time of saccadic eye movement. In the current study, we measured the accuracy and sensitivity of orientation discrimination of bar stimuli presented during fixation and before saccadic eye movements. Human participants compared the orientations of the test and reference bar stimuli with the head erect in a two-interval forced choice task. For the targets presented during steady fixation, the accuracy and sensitivity of orientation discrimination were better near the cardinal than oblique axes, a perceptual anisotropy known as the oblique effect. For the targets presented during the 100 ms interval immediately before a saccade was executed, the anisotropy decreased mainly due to reduction in sensitivity for cardinal orientations. Directing attention to the goal location of the impending saccade emulated the saccadic effects on orientation discrimination for the targets at saccadic goal, suggesting that the saccadic effects on orientation discrimination are partly mediated by the shift of spatial attention that accompanies the saccade. These results were in line with the anti-oblique effect that perceptual judgment of motion direction along the oblique angle becomes relatively accurate for motion targets presented before saccadic eye movements [Lee, J., & Lee, C. (2005). Changes in visual motion perception before saccadic eye movements. Vision Research, 45(11), 1447-1457].

Perisaccadic Parietal and Occipital Gamma Power in Light and in Complete Darkness

Our objective was to determine perisaccadic gamma range oscillations in the EEG during voluntary saccades in humans. We evaluated occipital perisaccadic gamma activity both in the presence and absence of visual input, when the observer was blindfolded. We quantified gamma power in the time periods before, during, and after horizontal saccades. The corresponding EEG was evaluated for individual saccades and the wavelet transformed EEG averaged for each time window, without averaging the EEG first. We found that, in both dark and light, parietal and occipital gamma power increased during the saccade and peaked prior to reaching new fixation. We show that this is not the result of muscle activity and not the result of visual input during saccades. Saccade direction affects the laterality of gamma power over posterior electrodes. Gamma power recorded over the posterior scalp increases during a saccade. The phasic modulation of gamma by saccades in darkness—when occipital activity is decoupled from visual input—provides electrophysiological evidence that voluntary saccades affect ongoing EEG. We suggest that saccade-phasic gamma modulation may contribute to short-term plasticity required to realign the visual space to the intended fixation point of a saccade and provides a mechanism for neuronal assembly formation prior to achieving the intended saccadic goal. The wavelet-transformed perisaccadic EEG could provide an electrophysiological tool applicable in humans for the purpose of fine analysis and potential separation of stages of ‘planning’ and ‘action’.

TMS Pulses on the Frontal Eye Fields Break Coupling Between Visuospatial Attention and Eye Movements

While preparing a saccadic eye movement, visual processing of the saccade goal is prioritized. Here, we provide evidence that the frontal eye fields (FEFs) are responsible for this coupling between eye movements and shifts of visuospatial attention. Functional magnetic resonance imaging (fMRI)–guided transcranial magnetic stimulation (TMS) was applied to the FEFs 30 ms before a discrimination target was presented at or next to the target of a saccade in preparation. Results showed that the well-known enhancement of discrimination performance on locations to which eye movements are being prepared was diminished by TMS contralateral to eye movement direction. Based on the present and other reports, we propose that saccade preparatory processes in the FEF affect selective visual processing within the visual cortex through feedback projections, in that way coupling saccade preparation and visuospatial attention.

Distinct causal influences of parietal versus frontal areas on human visual cortex: evidence from concurrent TMS-fMRI

It has often been proposed that regions of the human parietal and/or frontal lobe may modulate activity in visual cortex, for example, during selective attention or saccade preparation. However, direct evidence for such causal claims is largely missing in human studies, and it remains unclear to what degree the putative roles of parietal and frontal regions in modulating visual cortex may differ. Here we used transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) concurrently, to show that stimulating right human intraparietal sulcus (IPS, at a site previously implicated in attention) elicits a pattern of activity changes in visual cortex that strongly depends on current visual context. Increased intensity of IPS TMS affected the blood oxygen level-dependent (BOLD) signal in V5/MT+ only when moving stimuli were present to drive this visual region, whereas TMS-elicited BOLD signal changes were observed in areas V1-V4 only during the absence of visual input. These influences of IPS TMS upon remote visual cortex differed significantly from corresponding effects of frontal (eye field) TMS, in terms of how they related to current visual input and their spatial topography for retinotopic areas V1-V4. Our results show directly that parietal and frontal regions can indeed have distinct patterns of causal influence upon functional activity in human visual cortex.

Figure-ground activity in V1 and guidance of saccadic eye movements

Every day we shift our gaze about 150.000 times mostly without noticing it. The direction of these gaze shifts are not random but directed by sensory information and internal factors. After each movement the eyes hold still for a brief moment so that visual information at the center of our gaze can be processed in detail. This means that visual information at the saccade target location is sufficient to accurately guide the gaze shift but yet is not sufficiently processed to be fully perceived. In this paper I will discuss the possible role of activity in the primary visual cortex (V1), in particular figure-ground activity, in oculo-motor behavior. Figure-ground activity occurs during the late response period of V1 neurons and correlates with perception. The strength of figure-ground responses predicts the direction and moment of saccadic eye movements. The superior colliculus, a gaze control center that integrates visual and motor signals, receives direct anatomical connections from V1. These projections may convey the perceptual information that is required for appropriate gaze shifts. In conclusion, figure-ground activity in V1 may act as an intermediate component linking visual and motor signals.

Strength of figure-ground activity in monkey primary visual cortex predicts saccadic reaction time in a delayed detection task

When and where are decisions made? In the visual system a saccade, which is a fast shift of gaze toward a target in the visual scene, is the behavioral outcome of a decision. Current neurophysiological data and reaction time models show that saccadic reaction times are determined by a build-up of activity in motor-related structures, such as the frontal eye fields. These structures depend on the sensory evidence of the stimulus. Here we use a delayed figure-ground detection task to show that late modulated activity in the visual cortex (V1) predicts saccadic reaction time. This predictive activity is part of the process of figure-ground segregation and is specific for the saccade target location. These observations indicate that sensory signals are directly involved in the decision of when and where to look.

Widely distributed magnetoencephalography spikes related to the planning and execution of human saccades

With sufficiently fast data sampling, ubiquitous sharp transients appear in magnetoencephalography (MEG) data. Initially, no known collective neuronal activity could explain MEG signal generation well above 100 Hz, so it was assumed that these transients were entirely composed of background electronic noise that could be eliminated by filtering and averaging. Recent studies at the cellular level provided evidence for synchronous synaptic input to dendrites and volleys of near-simultaneous action potentials. MEG studies have also identified high-frequency oscillations well above 200 Hz after averaging large number of somatosensory evoked responses. In this study, we searched for evidence of high-frequency neuronal activity in the raw MEG signal using the highest sampling rate available with our hardware. Two human subjects participated in three experiments using visual cues to define planning, preparation, and execution or inhibition of saccades. Tomographic analysis identified "MEG spikes" that were widely distributed across the cortex, cerebellum, and brainstem during cue presentations and saccades. Here we demonstrate how these MEG spikes can be recorded and localized in real time and show that task demands influence their properties. The MEG spikes were organized into feedforward and corollary discharge sequences that could, when combined with the slower activity-linked processing in discrete brain areas over long periods, lasting hundreds of milliseconds. Preparation for impending saccade began as soon as relevant information became available. Cues providing partial information initiated competing motor programs for as yet undecided future actions that were maintained until cues with new information resolved the uncertainty.

Visual Motion Perception at the Time of Saccadic Eye Movements and its Relation to Spatial Mislocalization

The same retinal image motion can be produced by a variety of combinations of eye and target motion. In natural conditions, extraretinal information disambiguates retinal information for motion perception. By controlling the timing of visual motion with respect to saccades, it was possible to appraise the roles for motion perception of the signal related to saccades occurring in the vicinity of visual motion. When a visual motion was seen before or after a saccade, its perceived direction was biased in the direction opposite to the saccade. The magnitude of the bias depended on the timing of the visual motion with respect to the saccade and the meridian of the visual motion. The bias appears to be independent of the deformation of visual space reported to occur before and after saccades.

Shortening and prolongation of saccade latencies following microsaccades

When the eyes fixate at a point in a visual scene, small saccades rapidly shift the image on the retina. The effect of these microsaccades on the latency of subsequent large-scale saccades may be twofold. First, microsaccades are associated with an enhancement of visual perception. Their occurrence during saccade target perception could, thus, decrease saccade latencies. Second, microsaccades are likely to indicate activity in fixation-related oculomotor neurons. These represent competitors to saccade-related cells in the interplay of gaze holding and shifting. Consequently, an increase in saccade latencies would be expected after microsaccades. Here, we present evidence for both aspects of microsaccadic impact on saccade latency. In a delayed response task, participants made saccades to visible or memorized targets. First, microsaccade occurrence up to 50 ms before target disappearance correlated with 18 ms (or 8%) faster saccades to memorized targets. Second, if microsaccades occurred shortly (i.e., <150 ms) before a saccade was required, mean saccadic reaction time in visual and memory trials was increased by about 40 ms (or 16%). Hence, microsaccades can have opposite consequences for saccade latencies, pointing at a differential role of these fixational eye movements in the preparation of saccade motor programs.

Attention-gated reinforcement learning of internal representations for classification

Animal learning is associated with changes in the efficacy of connections between neurons. The rules that govern this plasticity can be tested in neural networks. Rules that train neural networks to map stimuli onto outputs are given by supervised learning and reinforcement learning theories. Supervised learning is efficient but biologically implausible. In contrast, reinforcement learning is biologically plausible but comparatively inefficient. It lacks a mechanism that can identify units at early processing levels that play a decisive role in the stimulus-response mapping. Here we show that this so-called credit assignment problem can be solved by a new role for attention in learning. There are two factors in our new learning scheme that determine synaptic plasticity: (1) a reinforcement signal that is homogeneous across the network and depends on the amount of reward obtained after a trial, and (2) an attentional feedback signal from the output layer that limits plasticity to those units at earlier processing levels that are crucial for the stimulus-response mapping. The new scheme is called attention-gated reinforcement learning (AGREL). We show that it is as efficient as supervised learning in classification tasks. AGREL is biologically realistic and integrates the role of feedback connections, attention effects, synaptic plasticity, and reinforcement learning signals into a coherent framework.

Extraretinal saccadic signals in human LGN and early retinotopic cortex

In the human LGN and V1, saccades in darkness lead to enhanced activity while saccades made during strong visual stimulation suppress activity [Sylvester, R., Haynes, J.D., and Rees, G., 2005. Saccades differentially modulate human LGN and V1 responses in the presence and absence of visual stimulation. Curr. Biol. 15, 37-41]. Here, we explored this differential modulation further using graded changes in the strength of visual stimulation by changing the mean luminance of a flickering visual stimulus. We replicate the finding of differential modulation of activity in human LGN and V1, and show that this relationship also holds in retinotopic areas V2 and V3. Suppression of visually evoked activity during saccades was detectable during strong visual stimulation, but not during weaker stimulation. This suggests that the activation of visual cortex by saccades in darkness represents a signal that persists irrespective of the state of visual stimulation, masking suppressive effects of saccades when visual stimulation is weak. Such a signal may represent a motor signal in a sensory area. We discuss the possible role of oculomotor corollary discharge in changes in visual perception that occur peri-saccadically, which contribute to the successful negation of the disruptive effects of saccades on our seamless visual experience of the world.

Synchrony dynamics in monkey V1 predict success in visual detection

Behavioral measures such as expectancy and attention have been associated with the strength of synchronous neural activity. On this basis, it is hypothesized that synchronous activity affects our ability to detect and recognize visual objects. To investigate the role of synchronous activity in visual perception, we studied the magnitude and precision of correlated activity, before and after stimulus presentation within the visual cortex (V1), in relation to a monkey's performance in a figure-ground discrimination task. We show that during the period of stimulus presentation a transition in synchronized activity occurs that is characterized by a reduction of the correlation peak height and width. Before stimulus onset, broad peak correlations are observed that change towards thin peak correlations after stimulus onset, due to a specific decrease of low-frequency components. The magnitude of the transition in correlated activity is larger, i.e. a stronger desynchronization occurs, when the animal perceives the stimulus correctly than when the animal fails to detect the stimulus. These results therefore show that a transition in synchronous firing is important for the detection of sensory stimuli. We hypothesize that the transition in synchrony reflects a change from loose and global neuronal interactions towards a finer temporal and spatial scale of neuronal interactions, and that such a change in neuronal interactions is required for figure-ground discrimination.

Rapid processing of retinal slip during saccades in macaque area MT

The primate middle temporal area (MT) is involved in the analysis and perception of visual motion, which is generated actively by eye and body movements and passively when objects move. We studied the responses of single cells in area MT of awake macaques, comparing the direction tuning and latencies of responses evoked by wide-field texture motion during fixation (passive viewing) and during rewarded, target-directed saccades and non-rewarded, spontaneous saccades over the same stationary texture (active viewing). We found that MT neurons have similar motion sensitivity and direction-selectivity for retinal slip associated with active and passive motion. No cells showed reversals in direction tuning between the active and passive viewing conditions. However, mean latencies were significantly different for saccade-evoked responses (30 ms) and stimulus-evoked responses (67 ms). Our results demonstrate that neurons in area MT retain their direction-selectivity and display reduced processing times during saccades. This rapid, accurate processing of peri-saccadic motion may facilitate post-saccadic ocular following reflexes or corrective saccades.

Chronic multiunit recordings in behaving animals: advantages and limitations

By simultaneous recording from neural responses at many different loci at the same time, we can understand the interaction between neurons, and thereby gain insight into the network properties of neural processing, instead of the functioning of individual neurons. Here we will discuss a method for recording in behaving animals that uses chronically implanted micro-electrodes that allow one to track neural responses over a long period of time. In a majority of cases, multiunit activity, which is the aggregate spiking activity of a number of neurons in the vicinity of an electrode tip, is recorded through these electrodes, and occasionally single neurons can be isolated. Here we compare the properties of multiunit responses to the responses of single neurons in the primary visual cortex. We also discuss the advantages and disadvantages of the multiunit signal as opposed to a signal of single neurons. We demonstrate that multiunit recording provides a reliable and useful technique in cases where the neurons at the electrodes have similar response properties. Multiunit recording is therefore especially valuable when task variables have an effect that is consistent across the population of neurons. In the primary visual cortex, this is the case for figure-ground segregation and visual attention. Multiunit recording also has clear advantages for cross-correlation analysis. We show that the cross-correlation function between multiunit signals gives a reliable estimate of the average single-unit cross-correlation function. By the use of multiunit recording, it becomes much easier to detect relatively weak interactions between neurons at different cortical locations.

Phosphene Induction and the Generation of Saccadic Eye Movements by Striate Cortex

The purpose of this review is to critically examine phosphene induction and saccadic eye movement generation by electrical microstimulation of striate cortex (area V1) in humans and monkeys. The following issues are addressed: 1) Properties of electrical stimulation as they pertain to the activation of V1 elements; 2) the induction of phosphenes in sighted and blind human subjects elicited by electrical stimulation using various stimulation parameters and electrode types; 3) the induction of phosphenes with electrical microstimulation of V1 in monkeys; 4) the generation of saccadic eye movements with electrical microstimulation of V1 in monkeys; and 5) the tasks involved for the development of a cortical visual prosthesis for the blind. In this review it is concluded that electrical microstimulation of area V1 in trained monkeys can be used to accelerate the development of an effective prosthetic device for the blind.