Changes in the response rate and response variability of area V4 neurons during the preparation of saccadic eye movements

Neural correlates of perisaccadic visual mislocalization in extrastriate cortex

When interacting with the visual world using saccadic eye movements (saccades), the perceived location of visual stimuli becomes biased, a phenomenon called perisaccadic mislocalization. However, the neural mechanism underlying this altered visuospatial perception and its potential link to other perisaccadic perceptual phenomena have not been established. Using the electrophysiological recording of extrastriate areas in four male macaque monkeys, combined with a computational model, we were able to quantify spatial bias around the saccade target (ST) based on the perisaccadic dynamics of extrastriate spatiotemporal sensitivity captured by a statistical model. This approach could predict the perisaccadic spatial bias around the ST, consistent with behavioral data, and revealed the precise neuronal response components underlying representational bias. These findings also establish the crucial role of increased sensitivity near the ST for neurons with receptive fields far from the ST in driving the ST spatial bias. Moreover, we showed that, by allocating more resources for visual target representation, visual areas enhance their representation of the ST location, even at the expense of transient distortions in spatial representation. This potential neural basis for perisaccadic ST representation also supports a general role for extrastriate neurons in creating the perception of stimulus location.

Coordinated Response Modulations Enable Flexible Use of Visual Information

We use sensory information in remarkably flexible ways. We can generalize by ignoring task-irrelevant features, report different features of a stimulus, and use different actions to report a perceptual judgment. These forms of flexible behavior are associated with small modulations of the responses of sensory neurons. While the existence of these response modulations is indisputable, efforts to understand their function have been largely relegated to theory, where they have been posited to change information coding or enable downstream neurons to read out different visual and cognitive information using flexible weights. Here, we tested these ideas using a rich, flexible behavioral paradigm, multi-neuron, multi-area recordings in primary visual cortex (V1) and mid-level visual area V4. We discovered that those response modulations in V4 (but not V1) contain the ingredients necessary to enable flexible behavior, but not via those previously hypothesized mechanisms. Instead, we demonstrated that these response modulations are precisely coordinated across the population such that downstream neurons have ready access to the correct information to flexibly guide behavior without making changes to information coding or synapses. Our results suggest a novel computational role for task-dependent response modulations: they enable flexible behavior by changing the information that gets out of a sensory area, not by changing information coding within it.

Decision-related activity and movement selection in primate visual cortex

Fluctuations in the activity of sensory neurons often predict perceptual decisions. This connection can be quantified with a metric called choice probability (CP), and there is a longstanding debate about whether CP reflects a causal influence on decisions or an echo of decision-making activity elsewhere in the brain. Here, we show that CP can reflect a third variable, namely, the movement used to indicate the decision. In a standard visual motion discrimination task, neurons in the middle temporal (MT) area of primate cortex responded more strongly during trials that involved a saccade toward their receptive fields. This variability accounted for much of the CP observed across the neuronal population, and it arose through training. Moreover, pharmacological inactivation of MT biased behavioral responses away from the corresponding visual field locations. These results demonstrate that training on a task with fixed sensorimotor contingencies introduces movement-related activity in sensory brain regions and that this plasticity can shape the neural circuitry of perceptual decision-making.

Pre-saccadic Neural Enhancements in Marmoset Area MT

Each time we make an eye movement, attention moves before the eyes, resulting in a perceptual enhancement at the target. Recent psychophysical studies suggest that this pre-saccadic attention enhances the visual features at the saccade target, whereas covert attention causes only spatially selective enhancements. While previous nonhuman primate studies have found that pre-saccadic attention does enhance neural responses spatially, no studies have tested whether changes in neural tuning reflect an automatic feature enhancement. Here we examined pre-saccadic attention using a saccade foraging task developed for marmoset monkeys (one male and one female). We recorded from neurons in the middle temporal area with peripheral receptive fields that contained a motion stimulus, which would either be the target of a saccade or a distracter as a saccade was made to another location. We established that marmosets, like macaques, show enhanced pre-saccadic neural responses for saccades toward the receptive field, including increases in firing rate and motion information. We then examined if the specific changes in neural tuning might support feature enhancements for the target. Neurons exhibited diverse changes in tuning but predominantly showed additive and multiplicative increases that were uniformly applied across motion directions. These findings confirm that marmoset monkeys, like macaques, exhibit pre-saccadic neural enhancements during saccade foraging tasks with minimal training requirements. However, at the level of individual neurons, the lack of feature-tuned enhancements is similar to neural effects reported during covert spatial attention.

Neural correlates of perisaccadic visual mislocalization in extrastriate cortex

When interacting with the visual world using saccadic eye movements (saccades), the perceived location of visual stimuli becomes biased, a phenomenon called perisaccadic mislocalization, which is indeed an exemplar of the brain's dynamic representation of the visual world. However, the neural mechanism underlying this altered visuospatial perception and its potential link to other perisaccadic perceptual phenomena have not been established. Using a combined experimental and computational approach, we were able to quantify spatial bias around the saccade target (ST) based on the perisaccadic dynamics of extrastriate spatiotemporal sensitivity captured by statistical models. This approach could predict the perisaccadic spatial bias around the ST, consistent with the psychophysical studies, and revealed the precise neuronal response components underlying representational bias. These findings also established the crucial role of response remapping toward ST representation for neurons with receptive fields far from the ST in driving the ST spatial bias. Moreover, we showed that, by allocating more resources for visual target representation, visual areas enhance their representation of the ST location, even at the expense of transient distortions in spatial representation. This potential neural basis for perisaccadic ST representation, also supports a general role for extrastriate neurons in creating the perception of stimulus location.

Neural activity in the mouse claustrum in a cross-modal sensory selection task

The claustrum, a subcortical nucleus forming extensive connections with the neocortex, has been implicated in sensory selection. Sensory-evoked claustrum activity is thought to modulate the neocortex's context-dependent response to sensory input. Recording from claustrum neurons while mice performed a tactile-visual sensory-selection task, we found that neurons in the anterior claustrum, including putative optotagged claustrocortical neurons projecting to the primary somatosensory cortex (S1), were rarely modulated by sensory input. Rather, they exhibited different types of direction-tuned motor responses. Furthermore, we found that claustrum neurons encoded upcoming movement during intertrial intervals and that pairs of claustrum neurons exhibiting synchronous firing were enriched for pairs preferring contralateral lick directions, suggesting that the activity of specific ensembles of similarly tuned claustrum neurons may modulate cortical activity. Chemogenetic inhibition of claustrocortical neurons decreased lick responses to inappropriate sensory stimuli. Altogether, our data indicate that the claustrum is integrated into higher-order premotor circuits recently implicated in decision-making.

Variability of neuronal responses in the posterior superior temporal sulcus predicts choice behavior during social interactions

Recent studies have shown that neural activity in a well-defined patch in the posterior superior temporal sulcus (the "gaze-following patch," GFP) of the primate brain is strongly modulated when the other's gaze attracts the observer's attention to locations/objects, the other is looking at. Changes of the mean discharge rate of neurons in the monkey GFP indicate that they are involved in two distinct computations: the allocation of spatial attention guided by the other's gaze vector and the suppression of gaze following if inappropriate in a given situation. Here, we asked if and how the discharge variability of neurons in the GFP is related to the task and if it carries information on behavioral performance. To this end, we calculated the Fano factor as a measure of across-trial discharge variability as a function of time. Our results show that all neurons exhibiting a task-related discharge-rate modulation also exhibit a stimulus onset-dependent drop in the Fano factor. Furthermore, the amplitude of the Fano factor reduction is modulated by task condition and the neuron's selectivity in this regard. We found that these effects are directly related to the monkeys' behavioral performance in that the Fano factor is predictive about upcoming correct or wrong decisions. Our results indicate that neuronal discharge variability as gauged by the Fano factor, hitherto primarily studied in the context of visual perception or motor control, is an informative measure also in studies of the neural underpinnings of complex social behavior.NEW & NOTEWORTHY Quenching of neural variability following stimulus onset is a widely accepted phenomenon. However, the relevance of quenching for the shaping of complex social behaviors remains to be explored. Here, we show that task selective neurons in the GFP exhibit a higher degree of variability quenching than their neighboring unselective neurons. Furthermore, we demonstrate that behavioral errors are not only associated with lower firing rates but also less variability quenching, suggesting that both facilitate optimal performance.

To look or not to look: dissociating presaccadic and covert spatial attention

Attention is a central neural process that enables selective and efficient processing of visual information. Individuals can attend to specific visual information either overtly, by making an eye movement to an object of interest, or covertly, without moving their eyes. We review behavioral, neuropsychological, neurophysiological, and computational evidence of presaccadic attentional modulations that occur while preparing saccadic eye movements, and highlight their differences from those of covert spatial endogenous (voluntary) and exogenous (involuntary) attention. We discuss recent studies and experimental procedures on how these different types of attention impact visual performance, alter appearance, differentially modulate the featural representation of basic visual dimensions (orientation and spatial frequency), engage different neural computations, and recruit partially distinct neural substrates. We conclude that presaccadic attention and covert attention are dissociable.

Sensorimotor strategies and neuronal representations for shape discrimination

Humans and other animals can identify objects by active touch, requiring the coordination of exploratory motion and tactile sensation. Both the motor strategies and neural representations employed could depend on the subject's goals. We developed a shape discrimination task that challenged head-fixed mice to discriminate concave from convex shapes. Behavioral decoding revealed that mice did this by comparing contacts across whiskers. In contrast, a separate group of mice performing a shape detection task simply summed up contacts over whiskers. We recorded populations of neurons in the barrel cortex, which processes whisker input, and found that individual neurons across the cortical layers encoded touch, whisker motion, and task-related signals. Sensory representations were task-specific: during shape discrimination, but not detection, neurons responded most to behaviorally relevant whiskers, overriding somatotopy. Thus, sensory cortex employs task-specific representations compatible with behaviorally relevant computations.

Movement preparation time determines movement variability

Faster movements are typically more variable-a speed-accuracy trade-off known as Fitts' law. Are movements that are initiated faster also more variable? Neurophysiological work has associated larger neural variability during motor preparation with longer reaction time (RT) and larger movement variability, implying that movement variability decreases with increasing RT. Here, we recorded over 30,000 reaching movements in 11 human participants who moved to visually cued targets. Half of the visual cues were accompanied by a beep to evoke a wide RT range in each participant. Results show that initial reach variability decreases with increasing RT, for voluntarily produced RTs up to ∼300 ms, whereas other kinematic aspects and endpoint accuracy remained unaffected. We conclude that movement preparation time determines initial movement variability. We suggest that the chosen movement preparation time reflects a trade-off between movement initiation and precision.NEW & NOTEWORTHY Fitts' law describes the speed-accuracy trade-off in the execution of human movements. We examined whether there is also a trade-off between movement planning time and initial movement precision. We show that shorter reaction times result in higher initial movement variability. In other words, movement preparation time determines movement variability.

Differential Contributions of Inhibitory Subnetwork to Visual Cortical Modulations Identified via Computational Model of Working Memory

Extrastriate visual neurons show no firing rate change during a working memory (WM) task in the absence of sensory input, but both αβ oscillations and spike phase locking are enhanced, as is the gain of sensory responses. This lack of change in firing rate is at odds with many models of WM, or attentional modulation of sensory networks. In this article we devised a computational model in which this constellation of results can be accounted for via selective activation of inhibitory subnetworks by a top-down working memory signal. We confirmed the model prediction of selective inhibitory activation by segmenting cells in the experimental neural data into putative excitatory and inhibitory cells. We further found that this inhibitory activation plays a dual role in influencing excitatory cells: it both modulates the inhibitory tone of the network, which underlies the enhanced sensory gain, and also produces strong spike-phase entrainment to emergent network oscillations. Using a phase oscillator model we were able to show that inhibitory tone is principally modulated through inhibitory network gain saturation, while the phase-dependent efficacy of inhibitory currents drives the phase locking modulation. The dual contributions of the inhibitory subnetwork to oscillatory and non-oscillatory modulations of neural activity provides two distinct ways for WM to recruit sensory areas, and has relevance to theories of cortical communication.

Different computations underlie overt presaccadic and covert spatial attention

Perception and action are tightly coupled: visual responses at the saccade target are enhanced right before saccade onset. This phenomenon, presaccadic attention, is a form of overt attention-deployment of visual attention with concurrent eye movements. Presaccadic attention is well-documented, but its underlying computational process remains unknown. This is in stark contrast to covert attention-deployment of visual attention without concurrent eye movements-for which the computational processes are well characterized by a normalization model. Here, a series of psychophysical experiments reveal that presaccadic attention modulates visual performance only via response gain changes. A response gain change was observed even when attention field size increased, violating the predictions of a normalization model of attention. Our empirical results and model comparisons reveal that the perceptual modulations by overt presaccadic and covert spatial attention are mediated through different computations.

Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Assessing within-trial and across-trial neural variability in macaque frontal eye fields and their relation to behaviour

The conventional approach to understanding neural responses underlying complex computations is to study across-trial averages of repeatedly performed computations from single neurons. When neurons perform complex computations, such as processing stimulus-related information or movement planning, it has been repeatedly shown, through measures such as the Fano factor (FF), that neural variability across trials decreases. However, multiple neurons contribute to a common computation on a single trial, rather than a single neuron contributing to a computation across multiple trials. Therefore, at the level of a single trial, the concept of FF loses significance. Here, using a combination of simulations and empirical data, we show that changes in the spiking regularity on single trials produce changes in FF. Further, at the behavioural level, the reaction time of the animal was faster when the neural spiking regularity both within and across trials was lower. Taken together, our results provide further constraints on how changes in spiking statistics help neurons optimally encode visual and saccade-related information across multiple timescales and its implication on behaviour.

Timing of response onset and offset in macaque V4: stimulus and task dependence

In the primate visual cortex, both the magnitude of the neuronal response and its timing can carry important information about the visual world, but studies typically focus only on response magnitude. Here, we examine the onset and offset latency of the responses of neurons in area V4 of awake, behaving macaques across several experiments in the context of a variety of stimuli and task paradigms. Our results highlight distinct contributions of stimuli and tasks to V4 response latency. We found that response onset latencies are shorter than typically cited (median = 75.5 ms), supporting a role for V4 neurons in rapid object and scene recognition functions. Moreover, onset latencies are longer for smaller stimuli and stimulus outlines, consistent with the hypothesis that longer latencies are associated with higher spatial frequency content. Strikingly, we found that onset latencies showed no significant dependence on stimulus occlusion, unlike in inferotemporal cortex, nor on task demands. Across the V4 population, onset latencies had a broad distribution, reflecting the diversity of feedforward, recurrent, and feedback connections that inform the responses of individual neurons. Response offset latencies, on the other hand, displayed the opposite tendency in their relationship to stimulus and task attributes: they are less influenced by stimulus appearance but are shorter in guided saccade tasks compared with fixation tasks. The observation that response latency is influenced by stimulus- and task-associated factors emphasizes a need to examine response timing alongside firing rate in determining the functional role of area V4.NEW & NOTEWORTHY Onset and offset timing of neuronal responses can provide information about visual environment and neuron's role in visual processing and its anatomical connectivity. In the first comprehensive examination of onset and offset latencies in the intermediate visual cortical area V4, we find neurons respond faster than previously reported, making them ideally suited to contribute to rapid object and scene recognition. While response onset reflects stimulus characteristics, timing of response offset is influenced more by behavioral task.

Neuromodulation of Brain State and Behavior

Neural activity and behavior are both notoriously variable, with responses differing widely between repeated presentation of identical stimuli or trials. Recent results in humans and animals reveal that these variations are not random in their nature, but may in fact be due in large part to rapid shifts in neural, cognitive, and behavioral states. Here we review recent advances in the understanding of rapid variations in the waking state, how variations are generated, and how they modulate neural and behavioral responses in both mice and humans. We propose that the brain has an identifiable set of states through which it wanders continuously in a nonrandom fashion, owing to the activity of both ascending modulatory and fast-acting corticocortical and subcortical-cortical neural pathways. These state variations provide the backdrop upon which the brain operates, and understanding them is critical to making progress in revealing the neural mechanisms underlying cognition and behavior.

Predicting Perceptual Decisions Using Visual Cortical Population Responses and Choice History

Our understanding of the neural basis of perceptual decision making has been built in part on relating co-fluctuations of single neuron responses to perceptual decisions on a trial-by-trial basis. The strength of this relationship is often compared across neurons or brain areas, recorded in different sessions, animals, or variants of a task. We sought to extend our understanding of perceptual decision making in three ways. First, we measured neuronal activity simultaneously in early [primary visual cortex (V1)] and midlevel (V4) visual cortex while macaque monkeys performed a fine orientation discrimination perceptual task. This allowed a direct comparison of choice signals in these two areas, including their dynamics. Second, we asked how our ability to predict animals' decisions would be improved by considering small simultaneously-recorded neuronal populations rather than individual units. Finally, we asked whether predictions would be improved by taking into account the animals' choice and reward histories, which can strongly influence decision making. We found that responses of individual V4 neurons were weakly predictive of decisions, but only in a brief epoch between stimulus offset and the indication of choice. In V1, few neurons showed significant decision-related activity. Analysis of neuronal population responses revealed robust choice-related information in V4 and substantially weaker signals in V1. Including choice- and reward-history information improved performance further, particularly when the recorded populations contained little decision-related information. Our work shows the power of using neuronal populations and decision history when relating neuronal responses to the perceptual decisions they are thought to underlie.SIGNIFICANCE STATEMENT Decades of research has provided a rich description of how visual information is represented in the visual cortex. Yet how cortical responses relate to visual perception remains poorly understood. Here we relate fluctuations in small neuronal population responses, recorded simultaneously in primary visual cortex (V1) and area V4 of monkeys, to perceptual reports in an orientation discrimination task. Choice-related signals were robust in V4, particularly late in the behavioral trial, but not in V1. Models that include both neuronal responses and choice-history information were able to predict a substantial portion of decisions. Our work shows the power of integrating information across neurons and including decision history in relating neuronal responses to perceptual decisions.

The Magnitude of Trial-By-Trial Neural Variability Is Reproducible over Time and across Tasks in Humans

Numerous studies have shown that neural activity in sensory cortices is remarkably variable over time and across trials even when subjects are presented with an identical repeating stimulus or task. This trial-by-trial neural variability is relatively large in the prestimulus period and considerably smaller (quenched) following stimulus presentation. Previous studies have suggested that the magnitude of neural variability affects behavior such that perceptual performance is better on trials and in individuals where variability quenching is larger. To what degree are neural variability magnitudes of individual subjects flexible or static? Here, we used EEG recordings from adult humans to demonstrate that neural variability magnitudes in visual cortex are remarkably consistent across different tasks and recording sessions. While magnitudes of neural variability differed dramatically across individual subjects, they were surprisingly stable across four tasks with different stimuli, temporal structures, and attentional/cognitive demands as well as across experimental sessions separated by one year. These experiments reveal that, in adults, neural variability magnitudes are mostly solidified individual characteristics that change little with task or time, and are likely to predispose individual subjects to exhibit distinct behavioral capabilities.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Firing Variability of Frontal Pole Neurons during a Cued Strategy Task

In previous reports, we described neuronal activity in the polar (PFp), dorsolateral (PFdl), and orbital (PFo) PFC as monkeys performed a cued strategy task with two spatial goals. On each trial, a cue instructed one of two strategies: Stay with the previous goal or shift to the alternative. A delay period followed each cue, and feedback followed each choice, also at a delay. Our initial analysis showed that the mean firing rate of a population of PFp cells encoded the goal chosen on a trial, but only near the time of feedback, not earlier in the trial. In contrast, PFdl cells encoded goals and strategies during the cue and delay periods, and PFo cells encoded strategies in those task periods. Both areas also signaled goals near feedback time. Here we analyzed trial-to-trial variability of neuronal firing, as measured by the Fano factor (FF): the ratio of variance to the mean. Goal-selective PFp neurons had two properties: (1) a lower FF from the beginning of the trial compared with PFp cells that did not encode goals and (2) a weak but significant inverse correlation between FF throughout a trial and the degree of goal selectivity at feedback time. Cells in PFdl and PFo showed neither of these properties. Our findings indicate that goal-selective PFp neurons were engaged in the task throughout a trial, although they only encoded goals near feedback time. Their lower FF could improve the ability of other cortical areas to decode its selected-goal signal.

Selective modulation of cortical state during spatial attention

Neocortical activity is permeated with endogenously generated fluctuations, but how these dynamics affect goal-directed behavior remains a mystery. We found that ensemble neural activity in primate visual cortex spontaneously fluctuated between phases of vigorous (On) and faint (Off) spiking synchronously across cortical layers. These On-Off dynamics, reflecting global changes in cortical state, were also modulated at a local scale during selective attention. Moreover, the momentary phase of local ensemble activity predicted behavioral performance. Our results show that cortical state is controlled locally within a cortical map according to cognitive demands and reveal the impact of these local changes in cortical state on goal-directed behavior.

An Eye in the Palm of Your Hand: Alterations in Visual Processing Near the Hand, a Mini-Review

Feedback within the oculomotor system improves visual processing at eye movement end points, also termed a visual grasp. We do not just view the world around us however, we also reach out and grab things with our hands. A growing body of literature suggests that visual processing in near-hand space is altered. The control systems for moving either the eyes or the hands rely on parallel networks of fronto-parietal regions, which have feedback connections to visual areas. Since the oculomotor system effects on visual processing occur through feedback, both through the motor plan and the motor efference copy, a parallel system where reaching and/or grasping motor-related activity also affects visual processing is likely. Areas in the posterior parietal cortex, for example, receive proprioceptive and visual information used to guide actions, as well as motor efference signals. This trio of information channels is all that would be necessary to produce spatial allocation of reach-related visual attention. We review evidence from behavioral and neurophysiological studies that support the hypothesis that feedback from the reaching and/or grasping motor control networks affects visual processing while noting ways in which it differs from that seen within the oculomotor system. We also suggest that object affordances may represent the neural mechanism through which certain object features are selected for preferential processing when stimuli are near the hand. Finally, we summarize the two effector-based feedback systems and discuss how having separate but parallel effector systems allows for efficient decoupling of eye and hand movements.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron's spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Enhanced sensory sampling precedes self-initiated locomotion in an electric fish

Cortical activity precedes self-initiated movements by several seconds in mammals; this observation has led into inquiries on the nature of volition. Preparatory neural activity is known to be associated with decision making and movement planning. Self-initiated locomotion has been linked to increased active sensory sampling; however, the precise temporal relationship between sensory acquisition and voluntary movement initiation has not been established. Based on long-term monitoring of sensory sampling activity that is readily observable in freely behaving pulse-type electric fish, we show that heightened sensory acquisition precedes spontaneous initiation of swimming. Gymnotus sp. revealed a bimodal distribution of electric organ discharge rate (EODR) demonstrating down- and up-states of sensory sampling and neural activity; movements only occurred during up-states and up-states were initiated before movement onset. EODR during voluntary swimming initiation exhibited greater trial-to-trial variability than the sound-evoked increases in EODR. The sampling variability declined after voluntary movement onset as previously observed for the neural variability associated with decision making in primates. Spontaneous movements occurred randomly without a characteristic timescale, and no significant temporal correlation was found between successive movement intervals. Using statistical analyses of spontaneous exploratory behaviours and associated preparatory sensory sampling increase, we conclude that electric fish exhibit key attributes of volitional movements, and that voluntary behaviours in vertebrates may generally be preceded by increased sensory sampling. Our results suggest that comparative studies of the neural basis of volition may therefore be possible in pulse-type electric fish, given the substantial homologies between the telencephali of teleost fish and mammals.

Visual attention: Linking prefrontal sources to neuronal and behavioral correlates

Attention is a means of flexibly selecting and enhancing a subset of sensory input based on the current behavioral goals. Numerous signatures of attention have been identified throughout the brain, and now experimenters are seeking to determine which of these signatures are causally related to the behavioral benefits of attention, and the source of these modulations within the brain. Here, we review the neural signatures of attention throughout the brain, their theoretical benefits for visual processing, and their experimental correlations with behavioral performance. We discuss the importance of measuring cue benefits as a way to distinguish between impairments on an attention task, which may instead be visual or motor impairments, and true attentional deficits. We examine evidence for various areas proposed as sources of attentional modulation within the brain, with a focus on the prefrontal cortex. Lastly, we look at studies that aim to link sources of attention to its neuronal signatures elsewhere in the brain.

Saccade execution suppresses discrimination at distractor locations rather than enhancing the saccade goal location

As we have limited processing abilities with respect to the plethora of visual information entering our brain, spatial selection mechanisms are crucial. These mechanisms result in both enhancing processing at a location of interest and in suppressing processing at other locations; together, they enable successful further processing of locations of interest. It has been suggested that saccade planning modulates these spatial selection mechanisms; however, the precise influence of saccades on the distribution of spatial resources underlying selection remains unclear. To this end, we compared discrimination performance at different locations (six) within a work space during different saccade tasks. We used visual discrimination performance as a behavioral measure of enhancement and suppression at the different locations. A total of 14 participants performed a dual discrimination/saccade countermanding task, which allowed us to specifically isolate the consequences of saccade execution. When a saccade was executed, discrimination performance at the cued location was never better than when fixation was maintained, suggesting that saccade execution did not enhance processing at a location more than knowing the likelihood of its appearance. However, discrimination was consistently lower at distractor (uncued) locations in all cases where a saccade was executed compared with when fixation was maintained. Based on these results, we suggest that saccade execution specifically suppresses distractor locations, whereas attention shifts (with or without an accompanying saccade) are involved in enhancing perceptual processing at the goal location.

Hand placement near the visual stimulus improves orientation selectivity in V2 neurons

Often, the brain receives more sensory input than it can process simultaneously. Spatial attention helps overcome this limitation by preferentially processing input from a behaviorally-relevant location. Recent neuropsychological and psychophysical studies suggest that attention is deployed to near-hand space much like how the oculomotor system can deploy attention to an upcoming gaze position. Here we provide the first neuronal evidence that the presence of a nearby hand enhances orientation selectivity in early visual processing area V2. When the hand was placed outside the receptive field, responses to the preferred orientation were significantly enhanced without a corresponding significant increase at the orthogonal orientation. Consequently, there was also a significant sharpening of orientation tuning. In addition, the presence of the hand reduced neuronal response variability. These results indicate that attention is automatically deployed to the space around a hand, improving orientation selectivity. Importantly, this appears to be optimal for motor control of the hand, as opposed to oculomotor mechanisms which enhance responses without sharpening orientation selectivity. Effector-based mechanisms for visual enhancement thus support not only the spatiotemporal dissociation of gaze and reach, but also the optimization of vision for their separate requirements for guiding movements.

Saccades and shifting receptive fields: anticipating consequences or selecting targets?

Saccadic eye movements cause frequent and substantial displacements of the retinal image, but those displacements go unnoticed. It has been widely assumed that this perceived stability emerges from the shifting of visual receptive fields from their current, presaccadic locations to their future, postsaccadic locations in anticipation of the retinal consequences of saccades. Although evidence consistent with this anticipatory remapping has accumulated over the years, more recent work suggests an alternative view. In this opinion article, we examine the evidence of presaccadic receptive field shifts and their relationship to the perceptual changes that accompany saccades. We argue that both reflect the selection of targets for saccades rather than the anticipation of a displaced retinal image.

Motor preparation attenuates neural variability and beta-band LFP in parietal cortex

Accumulative evidence shows that neural variability is meaningful and powerful during brain information processing, but how cognitive state influences neural variability is still unclear. We studied neural variability during motor preparation in lateral intraparietal area (LIP), a brain area closely involved in saccade generation. During motor preparation, we observed significant variability decline, and the decline highly correlated with beta-band local field potential (LFP) fluctuations. Furthermore, we found similar variance-LFP correlations in both the memory-guided saccade task and the visually-guided saccade task. These results indicate a possible linkage between beta-band LFP and trial-to-trial neural variability.

Rapid shape detection signals in area V4

Vision in foveate animals is an active process that requires rapid and constant decision-making. For example, when a new object appears in the visual field, we can quickly decide to inspect it by directing our eyes to the object's location. We studied the contribution of primate area V4 to these types of rapid foveation decisions. Animals performed a reaction time task that required them to report when any shape appeared within a peripherally-located noisy stimulus by making a saccade to the stimulus location. We found that about half of the randomly sampled V4 neurons not only rapidly and precisely represented the appearance of this shape, but they were also predictive of the animal's saccades. A neuron's ability to predict the animal's saccades was not related to the specificity with which the cell represented a single type of shape but rather to its ability to signal whether any shape was present. This relationship between sensory sensitivity and behavioral predictiveness was not due to global effects such as alertness, as it was equally likely to be observed for cells with increases and decreases in firing rate. Careful analysis of the timescales of reliability in these neurons implies that they reflect both feedforward and feedback shape detecting processes. In approximately 7% of our recorded sample, individual neurons were able to predict both the delay and precision of the animal's shape detection performance. This suggests that a subset of V4 neurons may have been directly and causally contributing to task performance and that area V4 likely plays a critical role in guiding rapid, form-based foveation decisions.

Eye movement preparation modulates neuronal responses in area V4 when dissociated from attentional demands

We examined whether the preparation of saccadic eye movements, when behaviorally dissociated from covert attention, modulates activity within visual cortex. We measured single-neuron and local field potential (LFP) responses to visual stimuli in area V4 while monkeys covertly attended a stimulus at one location and prepared saccades to a potential target at another. In spite of the irrelevance of visual information at the saccade target, visual activity at that location was modulated at least as much as, and often more than, activity at the covertly attended location. Modulations of activity at the attended and saccade target locations were qualitatively similar and included increased response magnitude, stimulus selectivity, and spiking reliability, as well as increased gamma and decreased low-frequency power of LFPs. These results demonstrate that saccade preparation is sufficient to modulate visual cortical representations and suggest that the interrelationship of oculomotor and attention-related mechanisms extends to posterior visual cortex.

A distinct contribution of the frontal eye field to the visual representation of saccadic targets

The responses of neurons within posterior visual cortex are enhanced when response field (RF) stimuli are targeted with saccadic eye movements. Although the motor-related activity within oculomotor structures seems a likely source of the enhancement, the origin of the modulation is unknown. We tested the role of the frontal eye field (FEF) in driving presaccadic modulation in area V4 by inactivating FEF neurons at retinotopically corresponding sites within the macaque monkey (Macaca mulatta) brain. As previously observed, FEF inactivation produced profound, and spatially specific, deficits in memory-guided saccades, and increased the latency, scatter, and duration of visually guided saccades. Despite the clear behavioral deficits, we found that rather than being eliminated or reduced by FEF inactivation, presaccadic enhancement of V4 activity was increased. FEF inactivation nonetheless diminished the stimulus discriminability of V4 visual responses both during fixation and in the presaccadic period. Thus, without input from the FEF, V4 neurons signaled more about the direction of saccades and less about the features of the saccadic target. In addition, FEF inactivation significantly increased the suppressive effects of non-RF stimuli on V4 activity. These results reveal multiple sources of presaccadic modulation in V4 and suggest that the FEF contributes uniquely to the presaccadic specification of visual target features.

Spatial representation and cognitive modulation of response variability in the lateral intraparietal area priority map

The lateral intraparietal area (LIP) in the macaque contains a priority-based representation of the visual scene. We previously showed that the mean spike rate of LIP neurons is strongly influenced by spatially wide-ranging surround suppression in a manner that effectively sharpens the priority map. Reducing response variability can also improve the precision of LIP's priority map. We show that when a monkey plans a visually guided delayed saccade with an intervening distractor, variability (measured by the Fano factor) decreases both for neurons representing the saccade goal and for neurons representing the broad spatial surround. The reduction in Fano factor is maximal for neurons representing the saccade goal and steadily decreases for neurons representing more distant locations. LIP Fano factor changes are behaviorally significant: increasing expected reward leads to lower variability for the LIP representation of both the target and distractor locations, and trials with shorter latency saccades are associated with lower Fano factors in neurons representing the surround. Thus, the LIP Fano factor reflects both stimulus and behavioral engagement. Quantitative modeling shows that the interaction between mean spike count and target-receptive field (RF) distance in the surround during the predistractor epoch is multiplicative: the Fano factor increases more steeply with mean spike count further away from the RF. A negative-binomial model for LIP spike counts captures these findings quantitatively, suggests underlying mechanisms based on trial-by-trial variations in mean spike rate or burst-firing patterns, and potentially provides a principled framework to account simultaneously for the previously observed unsystematic relationships between spike rate and variability in different brain areas.

Average is optimal: an inverted-U relationship between trial-to-trial brain activity and behavioral performance

It is well known that even under identical task conditions, there is a tremendous amount of trial-to-trial variability in both brain activity and behavioral output. Thus far the vast majority of event-related potential (ERP) studies investigating the relationship between trial-to-trial fluctuations in brain activity and behavioral performance have only tested a monotonic relationship between them. However, it was recently found that across-trial variability can correlate with behavioral performance independent of trial-averaged activity. This finding predicts a U- or inverted-U- shaped relationship between trial-to-trial brain activity and behavioral output, depending on whether larger brain variability is associated with better or worse behavior, respectively. Using a visual stimulus detection task, we provide evidence from human electrocorticography (ECoG) for an inverted-U brain-behavior relationship: When the raw fluctuation in broadband ECoG activity is closer to the across-trial mean, hit rate is higher and reaction times faster. Importantly, we show that this relationship is present not only in the post-stimulus task-evoked brain activity, but also in the pre-stimulus spontaneous brain activity, suggesting anticipatory brain dynamics. Our findings are consistent with the presence of stochastic noise in the brain. They further support attractor network theories, which postulate that the brain settles into a more confined state space under task performance, and proximity to the targeted trajectory is associated with better performance.

The human brain is notoriously “noisy”. Even with identical physical sensory inputs and task demands, brain responses and behavioral output vary tremendously from trial to trial. Such brain and behavioral variability and the relationship between them have been the focus of intense neuroscience research for decades. Traditionally, it is thought that the relationship between trial-to-trial brain activity and behavioral performance is monotonic: the highest or lowest brain activity levels are associated with the best behavioral performance. Using invasive recordings in neurosurgical patients, we demonstrate an inverted-U relationship between brain and behavioral variability. Under such a relationship, moderate brain activity is associated with the best performance, while both very low and very high brain activity levels are predictive of compromised performance. These results have significant implications for our understanding of brain functioning. They further support recent theoretical frameworks that view the brain as an active nonlinear dynamical system instead of a passive signal-processing device.

Variability of human corticospinal excitability tracks the state of action preparation

Task-evoked trial-by-trial variability is a ubiquitous property of neural responses, yet its functional role remains largely unclear. Recent work in nonhuman primates shows that the temporal structure of neural variability in several brain regions is task-related. For example, trial-by-trial variability in premotor cortex tracks motor preparation with increasingly consistent firing rates and thus a decline in variability before movement onset. However, whether noninvasive measures of the variability of population activity available from humans can similarly track the preparation of actions remains unknown. We tested this by using single-pulse transcranial magnetic stimulation (TMS) over primary motor cortex (M1) to measure corticospinal excitability (CSE) at different times during action preparation. First, we established the basic properties of intrinsic CSE variability at rest. Then, during the task, responses (left or right button presses) were either directly instructed (forced choice) or resulted from a value decision (choice). Before movement onset, we observed a temporally specific task-related decline in CSE variability contralateral to the responding hand. This decline was stronger in fast-response compared with slow-response trials, consistent with data in nonhuman primates. For the nonresponding hand, CSE variability also decreased, but only in choice trials, and earlier compared with the responding hand, possibly reflecting choice-specific suppression of unselected actions. These findings suggest that human CSE variability measured by TMS over M1 tracks the state of motor preparation, and may reflect the optimization of preparatory population activity. This provides novel avenues in humans to assess the dynamics of action preparation but also more complex processes, such as choice-to-action transformations.

Response variability of frontal eye field neurons modulates with sensory input and saccade preparation but not visual search salience

Discharge rate modulation of frontal eye field (FEF) neurons has been identified with a representation of visual search salience (physical conspicuity and behavioral relevance) and saccade preparation. We tested whether salience or saccade preparation are evident in the trial-to-trial variability of discharge rate. We quantified response variability via the Fano factor in FEF neurons recorded in monkeys performing efficient and inefficient visual search tasks. Response variability declined following stimulus presentation in most neurons, but despite clear discharge rate modulation, variability did not change with target salience. Instead, we found that response variability was modulated by stimulus luminance and the number of items in the visual field independently of attentional demands. Response variability declined to a minimum before saccade initiation, and presaccadic response variability was directionally tuned. In addition, response variability was correlated with the response time of memory-guided saccades. These results indicate that the trial-by-trial response variability of FEF neurons reflects saccade preparation and the strength of sensory input, but not visual search salience or attentional allocation.

Neuronal variability of MSTd neurons changes differentially with eye movement and visually related variables

Neurons in macaque cortical area MSTd are driven by visual motion and eye movement related signals. This multimodal characteristic makes MSTd an ideal system for studying the dependence of neuronal activity on different variables. Here, we analyzed the temporal structure of spiking patterns during visual motion stimulation using 2 distinct behavioral paradigms: fixation (FIX) and optokinetic response. For the FIX condition, inter- and intra-trial variability of spiking activity decreased with increasing stimulus strength, complying with a recent neurophysiological study reporting stimulus-related decline of neuronal variability. In contrast, for the optokinetic condition variability increased together with increasing eye velocity while retinal image velocity remained low. Analysis of stimulus signal variability revealed a correlation between the normalized variance of image velocity and neuronal variability, but no correlation with normalized eye velocity variance. We further show that the observed difference in neuronal variability allows classifying spike trains according to the paradigm used, even when mean firing rates (FRs) were similar. The stimulus-dependence of neuronal variability may result from the local network structure and/or the variability characteristics of the input signals, but may also reflect additional timing-based mechanisms independent of the neuron's mean FR and related to the modality driving the neuron.

Dissociation of response variability from firing rate effects in frontal eye field neurons during visual stimulation, working memory, and attention

Recent studies suggest that trial-to-trial variability of neuronal spiking responses may provide important information about behavioral state. Observed changes in variability during sensory stimulation, attention, motor preparation, and visual discrimination suggest that variability may reflect the engagement of neurons in a behavioral task. We examined changes in spiking variability of frontal eye field (FEF) neurons in a change detection task requiring monkeys to remember a visually cued location and direct attention to that location while ignoring distracters elsewhere. In this task, the firing rates (FRs) of FEF neurons not only continuously reflect the location of the remembered cue and select targets, but also predict detection performance on a trial-by-trial basis. Changes in FEF response variability, as measured by the Fano factor (FF), showed clear dissociations from changes in FR. The FF declined in response to visual stimulation at all tested locations, even in the opposite hemifield, indicating much broader spatial tuning of the FF compared with the FR. Furthermore, despite robust spatial modulation of the FR throughout all epochs of the task, spatial tuning of the FF did not persist throughout the delay period, nor did it show attentional modulation. These results indicate that changes in variability, at least in the FEF, are most effectively driven by visual stimulation, while behavioral engagement is not sufficient. Instead, changes in variability may reflect shifts in the balance between feedforward and recurrent sources of excitatory drive.

A cortical substrate for memory-guided orienting in the rat

Anatomical, stimulation, and lesion data have suggested a homology between the rat frontal orienting fields (FOF) (centered at +2 AP, ±1.3 ML mm from Bregma) and primate frontal cortices such as the frontal or supplementary eye fields. We investigated the functional role of the FOF using rats trained to perform a memory-guided orienting task, in which there was a delay period between the end of a sensory stimulus instructing orienting direction and the time of the allowed motor response. Unilateral inactivation of the FOF resulted in impaired contralateral responses. Extracellular recordings of single units revealed that 37% of FOF neurons had delay period firing rates that predicted the direction of the rats' later orienting motion. Our data provide the first electrophysiological and pharmacological evidence supporting the existence in the rat, as in the primate, of a frontal cortical area involved in the preparation and/or planning of orienting responses.

Practicing coarse orientation discrimination improves orientation signals in macaque cortical area v4

Practice improves the performance in visual tasks, but mechanisms underlying this adult brain plasticity are unclear. Single-cell studies reported no [1], weak [2], or moderate [3, 4] perceptual learning-related changes in macaque visual areas V1 and V4, whereas none were found in middle temporal (MT) [5]. These conflicting results and modeling of human (e.g., [6, 7]) and monkey data [8] suggested that changes in the readout of visual cortical signals underlie perceptual learning, rather than changes in these signals. In the V4 learning studies, monkeys discriminated small differences in orientation, whereas in the MT study, the animals discriminated opponent motion directions. Analogous to the latter study, we trained monkeys to discriminate static orthogonal orientations masked by noise. V4 neurons showed robust increases in their capacity to discriminate the trained orientations during the course of the training. This effect was observed during discrimination and passive fixation but specifically for the trained orientations. The improvement in neural discrimination was due to decreased response variability and an increase of the difference between the mean responses for the two trained orientations. These findings demonstrate that perceptual learning in a coarse discrimination task indeed can change the response properties of a cortical sensory area.

A dynamical systems view of motor preparation: implications for neural prosthetic system design

Neural prosthetic systems aim to help disabled patients suffering from a range of neurological injuries and disease by using neural activity from the brain to directly control assistive devices. This approach in effect bypasses the dysfunctional neural circuitry, such as an injured spinal cord. To do so, neural prostheses depend critically on a scientific understanding of the neural activity that drives them. We review here several recent studies aimed at understanding the neural processes in premotor cortex that precede arm movements and lead to the initiation of movement. These studies were motivated by hypotheses and predictions conceived of within a dynamical systems perspective. This perspective concentrates on describing the neural state using as few degrees of freedom as possible and on inferring the rules that govern the motion of that neural state. Although quite general, this perspective has led to a number of specific predictions that have been addressed experimentally. It is hoped that the resulting picture of the dynamical role of preparatory and movement-related neural activity will be particularly helpful to the development of neural prostheses, which can themselves be viewed as dynamical systems under the control of the larger dynamical system to which they are attached.

Trial-to-trial variability of the prefrontal neurons reveals the nature of their engagement in a motion discrimination task

During motion discrimination tasks, many prefrontal cortex (PFC) neurons are strongly modulated by the behavioral context, suggesting their involvement in sensory discriminations. Recent studies suggest that trial-to-trial variability of spiking activity characteristic of cortical neurons could be a source of information about the state of neurons and their participation in behavioral tasks. We tested this hypothesis by examining the variability of putative pyramidal PFC neurons, a likely source of top-down influences. The variability of these neurons was calculated as a ratio of spike count variance to its mean (fano factor, FF), while monkeys compared the directions of two moving stimuli, sample and test, separated by a delay. We found that the FF tracked consecutive components of the task, dropping rapidly with the onset of stimuli being discriminated and declining more slowly before each salient event of the trial: The sample, the test, and the response. These time-dependent signals were less consistent in direction selective neurons and were largely absent during passive fixation. Furthermore, neurons with test responses that reflected the remembered sample decreased their FF well before the test, revealing the predictive nature of response variability, an effect present only during the active task. The FF was also sensitive to behavioral performance, exhibiting different temporal dynamics on error trials. These changes did not depend on firing rates and were often the only metric correlated with task demands. Our results demonstrate that trial-to-trial variability provides a sensitive measure of the engagement of putative pyramidal PFC neurons in circuits subserving discrimination tasks.