Attentional modulation of visual responses by flexible input gain

Attention Selectively Gates Afferent Signal Transmission to Area V4

Selective attention allows focusing on only part of the incoming sensory information. Neurons in the extrastriate visual cortex reflect such selective processing when different stimuli are simultaneously present in their large receptive fields. Their spiking response then resembles the response to the attended stimulus when presented in isolation. Unclear is where in the neuronal pathway attention intervenes to achieve such selective signal routing and processing. To investigate this question, we tagged two equivalent visual stimuli by independent broadband luminance noise and used the spectral coherence of these behaviorally irrelevant signals with the field potential of a local neuronal population in male macaque monkeys' area V4 as a measure for their respective causal influences. This new experimental paradigm revealed that signal transmission was considerably weaker for the not-attended stimulus. Furthermore, our results show that attention does not need to modulate responses in the input populations sending signals to V4 to selectively represent a stimulus, nor do they suggest a change of the V4 neurons' output gain depending on their feature similarity with the stimuli. Our results rather imply that selective attention uses a gating mechanism comprising the synaptic "inputs" that transmit signals from upstream areas into the V4 neurons. A minimal model implementing attention-dependent routing by gamma-band synchrony replicated the attentional gating effect and the signals' spectral transfer characteristics. It supports the proposal that selective interareal gamma-band synchrony subserves signal routing and explains our experimental finding that attention selectively gates signals already at the level of afferent synaptic input.SIGNIFICANCE STATEMENT Depending on the behavioral context, the brain needs to channel the flow of information through its networks of massively interconnected neurons. We designed an experiment that allows to causally assess routing of information originating from an attended object. We found that attention "gates" signals at the interplay between afferent fibers and the local neurons. A minimal model demonstrated that coherent gamma-rhythmic activity (∼60 Hz) between local neurons and their afferent-providing input neurons can realize the gating. Importantly, the attended signals did not need to be amplified already in an earlier processing stage, nor did they get amplified by a simple output response modulation. The method provides a useful tool to study mechanisms of dynamic network configuration underlying cognitive processes.

Strategic deployment of feature-based attentional gain in primate visual cortex

Attending to visual stimuli enhances the gain of those neurons in primate visual cortex that preferentially respond to the matching locations and features (on-target gain). Although this is well suited to enhance the neuronal representation of attended stimuli, it is nonoptimal under difficult discrimination conditions, as in the presence of similar distractors. In such cases, directing attention to neighboring neuronal populations (off-target gain) has been shown to be the most efficient strategy, but although such a strategic deployment of attention has been shown behaviorally, its underlying neural mechanisms are unknown. Here, we investigated how attention affects the population responses of neurons in the middle temporal (MT) visual area of rhesus monkeys to bidirectional movement inside the neurons' receptive field (RF). The monkeys were trained to focus their attention onto the fixation spot or to detect a direction or speed change in one of the motion directions (the "target"), ignoring the distractor motion. Population activity profiles were determined by systematically varying the patterns' directions while maintaining a constant angle between them. As expected, the response profiles show a peak for each of the 2 motion directions. Switching spatial attention from the fixation spot into the RF enhanced the peak representing the attended stimulus and suppressed the distractor representation. Importantly, the population data show a direction-dependent attentional modulation that does not peak at the target feature but rather along the slopes of the activity profile representing the target direction. Our results show that attentional gains are strategically deployed to optimize the discriminability of target stimuli, in line with an optimal gain mechanism proposed by Navalpakkam and Itti.

Neuronal Effects of Spatial and Feature Attention Differ Due to Normalization

Although spatial and feature attention have differing effects on neuronal responses in visual cortex, it remains unclear why. Response normalization has been implicated in both types of attention (Carandini and Heeger, 2011), and single-unit studies have demonstrated that the magnitude of spatial attention effects on neuronal responses covaries with the magnitude of normalization effects. However, the relationship between feature attention and normalization remains largely unexplored. We recorded from individual neurons in the middle temporal area of rhesus monkeys using a task that allowed us to isolate the effects of feature attention, spatial attention, and normalization on the responses of each neuron. We found that the magnitudes of neuronal response modulations due to spatial attention and feature attention are correlated; however, whereas modulations due to spatial attention are correlated with normalization strength, those due to feature attention are not. Additionally, spatial attention modulations are stronger with multiple stimuli in the receptive field, whereas feature attention modulations are not. These findings are captured by a model in which spatial and feature attention share common top-down attention signals that nonetheless result in differing sensory neuron response modulations because of a spatially tuned sensory normalization mechanism. This model explains previously reported commonalities and differences between these two types of attention by clarifying the relationship between top-down attention signals and sensory normalization. We conclude that similar top-down signals to visual cortex can have distinct effects on neuronal responses due to distinct interactions with sensory mechanisms.SIGNIFICANCE STATEMENT Subjects use attention to improve their visual perception in several ways, including by attending to a location in space or to a visual feature. Prior studies have found both commonalities and differences between the effects of spatial and feature attention on neuronal responses in visual cortex, although it is unclear what mechanisms could explain this range of effects. Normalization, a computation by which neuronal responses are modified by stimulus context, has been implicated in many neuronal mechanisms throughout the brain. Here we propose that normalization provides a simple explanation for how spatial and feature attention could share common top-down attention signals that still affect sensory neuron responses differently.

A Normalization Circuit Underlying Coding of Spatial Attention in Primate Lateral Prefrontal Cortex

Lateral prefrontal cortex (LPFC) neurons signal the allocation of voluntary attention; however, the neural computations underlying this function remain unknown. To investigate this, we recorded from neuronal ensembles in the LPFC of two Macaca fascicularis performing a visuospatial attention task. LPFC neural responses to a single stimulus were normalized when additional stimuli/distracters appeared across the visual field and were well-characterized by an averaging computation. Deploying attention toward an individual stimulus surrounded by distracters shifted neural activity from an averaging regime toward a regime similar to that when the attended stimulus was presented in isolation (winner-take-all; WTA). However, attentional modulation is both qualitatively and quantitatively dependent on a neuron’s visuospatial tuning. Our results show that during attentive vision, LPFC neuronal ensemble activity can be robustly read out by downstream areas to generate motor commands, and/or fed back into sensory areas to filter out distracter signals in favor of target signals.

Attention-related changes in correlated neuronal activity arise from normalization mechanisms

Attention is believed to enhance perception by altering the correlations between pairs of neurons. How attention changes neuronal correlations is unknown. Using multi-electrodes in primate visual cortex, we measured spike-count correlations when single or multiple stimuli were presented, and stimuli were attended or unattended. When stimuli were unattended, adding a suppressive, non-preferred, stimulus beside a preferred stimulus increased spike-count correlations between pairs of similarly-tuned neurons, but decreased spike-count correlations between pairs of oppositely-tuned neurons. These changes are explained by a stochastic normalization model containing populations of oppositely-tuned, mutually-suppressive neurons. Importantly, this model also explains why attention decreased (attend preferred stimulus) or increased (attend non-preferred stimulus) correlations: as an indirect consequence of attention-related changes in the inputs to normalization mechanisms. Our findings link normalization mechanisms to correlated neuronal activity and attention, showing that normalization mechanisms shape response correlations and that these correlations change when attention biases normalization mechanisms.

Temporally evolving gain mechanisms of attention in macaque area V4

Cognitive attention and perceptual saliency jointly govern our interaction with the environment. Yet, we still lack a universally accepted account of the interplay between attention and luminance contrast, a fundamental dimension of saliency. We measured the attentional modulation of V4 neurons' contrast response functions (CRFs) in awake, behaving macaque monkeys and applied a new approach that emphasizes the temporal dynamics of cell responses. We found that attention modulates CRFs via different gain mechanisms during subsequent epochs of visually driven activity: an early contrast-gain, strongly dependent on prestimulus activity changes (baseline shift); a time-limited stimulus-dependent multiplicative modulation, reaching its maximal expression around 150 ms after stimulus onset; and a late resurgence of contrast-gain modulation. Attention produced comparable time-dependent attentional gain changes on cells heterogeneously coding contrast, supporting the notion that the same circuits mediate attention mechanisms in V4 regardless of the form of contrast selectivity expressed by the given neuron. Surprisingly, attention was also sometimes capable of inducing radical transformations in the shape of CRFs. These findings offer important insights into the mechanisms that underlie contrast coding and attention in primate visual cortex and a new perspective on their interplay, one in which time becomes a fundamental factor.NEW & NOTEWORTHY We offer an innovative perspective on the interplay between attention and luminance contrast in macaque area V4, one in which time becomes a fundamental factor. We place emphasis on the temporal dynamics of attentional effects, pioneering the notion that attention modulates contrast response functions of V4 neurons via the sequential engagement of distinct gain mechanisms. These findings advance understanding of attentional influences on visual processing and help reconcile divergent results in the literature.

An Extended Normalization Model of Attention Accounts for Feature-Based Attentional Enhancement of Both Response and Coherence Gain

Paying attention to a sensory feature improves its perception and impairs that of others. Recent work has shown that a Normalization Model of Attention (NMoA) can account for a wide range of physiological findings and the influence of different attentional manipulations on visual performance. A key prediction of the NMoA is that attention to a visual feature like an orientation or a motion direction will increase the response of neurons preferring the attended feature (response gain) rather than increase the sensory input strength of the attended stimulus (input gain). This effect of feature-based attention on neuronal responses should translate to similar patterns of improvement in behavioral performance, with psychometric functions showing response gain rather than input gain when attention is directed to the task-relevant feature. In contrast, we report here that when human subjects are cued to attend to one of two motion directions in a transparent motion display, attentional effects manifest as a combination of input and response gain. Further, the impact on input gain is greater when attention is directed towards a narrow range of motion directions than when it is directed towards a broad range. These results are captured by an extended NMoA, which either includes a stimulus-independent attentional contribution to normalization or utilizes direction-tuned normalization. The proposed extensions are consistent with the feature-similarity gain model of attention and the attentional modulation in extrastriate area MT, where neuronal responses are enhanced and suppressed by attention to preferred and non-preferred motion directions respectively.

We report a pattern of feature-based attentional effects on human psychophysical performance, which cannot be accounted for by the Normalization Model of Attention using biologically plausible parameters. Specifically, this prominent model of attentional modulation predicts that attention to a visual feature like a specific motion direction will lead to a response gain in the input-response function, rather than the input gain that we actually observe. In our data, the input gain is greater when attention is directed towards a narrow range of motion directions, again contrary to the model’s prediction. We therefore propose two physiologically testable extensions of the model that include direction-tuned normalization mechanisms of attention. Both extensions account for our data without affecting the previously demonstrated successful performance of the NMoA.

Functional states of rat cortical circuits during the unpredictable availability of a reward-related cue

Proper performance of acquired abilities can be disturbed by the unexpected occurrence of external changes. Rats trained with an operant conditioning task (to press a lever in order to obtain a food pellet) using a fixed-ratio (1:1) schedule were subsequently placed in a Skinner box in which the lever could be removed randomly. Field postsynaptic potentials (fPSPs) were chronically evoked in perforant pathway-hippocampal CA1 (PP-CA1), CA1-subiculum (CA1-SUB), CA1-medial prefrontal cortex (CA1-mPFC), mPFC-nucleus accumbens (mPFC-NAc), and mPFC-basolateral amygdala (mPFC-BLA) synapses during lever IN and lever OUT situations. While lever presses were accompanied by a significant increase in fPSP slopes at the five synapses, the unpredictable absence of the lever were accompanied by decreased fPSP slopes in all, except PP-CA1 synapses. Spectral analysis of local field potentials (LFPs) recorded when the animal approached the corresponding area in the lever OUT situation presented lower spectral powers than during lever IN occasions for all recording sites, apart from CA1. Thus, the unpredictable availability of a reward-related cue modified the activity of cortical and subcortical areas related with the acquisition of operant learning tasks, suggesting an immediate functional reorganization of these neural circuits to address the changed situation and to modify ongoing behaviors accordingly.

Attention operates uniformly throughout the classical receptive field and the surround

Shifting attention among visual stimuli at different locations modulates neuronal responses in heterogeneous ways, depending on where those stimuli lie within the receptive fields of neurons. Yet how attention interacts with the receptive-field structure of cortical neurons remains unclear. We measured neuronal responses in area V4 while monkeys shifted their attention among stimuli placed in different locations within and around neuronal receptive fields. We found that attention interacts uniformly with the spatially-varying excitation and suppression associated with the receptive field. This interaction explained the large variability in attention modulation across neurons, and a non-additive relationship among stimulus selectivity, stimulus-induced suppression and attention modulation that has not been previously described. A spatially-tuned normalization model precisely accounted for all observed attention modulations and for the spatial summation properties of neurons. These results provide a unified account of spatial summation and attention-related modulation across both the classical receptive field and the surround.

DOI:http://dx.doi.org/10.7554/eLife.17256.001

At any moment, our brain receives an enormous amount of information from our senses. However, we are not aware of all of this information; only the information we decide to focus on is perceived in detail. This ability to focus our attention is important for survival.

The neurons involved in vision respond best to information that comes from a small ‘window’ in what is being seen. When something appears in this window (known as the neuron’s receptive field), the activity of the neuron either increases or decreases. How does focusing attention on an object change the neuron’s response? Verhoef and Maunsell investigated this question by recording electrical activity in an area of the brain called V4 in monkeys as they focused their attention on objects in different locations of the neuron’s receptive field.

The recordings show that a single rule determines when attention influences a neuron’s activity. If an object inside the neuron’s receptive field decreases the activity of the neuron, then attention can change that neuron’s activity. Attention then changes the activity of the neuron by either removing or further boosting the influence of these objects.

Verhoef and Maunsell then developed a mathematical model based on these results, and found that the model could explain why the activity of a neuron changes when attention is focused on objects at different locations in its receptive field. The next step is to understand exactly how the brain works to either remove or boost the influence of an object that causes a neuron’s activity to decrease.

DOI:http://dx.doi.org/10.7554/eLife.17256.002

Attention enhances stimulus representations in macaque visual cortex without affecting their signal-to-noise level

The magnitude of the attentional modulation of neuronal responses in visual cortex varies with stimulus contrast. Whether the strength of these attentional influences is similarly dependent on other stimulus properties is unknown. Here we report the effect of spatial attention on responses in the medial-temporal area (MT) of macaque visual cortex to moving random dots pattern of various motion coherences, i.e. signal-to-noise ratios. Our data show that allocating spatial attention causes a gain change in MT neurons. The magnitude of this attentional modulation is independent of the attended stimulus’ motion coherence, creating a multiplicative scaling of the neuron’s coherence-response function. This is consistent with the characteristics of gain models of attentional modulation and suggests that attention strengthens the neuronal representation of behaviorally relevant visual stimuli relative to unattended stimuli, but without affecting their signal-to-noise ratios.

A Feedback Model of Attention Explains the Diverse Effects of Attention on Neural Firing Rates and Receptive Field Structure

Visual attention has many effects on neural responses, producing complex changes in firing rates, as well as modifying the structure and size of receptive fields, both in topological and feature space. Several existing models of attention suggest that these effects arise from selective modulation of neural inputs. However, anatomical and physiological observations suggest that attentional modulation targets higher levels of the visual system (such as V4 or MT) rather than input areas (such as V1). Here we propose a simple mechanism that explains how a top-down attentional modulation, falling on higher visual areas, can produce the observed effects of attention on neural responses. Our model requires only the existence of modulatory feedback connections between areas, and short-range lateral inhibition within each area. Feedback connections redistribute the top-down modulation to lower areas, which in turn alters the inputs of other higher-area cells, including those that did not receive the initial modulation. This produces firing rate modulations and receptive field shifts. Simultaneously, short-range lateral inhibition between neighboring cells produce competitive effects that are automatically scaled to receptive field size in any given area. Our model reproduces the observed attentional effects on response rates (response gain, input gain, biased competition automatically scaled to receptive field size) and receptive field structure (shifts and resizing of receptive fields both spatially and in complex feature space), without modifying model parameters. Our model also makes the novel prediction that attentional effects on response curves should shift from response gain to contrast gain as the spatial focus of attention drifts away from the studied cell.

Exerting visual attention results in profound changes in the activity of neurons in visual areas of the brain. Attention increases the firing of some neurons, decreases that of others, moves and resizes the receptive fields of individual neurons, and changes their preferred features according to what is being attended. How are these complex, subtle effects generated? While several models explain various subsets of these effects, a consistent explanation compatible with anatomical and physiological observations remains elusive. Here we show that the apparently complex and multifaceted effects of attention on neural responses can be explained as the automatic consequence of a top-down modulation, falling on higher visual areas (as suggested by anatomical observations), and interacting with short-range inhibition and feedback connections between areas. Our model only assumes the existence of well-known features of brain organization (reciprocal inter-area connections, mutual inhibition between neighboring neurons) to explain a wide range of attentional effects, including apparently finely-tuned effects (complex shifts in feature preferences, automatic scaling of competitive effects to receptive field size, resizing or shifting of receptive fields, etc). Our model also makes novel, testable predictions about the effect of certain attentional manipulations on neural responses.

Canonical computations of cerebral cortex

The idea that there is a fundamental cortical circuit that performs canonical computations remains compelling though far from proven. Here we review evidence for two canonical operations within sensory cortical areas: a feedforward computation of selectivity; and a recurrent computation of gain in which, given sufficiently strong external input, perhaps from multiple sources, intracortical input largely, but not completely, cancels this external input. This operation leads to many characteristic cortical nonlinearities in integrating multiple stimuli. The cortical computation must combine such local processing with hierarchical processing across areas. We point to important changes in moving from sensory cortex to motor and frontal cortex and the possibility of substantial differences between cortex in rodents vs. species with columnar organization of selectivity.

Deconstructing Interocular Suppression: Attention and Divisive Normalization

In interocular suppression, a suprathreshold monocular target can be rendered invisible by a salient competitor stimulus presented in the other eye. Despite decades of research on interocular suppression and related phenomena (e.g., binocular rivalry, flash suppression, continuous flash suppression), the neural processing underlying interocular suppression is still unknown. We developed and tested a computational model of interocular suppression. The model included two processes that contributed to the strength of interocular suppression: divisive normalization and attentional modulation. According to the model, the salient competitor induced a stimulus-driven attentional modulation selective for the location and orientation of the competitor, thereby increasing the gain of neural responses to the competitor and reducing the gain of neural responses to the target. Additional suppression was induced by divisive normalization in the model, similar to other forms of visual masking. To test the model, we conducted psychophysics experiments in which both the size and the eye-of-origin of the competitor were manipulated. For small and medium competitors, behavioral performance was consonant with a change in the response gain of neurons that responded to the target. But large competitors induced a contrast-gain change, even when the competitor was split between the two eyes. The model correctly predicted these results and outperformed an alternative model in which the attentional modulation was eye specific. We conclude that both stimulus-driven attention (selective for location and feature) and divisive normalization contribute to interocular suppression.

In interocular suppression, a visible target presented in one eye can be rendered invisible by a competing image (the competitor) presented in the other eye. This phenomenon is a striking demonstration of the discrepancy between physical inputs to the visual system and perception, and it also allows neuroscientists to study how perceptual systems regulate competing information. Interocular suppression has been explained by mutually suppressive interactions (modeled by divisive normalization) between neurons that respond differentially to the two eyes. Attention, which selects relevant information in natural viewing condition, has also been found to play a role in interocular suppression. But the specific role of attentional modulation is still an open question. In this study, we proposed a computational model of interocular suppression integrating both attentional modulation and divisive normalization. By modeling the hypothetical neural responses and fitting the model to psychophysical data, we showed that interocular suppression involves an attentional modulation selective for the orientation of the competitor, and covering the spatial extent of the competitor. We conclude that both attention and divisive normalization contribute to interocular suppression, and that their impacts are distinguishable.

Attention and normalization circuits in macaque V1

Attention affects neuronal processing and improves behavioural performance. In extrastriate visual cortex these effects have been explained by normalization models, which assume that attention influences the circuit that mediates surround suppression. While normalization models have been able to explain attentional effects, their validity has rarely been tested against alternative models. Here we investigate how attention and surround/mask stimuli affect neuronal firing rates and orientation tuning in macaque V1. Surround/mask stimuli provide an estimate to what extent V1 neurons are affected by normalization, which was compared against effects of spatial top down attention. For some attention/surround effect comparisons, the strength of attentional modulation was correlated with the strength of surround modulation, suggesting that attention and surround/mask stimulation (i.e. normalization) might use a common mechanism. To explore this in detail, we fitted multiplicative and additive models of attention to our data. In one class of models, attention contributed to normalization mechanisms, whereas in a different class of models it did not. Model selection based on Akaike's and on Bayesian information criteria demonstrated that in most cells the effects of attention were best described by models where attention did not contribute to normalization mechanisms. This demonstrates that attentional influences on neuronal responses in primary visual cortex often bypass normalization mechanisms.

Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene with eye movements

How does the brain maintain stable fusion of 3D scenes when the eyes move? Every eye movement causes each retinal position to process a different set of scenic features, and thus the brain needs to binocularly fuse new combinations of features at each position after an eye movement. Despite these breaks in retinotopic fusion due to each movement, previously fused representations of a scene in depth often appear stable. The 3D ARTSCAN neural model proposes how the brain does this by unifying concepts about how multiple cortical areas in the What and Where cortical streams interact to coordinate processes of 3D boundary and surface perception, spatial attention, invariant object category learning, predictive remapping, eye movement control, and learned coordinate transformations. The model explains data from single neuron and psychophysical studies of covert visual attention shifts prior to eye movements. The model further clarifies how perceptual, attentional, and cognitive interactions among multiple brain regions (LGN, V1, V2, V3A, V4, MT, MST, PPC, LIP, ITp, ITa, SC) may accomplish predictive remapping as part of the process whereby view-invariant object categories are learned. These results build upon earlier neural models of 3D vision and figure-ground separation and the learning of invariant object categories as the eyes freely scan a scene. A key process concerns how an object's surface representation generates a form-fitting distribution of spatial attention, or attentional shroud, in parietal cortex that helps maintain the stability of multiple perceptual and cognitive processes. Predictive eye movement signals maintain the stability of the shroud, as well as of binocularly fused perceptual boundaries and surface representations.

Featural and temporal attention selectively enhance task-appropriate representations in human primary visual cortex

Our perceptions are often shaped by focusing our attention towards specific features or periods of time irrespective of location. Here we explore the physiological bases of these non-spatial forms of attention by imaging brain activity while subjects perform a challenging change-detection task. The task employs a continuously varying visual stimulus that, for any moment in time, selectively activates functionally distinct subpopulations of primary visual cortex (V1) neurons. When subjects are cued to the timing and nature of the change, the mapping of orientation preference across V1 systematically shifts towards the cued stimulus just prior to its appearance. A simple linear model can explain this shift: attentional changes are selectively targeted towards neural subpopulations, representing the attended feature at the times the feature was anticipated. Our results suggest that featural attention is mediated by a linear change in the responses of task-appropriate neurons across cortex during appropriate periods of time.

Infrared neural stimulation of primary visual cortex in non-human primates

Infrared neural stimulation (INS) is an alternative neurostimulation modality that uses pulsed infrared light to evoke spatially precise neural activity that does not require direct contact with neural tissue. With these advantages INS has the potential to increase our understanding of specific neural pathways and impact current diagnostic and therapeutic clinical applications. In order to develop this technique, we investigate the feasibility of INS (λ=1.875μm, fiber diameter=100-400μm) to activate and modulate neural activity in primary visual cortex (V1) of Macaque monkeys. Infrared neural stimulation was found to evoke localized neural responses as evidenced by both electrophysiology and intrinsic signal optical imaging (OIS). Single unit recordings acquired during INS indicated statistically significant increases in neuron firing rates that demonstrate INS evoked excitatory neural activity. Consistent with this, INS stimulation led to focal intensity-dependent reflectance changes recorded with OIS. We also asked whether INS is capable of stimulating functionally specific domains in visual cortex and of modulating visually evoked activity in visual cortex. We found that application of INS via 100μm or 200μm fiber optics produced enhancement of visually evoked OIS response confined to the eye column where INS was applied and relative suppression of the other eye column. Stimulating the cortex with a 400μm fiber, exceeding the ocular dominance width, led to relative suppression, consistent with involvement of inhibitory surrounds. This study is the first to demonstrate that INS can be used to either enhance or diminish visual cortical response and that this can be done in a functional domain specific manner. INS thus holds great potential for use as a safe, non-contact, focally specific brain stimulation technology in primate brains.

Inattention blindness to motion in middle temporal area

Subjects naturally form and use expectations to solve familiar tasks, but the accuracy of these expectations and the neuronal mechanisms by which these expectations enhance behavior are unclear. We trained animals (Macaca mulatta) in a challenging perceptual task in which the likelihood of a very brief pulse of motion was consistently modulated over time and space. Pulse likelihood had dramatic effects on behavior: unexpected pulses were nearly invisible to the animals. To examine the neuronal basis of such inattention blindness, we recorded from single neurons in the middle temporal (MT) area, an area related to motion perception. Fluctuations in how reliably MT neurons both signaled stimulus events and predicted behavioral choices were highly correlated with changes in performance over the course of individual trials. A simple neuronal pooling model reveals that the dramatic behavioral effects of attention in this task can be completely explained by changes in the reliability of a small number of MT neurons.

Attention as reward-driven optimization of sensory processing

Attention causes diverse changes to visual neuron responses, including alterations in receptive field structure, and firing rates. A common theoretical approach to investigate why sensory neurons behave as they do is based on the efficient coding hypothesis: that sensory processing is optimized toward the statistics of the received input. We extend this approach to account for the influence of task demands, hypothesizing that the brain learns a probabilistic model of both the sensory input and reward received for performing different actions. Attention-dependent changes to neural responses reflect optimization of this internal model to deal with changes in the sensory environment (stimulus statistics) and behavioral demands (reward statistics). We use this framework to construct a simple model of visual processing that is able to replicate a number of attention-dependent changes to the responses of neurons in the midlevel visual cortices. The model is consistent with and provides a normative explanation for recent divisive normalization models of attention (Reynolds & Heeger, 2009).

Strength of gamma rhythm depends on normalization

Manipulating a divisive normalization mechanism independently of attention in monkeys suggests that gamma power reflects excitation-inhibition interactions rather than plays a functional role in attentional processing.

Neuronal assemblies often exhibit stimulus-induced rhythmic activity in the gamma range (30–80 Hz), whose magnitude depends on the attentional load. This has led to the suggestion that gamma rhythms form dynamic communication channels across cortical areas processing the features of behaviorally relevant stimuli. Recently, attention has been linked to a normalization mechanism, in which the response of a neuron is suppressed (normalized) by the overall activity of a large pool of neighboring neurons. In this model, attention increases the excitatory drive received by the neuron, which in turn also increases the strength of normalization, thereby changing the balance of excitation and inhibition. Recent studies have shown that gamma power also depends on such excitatory–inhibitory interactions. Could modulation in gamma power during an attention task be a reflection of the changes in the underlying excitation–inhibition interactions? By manipulating the normalization strength independent of attentional load in macaque monkeys, we show that gamma power increases with increasing normalization, even when the attentional load is fixed. Further, manipulations of attention that increase normalization increase gamma power, even when they decrease the firing rate. Thus, gamma rhythms could be a reflection of changes in the relative strengths of excitation and normalization rather than playing a functional role in communication or control.

Brain signals often show a stimulus-induced rhythm in the “gamma” band (30–80 Hz) whose magnitude depends on attentional load, leading to suggestions that gamma rhythm plays a functional role in routing signals across cortical areas. However, gamma power also depends on simple stimulus features such as size or contrast, which suggests that gamma could arise from basic cortical processes involving excitation–inhibition interactions. One such process is divisive normalization, a mechanism that suppresses the response of a neuron by the overall activity of a large pool of neighboring neurons. Recent studies have shown that attention increases the strength of both excitation and normalization. We hypothesized that the increase in gamma power in an attention task is due to the effect of attention on excitation and normalization. By manipulating the normalization strength independent of attentional load in macaque monkeys, we show that gamma power increases with increasing normalization, even when attentional load is held fixed. Thus, gamma rhythms could be a reflection of changes in the relative strengths of excitation and normalization rather than playing a functional role in communication or control.

Contribution of cholinergic and GABAergic mechanisms to direction tuning, discriminability, response reliability, and neuronal rate correlations in macaque middle temporal area

Previous studies have investigated the effects of acetylcholine (ACh) on neuronal tuning, coding, and attention in primary visual cortex, but its contribution to coding in extrastriate cortex is unexplored. Here we investigate the effects of ACh on tuning properties of macaque middle temporal area MT neurons and contrast them with effects of gabazine, a GABA(A) receptor blocker. ACh increased neuronal activity, it had no effect on tuning width, but it significantly increased the direction discriminability of a neuron. Gabazine equally increased neuronal activity, but it widened tuning curves and decreased the direction discriminability of a neuron. Although gabazine significantly reduced response reliability, ACh application had little effect on response reliability. Finally, gabazine increased noise correlation of simultaneously recorded neurons, whereas ACh reduced it. Thus, both drugs increased firing rates, but only ACh application improved neuronal tuning and coding in line with effects seen in studies in which attention was selectively manipulated.

Suppressive surrounds of receptive fields in monkey frontal eye field

A critical step in determining how a neuron contributes to visual processing is determining its visual receptive field (RF). While recording from neurons in frontal eye field (FEF) of awake monkeys (Macaca mulatta), we probed the visual field with small spots of light and found excitatory RFs that decreased in strength from RF center to periphery. However, presenting stimuli with different diameters centered on the RF revealed suppressive surrounds that overlapped the previously determined excitatory RF and reduced responses by 84%, on average. Consequently, in that overlap area, stimulation produced excitation or suppression, depending on the stimulus. Strong stimulation of the RF periphery with annular stimuli allowed us to quantify this effect. A modified difference of gaussians model that independently varied center and surround activation accounted for the nonlinear activity in the overlap area. Our results suggest that (1) the suppressive surrounds found in FEF are fundamentally the same as those in V1 except for the size and strength of excitatory and suppressive mechanisms, (2) methodically assaying suppressive surrounds in FEF is essential for correctly interpreting responses to large and/or peripheral stimuli and therefore understanding the effects of stimulus context, and (3) regulating the relative strength of the surround clearly changes neuronal responses and may therefore play a significant part in the neuronal changes resulting from visual attention and stimulus salience.

Dissociating activity in the lateral intraparietal area from value using a visual foraging task

We make decisions about where to look approximately three times per second in normal viewing. It has been suggested that eye movements may be guided by activity in the lateral intraparietal area (LIP), which is thought to represent the relative value of objects in space. However, it is not clear how values for saccade goal selection are prioritized while free-viewing in a cluttered visual environment. To address this question, we compared the neural responses of LIP neurons in two subjects with their saccadic behavior and three estimates of stimulus value. These measures were extracted from the subjects' performance in a visual foraging task, in which we parametrically controlled the number of objects on the screen. We found that the firing rates of LIP neurons did not correlate well with the animals' behavior or any of our estimated measures of value. However, if the LIP activity was further normalized, it became highly correlated with the animals' decisions. These data suggest that LIP activity does not represent value in complex environments, but that the value can easily be extracted with one further step of processing. We propose that activity in LIP represents attentional priority and that the downstream normalization of this activity is an essential process in guiding action.

Optimal deployment of attentional gain during fine discriminations

Most models assume that top-down attention enhances the gain of sensory neurons tuned to behaviorally relevant stimuli (on-target gain). However, theoretical work suggests that when targets and distracters are highly similar, attention should enhance the gain of neurons that are tuned away from the target, because these neurons better discriminate neighboring features (off-target gain). While it is established that off-target neurons support difficult fine discriminations, it is unclear if top-down attentional gain can be optimally applied to informative off-target sensory neurons or if gain is always applied to on-target neurons, regardless of task demands. To test the optimality of attentional gain in human visual cortex, we used functional magnetic resonance imaging and an encoding model to estimate the response profile across a set of hypothetical orientation-selective channels during a difficult discrimination task. The results suggest that top-down attention can adaptively modulate off-target neural populations, but only when the discriminanda are precisely specified in advance. Furthermore, logistic regression revealed that activation levels in off-target orientation channels predicted behavioral accuracy on a trial-by-trial basis. Overall, these data suggest that attention does not only increase the gain of sensory-evoked responses, but may bias population response profiles in an optimal manner that respects both the tuning properties of sensory neurons and the physical characteristics of the stimulus array.

Feature-based attention enhances performance by increasing response gain

Covert spatial attention can increase contrast sensitivity either by changes in contrast gain or by changes in response gain, depending on the size of the attention field and the size of the stimulus (Herrmann et al., 2010), as predicted by the normalization model of attention (Reynolds & Heeger, 2009). For feature-based attention, unlike spatial attention, the model predicts only changes in response gain, regardless of whether the featural extent of the attention field is small or large. To test this prediction, we measured the contrast dependence of feature-based attention. Observers performed an orientation-discrimination task on a spatial array of grating patches. The spatial locations of the gratings were varied randomly so that observers could not attend to specific locations. Feature-based attention was manipulated with a 75% valid and 25% invalid pre-cue, and the featural extent of the attention field was manipulated by introducing uncertainty about the upcoming grating orientation. Performance accuracy was better for valid than for invalid pre-cues, consistent with a change in response gain, when the featural extent of the attention field was small (low uncertainty) or when it was large (high uncertainty) relative to the featural extent of the stimulus. These results for feature-based attention clearly differ from results of analogous experiments with spatial attention, yet both support key predictions of the normalization model of attention.

Neural dynamics of object-based multifocal visual spatial attention and priming: object cueing, useful-field-of-view, and crowding

How are spatial and object attention coordinated to achieve rapid object learning and recognition during eye movement search? How do prefrontal priming and parietal spatial mechanisms interact to determine the reaction time costs of intra-object attention shifts, inter-object attention shifts, and shifts between visible objects and covertly cued locations? What factors underlie individual differences in the timing and frequency of such attentional shifts? How do transient and sustained spatial attentional mechanisms work and interact? How can volition, mediated via the basal ganglia, influence the span of spatial attention? A neural model is developed of how spatial attention in the where cortical stream coordinates view-invariant object category learning in the what cortical stream under free viewing conditions. The model simulates psychological data about the dynamics of covert attention priming and switching requiring multifocal attention without eye movements. The model predicts how "attentional shrouds" are formed when surface representations in cortical area V4 resonate with spatial attention in posterior parietal cortex (PPC) and prefrontal cortex (PFC), while shrouds compete among themselves for dominance. Winning shrouds support invariant object category learning, and active surface-shroud resonances support conscious surface perception and recognition. Attentive competition between multiple objects and cues simulates reaction-time data from the two-object cueing paradigm. The relative strength of sustained surface-driven and fast-transient motion-driven spatial attention controls individual differences in reaction time for invalid cues. Competition between surface-driven attentional shrouds controls individual differences in detection rate of peripheral targets in useful-field-of-view tasks. The model proposes how the strength of competition can be mediated, though learning or momentary changes in volition, by the basal ganglia. A new explanation of crowding shows how the cortical magnification factor, among other variables, can cause multiple object surfaces to share a single surface-shroud resonance, thereby preventing recognition of the individual objects.

Expansion of MT neurons excitatory receptive fields during covert attentive tracking

Primates can attentively track moving objects while keeping gaze stationary. The neural mechanisms underlying this ability are poorly understood. We investigated this issue by recording responses of neurons in area MT of two rhesus monkeys while they performed two different tasks. During the Attend-Fixation task, two moving random dot patterns (RDPs) translated across a screen at the same speed and in the same direction while the animals directed gaze to a fixation spot and detected a change in its luminance. During the Tracking task, the animals kept gaze on the fixation spot and attentively tracked the two RDPs to report a change in the local speed of one of the patterns' dots. In both conditions, neuronal responses progressively increased as the RDPs entered the neurons' receptive field (RF), peaked when they reached its center, and decreased as they translated away. This response profile was well described by a Gaussian function with its center of gravity indicating the RF center and its flanks the RF excitatory borders. During Tracking, responses were increased relative to Attend-Fixation, causing the Gaussian profiles to expand. Such increases were proportionally larger in the RF periphery than at its center, and were accompanied by a decrease in the trial-to-trial response variability (Fano factor) relative to Attend-Fixation. These changes resulted in an increase in the neurons' performance at detecting targets at longer distances from the RF center. Our results show that attentive tracking dynamically changes MT neurons' RF profiles, ultimately improving the neurons' ability to encode the tracked stimulus features.

Pulsed infrared light alters neural activity in rat somatosensory cortex in vivo

Pulsed infrared light has shown promise as an alternative to electrical stimulation in applications where contact free or high spatial precision stimulation is desired. Infrared neural stimulation (INS) is well characterized in the peripheral nervous system; however, to date, research has been limited in the central nervous system. In this study, pulsed infrared light (λ=1.875 μm, pulse width=250 μs, radiant exposure=0.01-0.55 J/cm(2), fiber size=400 μm, repetition rate=50-200 Hz) was used to stimulate the somatosensory cortex of anesthetized rats, and its efficacy was assessed using intrinsic optical imaging and electrophysiology techniques. INS was found to evoke an intrinsic response of similar magnitude to that evoked by tactile stimulation (0.3-0.4% change in intrinsic signal magnitude). A maximum deflection in the intrinsic signal was measured to range from 0.05% to 0.4% in response to INS, and the activated region of cortex measured approximately 2mm in diameter. The intrinsic signal magnitude increased with faster laser repetition rates and increasing radiant exposures. Single unit recordings indicated a statistically significant decrease in neuronal firing that was observed at the onset of INS stimulation (0.5s stimulus) and continued up to 1s after stimulation onset. The pattern of neuronal firing differed from that observed during tactile stimulation, potentially due to a different spatial integration field of the pulsed infrared light compared to tactile stimulation. The results demonstrate that INS can be used safely and effectively to manipulate neuronal firing.

Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech

Humans are remarkably adept at understanding speech, even when it is contaminated by noise. Multisensory integration may explain some of this ability: combining independent information from the auditory modality (vocalizations) and the visual modality (mouth movements) reduces noise and increases accuracy. Converging evidence suggests that the superior temporal sulcus (STS) is a critical brain area for multisensory integration, but little is known about its role in the perception of noisy speech. Behavioral studies have shown that perceptual judgments are weighted by the reliability of the sensory modality: more reliable modalities are weighted more strongly, even if the reliability changes rapidly. We hypothesized that changes in the functional connectivity of STS with auditory and visual cortex could provide a neural mechanism for perceptual reliability weighting. To test this idea, we performed five blood oxygenation level-dependent functional magnetic resonance imaging and behavioral experiments in 34 healthy subjects. We found increased functional connectivity between the STS and auditory cortex when the auditory modality was more reliable (less noisy) and increased functional connectivity between the STS and visual cortex when the visual modality was more reliable, even when the reliability changed rapidly during presentation of successive words. This finding matched the results of a behavioral experiment in which the perception of incongruent audiovisual syllables was biased toward the more reliable modality, even with rapidly changing reliability. Changes in STS functional connectivity may be an important neural mechanism underlying the perception of noisy speech.

Spatial summation in macaque parietal area 7a follows a winner-take-all rule

While neurons in posterior parietal cortex have been found to signal the presence of a salient stimulus among multiple items in a display, spatial summation within their receptive field in the absence of an attentional bias has never been investigated. This information, however, is indispensable when one investigates the mechanisms of spatial attention and competition between multiple visual objects. To examine the spatial summation rule in parietal area 7a neurons, we trained rhesus monkeys to fixate on a central cross while two identical stimuli were briefly displayed in a neuron's receptive field. The response to a pair of dots was compared with the responses to the same dots when they were presented individually. The scaling and power parameters of a generalized summation algorithm varied greatly, both across neurons and across combinations of stimulus locations. However, the averaged response of the recorded population of 7a neurons was consistent with a winner-take-all rule for spatial summation. A control experiment where a monkey covertly attended to both stimuli simultaneously suggests that attention introduces additional competition by facilitating the less optimal stimulus. Thus an averaging stage is introduced between ∼ 200 and 300 ms of the response to a pair of stimuli. In short, the summation algorithm over the population of area 7a neurons carries the signature of a winner-take-all operation, with spatial attention possibly influencing the temporal dynamics of stimulus competition, that is the moment that the "winner" takes "victory" over the "loser" stimulus.

The neural basis of visual attention

Visual attention is the mechanism the nervous system uses to highlight specific locations, objects or features within the visual field. This can be accomplished by making an eye movement to bring the object onto the fovea (overt attention) or by increased processing of visual information in neurons representing more peripheral regions of the visual field (covert attention). This review will examine two aspects of visual attention: the changes in neural responses within visual cortices due to the allocation of covert attention; and the neural activity in higher cortical areas involved in guiding the allocation of attention. The first section will highlight processes that occur during visual spatial attention and feature-based attention in cortical visual areas and several related models that have recently been proposed to explain this activity. The second section will focus on the parietofrontal network thought to be involved in targeting eye movements and allocating covert attention. It will describe evidence that the lateral intraparietal area, frontal eye field and superior colliculus are involved in the guidance of visual attention, and describe the priority map model, which is thought to operate in at least several of these areas.

The effect of attention on neuronal responses to high and low contrast stimuli

It remains unclear how attention affects the tuning of individual neurons in visual cerebral cortex. Some observations suggest that attention preferentially enhances responses to low contrast stimuli, whereas others suggest that attention proportionally affects responses to all stimuli. Resolving how attention affects responses to different stimuli is essential for understanding the mechanism by which it acts. To explore the effects of attention on stimuli of different contrasts, we recorded from individual neurons in the middle temporal visual area (MT) of rhesus monkeys while shifting their attention between preferred and nonpreferred stimuli within their receptive fields. This configuration results in robust attentional modulation that makes it possible to readily distinguish whether attention acts preferentially on low contrast stimuli. We found no evidence for greater enhancement of low contrast stimuli. Instead, the strong attentional modulations were well explained by a model in which attention proportionally enhances responses to stimuli of all contrasts. These data, together with observations on the effects of attention on responses to other stimulus dimensions, suggest that the primary effect of attention in visual cortex may be to simply increase the strength of responses to all stimuli by the same proportion.

Attentional modulation of MT neurons with single or multiple stimuli in their receptive fields

Descriptions of how attention modulates neuronal responses suggest that the strength of its effects depends on stimulus conditions. Attention to an isolated stimulus in the receptive field of an individual neuron typically produces a moderate enhancement of the cell's response, but neuronal responses are often strongly modulated when attention is shifted between multiple stimuli that lie within the receptive field. However, previous reports have not compared these stimulus effects under equivalent conditions, so differences in task difficulty could have been responsible for much of the difference. Consequently, the quantitative effects of stimulus conditions have remained unknown, and it has not been possible to address the question of whether the differences that have been observed could be explained by a single mechanism. We measured the attentional modulation of the responses of 70 single neurons in area MT of two rhesus monkeys using a task designed to keep attention stable across different stimulus configurations. We found that attentional modulation was indeed much stronger when more than one stimulus was within the receptive field. Nevertheless, the broad range of attentional modulations seen across the different conditions could be readily explained by single mechanism. The neurophysiological data from all stimulus conditions were well fit by a model in which attention acts via a response normalization mechanism (Lee and Maunsell, 2009). Collectively, these results validate previous impressions of the effects of stimulus configuration on attentional modulation, and add support to hypothesis that attention modulation depends on a response normalization mechanism.

Attention reduces stimulus-driven gamma frequency oscillations and spike field coherence in V1

Rhythmic activity of neuronal ensembles has been proposed to play an important role in cognitive functions such as attention, perception, and memory. Here we investigate whether rhythmic activity in V1 of the macaque monkey (macaca mulatta) is affected by top-down visual attention. We measured the local field potential (LFP) and V1 spiking activity while monkeys performed an attention-demanding detection task. We show that gamma oscillations were strongly modulated by the stimulus and by attention. Stimuli that engaged inhibitory mechanisms induced the largest gamma LFP oscillations and the largest spike field coherence. Directing attention toward a visual stimulus at the receptive field of the recorded neurons decreased LFP gamma power and gamma spike field coherence. This decrease could reflect an attention-mediated reduction of surround inhibition. Changes in synchrony in V1 would thus be a byproduct of reduced inhibitory drive, rather than a mechanism that directly aids perceptual processing.

Attention differentially modulates similar neuronal responses evoked by varying contrast and direction stimuli in area MT

The effects of attention on the responses of visual neurons have been described as a scaling or additive modulation independent of stimulus features and contrast, or as a contrast-dependent modulation. We explored these alternatives by recording neuronal responses in macaque area MT to moving stimuli that evoked similar firing rates but varied in contrast and direction. We presented two identical pairs of stimuli, one inside the neurons' receptive field and the other outside, in the opposite hemifield. One stimulus of each pair always had high contrast and moved in the recorded cell's antipreferred direction (AP pattern), while the other (test pattern) could either move in the cell's preferred direction and vary in contrast, or have the same contrast as the AP pattern and vary in direction. For different stimulus pairs evoking similar responses, switching attention between the two AP patterns, or directing attention from a fixation spot to the AP pattern inside or outside the receptive field, produced a stronger suppression of responses to varying contrast pairs, reaching a maximum ( approximately 20%) at intermediate contrast. For invariable contrast pairs, switching attention from the fixation spot to the AP pattern produced a modulation that ranged from 10% suppression when the test pattern moved in the cells preferred direction to 14% enhancement when it moved in a direction 90 degrees away from that direction. Our results are incompatible with a scaling or additive modulation of MT neurons' response by attention, but support models where spatial and feature-based attention modulate input signals into the area normalization circuit.

Attention directed by expectations enhances receptive fields in cortical area MT

Expectations, especially those formed on the basis of extensive training, can substantially enhance visual performance. However, it is not clear that the physiological mechanisms underlying this enhancement are identical to those examined by experiments in which attention is directed by explicit instructions rather than strong expectations. To study the changes in visual representations associated with strong expectations, we trained animals to detect a brief motion pulse that was embedded in noise. Because the nature of the pulse and the statistics of its appearance were well known to the animals, they formed strong expectations which determined their behavioral performance. We used white-noise methods to infer the receptive field structure of single neurons in area MT while they were performing this task. Incorporating non-linearities, we compared receptive fields during periods of time when the animals were expecting the motion pulse with periods of time when they were not. We found receptive field changes consistent with an increased reliability in signaling pulse occurrence. Moreover, these changes were not consistent with a simple gain modulation. The results suggest that strong expectations can create very specific changes in the visual representations at a cellular level to enhance performance.

Attentional modulation of adaptation in V4

Adaptation and visual attention are two processes that alter neural responses to luminance contrast. Rapid contrast adaptation changes response size and dynamics at all stages of visual processing, while visual attention has been shown to modulate both contrast gain and response gain in macaque extrastriate visual cortex. Because attention aims to enhance behaviorally relevant sensory responses while adaptation acts to attenuate neural activity, the question we asked is, how does attention alter adaptation? We present here single-unit recordings from V4 of two rhesus macaques performing a cued target detection task. The study was designed to characterize the effects of attention on the size and dynamics of a sequence of responses produced by a series of flashed oriented gratings parametric in luminance contrast. We found that the effect of attention on the response dynamics of V4 neurons is inconsistent with a mechanism that only alters the effective stimulus contrast, or only rescales the gain of the response. Instead, the action of attention modifies contrast gain early in the task, and modifies both response gain and contrast gain later in the task. We also show that responses to attended stimuli are more closely locked to the stimulus cycle than unattended responses, and that attended responses show less of the phase lag produced by adaptation than unattended responses. The phase advance generated by attention of the adapted responses suggests that the attentional gain control operates in some ways like a contrast gain control utilizing a neural measure of contrast to influence dynamics.