Effects of task difficulty and target likelihood in area V4 of macaque monkeys

Attentional modulation of MT neurons with single or multiple stimuli in their receptive fields

Descriptions of how attention modulates neuronal responses suggest that the strength of its effects depends on stimulus conditions. Attention to an isolated stimulus in the receptive field of an individual neuron typically produces a moderate enhancement of the cell's response, but neuronal responses are often strongly modulated when attention is shifted between multiple stimuli that lie within the receptive field. However, previous reports have not compared these stimulus effects under equivalent conditions, so differences in task difficulty could have been responsible for much of the difference. Consequently, the quantitative effects of stimulus conditions have remained unknown, and it has not been possible to address the question of whether the differences that have been observed could be explained by a single mechanism. We measured the attentional modulation of the responses of 70 single neurons in area MT of two rhesus monkeys using a task designed to keep attention stable across different stimulus configurations. We found that attentional modulation was indeed much stronger when more than one stimulus was within the receptive field. Nevertheless, the broad range of attentional modulations seen across the different conditions could be readily explained by single mechanism. The neurophysiological data from all stimulus conditions were well fit by a model in which attention acts via a response normalization mechanism (Lee and Maunsell, 2009). Collectively, these results validate previous impressions of the effects of stimulus configuration on attentional modulation, and add support to hypothesis that attention modulation depends on a response normalization mechanism.

Chronic cellular imaging of mouse visual cortex during operant behavior and passive viewing

Nearby neurons in mammalian neocortex demonstrate a great diversity of cell types and connectivity patterns. The importance of this diversity for computation is not understood. While extracellular recording studies in visual cortex have provided a particularly rich description of behavioral modulation of neural activity, new methods are needed to dissect the contribution of specific circuit elements in guiding visual perception. Here, we describe a method for three-dimensional cellular imaging of neural activity in the awake mouse visual cortex during active discrimination and passive viewing of visual stimuli. Head-fixed mice demonstrated robust discrimination for many hundred trials per day after initial task acquisition. To record from multiple neurons during operant behavior with single-trial resolution and minimal artifacts, we built a sensitive microscope for two-photon calcium imaging, capable of rapid tracking of neurons in three dimensions. We demonstrate stable recordings of cellular calcium activity during discrimination behavior across hours, days, and weeks, using both synthetic and genetically encoded calcium indicators. When combined with molecular and genetic technologies in mice (e.g., cell-type specific transgenic labeling), this approach allows the identification of neuronal classes in vivo. Physiological measurements from distinct classes of neighboring neurons will enrich our understanding of the coordinated roles of diverse elements of cortical microcircuits in guiding sensory perception and perceptual learning. Further, our method provides a high-throughput, chronic in vivo assay of behavioral influences on cellular activity that is applicable to a wide range of mouse models of neurologic disease.

Attention differentially modulates similar neuronal responses evoked by varying contrast and direction stimuli in area MT

The effects of attention on the responses of visual neurons have been described as a scaling or additive modulation independent of stimulus features and contrast, or as a contrast-dependent modulation. We explored these alternatives by recording neuronal responses in macaque area MT to moving stimuli that evoked similar firing rates but varied in contrast and direction. We presented two identical pairs of stimuli, one inside the neurons' receptive field and the other outside, in the opposite hemifield. One stimulus of each pair always had high contrast and moved in the recorded cell's antipreferred direction (AP pattern), while the other (test pattern) could either move in the cell's preferred direction and vary in contrast, or have the same contrast as the AP pattern and vary in direction. For different stimulus pairs evoking similar responses, switching attention between the two AP patterns, or directing attention from a fixation spot to the AP pattern inside or outside the receptive field, produced a stronger suppression of responses to varying contrast pairs, reaching a maximum ( approximately 20%) at intermediate contrast. For invariable contrast pairs, switching attention from the fixation spot to the AP pattern produced a modulation that ranged from 10% suppression when the test pattern moved in the cells preferred direction to 14% enhancement when it moved in a direction 90 degrees away from that direction. Our results are incompatible with a scaling or additive modulation of MT neurons' response by attention, but support models where spatial and feature-based attention modulate input signals into the area normalization circuit.

Frontal eye field activity enhances object identification during covert visual search

We investigated the link between neuronal activity in the frontal eye field (FEF) and the enhancement of visual processing associated with covert spatial attention in the absence of eye movements. We correlated activity recorded in the FEF of monkeys manually reporting the identity of a visual search target to performance accuracy and reaction time. Monkeys were cued to the most probable target location with a cue array containing a popout color singleton. Neurons exhibited spatially selective responses for the popout cue stimulus and for the target of the search array. The magnitude of activity related to the location of the cue prior to the presentation of the search array was correlated with trends in behavioral performance across valid, invalid, and neutral cue trial conditions. However, the speed and accuracy of the behavioral report on individual trials were predicted by the magnitude of spatial selectivity related to the target to be identified, not for the spatial cue. A minimum level of selectivity was necessary for target detection and a higher level for target identification. Muscimol inactivation of FEF produced spatially selective perceptual deficits in the covert search task that were correlated with the effectiveness of the inactivation and were strongest on invalid cue trials that require an endogenous attention shift. These results demonstrate a strong functional link between FEF activity and covert spatial attention and suggest that spatial signals from FEF directly influence visual processing during the time that a stimulus to be identified is being processed by the visual system.

Attention directed by expectations enhances receptive fields in cortical area MT

Expectations, especially those formed on the basis of extensive training, can substantially enhance visual performance. However, it is not clear that the physiological mechanisms underlying this enhancement are identical to those examined by experiments in which attention is directed by explicit instructions rather than strong expectations. To study the changes in visual representations associated with strong expectations, we trained animals to detect a brief motion pulse that was embedded in noise. Because the nature of the pulse and the statistics of its appearance were well known to the animals, they formed strong expectations which determined their behavioral performance. We used white-noise methods to infer the receptive field structure of single neurons in area MT while they were performing this task. Incorporating non-linearities, we compared receptive fields during periods of time when the animals were expecting the motion pulse with periods of time when they were not. We found receptive field changes consistent with an increased reliability in signaling pulse occurrence. Moreover, these changes were not consistent with a simple gain modulation. The results suggest that strong expectations can create very specific changes in the visual representations at a cellular level to enhance performance.

Spatial attention modulates center-surround interactions in macaque visual area v4

In natural viewing, a visual stimulus that is the target of attention is generally surrounded by many irrelevant distracters. Stimuli falling in the receptive field surround can influence the neuronal response evoked by a stimulus appearing within the classical receptive field. Such modulation by task-irrelevant distracters may degrade the target-related neuronal signal. We therefore examined whether directing attention to a target stimulus can reduce the influence of task-irrelevant distracters on neuronal response. We find that in area V4 attention to a stimulus within a neuron's receptive field filters out a large fraction of the suppression induced by distracters appearing in the surround. When attention is instead directed to the surround stimulus, suppression is increased, thereby filtering out part of the neuronal response to the irrelevant distracter positioned within the receptive field. These findings demonstrate that attention modulates the neural mechanisms that give rise to center-surround interactions.

Task difficulty and performance induce diverse adaptive patterns in gain and shape of primary auditory cortical receptive fields

Attention is essential for navigating complex acoustic scenes, when the listener seeks to extract a foreground source while suppressing background acoustic clutter. This study explored the neural correlates of this perceptual ability by measuring rapid changes of spectrotemporal receptive fields (STRFs) in primary auditory cortex during detection of a target tone embedded in noise. Compared with responses in the passive state, STRF gain decreased during task performance in most cells. By contrast, STRF shape changes were excitatory and specific, and were strongest in cells with best frequencies near the target tone. The net effect of these adaptations was to accentuate the representation of the target tone relative to the noise by enhancing responses of near-target cells to the tone during high-signal-to-noise ratio (SNR) tasks while suppressing responses of far-from-target cells to the masking noise in low-SNR tasks. These adaptive STRF changes were largest in high-performance sessions, confirming a close correlation with behavior.

Attentional modulation of visual responses by flexible input gain

Although it is clear that sensory responses in the cortex can be strongly modulated by stimuli outside of classical receptive fields as well as by extraretinal signals such as attention and anticipation, the exact rules governing the neuronal integration of sensory and behavioral signals remain unclear. For example, most experiments studying sensory interactions have not explored attention, while most studies of attention have relied on the responses to relatively limited sets of stimuli. However, a recent study of V4 responses, in which location, orientation, and spatial attention were systematically varied, suggests that attention can both facilitate and suppress specific sensory inputs to a neuron according to behavioral relevance. To explore the implications of such input gain, we modeled the effects of a center-surround organization of attentional modulation using existing receptive field models of sensory integration. The model is consistent with behavioral measurements of a suppressive effect that surrounds the facilitatory locus of spatial attention. When this center-surround modulation is incorporated into realistic models of sensory integration, it is able to explain seemingly disparate observations of attentional effects in the neurophysiological literature, including spatial shifts in receptive field position and the preferential modulation of low contrast stimuli. The model is also consistent with recent formulations of attention to features in which gain is variably applied among cells with different receptive field properties. Consistent with functional imaging results, the model predicts that spatial attention effects will vary between different visual areas and suggests that attention may act through a common mechanism of selective and flexible gain throughout the visual system.

Early stages of melody processing: stimulus-sequence and task-dependent neuronal activity in monkey auditory cortical fields A1 and R

To explore the effects of acoustic and behavioral context on neuronal responses in the core of auditory cortex (fields A1 and R), two monkeys were trained on a go/no-go discrimination task in which they learned to respond selectively to a four-note target (S+) melody and withhold response to a variety of other nontarget (S-) sounds. We analyzed evoked activity from 683 units in A1/R of the trained monkeys during task performance and from 125 units in A1/R of two naive monkeys. We characterized two broad classes of neural activity that were modulated by task performance. Class I consisted of tone-sequence-sensitive enhancement and suppression responses. Enhanced or suppressed responses to specific tonal components of the S+ melody were frequently observed in trained monkeys, but enhanced responses were rarely seen in naive monkeys. Both facilitatory and suppressive responses in the trained monkeys showed a temporal pattern different from that observed in naive monkeys. Class II consisted of nonacoustic activity, characterized by a task-related component that correlated with bar release, the behavioral response leading to reward. We observed a significantly higher percentage of both Class I and Class II neurons in field R than in A1. Class I responses may help encode a long-term representation of the behaviorally salient target melody. Class II activity may reflect a variety of nonacoustic influences, such as attention, reward expectancy, somatosensory inputs, and/or motor set and may help link auditory perception and behavioral response. Both types of neuronal activity are likely to contribute to the performance of the auditory task.

Attention during active visual tasks: counting, pointing, or simply looking

Visual attention and saccades are typically studied in artificial situations, with stimuli presented to the steadily fixating eye, or saccades made along specified paths. By contrast, in real-world tasks saccadic patterns are constrained only by the demands of the motivating task. We studied attention during pauses between saccades made to perform three free-viewing tasks: counting dots, pointing to the same dots with a visible cursor, or simply looking at the dots using a freely-chosen path. Attention was assessed by the ability to identify the orientation of a briefly-presented Gabor probe. All primary tasks produced losses in identification performance, with counting producing the largest losses, followed by pointing and then looking-only. Looking-only resulted in a 37% increase in contrast thresholds in the orientation task. Counting produced more severe losses that were not overcome by increasing Gabor contrast. Detection or localization of the Gabor, unlike identification, were largely unaffected by any of the primary tasks. Taken together, these results show that attention is required to control saccades, even with freely-chosen paths, but the attentional demands of saccades are less than those attached to tasks such as counting, which have a significant cognitive load. Counting proved to be a highly demanding task that either exhausted momentary processing capacity (e.g., working memory or executive functions), or, alternatively, encouraged a strategy of filtering out all signals irrelevant to counting itself. The fact that the attentional demands of saccades (as well as those of detection/localization) are relatively modest makes it possible to continually adjust both the spatial and temporal pattern of saccades so as to re-allocate attentional resources as needed to handle the complex and multifaceted demands of real-world environments.

Task difficulty modulates the activity of specific neuronal populations in primary visual cortex

Spatial attention enhances our ability to detect stimuli at restricted regions of the visual field. This enhancement is thought to depend on the difficulty of the task being performed, but the underlying neuronal mechanisms for this dependency remain largely unknown. We found that task difficulty modulates neuronal firing rate at the earliest stages of cortical visual processing (area V1) in monkey (Macaca mulatta). These modulations were spatially specific: increasing task difficulty enhanced V1 neuronal firing rate at the focus of attention and suppressed it in regions surrounding the focus. Moreover, we found that response enhancement and suppression are mediated by distinct populations of neurons that differ in direction selectivity, spike width, interspike-interval distribution and contrast sensitivity. Our results provide strong support for center-surround models of spatial attention and suggest that task difficulty modulates the activity of specific populations of neurons in the primary visual cortex.

Spatial summation can explain the attentional modulation of neuronal responses to multiple stimuli in area V4

Although many studies have shown that the activity of individual neurons in a variety of visual areas is modulated by attention, a fundamental question remains unresolved: can attention alter the visual representations of individual neurons? One set of studies, primarily relying on the attentional modulations observed when a single stimulus is presented within the receptive field of a neuron, suggests that neuronal selectivities, such as orientation or direction tuning, are not fundamentally altered by attention (Salinas and Abbott, 1997; McAdams and Maunsell, 1999; Treue and Martinez Trujillo, 1999). Another set of studies, relying on modulations observed when multiple stimuli are presented within a receptive field, suggests that attention can alter the weighting of sensory inputs (Moran and Desimone, 1985; Luck et al., 1997; Reynolds et al., 1999; Chelazzi et al., 2001). In these studies, when preferred and nonpreferred stimuli are simultaneously presented, responses are much stronger when attention is directed to the preferred stimulus than when it is directed to the nonpreferred stimulus. In this study, we recorded neuronal responses from individual neurons in visual cortical area V4 to both single and paired stimuli with a variety of attentional allocations and stimulus combinations. For each neuron studied, we constructed a quantitative model of input summation and then tested various models of attention. In many neurons, we are able to explain neuronal responses across the entire range of stimuli and attentional allocations tested. Specifically, we are able to reconcile seemingly inconsistent observations of single and paired stimuli attentional modulation with a new model in which attention can facilitate or suppress specific inputs to a neuron but does not fundamentally alter the integration of these inputs.

Modulating the granularity of category formation by global cortical States

The unsupervised categorization of sensory stimuli is typically attributed to feedforward processing in a hierarchy of cortical areas. This purely sensory-driven view of cortical processing, however, ignores any internal modulation, e.g., by top-down attentional signals or neuromodulator release. To isolate the role of internal signaling on category formation, we consider an unbroken continuum of stimuli without intrinsic category boundaries. We show that a competitive network, shaped by recurrent inhibition and endowed with Hebbian and homeostatic synaptic plasticity, can enforce stimulus categorization. The degree of competition is internally controlled by the neuronal gain and the strength of inhibition. Strong competition leads to the formation of many attracting network states, each being evoked by a distinct subset of stimuli and representing a category. Weak competition allows more neurons to be co-active, resulting in fewer but larger categories. We conclude that the granularity of cortical category formation, i.e., the number and size of emerging categories, is not simply determined by the richness of the stimulus environment, but rather by some global internal signal modulating the network dynamics. The model also explains the salient non-additivity of visual object representation observed in the monkey inferotemporal (IT) cortex. Furthermore, it offers an explanation of a previously observed, demand-dependent modulation of IT activity on a stimulus categorization task and of categorization-related cognitive deficits in schizophrenic patients.

Spatial attention does not strongly modulate neuronal responses in early human visual cortex

Attention can dramatically enhance behavioral performance based on a visual stimulus, but the degree to which attention modulates activity in early visual cortex is unclear. Whereas single-unit studies of spatial attention in monkeys have repeatedly revealed relatively modest attentional modulations in V1, human functional magnetic resonance imaging studies demonstrate a large attentional enhancement of the blood oxygen level-dependent (BOLD) signal in V1. To explore this discrepancy, we used intracranial electrodes to directly measure the effect of spatial attention on the responses of neurons near the human occipital pole. We found that spatial attention does not robustly modulate stimulus-driven local field potentials in early human visual cortex, but instead produces modest modulations that are consistent with those seen in monkey neurophysiology experiments. This finding suggests that the neuronal activity that underlies visual attention in humans is similar to that found in other primates and that behavioral state may alter the linear relationship between neuronal activity and BOLD.

Auditory attention—focusing the searchlight on sound

Some fifty years after the first physiological studies of auditory attention, the field is now ripening, with exciting recent insights into the psychophysics, psychology, and neural basis of auditory attention. Current research seeks to unravel the complex interactions of pre-attentive and attentive processing of the acoustic scene, the role of auditory attention in mediating receptive-field plasticity in both auditory spatial and auditory feature processing, the contrasts and parallels between auditory and visual attention pathways and mechanisms, the interplay of bottom-up and top-down attentional mechanisms, the influential role of attention, goals, and expectations in shaping auditory processing, and the orchestration of diverse attentional effects at multiple levels from the cochlea to the cortex.

Fundamental components of attention

A mechanistic understanding of attention is necessary for the elucidation of the neurobiological basis of conscious experience. This chapter presents a framework for thinking about attention that facilitates the analysis of this cognitive process in terms of underlying neural mechanisms. Four processes are fundamental to attention: working memory, top-down sensitivity control, competitive selection, and automatic bottom-up filtering for salient stimuli. Each process makes a distinct and essential contribution to attention. Voluntary control of attention involves the first three processes (working memory, top-down sensitivity control, and competitive selection) operating in a recurrent loop. Recent results from neurobiological research on attention are discussed within this framework.