Optimal deployment of attentional gain during fine discriminations

Normalization in human somatosensory cortex

Functional magnetic resonance imaging (fMRI) was used to measure activity in human somatosensory cortex and to test for cross-digit suppression. Subjects received stimulation (vibration of varying amplitudes) to the right thumb (target) with or without concurrent stimulation of the right middle finger (mask). Subjects were less sensitive to target stimulation (psychophysical detection thresholds were higher) when target and mask digits were stimulated concurrently compared with when the target was stimulated in isolation. fMRI voxels in a region of the left postcentral gyrus each responded when either digit was stimulated. A regression model (called a forward model) was used to separate the fMRI measurements from these voxels into two hypothetical channels, each of which responded selectively to only one of the two digits. For the channel tuned to the target digit, responses in the left postcentral gyrus increased with target stimulus amplitude but were suppressed by concurrent stimulation to the mask digit, evident as a shift in the gain of the response functions. For the channel tuned to the mask digit, a constant baseline response was evoked for all target amplitudes when the mask was absent and responses decreased with increasing target amplitude when the mask was concurrently presented. A computational model based on divisive normalization provided a good fit to the measurements for both mask-absent and target + mask stimulation. We conclude that the normalization model can explain cross-digit suppression in human somatosensory cortex, supporting the hypothesis that normalization is a canonical neural computation.

Visual attention mitigates information loss in small- and large-scale neural codes

The visual system transforms complex inputs into robust and parsimonious neural codes that efficiently guide behavior. Because neural communication is stochastic, the amount of encoded visual information necessarily decreases with each synapse. This constraint requires that sensory signals are processed in a manner that protects information about relevant stimuli from degradation. Such selective processing--or selective attention--is implemented via several mechanisms, including neural gain and changes in tuning properties. However, examining each of these effects in isolation obscures their joint impact on the fidelity of stimulus feature representations by large-scale population codes. Instead, large-scale activity patterns can be used to reconstruct representations of relevant and irrelevant stimuli, thereby providing a holistic understanding about how neuron-level modulations collectively impact stimulus encoding.

Do humans make good decisions?

Human performance on perceptual classification tasks approaches that of an ideal observer, but economic decisions are often inconsistent and intransitive, with preferences reversing according to the local context. We discuss the view that suboptimal choices may result from the efficient coding of decision-relevant information, a strategy that allows expected inputs to be processed with higher gain than unexpected inputs. Efficient coding leads to 'robust' decisions that depart from optimality but maximise the information transmitted by a limited-capacity system in a rapidly-changing world. We review recent work showing that when perceptual environments are variable or volatile, perceptual decisions exhibit the same suboptimal context-dependence as economic choices, and we propose a general computational framework that accounts for findings across the two domains.

Encoding of graded changes in spatial specificity of prior cues in human visual cortex

Prior information about the relevance of spatial locations can vary in specificity; a single location, a subset of locations, or all locations may be of potential importance. Using a contrast-discrimination task with four possible targets, we asked whether performance benefits are graded with the spatial specificity of a prior cue and whether we could quantitatively account for behavioral performance with cortical activity changes measured by blood oxygenation level-dependent (BOLD) imaging. Thus we changed the prior probability that each location contained the target from 100 to 50 to 25% by cueing in advance 1, 2, or 4 of the possible locations. We found that behavioral performance (discrimination thresholds) improved in a graded fashion with spatial specificity. However, concurrently measured cortical responses from retinotopically defined visual areas were not strictly graded; response magnitude decreased when all 4 locations were cued (25% prior probability) relative to the 100 and 50% prior probability conditions, but no significant difference in response magnitude was found between the 100 and 50% prior probability conditions for either cued or uncued locations. Also, although cueing locations increased responses relative to noncueing, this cue sensitivity was not graded with prior probability. Furthermore, contrast sensitivity of cortical responses, which could improve contrast discrimination performance, was not graded. Instead, an efficient-selection model showed that even if sensory responses do not strictly scale with prior probability, selection of sensory responses by weighting larger responses more can result in graded behavioral performance benefits with increasing spatial specificity of prior information.

Induced α rhythms track the content and quality of visual working memory representations with high temporal precision

Past work has suggested that neuronal oscillations coordinate the cellular assemblies that represent items in working memory (WM). In line with this hypothesis, we show that the spatial distribution of power in the alpha frequency band (8-12 Hz) can be used to decode the content and quality of the representations stored in visual WM. We acquired EEG data during an orientation WM task, and used a forward encoding model of orientation selectivity to reconstruct orientation-specific response profiles (termed channel tuning functions, or CTFs) that tracked the orientation of the memorandum during both encoding and delay periods of the trial. Critically, these EEG-based CTFs were robust predictors of both between- and within-subject differences in mnemonic precision, showing that EEG-based CTFs provide a sensitive measure of the quality of sensory population codes. Experiments 2 and 3 established that these EEG-based CTFs are contingent on the voluntary storage goals of the observer. When observers were given a postsample cue to store or drop the memorandum, the resulting CTF was sustained in the "store" condition and rapidly eliminated following the "drop" cue. When observers were instructed to store one of two simultaneously presented stimuli, only the stored item was represented in a sustained fashion throughout the delay period. These findings suggest that the oscillatory activity in the alpha frequency band plays a central role in the active storage of information in visual WM, and demonstrate a powerful approach for tracking the precision of on-line memories with high temporal resolution.

Enhanced attentional gain as a mechanism for generalized perceptual learning in human visual cortex

Learning to better discriminate a specific visual feature (i.e., a specific orientation in a specific region of space) has been associated with plasticity in early visual areas (sensory modulation) and with improvements in the transmission of sensory information from early visual areas to downstream sensorimotor and decision regions (enhanced readout). However, in many real-world scenarios that require perceptual expertise, observers need to efficiently process numerous exemplars from a broad stimulus class as opposed to just a single stimulus feature. Some previous data suggest that perceptual learning leads to highly specific neural modulations that support the discrimination of specific trained features. However, the extent to which perceptual learning acts to improve the discriminability of a broad class of stimuli via the modulation of sensory responses in human visual cortex remains largely unknown. Here, we used functional MRI and a multivariate analysis method to reconstruct orientation-selective response profiles based on activation patterns in the early visual cortex before and after subjects learned to discriminate small offsets in a set of grating stimuli that were rendered in one of nine possible orientations. Behavioral performance improved across 10 training sessions, and there was a training-related increase in the amplitude of orientation-selective response profiles in V1, V2, and V3 when orientation was task relevant compared with when it was task irrelevant. These results suggest that generalized perceptual learning can lead to modified responses in the early visual cortex in a manner that is suitable for supporting improved discriminability of stimuli drawn from a large set of exemplars.

Visual crowding cannot be wholly explained by feature pooling

Visual perception is dramatically impaired when a peripheral target is embedded within clutter, a phenomenon known as visual crowding. Despite decades of study, the mechanisms underlying crowding remain a matter of debate. Feature pooling models assert that crowding results from a compulsory pooling (e.g., averaging) of target and distractor features. This view has been extraordinarily influential in recent years, so much so that crowding is typically regarded as synonymous with pooling. However, many demonstrations of feature pooling can also be accommodated by a probabilistic substitution model where observers occasionally report a distractor as the target. Here, we directly compared pooling and substitution using an analytical approach sensitive to both alternatives. In four experiments, we asked observers to report the precise orientation of a target stimulus flanked by two irrelevant distractors. In all cases, the observed data were well described by a quantitative model that assumes probabilistic substitution, and poorly described by a quantitative model that assumes that targets and distractors are averaged. These results challenge the widely held assumption that crowding can be wholly explained by compulsory pooling.

Optimal attentional modulation of a neural population

Top-down attention has often been separately studied in the contexts of either optimal population coding or biasing of visual search. Yet, both are intimately linked, as they entail optimally modulating sensory variables in neural populations according to top-down goals. Designing experiments to probe top-down attentional modulation is difficult because non-linear population dynamics are hard to predict in the absence of a concise theoretical framework. Here, we describe a unified framework that encompasses both contexts. Our work sheds light onto the ongoing debate on whether attention modulates neural response gain, tuning width, and/or preferred feature. We evaluate the framework by conducting simulations for two tasks: (1) classification (discrimination) of two stimuli s a and s b and (2) searching for a target T among distractors D. Results demonstrate that all of gain, tuning, and preferred feature modulation happen to different extents, depending on stimulus conditions and task demands. The theoretical analysis shows that task difficulty (linked to difference Δ between s a and s b , or T, and D) is a crucial factor in optimal modulation, with different effects in discrimination vs. search. Further, our framework allows us to quantify the relative utility of neural parameters. In easy tasks (when Δ is large compared to the density of the neural population), modulating gains and preferred features is sufficient to yield nearly optimal performance; however, in difficult tasks (smaller Δ), modulating tuning width becomes necessary to improve performance. This suggests that the conflicting reports from different experimental studies may be due to differences in tasks and in their difficulties. We further propose future electrophysiology experiments to observe different types of attentional modulation in a same neuron.

Functional size of human visual area V1: a neural correlate of top-down attention

Heavy demands are placed on the brain's attentional capacity when selecting a target item in a cluttered visual scene, or when reading. It is widely accepted that such attentional selection is mediated by top-down signals from higher cortical areas to early visual areas such as the primary visual cortex (V1). Further, it has also been reported that there is considerable variation in the surface area of V1. This variation may impact on either the number or specificity of attentional feedback signals and, thereby, the efficiency of attentional mechanisms. In this study, we investigated whether individual differences between humans performing attention-demanding tasks can be related to the functional area of V1. We found that those with a larger representation in V1 of the central 12° of the visual field as measured using BOLD signals from fMRI were able to perform a serial search task at a faster rate. In line with recent suggestions of the vital role of visuo-spatial attention in reading, the speed of reading showed a strong positive correlation with the speed of visual search, although it showed little correlation with the size of V1. The results support the idea that the functional size of the primary visual cortex is an important determinant of the efficiency of selective spatial attention for simple tasks, and that the attentional processing required for complex tasks like reading are to a large extent determined by other brain areas and inter-areal connections.

Differing effects of attention in single-units and populations are well predicted by heterogeneous tuning and the normalization model of attention

Single-unit measurements have reported many different effects of attention on contrast-response (e.g., contrast-gain, response-gain, additive-offset dependent on visibility), while functional imaging measurements have more uniformly reported increases in response across all contrasts (additive-offset). The normalization model of attention elegantly predicts the diversity of effects of attention reported in single-units well-tuned to the stimulus, but what predictions does it make for more realistic populations of neurons with heterogeneous tuning? Are predictions in accordance with population-scale measurements? We used functional imaging data from humans to determine a realistic ratio of attention-field to stimulus-drive size (a key parameter for the model) and predicted effects of attention in a population of model neurons with heterogeneous tuning. We found that within the population, neurons well-tuned to the stimulus showed a response-gain effect, while less-well-tuned neurons showed a contrast-gain effect. Averaged across the population, these disparate effects of attention gave rise to additive-offsets in contrast-response, similar to reports in human functional imaging as well as population averages of single-units. Differences in predictions for single-units and populations were observed across a wide range of model parameters (ratios of attention-field to stimulus-drive size and the amount of baseline response modifiable by attention), offering an explanation for disparity in physiological reports. Thus, by accounting for heterogeneity in tuning of realistic neuronal populations, the normalization model of attention can not only predict responses of well-tuned neurons, but also the activity of large populations of neurons. More generally, computational models can unify physiological findings across different scales of measurement, and make links to behavior, but only if factors such as heterogeneous tuning within a population are properly accounted for.

Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices

Computational theories propose that attention modulates the topographical landscape of spatial ‘priority’ maps in regions of visual cortex so that the location of an important object is associated with higher activation levels. While single-unit recording studies have demonstrated attention-related increases in the gain of neural responses and changes in the size of spatial receptive fields, the net effect of these modulations on the topography of region-level priority maps has not been investigated. Here, we used fMRI and a multivariate encoding model to reconstruct spatial representations of attended and ignored stimuli using activation patterns across entire visual areas. These reconstructed spatial representations reveal the influence of attention on the amplitude and size of stimulus representations within putative priority maps across the visual hierarchy. Our results suggest that attention increases the amplitude of stimulus representations in these spatial maps, particularly in higher visual areas, but does not substantively change their size.

Attending multiple items decreases the selectivity of population responses in human primary visual cortex

Multiple studies have documented an inverse relationship between the number of to-be-attended or remembered items in a display ("set size") and task performance. The neural source of this decline in cognitive performance is currently under debate. Here, we used a combination of fMRI and a forward encoding model of orientation selectivity to generate population tuning functions for each of two stimuli while human observers attended either one or both items. We observed (1) clear population tuning functions for the attended item(s) that peaked at the stimulus orientation and decreased monotonically as the angular distance from this orientation increased, (2) a set-size-dependent decline in the relative precision of orientation-specific population responses, such that attending two items yielded a decline in selectivity of the population tuning function for each item, and (3) that the magnitude of the loss of precision in population tuning functions predicted individual differences in the behavioral cost of attending an additional item. These findings demonstrate that attending multiple items degrades the precision of perceptual representations for the target items and provides a straightforward account for the associated impairments in visually guided behavior.

How attention extracts objects from noise

The visual system is remarkably proficient at extracting relevant object information from noisy, cluttered environments. Although attention is known to enhance sensory processing, the mechanisms by which attention extracts relevant information from noise are not well understood. According to the perceptual template model, attention may act to amplify responses to all visual input, or it may act as a noise filter, dampening responses to irrelevant visual noise. Amplification allows for improved performance in the absence of visual noise, whereas a noise-filtering mechanism can only improve performance if the target stimulus appears in noise. Here, we used fMRI to investigate how attention modulates cortical responses to objects at multiple levels of the visual pathway. Participants viewed images of faces, houses, chairs, and shoes, presented in various levels of visual noise. We used multivoxel pattern analysis to predict the viewed object category, for attended and unattended stimuli, from cortical activity patterns in individual visual areas. Early visual areas, V1 and V2, exhibited a benefit of attention only at high levels of visual noise, suggesting that attention operates via a noise-filtering mechanism at these early sites. By contrast, attention led to enhanced processing of noise-free images (i.e., amplification) only in higher visual areas, including area V4, fusiform face area, mid-Fusiform area, and the lateral occipital cortex. Together, these results suggest that attention improves people's ability to discriminate objects by de-noising visual input in early visual areas and amplifying this noise-reduced signal at higher stages of visual processing.

Individual differences in attention strategies during detection, fine discrimination, and coarse discrimination

Interacting with the environment requires the ability to flexibly direct attention to relevant features. We examined the degree to which individuals attend to visual features within and across Detection, Fine Discrimination, and Coarse Discrimination tasks. Electroencephalographic (EEG) responses were measured to an unattended peripheral flickering (4 or 6 Hz) grating while individuals (n = 33) attended to orientations that were offset by 0°, 10°, 20°, 30°, 40°, and 90° from the orientation of the unattended flicker. These unattended responses may be sensitive to attentional gain at the attended spatial location, since attention to features enhances early visual responses throughout the visual field. We found no significant differences in tuning curves across the three tasks in part due to individual differences in strategies. We sought to characterize individual attention strategies using hierarchical Bayesian modeling, which grouped individuals into families of curves that reflect attention to the physical target orientation ("on-channel") or away from the target orientation ("off-channel") or a uniform distribution of attention. The different curves were related to behavioral performance; individuals with "on-channel" curves had lower thresholds than individuals with uniform curves. Individuals with "off-channel" curves during Fine Discrimination additionally had lower thresholds than those assigned to uniform curves, highlighting the perceptual benefits of attending away from the physical target orientation during fine discriminations. Finally, we showed that a subset of individuals with optimal curves ("on-channel") during Detection also demonstrated optimal curves ("off-channel") during Fine Discrimination, indicating that a subset of individuals can modulate tuning optimally for detection and discrimination.

Near-real-time feature-selective modulations in human cortex

For neural activity to be linked with cognitive function, information is needed about both the temporal dynamics and the content of neural codes. Traditionally, recording of single neurons in animals has been the primary means of obtaining high temporal resolution and precise information about neural tuning properties such as selectivity for different sensory features. Recent functional magnetic resonance imaging (fMRI) studies in humans have been able to measure feature selectivity within specific subregions of sensory cortex (e.g., orientation selectivity in primary visual cortex, or V1) [1, 2]. However, investigating the neural mechanisms that support cognitive processing-which often occurs rapidly on a subsecond scale-with a temporally insensitive method such as fMRI severely limits the types of inferences that can be drawn. Here, we describe a new method for tracking the rapid temporal evolution of feature-selective information processing with scalp recordings of electroencephalography (EEG). We generate orientation-selective response profiles based on the spatially distributed pattern of steady-state visual evoked potential (SSVEP) responses to flickering visual stimuli. Using this approach, we report a multiplicative attentional modulation of these feature-selective response profiles with a temporal resolution of 24-120 ms, which is far faster than that achieved with fMRI. Finally, we show that behavioral performance on a discrimination task can be predicted based on the amplitude of these temporally precise feature-selective response profiles. This method thus provides a high-temporal-resolution metric that can be used to track the influence of cognitive manipulations on feature-selective information processing in human cortex.

Perceptual learning selectively refines orientation representations in early visual cortex

Although practice has long been known to improve perceptual performance, the neural basis of this improvement in humans remains unclear. Using fMRI in conjunction with a novel signal detection-based analysis, we show that extensive practice selectively enhances the neural representation of trained orientations in the human visual cortex. Twelve observers practiced discriminating small changes in the orientation of a laterally presented grating over 20 or more daily 1 h training sessions. Training on average led to a twofold improvement in discrimination sensitivity, specific to the trained orientation and the trained location, with minimal improvement found for untrained orthogonal orientations or for orientations presented in the untrained hemifield. We measured the strength of orientation-selective responses in individual voxels in early visual areas (V1-V4) using signal detection measures, both before and after training. Although the overall amplitude of the BOLD response was no greater after training, practice nonetheless specifically enhanced the neural representation of the trained orientation at the trained location. This training-specific enhancement of orientation-selective responses was observed in the primary visual cortex (V1) as well as higher extrastriate visual areas V2-V4, and moreover, reliably predicted individual differences in the behavioral effects of perceptual learning. These results demonstrate that extensive training can lead to targeted functional reorganization of the human visual cortex, refining the cortical representation of behaviorally relevant information.

Attention selects informative neural populations in human V1

In a neural population driven by a simple grating stimulus, different subpopulations are maximally informative about changes to the grating's orientation and contrast. In theory, observers should attend to the optimal subpopulation when switching between orientation and contrast discrimination tasks. Here we used source-imaged, steady-state visual evoked potentials and visual psychophysics to determine whether this is the case. Observers fixated centrally while static targets were presented bilaterally along with a cue indicating task type (contrast or orientation modulation detection) and task location (left or right). Changes in neuronal activity were measured by quantifying frequency-tagged responses from flickering "reporter" gratings surrounding the targets. To determine the orientation tuning of attentionally modulated neurons, we measured responses for three different probe-reporter angles: 0, 20, and 45°. We estimated frequency-tagged cortical activity using a minimum norm inverse procedure combined with realistic MR-derived head models and retinotopically mapped visual areas. Estimates of neural activity from regions of interest centered on V1 showed that attention to a spatial location clearly increased the amplitude of the neural response in that location. More importantly, the pattern of modulation depended on the task. For orientation discrimination, attentional modulation showed a sharp peak in the population tuned 20° from the target orientation, whereas for contrast discrimination the enhancement was more broadly tuned. Similar tuning functions for orientation and contrast discrimination were obtained from psychophysical adaptation studies. These findings indicate that humans attend selectively to the most informative neural population and that these populations change depending on the nature of the task.

Perceptual consequences of feature-based attentional enhancement and suppression

Feature-based attention has been shown to enhance the responses of neurons tuned to an attended feature while simultaneously suppressing responses of neurons tuned to unattended features. However, the influence of these suppressive neuronal-level modulations on perception is not well understood. Here, we investigated the perceptual consequences of feature-based attention by having subjects judge which of four random dot patterns (RDPs) contained a motion signal (Experiment 1) or which of four RDPs contained the most salient nonrandom motion signal (Experiment 2). Subjects viewed pre-cues which validly, invalidly, or neutrally cued the direction of the target RDP. Behavioral data were fit using the linear ballistic accumulator (LBA) model; the model design that best described the data revealed that the rate of sensory evidence accumulation (drift rate) was highest on valid trials and systematically decreased until the cued direction and the target direction were orthogonal. These results demonstrate behavioral correlates of both feature-based attentional enhancement and suppression.

Exploring the relationship between perceptual learning and top-down attentional control

Here, we review the role of top-down attention in both the acquisition and the expression of perceptual learning, as well as the role of learning in more efficiently guiding attentional modulations. Although attention often mediates learning at the outset of training, many of the characteristic behavioral and neural changes associated with learning can be observed even when stimuli are task irrelevant and ignored. However, depending on task demands, attention can override the effects of perceptual learning, suggesting that even if top-down factors are not strictly necessary to observe learning, they play a critical role in determining how learning-related changes in behavior and neural activity are ultimately expressed. In turn, training may also act to optimize the effectiveness of top-down attentional control by improving the efficiency of sensory gain modulations, regulating intrinsic noise, and altering the read-out of sensory information.