The normalization model of attention

Using neuronal populations to study the mechanisms underlying spatial and feature attention

Visual attention affects both perception and neuronal responses. Whether the same neuronal mechanisms mediate spatial attention, which improves perception of attended locations, and nonspatial forms of attention has been a subject of considerable debate. Spatial and feature attention have similar effects on individual neurons. Because visual cortex is retinotopically organized, however, spatial attention can comodulate local neuronal populations, whereas feature attention generally requires more selective modulation. We compared the effects of feature and spatial attention on local and spatially separated populations by recording simultaneously from dozens of neurons in both hemispheres of V4. Feature and spatial attention affect the activity of local populations similarly, modulating both firing rates and correlations between pairs of nearby neurons. However, whereas spatial attention appears to act on local populations, feature attention is coordinated across hemispheres. Our results are consistent with a unified attentional mechanism that can modulate the responses of arbitrary subgroups of neurons.

Contrast gain control abnormalities in idiopathic generalized epilepsy

OBJECTIVE

The origin of neural hyperexcitability underlying idiopathic generalized epilepsy (IGE) is not known. The objective of this study is to identify evidence of hyperexcitability in precisely measured visual evoked responses and to understand the nature of changes in excitation and inhibition that lead to altered responses in human patients with IGE.

METHODS

Steady-state visual-evoked potentials (VEPs) to contrast reversing gratings were recorded over a wide range of stimulus contrast. VEPs were analyzed at the pattern reversal rate using spectral analysis. Ten patients with IGE and 13 healthy subjects participated. All subjects had normal visual acuity and had no history of photic-induced seizures or photoparoxysmal electroencephalograph (EEG) activity.

RESULTS

At a group level, the amplitude of visual responses did not saturate at high stimulus contrast in patients, as it did in the control subjects. This reflects an abnormality in neuronal gain control. The VEPs did not have sufficient power to reliably distinguish patients from controls at an individual level. Parametric modeling using a standard gain control framework showed that the abnormality lay in reduced inhibition from neighboring neurons rather than increased excitatory response to the stimulus.

INTERPRETATION

Visual evoked responses reveal changes in a fundamental mechanism regulating neuronal sensitivity. These changes may give rise to hyperexcitability underlying generalized epilepsy.

Modeling Single-Trial ERP Reveals Modulation of Bottom-Up Face Visual Processing by Top-Down Task Constraints (in Some Subjects)

We studied how task constraints modulate the relationship between single-trial event-related potentials (ERPs) and image noise. Thirteen subjects performed two interleaved tasks: on different blocks, they saw the same stimuli, but they discriminated either between two faces or between two colors. Stimuli were two pictures of red or green faces that contained from 10 to 80% of phase noise, with 10% increments. Behavioral accuracy followed a noise dependent sigmoid in the identity task but was high and independent of noise level in the color task. EEG data recorded concurrently were analyzed using a single-trial ANCOVA: we assessed how changes in task constraints modulated ERP noise sensitivity while regressing out the main ERP differences due to identity, color, and task. Single-trial ERP sensitivity to image phase noise started at about 95-110 ms post-stimulus onset. Group analyses showed a significant reduction in noise sensitivity in the color task compared to the identity task from about 140 ms to 300 ms post-stimulus onset. However, statistical analyses in every subject revealed different results: significant task modulation occurred in 8/13 subjects, one showing an increase and seven showing a decrease in noise sensitivity in the color task. Onsets and durations of effects also differed between group and single-trial analyses: at any time point only a maximum of four subjects (31%) showed results consistent with group analyses. We provide detailed results for all 13 subjects, including a shift function analysis that revealed asymmetric task modulations of single-trial ERP distributions. We conclude that, during face processing, bottom-up sensitivity to phase noise can be modulated by top-down task constraints, in a broad window around the P2, at least in some subjects.

Neural substrates of cognitive capacity limitations

Cognition has a severely limited capacity: Adult humans can retain only about four items "in mind". This limitation is fundamental to human brain function: Individual capacity is highly correlated with intelligence measures and capacity is reduced in neuropsychiatric diseases. Although human capacity limitations are well studied, their mechanisms have not been investigated at the single-neuron level. Simultaneous recordings from monkey parietal and frontal cortex revealed that visual capacity limitations occurred immediately upon stimulus encoding and in a bottom-up manner. Capacity limitations were found to reflect a dual model of working memory. The left and right halves of visual space had independent capacities and thus are discrete resources. However, within each hemifield, neural information about successfully remembered objects was reduced by adding further objects, indicating that resources are shared. Together, these results suggest visual capacity limitation is due to discrete, slot-like, resources, each containing limited pools of neural information that can be divided among objects.

Spatial attention, feature-based attention, and saccades: three sides of one coin?

The last three decades has seen a steady growth of neuroscience research aimed at understanding the functions and sources of top-down attentional modulation in the brain. This correlates with recognition that attention may be a necessary component of sensory systems to support natural behaviors in natural environments. Complexity and clutter are two of the most recognizable hallmarks of natural environments, which can simultaneously contain vitally important and completely irrelevant stimuli. Attention serves as an adaptive filter providing each sensory modality preferential processing routes for important stimuli while suppressing responses to distracters, thus optimizing use of limited neural resources. In other words, attention is the family of mechanisms by which organisms are able to effectively and selectively allocate limited neural resources to achieve specific behavioral goals. This review provides some historical context for considering attentional frameworks and modern neurophysiological attention research, focusing on visual attention. A taxonomy of common attentional effects and neural mechanisms is provided, along with consideration of the specific relationship between attention and saccade planning. We examine the validity of premotor theories of attention, which posit that attention and saccade planning are one and the same. While there is strong evidence that attention and oculomotor planning are similar, with shared neural substrates, there is also evidence that these two functions are not synonymous. Finally, we examine neurophysiological explanations for dysfunction in attention-deficit/hyperactivity disorder and the hypothesis that social impairment in autism spectrum disorders is partially attributable to perturbations of attentional control circuitry.

Signaling of the strongest stimulus in the owl optic tectum

Essential to the selection of the next target for gaze or attention is the ability to compare the strengths of multiple competing stimuli (bottom-up information) and to signal the strongest one. Although the optic tectum (OT) has been causally implicated in stimulus selection, how it computes the strongest stimulus is unknown. Here, we demonstrate that OT neurons in the barn owl systematically encode the relative strengths of simultaneously occurring stimuli independently of sensory modality. Moreover, special "switch-like" responses of a subset of neurons abruptly increase when the stimulus inside their receptive field becomes the strongest one. Such responses are not predicted by responses to single stimuli and, indeed, are eliminated in the absence of competitive interactions. We demonstrate that this sensory transformation substantially boosts the representation of the strongest stimulus by creating a binary discrimination signal, thereby setting the stage for potential winner-take-all target selection for gaze and attention.

Strength of response suppression to distracter stimuli determines attentional-filtering performance in primate prefrontal neurons

Neurons in the primate dorsolateral prefrontal cortex (dlPFC) filter attended [corrected] targets distinctly from distracters through their response rates. The extent to which this ability correlates with the organism's performance, and the neural processes underlying it, remain unclear. We trained monkeys to attend to a visual target that differed in rank along a color-ordinal scale from that of a distracter. The animals' performance at focusing attention on the target and filtering out the distracter improved as ordinal distance between the stimuli increased. Importantly, dlPFC neurons also improved their filtering performance with increasing ordinal target-distracter distance; they built up their response rate in anticipation of the target-distracter onset, and then units encoding target representations increased their firing rate by similar amounts, whereas units encoding distracter representations gradually suppressed their rates as the interstimulus ordinal distance increased. These results suggest that attentional-filtering performance in primates relies upon dlPFC neurons' ability to suppress distracter representations.

Attention does more than modulate suppressive interactions: attending to multiple items

Directing attention to a visual item enhances its representations, making it more likely to guide behavior (Corbetta et al. 1991). Attention is thought to produce this enhancement by biasing suppressive interactions among multiple items in visual cortex in favor of the attended item (e.g., Desimone and Duncan 1995; Reynolds and Heeger 2009). We ask whether target enhancement and modulation of suppressive interactions are in fact inextricably linked or whether they can be decoupled. In particular, we ask whether simultaneously directing attention to multiple items may be one means of dissociating the influence of attention-related enhancement from the effects of inter-item suppression. When multiple items are attended, suppressive interactions in visual cortex limit the effectiveness with which attention may act on their representations, presumably because "biasing" the interactions in favor of a single item is no longer possible (Scalf and Beck 2010). In this experiment, we directly investigate whether applying attention to multiple competing stimulus items has any influence on either their evoked signal or their suppressive interactions. Both BOLD signal evoked by the items in V4 and behavioral responses to those items were significantly compromised by simultaneous presentation relative to simultaneous presentation, indicating that when the items appeared at the same time, they interacted in a mutually suppressive manner that compromised their ability to guide behavior. Attention significantly enhanced signal in V4. The attentional status of the items, however, had no influence on the suppressive effects of simultaneous presentation. To our knowledge, these data are the first to explicitly decouple the effects of top-down attention from those of inter-item suppression.

An attentional blink for nontargets?

In a typical attentional blink experiment, viewers try to detect two target items among distractors in a Rapid Serial Visual Presentation (RSVP): processing of the first target impairs participants' ability to recall a subsequent target at short stimulus onset asynchronies (SOAs). However, little is known about whether target detection interferes with memory for nontarget items. To answer this question, in two experiments, we employed a novel dual-task procedure: participants searched for a word target (e.g., "a four-footed animal") and then were tested for recognition of nontarget words. Detection of the target word, which was present on half the trials, produced a standard attentional blink effect on memory for nontarget words, with lag 1 sparing followed by an attentional blink at longer lags. This result shows that target processing has a generalized effect on processing of later events, not only other targets.

Bistable percepts in the brain: FMRI contrasts monocular pattern rivalry and binocular rivalry

The neural correlates of binocular rivalry have been actively debated in recent years, and are of considerable interest as they may shed light on mechanisms of conscious awareness. In a related phenomenon, monocular rivalry, a composite image is shown to both eyes. The subject experiences perceptual alternations in which the two stimulus components alternate in clarity or salience. The experience is similar to perceptual alternations in binocular rivalry, although the reduction in visibility of the suppressed component is greater for binocular rivalry, especially at higher stimulus contrasts. We used fMRI at 3T to image activity in visual cortex while subjects perceived either monocular or binocular rivalry, or a matched non-rivalrous control condition. The stimulus patterns were left/right oblique gratings with the luminance contrast set at 9%, 18% or 36%. Compared to a blank screen, both binocular and monocular rivalry showed a U-shaped function of activation as a function of stimulus contrast, i.e. higher activity for most areas at 9% and 36%. The sites of cortical activation for monocular rivalry included occipital pole (V1, V2, V3), ventral temporal, and superior parietal cortex. The additional areas for binocular rivalry included lateral occipital regions, as well as inferior parietal cortex close to the temporoparietal junction (TPJ). In particular, higher-tier areas MT+ and V3A were more active for binocular than monocular rivalry for all contrasts. In comparison, activation in V2 and V3 was reduced for binocular compared to monocular rivalry at the higher contrasts that evoked stronger binocular perceptual suppression, indicating that the effects of suppression are not limited to interocular suppression in V1.

Modulation of visual physiology by behavioral state in monkeys, mice, and flies

When a monkey attends to a visual stimulus, neurons in visual cortex respond differently to that stimulus than when the monkey attends elsewhere. In the 25 years since the initial discovery, the study of attention in primates has been central to understanding flexible visual processing. Recent experiments demonstrate that visual neurons in mice and fruit flies are modulated by locomotor behaviors, like running and flying, in a manner that resembles attention-based modulations in primates. The similar findings across species argue for a more generalized view of state-dependent sensory processing and for a renewed dialogue among vertebrate and invertebrate research communities.

Pulsed out of awareness: EEG alpha oscillations represent a pulsed-inhibition of ongoing cortical processing

Alpha oscillations are ubiquitous in the brain, but their role in cortical processing remains a matter of debate. Recently, evidence has begun to accumulate in support of a role for alpha oscillations in attention selection and control. Here we first review evidence that 8-12 Hz oscillations in the brain have a general inhibitory role in cognitive processing, with an emphasis on their role in visual processing. Then, we summarize the evidence in support of our recent proposal that alpha represents a pulsed-inhibition of ongoing neural activity. The phase of the ongoing electroencephalography can influence evoked activity and subsequent processing, and we propose that alpha exerts its inhibitory role through alternating microstates of inhibition and excitation. Finally, we discuss evidence that this pulsed-inhibition can be entrained to rhythmic stimuli in the environment, such that preferential processing occurs for stimuli at predictable moments. The entrainment of preferential phase may provide a mechanism for temporal attention in the brain. This pulsed inhibitory account of alpha has important implications for many common cognitive phenomena, such as the attentional blink, and seems to indicate that our visual experience may at least some times be coming through in waves.

Object decoding with attention in inferior temporal cortex

Recognizing objects in cluttered scenes requires attentional mechanisms to filter out distracting information. Previous studies have found several physiological correlates of attention in visual cortex, including larger responses for attended objects. However, it has been unclear whether these attention-related changes have a large impact on information about objects at the neural population level. To address this question, we trained monkeys to covertly deploy their visual attention from a central fixation point to one of three objects displayed in the periphery, and we decoded information about the identity and position of the objects from populations of ∼ 200 neurons from the inferior temporal cortex using a pattern classifier. The results show that before attention was deployed, information about the identity and position of each object was greatly reduced relative to when these objects were shown in isolation. However, when a monkey attended to an object, the pattern of neural activity, represented as a vector with dimensionality equal to the size of the neural population, was restored toward the vector representing the isolated object. Despite this nearly exclusive representation of the attended object, an increase in the salience of nonattended objects caused "bottom-up" mechanisms to override these "top-down" attentional enhancements. The method described here can be used to assess which attention-related physiological changes are directly related to object recognition, and should be helpful in assessing the role of additional physiological changes in the future.

A normalization model of multisensory integration

Responses of neurons that integrate multiple sensory inputs are traditionally characterized in terms of a set of empirical principles. However, a simple computational framework that accounts for these empirical features of multisensory integration has not been established. We propose that divisive normalization, acting at the stage of multisensory integration, can account for many of the empirical principles of multisensory integration shown by single neurons, such as the principle of inverse effectiveness and the spatial principle. This model, which uses a simple functional operation (normalization) for which there is considerable experimental support, also accounts for the recent observation that the mathematical rule by which multisensory neurons combine their inputs changes with cue reliability. The normalization model, which makes a strong testable prediction regarding cross-modal suppression, may therefore provide a simple unifying computational account of the important features of multisensory integration by neurons.

Behavior and neural basis of near-optimal visual search

The ability to search efficiently for a target in a cluttered environment is one of the most remarkable functions of the nervous system. This task is difficult under natural circumstances, as the reliability of sensory information can vary greatly across space and time and is typically a priori unknown to the observer. In contrast, visual-search experiments commonly use stimuli of equal and known reliability. In a target detection task, we randomly assigned high or low reliability to each item on a trial-by-trial basis. An optimal observer would weight the observations by their trial-to-trial reliability and combine them using a specific nonlinear integration rule. We found that humans were near-optimal, regardless of whether distractors were homogeneous or heterogeneous and whether reliability was manipulated through contrast or shape. We present a neural-network implementation of near-optimal visual search based on probabilistic population coding. The network matched human performance.

Distinguishing blocking from attenuation in visual selective attention

Sensory information must be processed selectively in order to represent the world and guide behavior. How does such selection occur? Here we consider two alternative classes of selection mechanisms: In blocking, unattended stimuli are blocked entirely from access to downstream processes, and in attenuation, unattended stimuli are reduced in strength but if strong enough can still access downstream processes. Existing evidence as to whether blocking or attenuation is a more accurate model of human performance is mixed. Capitalizing on a general distinction between blocking and attenuation-blocking cannot be overcome by strong stimuli, whereas attenuation can-we measured how attention interacted with the strength of stimuli in two spatial selection paradigms, spatial filtering and spatial monitoring. The evidence was consistent with blocking for the filtering paradigm and with attenuation for the monitoring paradigm. This approach provides a general measure of the fate of unattended stimuli.

Improvement and impairment of visually guided behavior through LTP- and LTD-like exposure-based visual learning

Cellular studies have focused on long-term potentiation (LTP) and long-term depression (LTD) to understand requirements for persistent changes in synaptic connections. Whereas LTP is induced through high-frequency intermittent stimulation, low-frequency stimulation evokes LTD. Because of the ubiquitous efficacy of these protocols, they are considered fundamental mechanisms underlying learning. Here we adapted LTP/LTD-like protocols to visual stimulation to alter human visually guided behavior. In a change-detection task, participants reported luminance changes against distracting orientation changes. Subsequently, they were exposed to passive visual high- or low-frequency stimulation of either the relevant luminance or irrelevant orientation feature. LTP-like high-frequency protocols using luminance improved ability to detect luminance changes, whereas low-frequency LTD-like stimulation impaired performance. In contrast, LTP-like exposure of the irrelevant orientation feature impaired performance, whereas LTD-like orientation stimulation improved it. LTP-like effects were present for 10 days, whereas LTD-like effects lasted for a shorter period of time. Our data demonstrate that instead of electrically stimulating synapses, selective behavioral changes are evoked in humans by using equivalently timed visual stimulation, suggesting that both LTD- and LTP-like protocols control human behavior but that the direction of changes is determined by the feature incorporated into the stimulation protocol.

Attention extracts signal in external noise: a BOLD fMRI study

On the basis of results from behavioral studies that spatial attention improves the exclusion of external noise in the target region, we predicted that attending to a spatial region would reduce the impact of external noise on the BOLD response in corresponding cortical areas, seen as reduced BOLD responses in conditions with large amounts of external noise but relatively low signal, and increased dynamic range of the BOLD response to variations in signal contrast. We found that, in the presence of external noise, covert attention reduced the trial-by-trial BOLD response by 15.5-18.9% in low signal contrast conditions in V1. It also increased the BOLD dynamic range in V1, V2, V3, V3A/B, and V4 by a factor of at least three. Overall, covert attention reduced the impact of external noise by about 73-85% in these early visual areas. It also increased the contrast gain by a factor of 2.6-3.8.

Visual attention: the past 25 years

This review focuses on covert attention and how it alters early vision. I explain why attention is considered a selective process, the constructs of covert attention, spatial endogenous and exogenous attention, and feature-based attention. I explain how in the last 25 years research on attention has characterized the effects of covert attention on spatial filters and how attention influences the selection of stimuli of interest. This review includes the effects of spatial attention on discriminability and appearance in tasks mediated by contrast sensitivity and spatial resolution; the effects of feature-based attention on basic visual processes, and a comparison of the effects of spatial and feature-based attention. The emphasis of this review is on psychophysical studies, but relevant electrophysiological and neuroimaging studies and models regarding how and where neuronal responses are modulated are also discussed.

Attentional modulation of temporal contrast sensitivity in human vision

Recent psychophysical studies have shown that attention can alter contrast sensitivities for temporally broadband stimuli such as flashed gratings. The present study examined the effect of attention on the contrast sensitivity for temporally narrowband stimuli with various temporal frequencies. Observers were asked to detect a drifting grating of 0–40 Hz presented gradually in the peripheral visual field with or without a concurrent letter identification task in the fovea. We found that removal of attention by the concurrent task reduced the contrast sensitivity for gratings with low temporal frequencies much more profoundly than for gratings with high temporal frequencies and for flashed gratings. The analysis revealed that the temporal contrast sensitivity function had a more band-pass shape with poor attention. Additional experiments showed that this was also true when the target was presented in various levels of luminance noise. These results suggest that regardless of the presence of external noise, attention extensively modulates visual sensitivity for sustained retinal inputs.

On the road to invariant recognition: explaining tradeoff and morph properties of cells in inferotemporal cortex using multiple-scale task-sensitive attentive learning

Visual object recognition is an essential accomplishment of advanced brains. Object recognition needs to be tolerant, or invariant, with respect to changes in object position, size, and view. In monkeys and humans, a key area for recognition is the anterior inferotemporal cortex (ITa). Recent neurophysiological data show that ITa cells with high object selectivity often have low position tolerance. We propose a neural model whose cells learn to simulate this tradeoff, as well as ITa responses to image morphs, while explaining how invariant recognition properties may arise in stages due to processes across multiple cortical areas. These processes include the cortical magnification factor, multiple receptive field sizes, and top-down attentive matching and learning properties that may be tuned by task requirements to attend to either concrete or abstract visual features with different levels of vigilance. The model predicts that data from the tradeoff and image morph tasks emerge from different levels of vigilance in the animals performing them. This result illustrates how different vigilance requirements of a task may change the course of category learning, notably the critical features that are attended and incorporated into learned category prototypes. The model outlines a path for developing an animal model of how defective vigilance control can lead to symptoms of various mental disorders, such as autism and amnesia.

How does the brain rapidly learn and reorganize view-invariant and position-invariant object representations in the inferotemporal cortex?

All primates depend for their survival on being able to rapidly learn about and recognize objects. Objects may be visually detected at multiple positions, sizes, and viewpoints. How does the brain rapidly learn and recognize objects while scanning a scene with eye movements, without causing a combinatorial explosion in the number of cells that are needed? How does the brain avoid the problem of erroneously classifying parts of different objects together at the same or different positions in a visual scene? In monkeys and humans, a key area for such invariant object category learning and recognition is the inferotemporal cortex (IT). A neural model is proposed to explain how spatial and object attention coordinate the ability of IT to learn invariant category representations of objects that are seen at multiple positions, sizes, and viewpoints. The model clarifies how interactions within a hierarchy of processing stages in the visual brain accomplish this. These stages include the retina, lateral geniculate nucleus, and cortical areas V1, V2, V4, and IT in the brain's What cortical stream, as they interact with spatial attention processes within the parietal cortex of the Where cortical stream. The model builds upon the ARTSCAN model, which proposed how view-invariant object representations are generated. The positional ARTSCAN (pARTSCAN) model proposes how the following additional processes in the What cortical processing stream also enable position-invariant object representations to be learned: IT cells with persistent activity, and a combination of normalizing object category competition and a view-to-object learning law which together ensure that unambiguous views have a larger effect on object recognition than ambiguous views. The model explains how such invariant learning can be fooled when monkeys, or other primates, are presented with an object that is swapped with another object during eye movements to foveate the original object. The swapping procedure is predicted to prevent the reset of spatial attention, which would otherwise keep the representations of multiple objects from being combined by learning. Li and DiCarlo (2008) have presented neurophysiological data from monkeys showing how unsupervised natural experience in a target swapping experiment can rapidly alter object representations in IT. The model quantitatively simulates the swapping data by showing how the swapping procedure fools the spatial attention mechanism. More generally, the model provides a unifying framework, and testable predictions in both monkeys and humans, for understanding object learning data using neurophysiological methods in monkeys, and spatial attention, episodic learning, and memory retrieval data using functional imaging methods in humans.

Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects

Visual attention is often understood as a modulatory field acting at early stages of processing, but the mechanisms that direct and fit the field to the attended object are not known. We show that a purely spatial attention field propagating downward in the neuronal network responsible for perceptual organization will be reshaped, repositioned, and sharpened to match the object's shape and scale. Key features of the model are grouping neurons integrating local features into coherent tentative objects, excitatory feedback to the same local feature neurons that caused grouping neuron activation, and inhibition between incompatible interpretations both at the local feature level and at the object representation level.

Sensorimotor integration in speech processing: computational basis and neural organization

Sensorimotor integration is an active domain of speech research and is characterized by two main ideas, that the auditory system is critically involved in speech production and that the motor system is critically involved in speech perception. Despite the complementarity of these ideas, there is little crosstalk between these literatures. We propose an integrative model of the speech-related "dorsal stream" in which sensorimotor interaction primarily supports speech production, in the form of a state feedback control architecture. A critical component of this control system is forward sensory prediction, which affords a natural mechanism for limited motor influence on perception, as recent perceptual research has suggested. Evidence shows that this influence is modulatory but not necessary for speech perception. The neuroanatomy of the proposed circuit is discussed as well as some probable clinical correlates including conduction aphasia, stuttering, and aspects of schizophrenia.

A framework for local cortical oscillation patterns

Oscillations are a pervasive feature of neuronal activity in the cerebral cortex. Here, we propose a framework for understanding local cortical oscillation patterns in cognition: two classes of network interactions underlying two classes of cognitive functions produce different local oscillation patterns. Local excitatory-inhibitory interactions shape neuronal representations of sensory, motor and cognitive variables, and produce local gamma-band oscillations. By contrast, the linkage of such representations by integrative functions such as decision-making is mediated by long-range cortical interactions, which yield more diverse local oscillation patterns often involving the beta range. This framework reconciles different cortical oscillation patterns observed in recent studies and helps to understand the link between cortical oscillations and the fMRI signal. Our framework highlights the notion that cortical oscillations index the specific circuit-level mechanisms of cognition.

Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech

Humans are remarkably adept at understanding speech, even when it is contaminated by noise. Multisensory integration may explain some of this ability: combining independent information from the auditory modality (vocalizations) and the visual modality (mouth movements) reduces noise and increases accuracy. Converging evidence suggests that the superior temporal sulcus (STS) is a critical brain area for multisensory integration, but little is known about its role in the perception of noisy speech. Behavioral studies have shown that perceptual judgments are weighted by the reliability of the sensory modality: more reliable modalities are weighted more strongly, even if the reliability changes rapidly. We hypothesized that changes in the functional connectivity of STS with auditory and visual cortex could provide a neural mechanism for perceptual reliability weighting. To test this idea, we performed five blood oxygenation level-dependent functional magnetic resonance imaging and behavioral experiments in 34 healthy subjects. We found increased functional connectivity between the STS and auditory cortex when the auditory modality was more reliable (less noisy) and increased functional connectivity between the STS and visual cortex when the visual modality was more reliable, even when the reliability changed rapidly during presentation of successive words. This finding matched the results of a behavioral experiment in which the perception of incongruent audiovisual syllables was biased toward the more reliable modality, even with rapidly changing reliability. Changes in STS functional connectivity may be an important neural mechanism underlying the perception of noisy speech.

Network anatomy and in vivo physiology of visual cortical neurons

In the cerebral cortex, local circuits consist of tens of thousands of neurons, each of which makes thousands of synaptic connections. Perhaps the biggest impediment to understanding these networks is that we have no wiring diagrams of their interconnections. Even if we had a partial or complete wiring diagram, however, understanding the network would also require information about each neuron's function. Here we show that the relationship between structure and function can be studied in the cortex with a combination of in vivo physiology and network anatomy. We used two-photon calcium imaging to characterize a functional property—the preferred stimulus orientation—of a group of neurons in the mouse primary visual cortex. We then used large-scale electron microscopy (EM) of serial thin sections to trace a portion of these neurons’ local network. Consistent with a prediction from recent physiological experiments, inhibitory interneurons received convergent anatomical input from nearby excitatory neurons with a broad range of preferred orientations, although weak biases could not be rejected.

Spatiotopic coding and remapping in humans

How our perceptual experience of the world remains stable and continuous in the face of continuous rapid eye movements still remains a mystery. This review discusses some recent progress towards understanding the neural and psychophysical processes that accompany these eye movements. We firstly report recent evidence from imaging studies in humans showing that many brain regions are tuned in spatiotopic coordinates, but only for items that are actively attended. We then describe a series of experiments measuring the spatial and temporal phenomena that occur around the time of saccades, and discuss how these could be related to visual stability. Finally, we introduce the concept of the spatio-temporal receptive field to describe the local spatiotopicity exhibited by many neurons when the eyes move.

Exogenous attention enhances 2nd-order contrast sensitivity

Natural scenes contain a rich variety of contours that the visual system extracts to segregate the retinal image into perceptually coherent regions. Covert spatial attention helps extract contours by enhancing contrast sensitivity for 1st-order, luminance-defined patterns at attended locations, while reducing sensitivity at unattended locations, relative to neutral attention allocation. However, humans are also sensitive to 2nd-order patterns such as spatial variations of texture, which are predominant in natural scenes and cannot be detected by linear mechanisms. We assess whether and how exogenous attention--the involuntary and transient capture of spatial attention--affects the contrast sensitivity of channels sensitive to 2nd-order, texture-defined patterns. Using 2nd-order, texture-defined stimuli, we demonstrate that exogenous attention increases 2nd-order contrast sensitivity at the attended location, while decreasing it at unattended locations, relative to a neutral condition. By manipulating both 1st- and 2nd-order spatial frequency, we find that the effects of attention depend both on 2nd-order spatial frequency of the stimulus and the observer's 2nd-order spatial resolution at the target location. At parafoveal locations, attention enhances 2nd-order contrast sensitivity to high, but not to low 2nd-order spatial frequencies; at peripheral locations attention also enhances sensitivity to low 2nd-order spatial frequencies. Control experiments rule out the possibility that these effects might be due to an increase in contrast sensitivity at the 1st-order stage of visual processing. Thus, exogenous attention affects 2nd-order contrast sensitivity at both attended and unattended locations.

Beyond multiple pattern analyzers modeled as linear filters (as classical V1 simple cells): useful additions of the last 25 years

This review briefly discusses processes that have been suggested in the last 25 years as important to the intermediate stages of visual processing of patterns. Five categories of processes are presented: (1) Higher-order processes including FRF structures; (2) Divisive contrast nonlinearities including contrast normalization; (3) Subtractive contrast nonlinearities including contrast comparison; (4) Non-classical receptive fields (surround suppression, cross-orientation inhibition); (5) Contour integration.

Reward sharpens orientation coding independently of attention

It has long been known that rewarding improves performance. However it is unclear whether this is due to high level modulations in the output modules of associated neural systems or due to low level mechanisms favoring more “generous” inputs? Some recent studies suggest that primary sensory areas, including V1 and A1, may form part of the circuitry of reward-based modulations, but there is no data indicating whether reward can be dissociated from attention or cross-trial forms of perceptual learning. Here we address this issue with a psychophysical dual task, to control attention, while perceptual performance on oriented targets associated with different levels of reward is assessed by measuring both orientation discrimination thresholds and behavioral tuning functions for tilt values near threshold. We found that reward, at any rate, improved performance. However, higher reward rates showed an improvement of orientation discrimination thresholds by about 50% across conditions and sharpened behavioral tuning functions. Data were unaffected by changing the attentional load and by dissociating the feature of the reward cue from the task-relevant feature. These results suggest that reward may act within the span of a single trial independently of attention by modulating the activity of early sensory stages through a improvement of the signal-to-noise ratio of task-relevant channels.

Action understanding and active inference

We have suggested that the mirror-neuron system might be usefully understood as implementing Bayes-optimal perception of actions emitted by oneself or others. To substantiate this claim, we present neuronal simulations that show the same representations can prescribe motor behavior and encode motor intentions during action-observation. These simulations are based on the free-energy formulation of active inference, which is formally related to predictive coding. In this scheme, (generalised) states of the world are represented as trajectories. When these states include motor trajectories they implicitly entail intentions (future motor states). Optimizing the representation of these intentions enables predictive coding in a prospective sense. Crucially, the same generative models used to make predictions can be deployed to predict the actions of self or others by simply changing the bias or precision (i.e. attention) afforded to proprioceptive signals. We illustrate these points using simulations of handwriting to illustrate neuronally plausible generation and recognition of itinerant (wandering) motor trajectories. We then use the same simulations to produce synthetic electrophysiological responses to violations of intentional expectations. Our results affirm that a Bayes-optimal approach provides a principled framework, which accommodates current thinking about the mirror-neuron system. Furthermore, it endorses the general formulation of action as active inference.

Spatial summation in macaque parietal area 7a follows a winner-take-all rule

While neurons in posterior parietal cortex have been found to signal the presence of a salient stimulus among multiple items in a display, spatial summation within their receptive field in the absence of an attentional bias has never been investigated. This information, however, is indispensable when one investigates the mechanisms of spatial attention and competition between multiple visual objects. To examine the spatial summation rule in parietal area 7a neurons, we trained rhesus monkeys to fixate on a central cross while two identical stimuli were briefly displayed in a neuron's receptive field. The response to a pair of dots was compared with the responses to the same dots when they were presented individually. The scaling and power parameters of a generalized summation algorithm varied greatly, both across neurons and across combinations of stimulus locations. However, the averaged response of the recorded population of 7a neurons was consistent with a winner-take-all rule for spatial summation. A control experiment where a monkey covertly attended to both stimuli simultaneously suggests that attention introduces additional competition by facilitating the less optimal stimulus. Thus an averaging stage is introduced between ∼ 200 and 300 ms of the response to a pair of stimuli. In short, the summation algorithm over the population of area 7a neurons carries the signature of a winner-take-all operation, with spatial attention possibly influencing the temporal dynamics of stimulus competition, that is the moment that the "winner" takes "victory" over the "loser" stimulus.

Consciousness and attention: on sufficiency and necessity

Recent research has slowly corroded a belief that selective attention and consciousness are so tightly entangled that they cannot be individually examined. In this review, we summarize psychophysical and neurophysiological evidence for a dissociation between top-down attention and consciousness. The evidence includes recent findings that show subjects can attend to perceptually invisible objects. More contentious is the finding that subjects can become conscious of an isolated object, or the gist of the scene in the near absence of top-down attention; we critically re-examine the possibility of "complete" absence of top-down attention. We also cover the recent flurry of studies that utilized independent manipulation of attention and consciousness. These studies have shown paradoxical effects of attention, including examples where top-down attention and consciousness have opposing effects, leading us to strengthen and revise our previous views. Neuroimaging studies with EEG, MEG, and fMRI are uncovering the distinct neuronal correlates of selective attention and consciousness in dissociative paradigms. These findings point to a functional dissociation: attention as analyzer and consciousness as synthesizer. Separating the effects of selective visual attention from those of visual consciousness is of paramount importance to untangle the neural substrates of consciousness from those for attention.

Recurrent antitopographic inhibition mediates competitive stimulus selection in an attention network

Topographically organized neurons represent multiple stimuli within complex visual scenes and compete for subsequent processing in higher visual centers. The underlying neural mechanisms of this process have long been elusive. We investigate an experimentally constrained model of a midbrain structure: the optic tectum and the reciprocally connected nucleus isthmi. We show that a recurrent antitopographic inhibition mediates the competitive stimulus selection between distant sensory inputs in this visual pathway. This recurrent antitopographic inhibition is fundamentally different from surround inhibition in that it projects on all locations of its input layer, except to the locus from which it receives input. At a larger scale, the model shows how a focal top-down input from a forebrain region, the arcopallial gaze field, biases the competitive stimulus selection via the combined activation of a local excitation and the recurrent antitopographic inhibition. Our findings reveal circuit mechanisms of competitive stimulus selection and should motivate a search for anatomical implementations of these mechanisms in a range of vertebrate attentional systems.

A neuronal population measure of attention predicts behavioral performance on individual trials

Visual attention improves perception for an attended location or feature and also modulates the responses of sensory neurons. In laboratory studies, the sensory stimuli and task instructions are held constant within an attentional condition, but despite experimenters' best efforts, attention likely varies from moment to moment. Because most previous studies have focused on single neurons, it has been impossible to use neuronal responses to identify attentional fluctuations and determine whether these are associated with changes in behavior. We show that an instantaneous measure of attention based on the responses of a modest number of neurons in area V4 of the rhesus monkey (Macaca mulatta) can reliably predict large changes in an animal's ability to perform a difficult psychophysical task. Unexpectedly, this measure shows that the amount of attention allocated at any moment to locations in opposite hemifields is uncorrelated, suggesting that animals allocate attention to each stimulus independently rather than moving their attentional focus from one location to another.

Attention, uncertainty, and free-energy

We suggested recently that attention can be understood as inferring the level of uncertainty or precision during hierarchical perception. In this paper, we try to substantiate this claim using neuronal simulations of directed spatial attention and biased competition. These simulations assume that neuronal activity encodes a probabilistic representation of the world that optimizes free-energy in a Bayesian fashion. Because free-energy bounds surprise or the (negative) log-evidence for internal models of the world, this optimization can be regarded as evidence accumulation or (generalized) predictive coding. Crucially, both predictions about the state of the world generating sensory data and the precision of those data have to be optimized. Here, we show that if the precision depends on the states, one can explain many aspects of attention. We illustrate this in the context of the Posner paradigm, using the simulations to generate both psychophysical and electrophysiological responses. These simulated responses are consistent with attentional bias or gating, competition for attentional resources, attentional capture and associated speed-accuracy trade-offs. Furthermore, if we present both attended and non-attended stimuli simultaneously, biased competition for neuronal representation emerges as a principled and straightforward property of Bayes-optimal perception.

Detecting contrast changes in invisible patterns during binocular rivalry

When dissimilar images are presented to the two eyes, the human visual system lapses into binocular rivalry, a unique perceptual state characterized by stochastic alternations in dominance of one of the two source images over the other. Probe targets delivered to an eye during suppression phases are more difficult to detect than probes delivered during dominance phases. Nearly all probe studies have involved presenting new stimulation (e.g., a spot of light) either superimposed on or replacing the suppressed stimulus. Here, we ask whether observers can detect a reduction in the contrast of the suppressed stimulus itself. In other words, can observers detect a probe that should make an already invisible stimulus even weaker? Specifically, we compared observers' ability to detect contrast increments and contrast decrements introduced within a rival pattern during dominance and suppression. Contrast increment thresholds were elevated across all pedestal contrasts when the increment was introduced during suppression compared to during dominance, replicating previous results. Contrast decrement thresholds measured during suppression were elevated to an even greater extent, but the fact that they were obtained at all establishes that observers were able to detect probes that should make an already invisible target even more difficult to perceive. In a second experiment, we found a similar pattern of results for contrast change detection in complex images of faces as well. Based on the resulting threshold-vs.-contrast functions, we suggest that, regardless of the complexity of the image, rivalry suppression modulates the neural contrast response function through a mixture of reduced overall response gain and a shift in the contrast gain.

When size matters: attention affects performance by contrast or response gain

Covert attention, the selective processing of visual information in the absence of eye movements, improves behavioral performance. Here, we show that attention, both exogenous (involuntary) and endogenous (voluntary), can affect performance by contrast or response gain changes, depending on the stimulus size and the relative size of the attention field. These two variables were manipulated in a cueing task while varying stimulus contrast. We observed a change in behavioral performance consonant with a change in contrast gain for small stimuli paired with spatial uncertainty, but a change in response gain for large stimuli presented at one location (no uncertainty) and surrounded by irrelevant flanking distracters. A complementary neuroimaging experiment revealed that observers’ attention field was wider with than without spatial uncertainty. Our results support key predictions of the normalization model of attention, and reconcile previous, seemingly contradictory, findings on the effects of visual attention.

Topographic maps of multisensory attention

The intraparietal sulcus (IPS) region is uniquely situated at the intersection of visual, somatosensory, and auditory association cortices, ideally located for processing of multisensory attention. We examined the internal architecture of the IPS region and its connectivity to other regions in the dorsal attention and cinguloinsular networks using maximal connectivity clustering. We show with resting state fMRI data from 58 healthy adolescent and young adult volunteers that points of maximal connectivity between the IPS and other regions in the dorsal attention and cinguloinsular networks are topographically organized, with at least seven maps of the IPS region in each hemisphere. Distinct clusters of the IPS exhibited differential connectivity to auditory, visual, somatosensory, and default mode networks, suggesting local specialization within the IPS region for different sensory modalities. In an independent task activation paradigm with 16 subjects, attention to different sensory modalities showed similar functional specialization within the left intraparietal sulcus region. The default mode network, in contrast, did not show a topographical relationship between regions in the network, but rather maximal connectivity in each region to a single central cluster of the other regions. The topographical architecture of multisensory attention may represent a mechanism for specificity in top-down control of attention from dorsolateral prefrontal and lateral orbitofrontal cortex and may represent an organizational unit for multisensory representations in the brain.

Separable codes for attention and luminance contrast in the primary visual cortex

The visual system encodes the features of visual stimuli as well as their behavioral relevance. Stimuli with a high luminance contrast evoke more activity in the visual cortex than stimuli with a low contrast. At the same time, attended stimuli evoke more activity than nonattended stimuli. There is a debate about how visual features and attention jointly determine neuronal activity in the visual cortex. Some studies suggested that attention increases apparent contrast (Reynolds et al., 2000), others that attention amplifies responses by a constant factor (Williford and Maunsell, 2006), and yet others that attention and contrast have largely additive effects (Buracas and Boynton, 2007; Thiele et al., 2009). The influence of attention on contrast sensitivity differs between neurons, raising the possibility that attention and contrast could be coded conjointly in a population of neurons. Here we investigate this possibility by recording neuronal activity at multiple sites in the primary visual cortex of macaque monkeys using multielectrode recording techniques and support vector machines to decode attended stimuli as well as stimulus contrast. We find that many, but not all, V1 neurons are influenced by attention and that the effects of attention and contrast are additive on average. Stimulus contrast can be decoded from neuronal responses not strongly modulated by attention, whereas the attended stimulus can be decoded as the difference in activity of cells that are influenced by attention and cells that are not. The success of the approach suggests that visual attention and stimulus contrast are represented by largely separable codes.

Visual saliency computations: mechanisms, constraints, and the effect of feedback

The primate visual system continuously selects spatial proscribed regions, features or objects for further processing. These selection mechanisms--collectively termed selective visual attention--are guided by intrinsic, bottom-up and by task-dependent, top-down signals. While much psychophysical research has shown that overt and covert attention is partially allocated based on saliency-driven exogenous signals, it is unclear how this is accomplished at the neuronal level. Recent electrophysiological experiments in monkeys point to the gradual emergence of saliency signals when ascending the dorsal visual stream and to the influence of top-down attention on these signals. To elucidate the neural mechanisms underlying these observations, we construct a biologically plausible network of spiking neurons to simulate the formation of saliency signals in different cortical areas. We find that saliency signals are rapidly generated through lateral excitation and inhibition in successive layers of neural populations selective to a single feature. These signals can be improved by feedback from a higher cortical area that represents a saliency map. In addition, we show how top-down attention can affect the saliency signals by disrupting this feedback through its action on the saliency map. While we find that saliency computations require dominant slow NMDA currents, the signal rapidly emerges from successive regions of the network. In conclusion, using a detailed spiking network model we find biophysical mechanisms and limitations of saliency computations which can be tested experimentally.

Neural basis of visual selective attention

Attentional modulation along the object-recognition pathway of the cortical visual system of primates has been shown to consist of enhanced representation of the retinal input at a specific location in space, or of objects located anywhere in the visual field which possess a critical object feature. Moreover, selective attention mechanisms allow the visual system to resolve competition among multiple objects in a crowded scene in favor of the object that is relevant for the current behavior. Finally, selective attention affects the spontaneous activity of neurons as well as their visually driven responses, and it does so not only by modulating the spiking activity of individual neurons, but also by modulating the degree of coherent firing within the critical neuronal populations. WIREs Cogni Sci 2011 2 392-407 DOI: 10.1002/wcs.117 For further resources related to this article, please visit the WIREs website.

Broadly tuned response properties of diverse inhibitory neuron subtypes in mouse visual cortex

Different subtypes of GABAergic neurons in sensory cortex exhibit diverse morphology, histochemical markers, and patterns of connectivity. These subtypes likely play distinct roles in cortical function, but their in vivo response properties remain unclear. We used in vivo calcium imaging, combined with immunohistochemical and genetic labels, to record visual responses in excitatory neurons and up to three distinct subtypes of GABAergic neurons (immunoreactive for parvalbumin, somatostatin, or vasoactive intestinal peptide) in layer 2/3 of mouse visual cortex. Excitatory neurons had sharp response selectivity for stimulus orientation and spatial frequency, while all GABAergic subtypes had broader selectivity. Further, bias in the responses of GABAergic neurons toward particular orientations or spatial frequencies tended to reflect net biases of the surrounding neurons. These results suggest that the sensory responses of layer 2/3 GABAergic neurons reflect the pooled activity of the surrounding population--a principle that may generalize across species and sensory modalities.