Integration of exogenous input into a dynamic salience map revealed by perturbing attention

Intracortical recordings reveal vision-to-action cortical gradients driving human exogenous attention

Exogenous attention, the process that makes external salient stimuli pop-out of a visual scene, is essential for survival. How attention-capturing events modulate human brain processing remains unclear. Here we show how the psychological construct of exogenous attention gradually emerges over large-scale gradients in the human cortex, by analyzing activity from 1,403 intracortical contacts implanted in 28 individuals, while they performed an exogenous attention task. The timing, location and task-relevance of attentional events defined a spatiotemporal gradient of three neural clusters, which mapped onto cortical gradients and presented a hierarchy of timescales. Visual attributes modulated neural activity at one end of the gradient, while at the other end it reflected the upcoming response timing, with attentional effects occurring at the intersection of visual and response signals. These findings challenge multi-step models of attention, and suggest that frontoparietal networks, which process sequential stimuli as separate events sharing the same location, drive exogenous attention phenomena such as inhibition of return.

Statistical learning of distractor shape modulates attentional capture

Physically salient but task-irrelevant stimuli have high attentional priority, although observers are able to capitalize on statistical regularities in the environment to more efficiently ignore such stimuli. Physically salient distractors that more frequently appear in a particular location are less distracting when they appear in this high probability location. Likewise, colors and orientations that are frequently associated with distractors become preferentially ignored with learning. Such statistically learned distractor suppression has been examined with respect to the frequency of elementary features across trials, and less is known about how statistics concerning the composition of distractor features within a trial influence attention, particularly with respect to how orientations combine to form shapes. Color, orientation, and location are also represented very early in vision, whereas more complex features such as shape are represented further downstream in the visual system; it remains unclear whether statistically leaned distractor suppression can operate over such downstream visual representations. In the present study, we demonstrate attentional capture by physically salient, shape-defined distractors that is reduced in magnitude for a high probability shape. Our findings demonstrate that statistical learning can modulate attentional priority at least at the level of basic shapes and is not restricted to modulations of priority at the earliest stages of visual information processing tied to elementary features.

Capture or suppression? Attentional allocation upon reward and loss-associated nonsalient distractors are supported by distinct neural mechanisms: An EEG study

Previous studies suggest that a reward-associated salient distractor can induce bottom-up attentional capture. Hitherto, the neurophysiological mechanisms underlying attentional allocation upon reward/loss associated nonsalient stimulus remain hardly investigated. The present study built the association between nonsalient stimuli and value, and tested it with a decision-making task. Consequently, we examined whether and how reward/loss-associated nonsalient stimuli (as distractors) influenced attentional allocation in a rapid serial visual presentation task. Behavioral analysis showed a significantly faster recognition of target in the loss condition compared to performance in the neutral/reward conditions. Electrophysiological results showed that reward-associated distractors induced a significant Pd component, while loss-associated distractors induced a significantly higher theta oscillation. These results demonstrated that subjects could proactively suppress reward-associated distractors. More importantly, we showed that attentional allocation upon reward/loss-associated nonsalient distractors is supported by distinct neural mechanisms.

Asymmetric Contributions of the Fronto-Parietal Network to Emotional Conflict in the Word–Face Interference Task

The fronto-parietal network is involved in top-down and bottom-up processes necessary to achieve cognitive control. We investigated the role of asymmetric enhancement of the left dorsolateral prefrontal cortex (lDLPFC) and right posterior parietal cortex (rPPC) in cognitive control under conditions of emotional conflict arising from emotional distractors. The effects of anodal tDCS over the lDLPFC/cathodal over the rPPC and the effects of anodal tDCS over the rPPC/cathodal over the lDLPFC were compared to sham tDCS in a double-blind design. The findings showed that anodal stimulation over the lDLPFC reduced interference from emotional distractors, but only when participants had already gained experience with the task. In contrast, having already performed the task only eliminated facilitation effects for positive stimuli. Importantly, anodal stimulation of the rPPC did not affect distractors’ interference. Therefore, the present findings indicate that the lDLPFC plays a crucial role in implementing top-down control to resolve emotional conflict, but that experience with the task is necessary to reveal this role.

Prior Experience Alters the Appearance of Blurry Object Borders

Object memories activated by borders serve as priors for figure assignment: figures are more likely to be perceived on the side of a border where a well-known object is sketched. Do object memories also affect the appearance of object borders? Memories represent past experience with objects; memories of well-known objects include many with sharp borders because they are often fixated. We investigated whether object memories affect appearance by testing whether blurry borders appear sharper when they are contours of well-known objects versus matched novel objects. Participants viewed blurry versions of one familiar and one novel stimulus simultaneously for 180 ms; then made comparative (Exp. 1) or equality judgments regarding perceived blur (Exps. 2–4). For equivalent levels of blur, the borders of well-known objects appeared sharper than those of novel objects. These results extend evidence for the influence of past experience to object appearance, consistent with dynamic interactive models of perception.

Threat reduces value-driven but not salience-driven attentional capture

What we direct our attention to is strongly influenced by both bottom-up and top-down processes. Moreover, the control of attention is biased by prior learning, such that attention is automatically captured by stimuli previously associated with either reward or threat. It is unknown whether value-oriented and threat-oriented mechanisms of selective information processing function independently of one another, or whether they interact with each other in the selection process. Here, we introduced the threat of electric shock into the value-driven attentional capture paradigm to examine whether the experience of threat influences the attention capturing quality of previously reward-associated stimuli. The results showed that value-driven attentional capture was blunted by the experience of threat. This contrasts with previous reports of threat potentiating attentional capture by physically salient stimuli, which we replicate here. Our findings demonstrate that threat selectively interferes with value-based but not salience-based attentional priority, consistent with a competitive relationship between value-based and threat-based information processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Selection history: How reward modulates selectivity of visual attention

Visual attention enables us to selectively prioritize or suppress information in the environment. Prominent models concerned with the control of visual attention differentiate between goal-directed, top-down and stimulus-driven, bottom-up control, with the former determined by current selection goals and the latter determined by physical salience. In the current review, we discuss recent studies that demonstrate that attentional selection does not need to be the result of top-down or bottom-up processing but, instead, is often driven by lingering biases due to the "history" of former attention deployments. This review mainly focuses on reward-based history effects; yet other types of history effects such as (intertrial) priming, statistical learning and affective conditioning are also discussed. We argue that evidence from behavioral, eye-movement and neuroimaging studies supports the idea that selection history modulates the topographical landscape of spatial "priority" maps, such that attention is biased toward locations having the highest activation on this map.

Covert shift of attention modulates the value encoding in the orbitofrontal cortex

During value-based decision making, we often evaluate the value of each option sequentially by shifting our attention, even when the options are presented simultaneously. The orbitofrontal cortex (OFC) has been suggested to encode value during value-based decision making. Yet it is not known how its activity is modulated by attention shifts. We investigated this question by employing a passive viewing task that allowed us to disentangle effects of attention, value, choice and eye movement. We found that the attention modulated OFC activity through a winner-take-all mechanism. When we attracted the monkeys’ attention covertly, the OFC neuronal activity reflected the reward value of the newly attended cue. The shift of attention could be explained by a normalization model. Our results strongly argue for the hypothesis that the OFC neuronal activity represents the value of the attended item. They provide important insights toward understanding the OFC’s role in value-based decision making.

Dissociable signatures of visual salience and behavioral relevance across attentional priority maps in human cortex

Computational models posit that visual attention is guided by activity within spatial maps that index the image-computable salience and the behavioral relevance of objects in the scene. These spatial maps are theorized to be instantiated as activation patterns across a series of retinotopic visual regions in occipital, parietal, and frontal cortex. Whereas previous research has identified sensitivity to either the behavioral relevance or the image-computable salience of different scene elements, the simultaneous influence of these factors on neural "attentional priority maps" in human cortex is not well understood. We tested the hypothesis that visual salience and behavioral relevance independently impact the activation profile across retinotopically organized cortical regions by quantifying attentional priority maps measured in human brains using functional MRI while participants attended one of two differentially salient stimuli. We found that the topography of activation in priority maps, as reflected in the modulation of region-level patterns of population activity, independently indexed the physical salience and behavioral relevance of each scene element. Moreover, salience strongly impacted activation patterns in early visual areas, whereas later visual areas were dominated by relevance. This suggests that prioritizing spatial locations relies on distributed neural codes containing graded representations of salience and relevance across the visual hierarchy. NEW & NOTEWORTHY We tested a theory which supposes that neural systems represent scene elements according to both their salience and their relevance in a series of "priority maps" by measuring functional MRI activation patterns across human brains and reconstructing spatial maps of the visual scene. We found that different regions indexed either the salience or the relevance of scene items, but not their interaction, suggesting an evolving representation of salience and relevance across different visual areas.

Visual cortical activity before and after cochlear implantation: A follow up ERP prospective study in deaf children

ERPs were recorded in response to presentation of static colored patterned stimuli in 25 children (19 to 80months of age at cochlear implantation, CI) with very early prelingual profound deafness (PreLD), 21 postlingual profoundly deaf children (PostLD) (34 to 180months of age at CI) and gender- and age-matched control hearing children. Recording sessions were performed before CI, then 6 and 24months after CI. Results showed that prelingual and, at a lesser degree, postlingual auditory deprivation altered cortical visual neural activity associated to colored shapes from both P1 and N1 cortical processing stages. The P1 and N1 amplitude modifications vanished about 24months after CI in both PreLD and PostLD deaf children. In PreLD the visual processing pattern becomes similar to the typical one essentially by an amplitude decrease of P1 on the left hemisphere together with an amplitude increase of the N1 on the right hemisphere. Finally, in PreLD, increased LH advantage over the RH in N1 amplitude on the cerebellar-occipito-parietal region before CI showed a significant inverse relationship with speech perception outcomes 3years after CI. Investigating early visual processing development and its neural substrates in deaf children would help to understand the variability of CI outcome, because their cortical visual organization diverged from the one of typically developing hearing children, and cannot be predicted from what is observed in deaf adults.

Task-context-dependent Linear Representation of Multiple Visual Objects in Human Parietal Cortex

A host of recent studies have reported robust representations of visual object information in the human parietal cortex, similar to those found in ventral visual cortex. In ventral visual cortex, both monkey neurophysiology and human fMRI studies showed that the neural representation of a pair of unrelated objects can be approximated by the averaged neural representation of the constituent objects shown in isolation. In this study, we examined whether such a linear relationship between objects exists for object representations in the human parietal cortex. Using fMRI and multivoxel pattern analysis, we examined object representations in human inferior and superior intraparietal sulcus, two parietal regions previously implicated in visual object selection and encoding, respectively. We also examined responses from the lateral occipital region, a ventral object processing area. We obtained fMRI response patterns to object pairs and their constituent objects shown in isolation while participants viewed these objects and performed a 1-back repetition detection task. By measuring fMRI response pattern correlations, we found that all three brain regions contained representations for both single object and object pairs. In the lateral occipital region, the representation for a pair of objects could be reliably approximated by the average representation of its constituent objects shown in isolation, replicating previous findings in ventral visual cortex. Such a simple linear relationship, however, was not observed in either parietal region examined. Nevertheless, when we equated the amount of task information present by examining responses from two pairs of objects, we found that representations for the average of two object pairs were indistinguishable in both parietal regions from the average of another two object pairs containing the same four component objects but with a different pairing of the objects (i.e., the average of AB and CD vs. that of AD and CB). Thus, when task information was held consistent, the same linear relationship may govern how multiple independent objects are represented in the human parietal cortex as it does in ventral visual cortex. These findings show that object and task representations coexist in the human parietal cortex and characterize one significant difference of how visual information may be represented in ventral visual and parietal regions.

Linking dopaminergic reward signals to the development of attentional bias: A positron emission tomographic study

The attention system is shaped by reward history, such that learned reward cues involuntarily draw attention. Recent research has begun to uncover the neural mechanisms by which learned reward cues compete for attention, implicating dopamine (DA) signaling within the dorsal striatum. How these elevated priority signals develop in the brain during the course of learning is less well understood, as is the relationship between value-based attention and the experience of reward during learning. We hypothesized that the magnitude of the striatal DA response to reward during learning contributes to the development of a learned attentional bias towards the cue that predicted it, and examined this hypothesis using positron emission tomography with [¹¹C]raclopride. We measured changes in dopamine release for rewarded versus unrewarded visual search for color-defined targets as indicated by the density and distribution of the available D₂/D₃ receptors. We then tested for correlations of individual differences in this measure of reward-related DA release to individual differences in the degree to which previously reward-associated but currently task-irrelevant stimuli impair performance in an attention task (i.e., value-driven attentional bias), revealing a significant relationship in the right anterior caudate. The degree to which reward-related DA release was right hemisphere lateralized was also predictive of later attentional bias. Our findings provide support for the hypothesis that value-driven attentional bias can be predicted from reward-related DA release during learning.

Reward processing in the value-driven attention network: reward signals tracking cue identity and location

Through associative reward learning, arbitrary cues acquire the ability to automatically capture visual attention. Previous studies have examined the neural correlates of value-driven attentional orienting, revealing elevated activity within a network of brain regions encompassing the visual corticostriatal loop [caudate tail, lateral occipital complex (LOC) and early visual cortex] and intraparietal sulcus (IPS). Such attentional priority signals raise a broader question concerning how visual signals are combined with reward signals during learning to create a representation that is sensitive to the confluence of the two. This study examines reward signals during the cued reward training phase commonly used to generate value-driven attentional biases. High, compared with low, reward feedback preferentially activated the value-driven attention network, in addition to regions typically implicated in reward processing. Further examination of these reward signals within the visual system revealed information about the identity of the preceding cue in the caudate tail and LOC, and information about the location of the preceding cue in IPS, while early visual cortex represented both location and identity. The results reveal teaching signals within the value-driven attention network during associative reward learning, and further suggest functional specialization within different regions of this network during the acquisition of an integrated representation of stimulus value.

Neural Mechanisms of Selective Visual Attention

Selective visual attention describes the tendency of visual processing to be confined largely to stimuli that are relevant to behavior. It is among the most fundamental of cognitive functions, particularly in humans and other primates for whom vision is the dominant sense. We review recent progress in identifying the neural mechanisms of selective visual attention. We discuss evidence from studies of different varieties of selective attention and examine how these varieties alter the processing of stimuli by neurons within the visual system, current knowledge of their causal basis, and methods for assessing attentional dysfunctions. In addition, we identify some key questions that remain in identifying the neural mechanisms that give rise to the selective processing of visual information.

Early Left Parietal Activity Elicited by Direct Gaze: A High-Density EEG Study

Gaze is one of the most important cues for human communication and social interaction. In particular, gaze contact is the most primary form of social contact and it is thought to capture attention. A very early-differentiated brain response to direct versus averted gaze has been hypothesized. Here, we used high-density electroencephalography to test this hypothesis. Topographical analysis allowed us to uncover a very early topographic modulation (40-80 ms) of event-related responses to faces with direct as compared to averted gaze. This modulation was obtained only in the condition where intact broadband faces-as opposed to high-pass or low-pas filtered faces-were presented. Source estimation indicated that this early modulation involved the posterior parietal region, encompassing the left precuneus and inferior parietal lobule. This supports the idea that it reflected an early orienting response to direct versus averted gaze. Accordingly, in a follow-up behavioural experiment, we found faster response times to the direct gaze than to the averted gaze broadband faces. In addition, classical evoked potential analysis showed that the N170 peak amplitude was larger for averted gaze than for direct gaze. Taken together, these results suggest that direct gaze may be detected at a very early processing stage, involving a parallel route to the ventral occipito-temporal route of face perceptual analysis.

Neuronal Mechanisms of Visual Attention

Advances on several fronts have refined our understanding of the neuronal mechanisms of attention. This review focuses on recent progress in understanding visual attention through single-neuron recordings made in behaving subjects. Simultaneous recordings from populations of individual cells have shown that attention is associated with changes in the correlated firing of neurons that can enhance the quality of sensory representations. Other work has shown that sensory normalization mechanisms are important for explaining many aspects of how visual representations change with attention, and these mechanisms must be taken into account when evaluating attention-related neuronal modulations. Studies comparing different brain structures suggest that attention is composed of several cognitive processes, which might be controlled by different brain regions. Collectively, these and other recent findings provide a clearer picture of how representations in the visual system change when attention shifts from one target to another.

The attention habit: how reward learning shapes attentional selection

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications.

The effects of short-term and long-term learning on the responses of lateral intraparietal neurons to visually presented objects

The lateral intraparietal area (LIP) is thought to play an important role in the guidance of where to look and pay attention. LIP can also respond selectively to differently shaped objects. We sought to understand to what extent short-term and long-term experience with visual orienting determines the responses of LIP to objects of different shapes. We taught monkeys to arbitrarily associate centrally presented objects of various shapes with orienting either toward or away from a preferred spatial location of a neuron. The training could last for less than a single day or for several months. We found that neural responses to objects are affected by such experience, but that the length of the learning period determines how this neural plasticity manifests. Short-term learning affects neural responses to objects, but these effects are only seen relatively late after visual onset; at this time, the responses to newly learned objects resemble those of familiar objects that share their meaning or arbitrary association. Long-term learning affects the earliest bottom-up responses to visual objects. These responses tend to be greater for objects that have been associated with looking toward, rather than away from, LIP neurons' preferred spatial locations. Responses to objects can nonetheless be distinct, although they have been similarly acted on in the past and will lead to the same orienting behavior in the future. Our results therefore indicate that a complete experience-driven override of LIP object responses may be difficult or impossible. We relate these results to behavioral work on visual attention.

Value-driven attentional priority signals in human basal ganglia and visual cortex

Goal-directed and stimulus-driven factors determine attentional priority through a well defined dorsal frontal-parietal and ventral temporal-parietal network of brain regions, respectively. Recent evidence demonstrates that reward-related stimuli also have high attentional priority, independent of their physical salience and goal-relevance. The neural mechanisms underlying such value-driven attentional control are unknown. Using human functional magnetic resonance imaging, we demonstrate that the tail of the caudate nucleus and extrastriate visual cortex respond preferentially to task-irrelevant but previously reward-associated objects, providing an attentional priority signal that is sensitive to reward history. The caudate tail has not been implicated in the control of goal-directed or stimulus-driven attention, but is well suited to mediate the value-driven control of attention. Our findings reveal the neural basis of value-based attentional priority.

The role of reward prediction in the control of attention

Previously rewarded stimuli involuntarily capture attention. The learning mechanisms underlying this value-driven attentional capture remain less understood. We tested whether theories of prediction-based associative reward learning explain the conditions under which reward feedback leads to value-based modulations of attentional priority. Across 4 experiments, we manipulated whether stimulus features served as unique predictors of reward outcomes. Participants received monetary rewards for correctly identifying a color-defined target in an initial search task (training phase) and then immediately completed a second, unrewarded visual search task in which color was irrelevant (test phase). In Experiments 1-3, monetary reward followed correct target selection during training, but critically, no target-defining features carried uniquely predictive information about reward outcomes. Under these conditions, we found no evidence of attentional capture by the previous target colors in the subsequent test phase. Conversely, when target colors in the training phase of Experiment 4 carried uniquely predictive information about reward magnitude, we observed significant attentional capture by the previously rewarded color. Our findings show that value-based attentional priority only develops for stimulus features that carry uniquely predictive information about reward, ruling out a purely motivational account and suggesting that mechanisms of reward prediction play an important role in shaping attentional priorities.

A spatially nonselective baseline signal in parietal cortex reflects the probability of a monkey's success on the current trial

We recorded the activity of neurons in the lateral intraparietal area of two monkeys while they performed two similar visual search tasks, one difficult, one easy. Each task began with a period of fixation followed by an array consisting of a single capital T and a number of lowercase t's. The monkey had to find the capital T and report its orientation, upright or inverted, with a hand movement. In the easy task the monkey could explore the array with saccades. In the difficult task the monkey had to continue fixating and find the capital T in the visual periphery. The baseline activity measured during the fixation period, at a time in which the monkey could not know if the impending task would be difficult or easy or where the target would appear, predicted the monkey's probability of success or failure on the task. The baseline activity correlated inversely with the monkey's recent history of success and directly with the intensity of the response to the search array on the current trial. The baseline activity was unrelated to the monkey's spatial locus of attention as determined by the location of the cue in a cued visual reaction time task. We suggest that rather than merely reflecting the noise in the system, the baseline signal reflects the cortical manifestation of modulatory state, motivational, or arousal pathways, which determine the efficiency of cortical sensorimotor processing and the quality of the monkey's performance.

The object advantage can be eliminated under equiluminant conditions

A key phenomenon supporting the existence of object-based attention is the object advantage, in which responses are faster for within-object, relative to equidistant between-object, shifts of attention. The origins of this effect have been variously ascribed to low-level "bottom-up" sensory processing and to a cognitive "top-down" strategy of within-object attention prioritization. The degree to which the object advantage depends on lower-level sensory processing was examined by differentially stimulating the magnocellular (M) and parvocellular (P) retino-geniculo-cortical visual pathways by using equiluminant and nonequiluminant conditions. We found that the object advantage can be eliminated when M activity is reduced using psychophysically equiluminant stimuli. This novel result in normal observers suggests that the origin of the object advantage is found in lower-level sensory processing rather than a general cognitive process, which should not be so sensitive to differential activation of the bottom-up P and M pathways. Eliminating the object advantage while maintaining a spatial-cueing advantage with reduced M activity suggests that the notion of independent M-driven spatial attention and P-driven object attention requires revision.

Shaping attention with reward: effects of reward on space- and object-based selection

The contribution of rewarded actions to automatic attentional selection remains obscure. We hypothesized that some forms of automatic orienting, such as object-based selection, can be completely abandoned in favor of a reward-maximizing strategy. In the two experiments reported here, we presented identical visual stimuli to observers while manipulating what was being rewarded (targets in different locations from the cue or in random object locations) and the type of reward received (money or points). We found that reward alone, not the objects, guides attentional selection and thus entirely predicts behavior. These results suggest that guidance of selective attention, although automatic, is flexible and can be adjusted in accordance with external nonsensory reward-based factors.

Amygdala, pulvinar, and inferior parietal cortex contribute to early processing of faces without awareness

The goals of the present study were 2-fold. First, we wished to investigate the neural correlates of stimulus-driven processing of stimuli strongly suppressed from awareness and in the absence of top-down influences. We accomplished this using a novel approach in which participants performed an orthogonal task atop a flash suppression noise image to prevent top-down search. Second, we wished to investigate the extent to which amygdala responses differentiate between suppressed stimuli (fearful faces and houses) based on their motivational relevance. Using continuous flash suppression (CFS) in conjunction with fMRI, we presented fearful faces, houses, and a no stimulus control to one eye while participants performed an orthogonal task that appeared atop the flashing Mondrian image presented to the opposite eye. In 29 adolescents, we show activation in subcortical regions, including the superior colliculus, amygdala, thalamus, and hippocampus for suppressed objects (fearful faces and houses) compared to a no stimulus control. Suppressed stimuli showed less activation compared to a no stimulus control in early visual cortex (EVC), indicating that object information was being suppressed from this region. Additionally, we find no activation in regions associated with conscious processing of these percepts (fusiform gyrus and/or parahippocampal cortex) as assessed by mean activations and multi-voxel patterns. A psychophysiological interaction analysis (PPI) that seeded the amygdala showed task-specific (fearful faces greater than houses) modulation of right pulvinar and left inferior parietal cortex. Taken together, our results support a role for the amygdala in stimulus-driven attentional guidance toward objects of relevance and a potential mechanism for successful suppression of rivalrous stimuli.

Spatial orienting in complex audiovisual environments

Previous studies on crossmodal spatial orienting typically used simple and stereotyped stimuli in the absence of any meaningful context. This study combined computational models, behavioural measures and functional magnetic resonance imaging to investigate audiovisual spatial interactions in naturalistic settings. We created short videos portraying everyday life situations that included a lateralised visual event and a co-occurring sound, either on the same or on the opposite side of space. Subjects viewed the videos with or without eye-movements allowed (overt or covert orienting). For each video, visual and auditory saliency maps were used to index the strength of stimulus-driven signals, and eye-movements were used as a measure of the efficacy of the audiovisual events for spatial orienting. Results showed that visual salience modulated activity in higher-order visual areas, whereas auditory salience modulated activity in the superior temporal cortex. Auditory salience modulated activity also in the posterior parietal cortex, but only when audiovisual stimuli occurred on the same side of space (multisensory spatial congruence). Orienting efficacy affected activity in the visual cortex, within the same regions modulated by visual salience. These patterns of activation were comparable in overt and covert orienting conditions. Our results demonstrate that, during viewing of complex multisensory stimuli, activity in sensory areas reflects both stimulus-driven signals and their efficacy for spatial orienting; and that the posterior parietal cortex combines spatial information about the visual and the auditory modality.

Thinking in spatial terms: decoupling spatial representation from sensorimotor control in monkey posterior parietal areas 7a and LIP

Perhaps the simplest and most complete description of the cerebral cortex is that it is a sensorimotor controller whose primary purpose is to represent stimuli and movements, and adaptively control the mapping between them. However, in order to think, the cerebral cortex has to generate patterns of neuronal activity that encode abstract, generalized information independently of ongoing sensorimotor events. A critical question confronting cognitive systems neuroscience at present therefore is how neural signals encoding abstract information emerge within the sensorimotor control networks of the brain. In this review, we approach that question in the context of the neural representation of space in posterior parietal cortex of non-human primates. We describe evidence indicating that parietal cortex generates a hierarchy of spatial representations with three basic levels: including (1) sensorimotor signals that are tightly coupled to stimuli or movements, (2) sensorimotor signals modified in strength or timing to mediate cognition (examples include attention, working memory, and decision-processing), as well as (3) signals that encode frankly abstract spatial information (such as spatial relationships or categories) generalizing across a wide diversity of specific stimulus conditions. Here we summarize the evidence for this hierarchy, and consider data showing that signals at higher levels derive from signals at lower levels. That in turn could help characterize neural mechanisms that derive a capacity for abstraction from sensorimotor experience.

Attention, learning, and the value of information

Despite many studies on selective attention, fundamental questions remain about its nature and neural mechanisms. Here I draw from the animal and machine learning fields that describe attention as a mechanism for active learning and uncertainty reduction and explore the implications of this view for understanding visual attention and eye movement control. I propose that a closer integration of these different views has the potential greatly to expand our understanding of oculomotor control and our ability to use this system as a window into high level but poorly understood cognitive functions, including the capacity for curiosity and exploration and for inferring internal models of the external world.

Affective salience can reverse the effects of stimulus-driven salience on eye movements in complex scenes

In natural vision both stimulus features and cognitive/affective factors influence an observer's attention. However, the relationship between stimulus-driven ("bottom-up") and cognitive/affective ("top-down") factors remains controversial: Can affective salience counteract strong visual stimulus signals and shift attention allocation irrespective of bottom-up features? Is there any difference between negative and positive scenes in terms of their influence on attention deployment? Here we examined the impact of affective factors on eye movement behavior, to understand the competition between visual stimulus-driven salience and affective salience and how they affect gaze allocation in complex scene viewing. Building on our previous research, we compared predictions generated by a visual salience model with measures indexing participant-identified emotionally meaningful regions of each image. To examine how eye movement behavior differs for negative, positive, and neutral scenes, we examined the influence of affective salience in capturing attention according to emotional valence. Taken together, our results show that affective salience can override stimulus-driven salience and overall emotional valence can determine attention allocation in complex scenes. These findings are consistent with the hypothesis that cognitive/affective factors play a dominant role in active gaze control.

The involvement of bottom-up saliency processing in endogenous inhibition of return

Participants are faster at detecting a visual target when it appears at a cued, as compared with an uncued, location. In general, a reversal of this cost-benefit pattern is observed after exogenous cuing when the cue-target interval exceeds approximately 250 ms (inhibition of return [IOR]), and not after endogenous cuing. We suggest that, usually, no IOR is found with endogenous cues because no bottom-up saliency-based orienting processes are claimed. Therefore, we developed an endogenous feature-based split-cue task to allow for endogenous saliency-based orienting. IOR was observed in the saliency-driven endogenous cuing condition, and not in the control condition that prevented saliency-based orienting. These results suggest that usage of saliency-based orienting processes in either endogenous or exogenous orienting warrants the appearance of IOR.

Cognitive functions of the posterior parietal cortex: top-down and bottom-up attentional control

Although much less is known about human parietal cortex than that of homologous monkey cortex, recent studies, employing neuroimaging, and neuropsychological methods, have begun to elucidate increasingly fine-grained functional and structural distinctions. This review is focused on recent neuroimaging and neuropsychological studies elucidating the cognitive roles of dorsal and ventral regions of parietal cortex in top-down and bottom-up attentional orienting, and on the interaction between the two attentional allocation mechanisms. Evidence is reviewed arguing that regions along the dorsal areas of the parietal cortex, including the superior parietal lobule (SPL) are involved in top-down attentional orienting, while ventral regions including the temporo-parietal junction (TPJ) are involved in bottom-up attentional orienting.

Attentional control: temporal relationships within the fronto-parietal network

Selective attention to particular aspects of incoming sensory information is enabled by a network of neural areas that includes frontal cortex, posterior parietal cortex, and, in the visual domain, visual sensory regions. Although progress has been made in understanding the relative contribution of these different regions to the process of visual attentional selection, primarily through studies using neuroimaging, rather little is known about the temporal relationships between these disparate regions. To examine this, participants viewed two rapid serial visual presentation (RSVP) streams of letters positioned to the left and right of fixation point. Before each run, attention was directed to either the left or the right stream. Occasionally, a digit appeared within the attended stream indicating whether attention was to be maintained within the same stream ('hold' condition) or to be shifted to the previously ignored stream ('shift' condition). By titrating the temporal parameters of the time taken to shift attention for each participant using a fine-grained psychophysics paradigm, we measured event-related potentials time-locked to the initiation of spatial shifts of attention. The results revealed that shifts of attention were evident earlier in the response recorded over frontal than over parietal electrodes and, importantly, that the early activity over frontal electrodes was associated with a successful shift of attention. We conclude that frontal areas are engaged early for the purpose of executing an attentional shift, likely triggering a cascade through the fronto-parietal network ultimately, resulting in the attentional modulation of sensory events in posterior cortices.

The role of the lateral intraparietal area in orienting attention and its implications for visual search

Orienting visual attention is of fundamental importance when viewing a visual scene. One of the areas thought to play a role in the guidance of this process is the posterior parietal cortex. In this review, we will describe the way the lateral intraparietal area (LIP) of the posterior parietal cortex acts as a priority map to help guide the allocation of covert attention and eye movements (overt attention). We will explain the concept of a priority map and then show that LIP activity is biased by both bottom-up stimulus-driven factors and top-down cognitive influences, and that this activity can be used to predict the locus of covert attention and initial saccadic latencies in simple visual search tasks. We will then describe evidence for how this system acts during covert visual search and how its activity could be used to optimize overt visual search performance.

Modulation of shifting receptive field activity in frontal eye field by visual salience

In the monkey frontal eye field (FEF), the sensitivity of some neurons to visual stimulation changes just before a saccade. Sensitivity shifts from the spatial location of its current receptive field (RF) to the location of that field after the saccade is completed (the future field, FF). These shifting RFs are thought to contribute to the stability of visual perception across saccades, and in this study we investigated whether the salience of the FF stimulus alters the magnitude of FF activity. We reduced the salience of the usually single flashed stimulus by adding other visual stimuli. We isolated 171 neurons in the FEF of 2 monkeys and did experiments on 50 that had FF activity. In 30% of these, that activity was higher before salience was reduced by adding stimuli. The mean magnitude reduction was 16%. We then determined whether the shifting RFs were more frequent in the central visual field, which would be expected if vision across saccades were only stabilized for the visual field near the fovea. We found no evidence of any skewing of the frequency of shifting receptive fields (or the effects of salience) toward the central visual field. We conclude that the salience of the FF stimulus makes a substantial contribution to the magnitude of FF activity in FEF. In so far as FF activity contributes to visual stability, the salience of the stimulus is probably more important than the region of the visual field in which it falls for determining which objects remain perceptually stable across saccades.

A pure salience response in posterior parietal cortex

When exploring a visual scene, some objects perceptually popout because of a difference of color, shape, or size. This bottom-up information is an important part of many models describing the allocation of visual attention. It has been hypothesized that the lateral intraparietal area (LIP) acts as a "priority map," integrating bottom-up and top-down information to guide the allocation of attention. Despite a large literature describing top-down influences in LIP, the presence of a pure salience response to a salient stimulus defined by its static features alone has not been reported. We compared LIP responses with colored salient stimuli and distractors in a passive fixation task. Many LIP neurons responded preferentially to 1 of the 2 colored stimuli, yet the mean responses to the salient stimuli were significantly higher than to distractors, independent of the features of the stimuli. These enhanced responses were significant within 75 ms, and the mean responses to salient and distractor stimuli were tightly correlated, suggesting a simple gain control. We propose that a pure salience signal rapidly appears in LIP by collating salience signals from earlier visual areas. This contributes to the creation of a priority map, which is used to guide attention and saccades.

Attention, intention, and priority in the parietal lobe

For many years there has been a debate about the role of the parietal lobe in the generation of behavior. Does it generate movement plans (intention) or choose objects in the environment for further processing? To answer this, we focus on the lateral intraparietal area (LIP), an area that has been shown to play independent roles in target selection for saccades and the generation of visual attention. Based on results from a variety of tasks, we propose that LIP acts as a priority map in which objects are represented by activity proportional to their behavioral priority. We present evidence to show that the priority map combines bottom-up inputs like a rapid visual response with an array of top-down signals like a saccade plan. The spatial location representing the peak of the map is used by the oculomotor system to target saccades and by the visual system to guide visual attention.

Spatial and non-spatial functions of the parietal cortex

Although the parietal cortex is traditionally associated with spatial attention and sensorimotor integration, recent evidence also implicates it in higher order cognitive functions. We review relevant results from neuron recording studies showing that inferior parietal neurons integrate information regarding target location with a variety of non-spatial signals. Some of these signals are modulatory and alter a stimulus-evoked response according to the action, category, or reward associated with the stimulus. Other non-spatial inputs act independently, encoding the context or rules of a task even before the presentation of a specific target. Despite the ubiquity of non-spatial information in individual neurons, reversible inactivation of the parietal lobe affects only spatial orienting of attention and gaze, but not non-spatial aspects of performance. This suggests that non-spatial signals contribute to an underlying spatial computation, possibly allowing the brain to determine which targets are worthy of attention or action in a given task context.

Neural correlates of developing and adapting behavioral biases in speeded choice reactions--an fMRI study on predictive motor coding

In reaction-time (RT) tasks with unequally probable stimuli, people respond faster and more accurately in high-probability trials than in low-probability trials. We used functional magnetic resonance imaging to investigate brain activity during the acquisition and adaptation of such biases. Participants responded to arrows pointing to either side with different and previously unknown probabilities across blocks, which were covertly reversed in the middle of some blocks. Changes in response bias were modeled using the development of the selective RT bias at the beginning of a block and after the reversal as parametric regressors. Both fresh development and reversal of an existing response bias were associated with bilateral activations in inferior parietal lobule, intraparietal sulcus, and supplementary motor cortex. Further activations were observed in right temporoparietal junction, dorsolateral prefrontal cortex, and dorsal premotor cortex. Only during initial development of biases at the beginning of a block, we observed additional activity in ventral premotor cortex and anterior insula, whereas the basal ganglia (bilaterally) were recruited when the bias was adapted to reversed probabilities. Taken together, these areas constitute a network that updates and applies implicit predictions to create an attention and motor bias according to environmental probabilities that transform into specific facilitation.

Visual saliency computations: mechanisms, constraints, and the effect of feedback

The primate visual system continuously selects spatial proscribed regions, features or objects for further processing. These selection mechanisms--collectively termed selective visual attention--are guided by intrinsic, bottom-up and by task-dependent, top-down signals. While much psychophysical research has shown that overt and covert attention is partially allocated based on saliency-driven exogenous signals, it is unclear how this is accomplished at the neuronal level. Recent electrophysiological experiments in monkeys point to the gradual emergence of saliency signals when ascending the dorsal visual stream and to the influence of top-down attention on these signals. To elucidate the neural mechanisms underlying these observations, we construct a biologically plausible network of spiking neurons to simulate the formation of saliency signals in different cortical areas. We find that saliency signals are rapidly generated through lateral excitation and inhibition in successive layers of neural populations selective to a single feature. These signals can be improved by feedback from a higher cortical area that represents a saliency map. In addition, we show how top-down attention can affect the saliency signals by disrupting this feedback through its action on the saliency map. While we find that saliency computations require dominant slow NMDA currents, the signal rapidly emerges from successive regions of the network. In conclusion, using a detailed spiking network model we find biophysical mechanisms and limitations of saliency computations which can be tested experimentally.

Microstimulation of posterior parietal cortex biases the selection of eye movement goals during search

People can find objects in a visual scene fast and effortlessly. It is thought that this may be accomplished by creating a map of the outside world that incorporates bottom-up sensory and top-down cognitive inputs--a priority map. Eye movements are made toward the location represented by the highest activity on the priority map. We hypothesized that the lateral intraparietal area (LIP) of posterior parietal cortex acts as such a map. To test this, we performed low current microstimulation on animals trained to perform a foraging task and asked whether we could bias the animals to make a saccade to a particular stimulus, by creating an artificial peak of activity at the location representing that stimulus on the map. We found that microstimulation slightly biased the animals to make saccades to visual stimuli at the stimulated location, without actively generating saccades. The magnitude of this effect was small, but it appeared to be similar for all visual stimuli. We interpret these results to mean that microstimulation slightly biased saccade goal selection to the object represented at the stimulated location in LIP.

Putative Pyramidal Neurons and Interneurons in the Monkey Parietal Cortex Make Different Contributions to the Performance of a Visual Grouping Task

Visual grouping of discrete elements is an important function for object recognition. We recently conducted an experiment to study neural correlates of visual grouping. We recorded neuronal activities while monkeys performed a grouping detection task in which they discriminated visual patterns composed of discrete dots arranged in a cross and detected targets in which dots with the same contrast were aligned horizontally or vertically. We found that some neurons in the lateral bank of the intraparietal sulcus exhibit activity related to visual grouping. In the present study, we analyzed how different types of neurons contribute to visual grouping. We classified the recorded neurons as putative pyramidal neurons or putative interneurons, depending on the duration of their action potentials. We found that putative pyramidal neurons exhibited selectivity for the orientation of the target, and this selectivity was enhanced by attention to a particular target orientation. By contrast, putative interneurons responded more strongly to the target stimuli than to the nontargets, regardless of the orientation of the target. These results suggest that different classes of parietal neurons contribute differently to the grouping of discrete elements.

The neural basis of visual attention

Visual attention is the mechanism the nervous system uses to highlight specific locations, objects or features within the visual field. This can be accomplished by making an eye movement to bring the object onto the fovea (overt attention) or by increased processing of visual information in neurons representing more peripheral regions of the visual field (covert attention). This review will examine two aspects of visual attention: the changes in neural responses within visual cortices due to the allocation of covert attention; and the neural activity in higher cortical areas involved in guiding the allocation of attention. The first section will highlight processes that occur during visual spatial attention and feature-based attention in cortical visual areas and several related models that have recently been proposed to explain this activity. The second section will focus on the parietofrontal network thought to be involved in targeting eye movements and allocating covert attention. It will describe evidence that the lateral intraparietal area, frontal eye field and superior colliculus are involved in the guidance of visual attention, and describe the priority map model, which is thought to operate in at least several of these areas.

Right temporoparietal junction activation by a salient contextual cue facilitates target discrimination

The right temporoparietal junction (R TPJ) is involved in stimulus-driven attentional control in response to the appearance of an unexpected target or a distractor that shares features with a task-relevant target. An unresolved question is whether these responses in R TPJ are due simply to the presence of a stimulus that is a potential target, or instead responds to any task-relevant information. Here, we addressed this issue by testing the sensitivity of R TPJ to a perceptually salient, non-target stimulus - a contextual cue. Although known to be a non-target, the contextual cue carried probabilistic information regarding the presence of a target in the opposite visual field. The contextual cue was therefore always of potential behavioral relevance, but only sometimes paired with a target. The appearance of the contextual cue alone increased activation in R TPJ, but more so when it appeared with a target. There was also greater connectivity between R TPJ and a network of attentional control and decision areas when the contextual cue was present. These results demonstrate that R TPJ is involved in the stimulus-driven representation of task-relevant information that can be used to engage an appropriate behavioral response.

A virtual reality-based FMRI study of reward-based spatial learning

Although temporo-parietal cortices mediate spatial navigation in animals and humans, the neural correlates of reward-based spatial learning are less well known. Twenty-five healthy adults performed a virtual reality fMRI task that required learning to use extra-maze cues to navigate an 8-arm radial maze and find hidden rewards. Searching the maze in the spatial learning condition compared to the control conditions was associated with activation of temporo-parietal regions, albeit not including the hippocampus. The receipt of rewards was associated with activation of the hippocampus in a control condition when using the extra-maze cues for navigation was rendered impossible by randomizing the spatial location of cues. Our novel experimental design allowed us to assess the differential contributions of the hippocampus and other temporo-parietal areas to searching and reward processing during reward-based spatial learning. This translational research will permit parallel studies in animals and humans to establish the functional similarity of learning systems across species; cellular and molecular studies in animals may then inform the effects of manipulations on these systems in humans, and fMRI studies in humans may inform the interpretation and relevance of findings in animals.

Integration of goal- and stimulus-related visual signals revealed by damage to human parietal cortex

Where we look is determined both by our current intentions and by the tendency of visually salient items to "catch our eye." After damage to parietal cortex, the normal process of directing attention is often profoundly impaired. Here, we tracked parietal patients' eye movements during visual search to separately map impairments in goal-directed orienting to targets versus stimulus-driven gaze shifts to salient but task-irrelevant probes. Deficits in these two distinct types of attentional selection are shown to be identical in both magnitude and spatial distribution, consistent with damage to a "priority map" that integrates goal- and stimulus-related signals to select visual targets. When goal-relevant and visually salient items compete for attention, the outcome depends on a biased competition in which the priority of contralesional targets is undervalued. On the basis of these findings, we further demonstrate that parietal patients' spatial bias (neglect) in goal-directed visual exploration can be corrected and even reversed by systematically manipulating the spatial distribution of stimulus salience in the visual array.

Attention as a decision in information space

Decision formation and attention are two fundamental processes through which we select, respectively, appropriate actions or sources of information. Although both functions have been studied in the oculomotor system, we lack a unified view explaining both forms of selection. We review evidence showing that parietal neurons encoding saccade motor decisions also carry signals of attention (perceptual selection) that are independent of the metrics, modality and reward of an action. We propose that attention implements a specialized form of decision based on the utility of information. Thus, oculomotor control depends on two interacting but distinct processes: attentional decisions that assign value to sources of information and motor decisions that flexibly link the selected information with action.

The consciousness circuit - an approach to the hard problem

This paper is the continuation of a study about Consciousness as a resulting function between attention and luminosity, presented at the Neuroscience International Congress, which was held in the city of Natal, in Brazil, in 2006. There it was described how visual Consciousness is generated by the interaction of the cortical area activity through FFS (Feedforward Sweep Processes), RP (Recurrent Processes) and WSRP (Wide Spread Recurrent Processes), and its relationship with the luminosity that hits the eyes. We have applied the reciprocal interaction model, which says that the eye reacts to luminosity through the regulation of sleep-awake states through EEG wave synchronization; this, in turn, is regulated by the thalamic cortical neural activity from the brainstem monoaminergic and cholinergic nuclei.Such an understanding has led us to construct a consciousness model which can be represented by an orthogonal graph. Through this model, we can represent all states of human consciousness (emotional consciousness, unconsciousness states, dreaming states, awareness states, pre-consciousness states and others) making it possible, in theory, to construct a Consciousness parameter which yields to the understanding of consciousness state observation without a subjective approach to experience.We have also applied on this orthogonal graph the Quantum Orch OR Model. According to it, subjective, phenomenal conscious vision depends on quantum computation in microtubules where the quantum is the smallest quantity of radiant energy, in a scale in which matter and energy interact. This model helps us to build the vertical axe of the consciousness orthogonal graph, a cholinergic-aminergic scale, in which activation triggers the quantum mechanism relevant to the consciousness phenomenon.

Influence and limitations of popout in the selection of salient visual stimuli by area V4 neurons

The neural mechanism of bottom-up attention and its relationship to top-down attention are poorly understood. Visual stimuli that differ from others in their component features are salient and tend to draw attention in a bottom-up manner. "Popout" stimuli differ uniformly from surrounding items and are more easily detected than stimuli composed of a conjunction of surrounding features. We compared the responses of single area V4 neurons to popout and conjunction stimuli appearing within the classical receptive field (CRF) and found that their responses are modulated by popout. This selectivity was more robust when larger numbers of surrounding items and multiple features were included in the display, and it was absent when only a few items were presented immediately outside the CRF. In addition, the popout modulation of V4 activity was eliminated when top-down attention was directed to locations outside of the CRFs during saccade preparation, indicating that the salience of popout stimuli is not sufficient to drive selection by V4 neurons. These results demonstrate that neurons in feature-selective cortex are influenced by bottom-up attention, but that this influence is limited by top-down attention.