Integration of exogenous input into a dynamic salience map revealed by perturbing attention

Frontal eye field activity enhances object identification during covert visual search

We investigated the link between neuronal activity in the frontal eye field (FEF) and the enhancement of visual processing associated with covert spatial attention in the absence of eye movements. We correlated activity recorded in the FEF of monkeys manually reporting the identity of a visual search target to performance accuracy and reaction time. Monkeys were cued to the most probable target location with a cue array containing a popout color singleton. Neurons exhibited spatially selective responses for the popout cue stimulus and for the target of the search array. The magnitude of activity related to the location of the cue prior to the presentation of the search array was correlated with trends in behavioral performance across valid, invalid, and neutral cue trial conditions. However, the speed and accuracy of the behavioral report on individual trials were predicted by the magnitude of spatial selectivity related to the target to be identified, not for the spatial cue. A minimum level of selectivity was necessary for target detection and a higher level for target identification. Muscimol inactivation of FEF produced spatially selective perceptual deficits in the covert search task that were correlated with the effectiveness of the inactivation and were strongest on invalid cue trials that require an endogenous attention shift. These results demonstrate a strong functional link between FEF activity and covert spatial attention and suggest that spatial signals from FEF directly influence visual processing during the time that a stimulus to be identified is being processed by the visual system.

Relationship between neural responses and visual grouping in the monkey parietal cortex

Visual grouping through the binding of multiple discrete elements is an important component of object perception, and neurological studies have shown that the posterior parietal cortex plays a vital role in that process. To study the neural mechanisms underlying visual grouping, we recorded neuronal activity from the lateral bank of the intraparietal sulcus (L-IPS) of monkeys while they performed a task that required them to discriminate among rapidly presented visual patterns composed of five black or white dots arranged in a cross. The monkeys had to detect the patterns in which dots with the same contrast were arranged either horizontally or vertically (target). Visual grouping was necessary for detection of the target, and we surmised that if L-IPS neurons are involved in visual grouping, they may selectively respond to the grouped objects. In addition, we manipulated the monkeys' attention to the grouping of the elements. We found that many L-IPS neurons showed selectivity for the orientation of the target stimuli, and that selectivity was enhanced by the top-down attention. Moreover, the selectivity correlated with behavioral performance. These results provide the first physiological evidence that L-IPS neurons make a crucial contribution to visual grouping by combining visual and attentional signals so as to bind discrete visual elements together.

Been there, seen that: a neural mechanism for performing efficient visual search

In everyday life, we efficiently find objects in the world by moving our gaze from one location to another. The efficiency of this process is brought about by ignoring items that are dissimilar to the target and remembering which target-like items have already been examined. We trained two animals on a visual foraging task in which they had to find a reward-loaded target among five task-irrelevant distractors and five potential targets. We found that both animals performed the task efficiently, ignoring the distractors and rarely examining a particular target twice. We recorded the single unit activity of 54 neurons in the lateral intraparietal area (LIP) while the animals performed the task. The responses of the neurons differentiated between targets and distractors throughout the trial. Further, the responses marked off targets that had been fixated by a reduction in activity. This reduction acted like inhibition of return in saliency map models; items that had been fixated would no longer be represented by high enough activity to draw an eye movement. This reduction could also be seen as a correlate of reward expectancy; after a target had been identified as not containing the reward the activity was reduced. Within a trial, responses to the remaining targets did not increase as they became more likely to yield a result, suggesting that only activity related to an event is updated on a moment-by-moment bases. Together, our data show that all the neural activity required to guide efficient search is present in LIP. Because LIP activity is known to correlate with saccade goal selection, we propose that LIP plays a significant role in the guidance of efficient visual search.

Reward modulates attention independently of action value in posterior parietal cortex

While numerous studies have explored the mechanisms of reward-based decisions (the choice of action based on expected gain), few have asked how reward influences attention (the selection of information relevant for a decision). Here we show that a powerful determinant of attentional priority is the association between a stimulus and an appetitive reward. A peripheral cue heralded the delivery of reward or no reward (these cues are termed herein RC+ and RC-, respectively); to experience the predicted outcome, monkeys made a saccade to a target that appeared unpredictably at the same or opposite location relative to the cue. Although the RC had no operant associations (did not specify the required saccade), they automatically biased attention, such that an RC+ attracted attention and an RC- repelled attention from its location. Neurons in the lateral intraparietal area (LIP) encoded these attentional biases, maintaining sustained excitation at the location of an RC+ and inhibition at the location of an RC-. Contrary to the hypothesis that LIP encodes action value, neurons did not encode the expected reward of the saccade. Moreover, at odds with an adaptive decision process, the cue-evoked biases interfered with the required saccade, and these biases increased rather than abating with training. After prolonged training, valence selectivity appeared at shorter latencies and automatically transferred to a novel task context, suggesting that training produced visual plasticity. The results suggest that reward predictors gain automatic attentional priority regardless of their operant associations, and this valence-specific priority is encoded in LIP independently of the expected reward of an action.

Visual processing, learning and feedback in the primate eye movement system

We present an overview of recent paradigms used for studying visual information and reward processing in the human and monkey oculomotor pathways. Current evidence indicates that eye movements made during visual search tasks rely on neural computations similar to those employed when eye movements are planned and executed to obtain explicit rewards. These data suggest that human eye movements originate from the processing of (predominantly visual) sensory information, feedback about previous errors, and expectations about factors, such as reward. We conclude that these properties make the saccadic system an ideal model for studying both the behavioral and neural mechanisms for human voluntary and involuntary choice behavior.

Functional significance of nonspatial information in monkey lateral intraparietal area

Although the parietal cortex is traditionally associated with spatial perception and motor planning, recent evidence shows that neurons in the lateral intraparietal area (LIP) carry both spatial and nonspatial signals. The functional significance of the nonspatial information in the parietal cortex is not understood. To address this question, we tested the effect of unilateral reversible inactivation of LIP on three behavioral tasks known to evoke nonspatial responses. Each task included a spatial component (target selection in the hemifield contralateral or ipsilateral to the inactivation) and a nonspatial component, namely limb motor planning, the estimation of elapsed time, and reward-based decisions. Although inactivation reliably impaired performance on all tasks, the deficits were spatially specific (restricted to contralateral target locations), and there were no effects on nonspatial aspects on performance. This suggests that modulatory nonspatial signals in LIP represent feedback about computations performed elsewhere rather than a primary role of LIP in these computations.

Anterior Intraparietal Sulcus is Sensitive to Bottom–Up Attention Driven by Stimulus Salience

Frontal eye fields (FEF) and anterior intraparietal sulcus (aIPS) are involved in the control of voluntary attention in humans, but their functional differences remain poorly understood. We examined the activity in these brain regions as a function of task-irrelevant changes in target and nontarget perceptual salience during a sustained spatial attention task. Both aIPS and FEF were engaged during selective attention. FEF, but not aIPS, was sensitive to the direction of spatial attention. Conversely, aIPS, but not FEF, was modulated by the relative perceptual salience of the target and nontarget stimuli. These results demonstrate separable roles for FEF and aIPS in attentional control with FEF more involved in goal-directed spatial attention and aIPS relatively more sensitive to bottom–up attentional influences driven by stimulus salience.

Left posterior parietal cortex participates in both task preparation and episodic retrieval

Optimal memory retrieval depends not only on the fidelity of stored information, but also on the attentional state of the subject. Factors such as mental preparedness to engage in stimulus processing can facilitate or hinder memory retrieval. The current study used functional magnetic resonance imaging (fMRI) to distinguish preparatory brain activity before episodic and semantic retrieval tasks from activity associated with retrieval itself. A catch-trial imaging paradigm permitted separation of neural responses to preparatory task cues and memory probes. Episodic and semantic task preparation engaged a common set of brain regions, including the bilateral intraparietal sulcus (IPS), left fusiform gyrus (FG), and the pre-supplementary motor area (pre-SMA). In the subsequent retrieval phase, the left IPS was among a set of frontoparietal regions that responded differently to old and new stimuli. In contrast, the right IPS responded to preparatory cues with little modulation during memory retrieval. The findings support a strong left-lateralization of retrieval success effects in left parietal cortex, and further indicate that left IPS performs operations that are common to both task preparation and memory retrieval. Such operations may be related to attentional control, monitoring of stimulus relevance, or retrieval.

Condition-dependent and condition-independent target selection in the macaque posterior parietal cortex

During a visual search, information about the visual attributes of an object and associated behavioral requirements is essential for discriminating a target object from others in the visual field. On the other hand, information about the object's position appears to be more important when orienting the eyes toward the target. To understand the neural mechanisms underlying such a transition (i.e., from nonspatial- to spatial-based target selection), we examined the dependence of neuronal activity in the macaque posterior parietal cortex (PPC) on visual sensory properties and ongoing task demands. Monkeys were trained to perform a visual search task in which either a shape or color singleton within an array was the target, depending on the ongoing search dimension. The visual properties and the task demands were manipulated by independently changing the stimulus features (shape and color), singleton type, and search dimension. We found that a subset of PPC neurons significantly discriminated the target from other stimuli only when the target was defined by a particular stimulus dimension and had specific stimulus features, such as a shape-singleton, bar stimulus (condition-dependent target selection), whereas another subset did so irrespective of the stimulus features and the target-defining dimension (condition-independent target selection). There was thus a great deal of variety in the neural representations specifying the locus of the target. The coexistence of these distinctly different types of target-discrimination processes suggests that the PPC may be situated at the level where the transition from nonspatial- to spatial-based target selection takes place.

A perceptual representation in the frontal eye field during covert visual search that is more reliable than the behavioral report

Neuronal activity in the frontal eye field (FEF) identifies locations of behaviorally important objects for guiding attention and eye movements. We recorded neural activity in the FEF of monkeys trained to manually turn a lever towards the location of a pop-out target of a visual search array without shifting gaze. We examined whether the reliability of the neural representation of the salient target location predicted the monkeys' accuracy of reporting target location. We found that FEF neurons reliably encoded the location of the target stimulus not only on correct trials but also on error trials. The representation of target location in FEF persisted until the manual behavioral report but did not increase in magnitude. This result suggests that, in the absence of an eye movement report, FEF encodes the perceptual information necessary to perform the task but does not accumulate this sensory evidence towards a perceptual decision threshold. These results provide physiological evidence that, under certain circumstances, accurate perceptual representations do not always lead to accurate behavioral reports and that variability in processes outside of perception must be considered to account for the variability in perceptual choice behavior.

Parietal stimulation decouples spatial and feature-based attention

Everyday visual scenes contain a vast quantity of information, only a fraction of which can guide our behavior. Properties such as the location, color and orientation of stimuli help us extract relevant information from complex scenes (Treisman and Gelade, 1980; Livingstone and Hubel, 1987). But how does the brain coordinate the selection of such different stimulus characteristics? Neuroimaging studies have revealed significant regions of overlapping activity in frontoparietal cortex during attention to locations and features, suggesting a global component to visual selection (Vandenberghe et al., 2001; Corbetta and Shulman, 2002; Giesbrecht et al., 2003; Slagter et al., 2007). At the same time, the neural consequences of spatial and feature-based attention differ markedly in early visual areas (Treue and Martinez-Trujillo, 2007), implying that selection may rely on more specific top-down processes. Here we probed the balance between specialized and generalized control by interrupting preparatory attention in the human parietal cortex with transcranial magnetic stimulation (TMS). We found that stimulation of the supramarginal gyrus (SMG) impaired spatial attention only, whereas TMS of the anterior intraparietal sulcus (aIPS) disrupted spatial and feature-based attention. The selection of different stimulus characteristics is thus mediated by distinct top-down mechanisms, which can be decoupled by cortical interference.

Parietal control of attentional guidance: the significance of sensory, motivational and motor factors

The lateral intraparietal area (LIP), a portion of monkey posterior parietal cortex, has been implicated in spatial attention. We review recent evidence showing that LIP encodes a priority map of the external environment that specifies the momentary locus of attention and is activated in a variety of behavioral tasks. The priority map in LIP is shaped by task-specific motor, cognitive and motivational variables, the functional significance of which is not entirely understood. We suggest that these modulations represent teaching signals by which the brain learns to identify attentional priority of various stimuli based on the task-specific associations between these stimuli, the required action and expected outcome.

Neuronal correlates of the set-size effect in monkey lateral intraparietal area

It has long been known that the brain is limited in the amount of sensory information that it can process at any given time. A well-known form of capacity limitation in vision is the set-size effect, whereby the time needed to find a target increases in the presence of distractors. The set-size effect implies that inputs from multiple objects interfere with each other, but the loci and mechanisms of this interference are unknown. Here we show that the set-size effect has a neural correlate in competitive visuo-visual interactions in the lateral intraparietal area, an area related to spatial attention and eye movements. Monkeys performed a covert visual search task in which they discriminated the orientation of a visual target surrounded by distractors. Neurons encoded target location, but responses associated with both target and distractors declined as a function of distractor number (set size). Firing rates associated with the target in the receptive field correlated with reaction time both within and across set sizes. The findings suggest that competitive visuo-visual interactions in areas related to spatial attention contribute to capacity limitations in visual searches.

Attention and active vision

Visual perception results from the interaction of incoming sensory signals and top down cognitive and motor signals. Here we focus on the representation of attended locations in parietal cortex and in earlier visual cortical areas. We review evidence that these spatial representations are modulated not only by selective attention but also by the intention to move the eyes. We describe recent experiments in monkey and human that elucidate the mechanisms and circuitry involved in updating, or remapping, the representations of salient stimuli. Two central ideas emerge. First, selective attention and remapping are closely intertwined, and together contribute to the percept of spatial stability. Second, remapping is accomplished not by a single area but by the participation of parietal, frontal and extrastriate cortex as well as subcortical structures. This neural circuitry is distinguished by significant redundancy and plasticity, suggesting that the updating of salient stimuli is fundamental for spatial stability and visuospatial behavior. We conclude that multiple processes and pathways contribute to active vision in the primate brain.

Task specific computations in attentional maps

The lateral intraparietal area (LIP), a portion of monkey posterior parietal cortex, has been implicated in spatial attention. We review recent evidence from our laboratory showing that LIP encodes a priority map of the external environment that specifies the momentary locus of attention and is activated in a variety of behavioral tasks. The priority map in LIP is shaped by task-specific variables. We suggest that the multifaceted responses in LIP represent mechanisms for allocating attention, and that the attentional system may flexibly configure itself to meet the cognitive, motor and motivational demands of individual tasks.

Resolving some confusions over attention and consciousness

There is presently an ongoing debate about the relation between attention and consciousness. Thus debate is being fuelled by results from experimental paradigms which probe various forms of the interaction between attention and consciousness, such as the attentional blink, object-substitution masking and change blindness. We present here simulations of these three paradigms which can all be produced from a single overarching control model of attention. This model helps to suggest an explanation of consciousness as created through attention, and helps to explore the complex nature of attention. It indicates how it is possible to accommodate the relevant experimental results without needing to regard consciousness and attention as independent processes.

Neural Mechanisms of Visual Attention: How Top-Down Feedback Highlights Relevant Locations

Attention helps us process potentially important objects by selectively increasing the activity of sensory neurons that represent the relevant locations and features of our environment. This selection process requires top-down feedback about what is important in our environment. We investigated how parietal cortical output influences neural activity in early sensory areas. Neural recordings were made simultaneously from the posterior parietal cortex and an earlier area in the visual pathway, the medial temporal area, of macaques performing a visual matching task. When the monkey selectively attended to a location, the timing of activities in the two regions became synchronized, with the parietal cortex leading the medial temporal area. Parietal neurons may thus selectively increase activity in earlier sensory areas to enable focused spatial attention.

From thought to action: the parietal cortex as a bridge between perception, action, and cognition

The lateral intraparietal area (LIP) is a subdivision of the inferior parietal lobe that has been implicated in the guidance of spatial attention. In a variety of tasks, LIP provides a "salience representation" of the external world-a topographic visual representation that encodes the locations of salient or behaviorally relevant objects. Recent neurophysiological experiments show that this salience representation incorporates information about multiple behavioral variables-such as a specific motor response, reward, or category membership-associated with the task-relevant object. This integration occurs in a wide variety of tasks, including those requiring eye or limb movements or goal-directed or nontargeting operant responses. Thus, LIP acts as a multifaceted behavioral integrator that binds visuospatial, motor, and cognitive information into a topographically organized signal of behavioral salience. By specifying attentional priority as a synthesis of multiple task demands, LIP operates at the interface of perception, action, and cognition.