Dissociating activity in the lateral intraparietal area from value using a visual foraging task

Pre-stimulus Alpha Activity Modulates Face and Object Processing in the Intra-Parietal Sulcus, a MEG Study

Face perception is crucial in all social animals. Recent studies have shown that pre-stimulus oscillations of brain activity modulate the perceptual performance of face vs. non-face stimuli, specifically under challenging conditions. However, it is unclear if this effect also occurs during simple tasks, and if so in which brain regions. Here we used magnetoencephalography (MEG) and a 1-back task in which participants decided if the two sequentially presented stimuli were the same or not in each trial. The aim of the study was to explore the effect of pre-stimulus alpha oscillation on the perception of face (human and monkey) and non-face stimuli. Our results showed that pre-stimulus activity in the left occipital face area (OFA) modulated responses in the intra-parietal sulcus (IPS) at around 170 ms after the presentation of human face stimuli. This effect was also found after participants were shown images of motorcycles. In this case, the IPS was modulated by pre-stimulus activity in the right OFA and the right fusiform face area (FFA). We conclude that pre-stimulus modulation of post-stimulus response also occurs during simple tasks and is therefore independent of behavioral responses.

Reward-driven modulation of spatial attention in the human frontal eye-field

Attentional selection and the decision of where to make an eye-movement are driven by various factors such as the representation of salience, task goal, and stimulus relevance, as well as expectations or predictions based on past experience. Brain systems implicated in these processes recruit cortico-subcortical areas including the Frontal Eye-Field (FEF), parietal cortex, or superior colliculus. How these areas interact to govern attention remains elusive. Priority maps of space have been observed in several brain regions, but the neural substrates where different sources of information are combined and integrated to guide attentional selection has not been elucidated. We investigated here the neural mechanisms subserving how reward cues influence the voluntary deployment of attention, in conditions where stimulus-driven capture and task-related goals compete for attention selection. Using fMRI in a visual search task in n = 23 participants, we found a selective modulation of FEF by the reward value of distractors during attentional shifts, particularly after high-predictive cueing to invalid locations. Reward information also modulated FEF connectivity to superior colliculus, striatum, and visual cortex. We conclude that FEF may occupy a central position within brain circuits integrating different sources of top-down biases for the generation of spatial saliency maps and guidance of selective attention.

The roles of the lateral intraparietal area and frontal eye field in guiding eye movements in free viewing search behavior

The lateral intraparietal area (LIP) and frontal eye field (FEF) have been shown to play significant roles in oculomotor control, yet most studies have found that the two areas behave similarly. To identify the unique roles each area plays in guiding eye movements, we recorded 200 LIP neurons and 231 FEF neurons from four animals performing a free viewing visual foraging task. We analyzed how neuronal responses were modulated by stimulus identity and the animals' choice of where to make a saccade. We additionally analyzed the comodulation of the sensory signals and the choice signal to identify how the sensory signals drove the choice. We found a clearly defined division of labor: LIP provided a stable map integrating task rules and stimulus identity, whereas FEF responses were dynamic, representing more complex information and, just before the saccade, were integrated with task rules and stimulus identity to decide where to move the eye.NEW & NOTEWORTHY The lateral intrapareital area (LIP) and frontal eye field (FEF) are known to contribute to guiding eye movements, but little is known about the unique roles that each area plays. Using a free viewing visual search task, we found that LIP provides a stable map of the visual world, integrating task rules and stimulus identity. FEF activity is consistently modulated by more complex information but, just before the saccade, integrates all the information to make the final decision about where to move.

Activity in LIP, But not V4, Matches Performance When Attention is Spread

The enhancement of neuronal responses in many visual areas while animals perform spatial attention tasks has widely been thought to be the neural correlate of visual attention, but it is unclear whether the presence or absence of this modulation contributes to our striking inability to notice changes in change blindness examples. We asked whether neuronal responses in visual area V4 and the lateral intraparietal area (LIP) in posterior parietal cortex could explain the limited ability of subjects to attend multiple items in a display. We trained animals to perform a change detection task in which they had to compare 2 arrays of stimuli separated briefly in time and found that each animal's performance decreased as function of set-size. Neuronal discriminability in V4 was consistent across set-sizes, but decreased for higher set-sizes in LIP. The introduction of a reward bias produced attentional enhancement in V4, but this could not explain the vast improvement in performance, whereas the enhancement in LIP responses could. We suggest that behavioral set-size effects and the marked improvement in performance with focused attention may not be related to response enhancement in V4 but, instead, may occur in or on the way to LIP.

Neurons in FEF Keep Track of Items That Have Been Previously Fixated in Free Viewing Visual Search

When searching a visual scene for a target, we tend not to look at items or locations we have already searched. It is thought that this behavior is driven by an inhibitory tagging mechanism that inhibits responses on priority maps to the relevant items. We hypothesized that this inhibitory tagging signal should be represented as an elevated response in neurons that keep track of stimuli that have been fixated. We recorded from 231 neurons in the frontal eye field (FEF) of 2 male animals performing a visual foraging task, in which they had to find a reward linked to one of five identical targets (Ts) among five distractors. We identified 38 neurons with activity that was significantly greater when the stimulus in the receptive field had been fixated previously in the trial than when it had not been fixated. The response to a fixated object began before the saccade ended, suggesting that this information is remapped. Unlike most FEF neurons, the activity in these cells was not suppressed during active fixation, had minimal motor responses, and did not change through the trial. Yet using traditional classifications from a memory-guided saccade, they were indistinguishable from the rest of the FEF population. We propose that these neurons keep track of any items that have been fixated within the trial and this signal is propagated by remapping. These neurons could be the source of the inhibitory tagging signal to parietal cortex, where a neuronal instantiation of inhibitory tagging is seen.SIGNIFICANCE STATEMENT When we search a scene for an item, we rarely examine the same location twice. It is thought that this is due to a neural mechanism that keeps track of the items at which we have looked. Here we identified a subset of neurons in the frontal eye field that preferentially responded to items that had been fixated earlier in the trial. These responses were remapped, appearing before the saccade even ended, and were not suppressed during maintained fixation. We propose that these neurons keep track of which items have been examined in search and could be the source of feedback that creates the inhibitory tagging seen in parietal cortex.

Internal but not external noise frees working memory resources

The precision with which visual information can be recalled from working memory declines as the number of items in memory increases. This finding has been explained in terms of the distribution of a limited representational resource between items. Here we investigated how the sensory strength of memoranda affects resource allocation. We manipulated signal strength of an orientation stimulus in two ways: we varied the internal (sensory) noise by adjusting stimulus contrast, and varied the external (stimulus) noise by altering the within-stimulus variability. Both manipulations had similar effects on the precision with which the orientation could be recalled, but differed in their impact on memory for other stimuli. These results indicate that increasing internal noise released resources that could be used to store other stimuli more precisely; increasing external noise had no such effect. We show that these observations can be captured by a simple neural model of working memory encoding, in which spiking activity takes on the role of the limited resource.

Investigations of visual short-term memory typically involve memorising clearly visible objects with elementary features, such as monochromatic disks or oriented bars. Results of such studies indicate that memory is allocated like a limited resource, i.e. shared out between objects. However, in daily life we are often confronted with visual features that are difficult to make out, like when an object is in shadow, or poorly-defined, like the color of a variegated leaf. Here we asked whether these kinds of features occupy as much memory resource as simple highly-visible objects. Our results demonstrate that reducing the sensory strength of a stimulus makes the quality of recall worse, but also takes up less resource so other objects can be remembered more precisely. Increasing the variability within a stimulus worsens recall, but has no effect on how other objects are remembered. These findings can be explained by considering how visual information is stored in populations of neurons: only the manipulation of sensory strength changes the amount of spiking activity dedicated to a stimulus.

Parietal neurons encode expected gains in instrumental information

In natural behavior, animals have access to multiple sources of information, but only a few of these sources are relevant for learning and actions. Beyond choosing an appropriate action, making good decisions entails the ability to choose the relevant information, but fundamental questions remain about the brain's information sampling policies. Recent studies described the neural correlates of seeking information about a reward, but it remains unknown whether, and how, neurons encode choices of instrumental information, in contexts in which the information guides subsequent actions. Here we show that parietal cortical neurons involved in oculomotor decisions encode, before an information sampling saccade, the reduction in uncertainty that the saccade is expected to bring for a subsequent action. These responses were distinct from the neurons' visual and saccadic modulations and from signals of expected reward or reward prediction errors. Therefore, even in an instrumental context when information and reward gains are closely correlated, individual cells encode decision variables that are based on informational factors and can guide the active sampling of action-relevant cues.

Reward expectation differentially modulates attentional behavior and activity in visual area V4

Neural activity in visual area V4 is enhanced when attention is directed into neuronal receptive fields. However, the source of this enhancement is unclear, as most physiological studies have manipulated attention by changing the absolute reward associated with a particular location as well as its value relative to other locations. We trained monkeys to discriminate the orientation of two stimuli presented simultaneously in different hemifields while we independently varied the reward magnitude associated with correct discrimination at each location. Behavioral measures of attention were controlled by the relative value of each location. By contrast, neurons in V4 were consistently modulated by absolute reward value, exhibiting increased activity, increased gamma-band power and decreased trial-to-trial variability whenever receptive field locations were associated with large rewards. These data challenge the notion that the perceptual benefits of spatial attention rely on increased signal-to-noise in V4. Instead, these benefits likely derive from downstream selection mechanisms.

LIP activity in the interstimulus interval of a change detection task biases the behavioral response

When looking around at the world, we can only attend to a limited number of locations. The lateral intraparietal area (LIP) is thought to play a role in guiding both covert attention and eye movements. In this study, we tested the involvement of LIP in both mechanisms with a change detection task. In the task, animals had to indicate whether an element changed during a blank in the trial by making a saccade to it. If no element changed, they had to maintain fixation. We examine how the animal's behavior is biased based on LIP activity prior to the presentation of the stimulus the animal must respond to. When the activity was high, the animal was more likely to make an eye movement toward the stimulus, even if there was no change; when the activity was low, the animal either had a slower reaction time or maintained fixation, even if a change occurred. We conclude that LIP activity is involved in both covert and overt attention, but when decisions about eye movements are to be made, this role takes precedence over guiding covert attention.

Spikes not slots: noise in neural populations limits working memory

This opinion article argues that noise (randomness) in neural activity is the limiting factor in visual working memory (WM), determining how accurately we can maintain stable internal representations of external stimuli. Sharing of a fixed amount of neural activity between items in memory explains why WM can be successfully described as a continuous resource. This contrasts with the popular conception of WM as comprising a limited number of memory slots, each holding a representation of one stimulus - I argue that this view is challenged by computational theory and the latest neurophysiological evidence.

Remapping, Spatial Stability, and Temporal Continuity: From the Pre-Saccadic to Postsaccadic Representation of Visual Space in LIP

As our eyes move, we have a strong percept that the world is stable in space and time; however, the signals in cortex coming from the retina change with each eye movement. It is not known how this changing input produces the visual percept we experience, although the predictive remapping of receptive fields has been described as a likely candidate. To explain how remapping accounts for perceptual stability, we examined responses of neurons in the lateral intraparietal area while animals performed a visual foraging task. When a stimulus was brought into the response field of a neuron that exhibited remapping, the onset of the postsaccadic representation occurred shortly after the saccade ends. Whenever a stimulus was taken out of the response field, the presaccadic representation abruptly ended shortly after the eyes stopped moving. In the 38% (20/52) of neurons that exhibited remapping, there was no more than 30 ms between the end of the presaccadic representation and the start of the postsaccadic representation and, in some neurons, and the population as a whole, it was continuous. We conclude by describing how this seamless shift from a presaccadic to postsaccadic representation could contribute to spatial stability and temporal continuity.

The what, where, and why of priority maps and their interactions with visual working memory

Priority maps are winner-take-all neural mechanisms thought to guide the allocation of covert and overt attention. Here, we go beyond this standard definition and argue that priority maps play a much broader role in controlling goal-directed behavior. We start by defining what priority maps are and where they might be found in the brain; we then ask why they exist-the function that they serve. We propose that this function is to communicate a goal state to the different effector systems, thereby guiding behavior. Within this framework, we speculate on how priority maps interact with visual working memory and introduce our common source hypothesis, the suggestion that this goal state is maintained in visual working memory and used to construct all of the priority maps controlling the various motor systems. Finally, we look ahead and suggest questions about priority maps that should be asked next.

Surprise signals in the supplementary eye field: rectified prediction errors drive exploration-exploitation transitions

Visual search is coordinated adaptively by monitoring and predicting the environment. The supplementary eye field (SEF) plays a role in oculomotor control and outcome evaluation. However, it is not clear whether the SEF is involved in adjusting behavioral modes based on preceding feedback. We hypothesized that the SEF drives exploration-exploitation transitions by generating "surprise signals" or rectified prediction errors, which reflect differences between predicted and actual outcomes. To test this hypothesis, we introduced an oculomotor two-target search task in which monkeys were required to find two valid targets among four identical stimuli. After they detected the valid targets, they exploited their knowledge of target locations to obtain a reward by choosing the two valid targets alternately. Behavioral analysis revealed two distinct types of oculomotor search patterns: exploration and exploitation. We found that two types of SEF neurons represented the surprise signals. The error-surprise neurons showed enhanced activity when the monkey received the first error feedback after the target pair change, and this activity was followed by an exploratory oculomotor search pattern. The correct-surprise neurons showed enhanced activity when the monkey received the first correct feedback after an error trial, and this increased activity was followed by an exploitative, fixed-type search pattern. Our findings suggest that error-surprise neurons are involved in the transition from exploitation to exploration and that correct-surprise neurons are involved in the transition from exploration to exploitation.

Transition of target-location signaling in activity of macaque lateral intraparietal neurons during delayed-response visual search

Neurons in the lateral intraparietal area (LIP) are involved in signaling the location of behaviorally relevant objects during visual discrimination and working memory maintenance. Although previous studies have examined these cognitive processes separately, they often appear as inseparable sequential processes in real-life situations. Little is known about how the neural representation of the target location is altered when both cognitive processes are continuously required for executing a task. We investigated this issue by recording single-unit activity from LIP of monkeys performing a delayed-response visual search task in which they were required to discriminate the target from distractors in the stimulus period, remember the location at which the extinguished target had been presented in the delay period, and make a saccade to that location in the response period. Target-location signaling was assessed using response modulations contingent on whether the target location was inside or opposite the receptive field. Although the population-averaged response modulation was consistent and changed only slightly during a trial, the across-neuron pattern of response modulations showed a marked and abrupt change around 170 ms after stimulus offset due to concurrent changes in the response modulations of a subset of LIP neurons, which manifested heterogeneous patterns of activity changes during the task. Our findings suggest that target-location signaling by the across-neuron pattern of LIP activity discretely changes after the stimulus disappearance under conditions that continuously require visual discrimination and working memory to perform a single behavioral task.

Extrafoveal preview benefit during free-viewing visual search in the monkey

Previous studies have shown that subjects require less time to process a stimulus at the fovea after a saccade if they have viewed the same stimulus in the periphery immediately prior to the saccade. This extrafoveal preview benefit indicates that information about the visual form of an extrafoveally viewed stimulus can be transferred across a saccade. Here, we extend these findings by demonstrating and characterizing a similar extrafoveal preview benefit in monkeys during a free-viewing visual search task. We trained two monkeys to report the orientation of a target among distractors by releasing one of two bars with their hand; monkeys were free to move their eyes during the task. Both monkeys took less time to indicate the orientation of the target after foveating it, when the target lay closer to the fovea during the previous fixation. An extrafoveal preview benefit emerged even if there was more than one intervening saccade between the preview and the target fixation, indicating that information about target identity could be transferred across more than one saccade and could be obtained even if the search target was not the goal of the next saccade. An extrafoveal preview benefit was also found for distractor stimuli. These results aid future physiological investigations of the extrafoveal preview benefit.

Separate evaluation of target facilitation and distractor suppression in the activity of macaque lateral intraparietal neurons during visual search

During visual search, neurons in the lateral intraparietal area (LIP) discriminate the target from distractors by exhibiting stronger activation when the target appears within the receptive field than when it appears outside the receptive field. It is generally thought that such target-discriminative activity is produced by the combination of target-related facilitation and distractor-related suppression. However, little is known about how the target-discriminative activity is constituted by these two types of neural modulation. To address this issue, we recorded activity from LIP of monkeys performing a visual search task that consisted of target-present and target-absent trials. Monkeys had to make a saccade to a target in the target-present trials, whereas they had to maintain fixation in the target-absent trials, in which only distractors were presented. By introducing the activity from the latter trials as neutral activity, we were able to separate the target-discriminative activity into target-related elevation and distractor-related reduction components. We found that the target-discriminative activity of most LIP neurons consisted of the combination of target-related elevation and distractor-related reduction or only target-related elevation. In contrast, target-discriminative activity composed of only distractor-related reduction was observed for very few neurons. We also found that, on average, target-related elevation was stronger and occurred earlier compared with distractor-related reduction. Finally, we consider possible underlying mechanisms, including lateral inhibitory interactions, responsible for target-discriminative activity in visual search. The present findings provide insight into how neuronal modulations shape target-discriminative activity during visual search.

Evidence for differential top-down and bottom-up suppression in posterior parietal cortex

When searching for an object, we usually avoid items that are visually different from the target and objects or places that have been searched already. Previous studies have shown that neural activity in the lateral intraparietal area (LIP) can be used to guide this behaviour; responses to task irrelevant stimuli or to stimuli that have been fixated previously in the trial are reduced compared with responses to potential targets. Here, we test the hypothesis that these reduced responses have a different genesis. Two animals were trained on a visual foraging task, in which they had to find a target among a number of physically identical potential targets (T) and task irrelevant distractors. We recorded neural activity and local field potentials (LFPs) in LIP while the animals performed the task. We found that LFP power was similar for potential targets and distractors but was greater in the alpha and low beta bands when a previously fixated T was in the response field. We interpret these data to suggest that the reduced single-unit response to distractors is a bottom-up feed-forward result of processing in earlier areas and the reduced response to previously fixated Ts is a result of active top-down suppression.

Signal multiplexing and single-neuron computations in lateral intraparietal area during decision-making

Previous work has revealed a remarkably direct neural correlate of decisions in the lateral intraparietal area (LIP). Specifically, firing rate has been observed to ramp up or down in a manner resembling the accumulation of evidence for a perceptual decision reported by making a saccade into (or away from) the neuron's response field (RF). However, this link between LIP response and decision formation emerged from studies where a saccadic target was always stimulating the RF during decisions, and where the neural correlate was the averaged activity of a restricted sample of neurons. Because LIP cells are (1) highly responsive to the presence of a visual stimulus in the RF, (2) heterogeneous, and (3) not clearly anatomically segregated from large numbers of neurons that fail selection criteria, the underlying neuronal computations are potentially obscured. To address this, we recorded single neuron spiking activity in LIP during a well-studied moving-dot direction-discrimination task and manipulated whether a saccade target was present in the RF during decision-making. We also recorded from a broad sample of LIP neurons, including ones conventionally excluded in prior studies. Our results show that cells multiplex decision signals with decision-irrelevant visual signals. We also observed disparate, repeating response "motifs" across neurons that, when averaged together, resemble traditional ramping decision signals. In sum, neural responses in LIP simultaneously carry decision signals and decision-irrelevant sensory signals while exhibiting diverse dynamics that reveal a broader range of neural computations than previously entertained.

Anticipatory remapping of attentional priority across the entire visual field

It has been suggested that one way we may create a stable percept of the visual world across multiple eye movements is to pass information from one set of neurons to another around the time of each eye movement. Previous studies have shown that some neurons in the lateral intraparietal area (LIP) exhibit anticipatory remapping: these neurons produce a visual response to a stimulus that will enter their receptive field after a saccade but before it actually does so. LIP responses during fixation are thought to represent attentional priority, behavioral relevance, or value. In this study, we test whether the remapped response represents this attentional priority by examining the activity of LIP neurons while animals perform a visual foraging task. We find that the population responds more to a target than to a distractor before the saccade even begins to bring the stimulus into the receptive field. Within 20 ms of the saccade ending, the responses in almost one-third of LIP neurons closely resemble the responses that will emerge during stable fixation. Finally, we show that, in these neurons and in the population as a whole, this remapping occurs for all stimuli in all locations across the visual field and for both long and short saccades. We conclude that this complete remapping of attentional priority across the visual field could underlie spatial stability across saccades.

Neural correlates and neural computations in posterior parietal cortex during perceptual decision-making

A recent line of work has found remarkable success in relating perceptual decision-making and the spiking activity in the macaque lateral intraparietal area (LIP). In this review, we focus on questions about the neural computations in LIP that are not answered by demonstrations of neural correlates of psychological processes. We highlight three areas of limitations in our current understanding of the precise neural computations that might underlie neural correlates of decisions: (1) empirical questions not yet answered by existing data; (2) implementation issues related to how neural circuits could actually implement the mechanisms suggested by both extracellular neurophysiology and psychophysics; and (3) ecological constraints related to the use of well-controlled laboratory tasks and whether they provide an accurate window on sensorimotor computation. These issues motivate the adoption of a more general "encoding-decoding framework" that will be fruitful for more detailed contemplation of how neural computations in LIP relate to the formation of perceptual decisions.

In monkeys making value-based decisions, LIP neurons encode cue salience and not action value

In monkeys deciding between alternative saccadic eye movements, lateral intraparietal (LIP) neurons representing each saccade fire at a rate proportional to the value of the reward expected upon its completion. This observation has been interpreted as indicating that LIP neurons encode saccadic value and that they mediate value-based decisions between saccades. Here, we show that LIP neurons representing a given saccade fire strongly not only if it will yield a large reward but also if it will incur a large penalty. This finding indicates that LIP neurons are sensitive to the motivational salience of cues. It is compatible neither with the idea that LIP neurons represent action value nor with the idea that value-based decisions take place in LIP neurons.

Neural dynamics and circuit mechanisms of decision-making

In this review, I briefly summarize current neurobiological studies of decision-making that bear on two general themes. The first focuses on the nature of neural representation and dynamics in a decision circuit. Experimental and computational results suggest that ramping-to-threshold in the temporal domain and trajectory of population activity in the state space represent a duality of perspectives on a decision process. Moreover, a decision circuit can display several different dynamical regimes, such as the ramping mode and the jumping mode with distinct defining properties. The second is concerned with the relationship between biologically-based mechanistic models and normative-type models. A fruitful interplay between experiments and these models at different levels of abstraction have enabled investigators to pose increasingly refined questions and gain new insights into the neural basis of decision-making. In particular, recent work on multi-alternative decisions suggests that deviations from rational models of choice behavior can be explained by established neural mechanisms.