Matching behavior and the representation of value in the parietal cortex

Ventromedial Prefrontal Cortex Damage Is Associated with Decreased Ventral Striatum Volume and Response to Reward

The ventral striatum and ventromedial prefrontal cortex (vmPFC) are two central nodes of the "reward circuit" of the brain. Human neuroimaging studies have demonstrated coincident activation and functional connectivity between these brain regions, and animal studies have demonstrated that the vmPFC modulates ventral striatum activity. However, there have been no comparable data in humans to address whether the vmPFC may be critical for the reward-related response properties of the ventral striatum. In this study, we used fMRI in five neurosurgical patients with focal vmPFC lesions to test the hypothesis that the vmPFC is necessary for enhancing ventral striatum responses to the anticipation of reward. In support of this hypothesis, we found that, compared with age- and gender-matched neurologically healthy subjects, the vmPFC-lesioned patients had reduced ventral striatal activity during the anticipation of reward. Furthermore, we observed that the vmPFC-lesioned patients had decreased volumes of the accumbens subregion of the ventral striatum. Together, these functional and structural neuroimaging data provide novel evidence for a critical role for the vmPFC in contributing to reward-related activity of the ventral striatum. These results offer new insight into the functional and structural interactions between key components of the brain circuitry underlying human affective function and decision-making.

SIGNIFICANCE STATEMENT

Maladaptive decision-making is a common problem across multiple mental health disorders. Developing new pathophysiologically based strategies for diagnosis and treatment thus requires a better understanding of the brain circuits responsible for adaptive decision-making and related psychological subprocesses (e.g., reward valuation, anticipation, and motivation). Animal studies provide evidence that these functions are mediated through direct interactions between two key nodes of a posited "reward circuit," the ventral striatum and the ventromedial prefrontal cortex (vmPFC). For the first time in humans, we demonstrate that damage to the vmPFC results in decreased ventral striatum activity during reward anticipation. These data provide unique evidence on the causal mechanisms by which the vmPFC and ventral striatum interact during the anticipation of rewards.

Object comparison in the lateral intraparietal area

We can search for and locate specific objects in our environment by looking for objects with similar features. Object recognition involves stimulus similarity responses in ventral visual areas and task-related responses in prefrontal cortex. We tested whether neurons in the lateral intraparietal area (LIP) of posterior parietal cortex could form an intermediary representation, collating information from object-specific similarity map representations to allow general decisions about whether a stimulus matches the object being looked for. We hypothesized that responses to stimuli would correlate with how similar they are to a sample stimulus. When animals compared two peripheral stimuli to a sample at their fovea, the response to the matching stimulus was similar, independent of the sample identity, but the response to the nonmatch depended on how similar it was to the sample: the more similar, the greater the response to the nonmatch stimulus. These results could not be explained by task difficulty or confidence. We propose that LIP uses its known mechanistic properties to integrate incoming visual information, including that from the ventral stream about object identity, to create a dynamic representation that is concise, low dimensional, and task relevant and that signifies the choice priorities in mental matching behavior.NEW & NOTEWORTHY Studies in object recognition have focused on the ventral stream, in which neurons respond as a function of how similar a stimulus is to their preferred stimulus, and on prefrontal cortex, where neurons indicate which stimulus is being looked for. We found that parietal area LIP uses its known mechanistic properties to form an intermediary representation in this process. This creates a perceptual similarity map that can be used to guide decisions in prefrontal areas.

Distinct Roles for the Amygdala and Orbitofrontal Cortex in Representing the Relative Amount of Expected Reward

The same reward can possess different motivational meaning depending upon its magnitude relative to other rewards. To study the neurophysiological mechanisms mediating assignment of motivational meaning, we recorded the activity of neurons in the amygdala and orbitofrontal cortex (OFC) of monkeys during a Pavlovian task in which the relative amount of liquid reward associated with one conditioned stimulus (CS) was manipulated by changing the reward amount associated with a second CS. Anticipatory licking tracked relative reward magnitude, implying that monkeys integrated information about recent rewards to adjust the motivational meaning of a CS. Upon changes in relative reward magnitude, neural responses to reward-predictive cues updated more rapidly in OFC than amygdala, and activity in OFC but not the amygdala was modulated by recent reward history. These results highlight a distinction between the amygdala and OFC in assessing reward history to support the flexible assignment of motivational meaning to sensory cues.

Prefrontal Contribution to Decision-Making under Free-Choice Conditions

Executive function is thought to be the coordinated operation of multiple neural processes and allows to accomplish a current goal flexibly. The most important function of the prefrontal cortex is the executive function. Among a variety of executive functions in which the prefrontal cortex participates, decision-making is one of the most important. Although the prefrontal contribution to decision-making has been examined using a variety of behavioral tasks, recent studies using fMRI have shown that the prefrontal cortex participates in decision-making under free-choice conditions. Since decision-making under free-choice conditions represents the very first stage for any kind of decision-making process, it is important that we understand its neural mechanism. Although few studies have examined this issue while a monkey performed a free-choice task, those studies showed that, when the monkey made a decision to subsequently choose one particular option, prefrontal neurons showing selectivity to that option exhibited transient activation just before presentation of the imperative cue. Further studies have suggested that this transient increase is caused by the irregular fluctuation of spontaneous firing just before cue presentation, which enhances the response to the cue and biases the strength of the neuron's selectivity to the option. In addition, this biasing effect was observed only in neurons that exhibited sustained delay-period activity, indicating that this biasing effect not only influences the animal's decision for an upcoming choice, but also is linked to working memory mechanisms in the prefrontal cortex.

Optimal decision making and matching are tied through diminishing returns

How individuals make decisions has been a matter of long-standing debate among economists and researchers in the life sciences. In economics, subjects are viewed as optimal decision makers who maximize their overall reward income. This framework has been widely influential, but requires a complete knowledge of the reward contingencies associated with a given choice situation. Psychologists and ecologists have observed that individuals tend to use a simpler "matching" strategy, distributing their behavior in proportion to relative rewards associated with their options. This article demonstrates that the two dominant frameworks of choice behavior are linked through the law of diminishing returns. The relatively simple matching can in fact provide maximal reward when the rewards associated with decision makers' options saturate with the invested effort. Such saturating relationships between reward and effort are hallmarks of the law of diminishing returns. Given the prevalence of diminishing returns in nature and social settings, this finding can explain why humans and animals so commonly behave according to the matching law. The article underscores the importance of the law of diminishing returns in choice behavior.

Vision and Action

Investigation of natural behavior has contributed a number of insights to our understanding of visual guidance of actions by highlighting the importance of behavioral goals and focusing attention on how vision and action play out in time. In this context, humans make continuous sequences of sensory-motor decisions to satisfy current behavioral goals, and the role of vision is to provide the relevant information for making good decisions in order to achieve those goals. This conceptualization of visually guided actions as a sequence of sensory-motor decisions has been formalized within the framework of statistical decision theory, which structures the problem and provides the context for much recent progress in vision and action. Components of a good decision include the task, which defines the behavioral goals, the rewards and costs associated with those goals, uncertainty about the state of the world, and prior knowledge.

Reminders of past choices bias decisions for reward in humans

We provide evidence that decisions are made by consulting memories for individual past experiences, and that this process can be biased in favour of past choices using incidental reminders. First, in a standard rewarded choice task, we show that a model that estimates value at decision-time using individual samples of past outcomes fits choices and decision-related neural activity better than a canonical incremental learning model. In a second experiment, we bias this sampling process by incidentally reminding participants of individual past decisions. The next decision after a reminder shows a strong influence of the action taken and value received on the reminded trial. These results provide new empirical support for a decision architecture that relies on samples of individual past choice episodes rather than incrementally averaged rewards in evaluating options and has suggestive implications for the underlying cognitive and neural mechanisms.

A cage-based training, cognitive testing and enrichment system optimized for rhesus macaques in neuroscience research

In neurophysiological studies with awake non-human primates (NHP), it is typically necessary to train the animals over a prolonged period of time on a behavioral paradigm before the actual data collection takes place. Rhesus monkeys (Macaca mulatta) are the most widely used primate animal models in system neuroscience. Inspired by existing joystick- or touch-screen-based systems designed for a variety of monkey species, we built and successfully employed a stand-alone cage-based training and testing system for rhesus monkeys (eXperimental Behavioral Intrument, XBI). The XBI is mobile and easy to handle by both experts and non-experts; animals can work with only minimal physical restraints, yet the ergonomic design successfully encourages stereotypical postures with a consistent positioning of the head relative to the screen. The XBI allows computer-controlled training of the monkeys with a large variety of behavioral tasks and reward protocols typically used in systems and cognitive neuroscience research.

Four eyes match better than two: Sharing of precise patch-use time among socially foraging domestic chicks

To examine how resource competition contributes to patch-use behaviour, we examined domestic chicks foraging in an I-shaped maze equipped with two terminal feeders. In a variable interval schedule, one feeder supplied grains three times more frequently than the other, and the sides were reversed midway through the experiment. The maze was partitioned into two lanes by a transparent wall, so that chicks fictitiously competed without actual interference. Stay time at feeders was compared among three groups. The "single" group contained control chicks; the "pair" group comprised the pairs of chicks tested in the fictitious competition; "mirror" included single chicks accompanied by their respective mirror images. Both "pair" and "mirror" chicks showed facilitated running. In terms of the patch-use ratio, "pair" chicks showed precise matching at approximately 3:1 with significant mutual dependence, whereas "single" and "mirror" chicks showed a comparable under-matching. The facilitated running increased visits to feeders, but failed to predict the patch-use ratio of the subject. At the reversal, quick switching occurred similarly in all groups, but the "pair" chicks revealed a stronger memory-based matching. Perceived competition therefore contributes to precise matching and lasting memory of the better feeder, in a manner dissociated from socially facilitated food search.

Parietal neurons encode expected gains in instrumental information

In natural behavior, animals have access to multiple sources of information, but only a few of these sources are relevant for learning and actions. Beyond choosing an appropriate action, making good decisions entails the ability to choose the relevant information, but fundamental questions remain about the brain's information sampling policies. Recent studies described the neural correlates of seeking information about a reward, but it remains unknown whether, and how, neurons encode choices of instrumental information, in contexts in which the information guides subsequent actions. Here we show that parietal cortical neurons involved in oculomotor decisions encode, before an information sampling saccade, the reduction in uncertainty that the saccade is expected to bring for a subsequent action. These responses were distinct from the neurons' visual and saccadic modulations and from signals of expected reward or reward prediction errors. Therefore, even in an instrumental context when information and reward gains are closely correlated, individual cells encode decision variables that are based on informational factors and can guide the active sampling of action-relevant cues.

Spatial attention during saccade decisions

Behavioral measures of decision making are usually limited to observations of decision outcomes. In the present study, we made use of the fact that oculomotor and sensory selection are closely linked to track oculomotor decision making before oculomotor responses are made. We asked participants to make a saccadic eye movement to one of two memorized target locations and observed that visual sensitivity increased at both the chosen and the nonchosen saccade target locations, with a clear bias toward the chosen target. The time course of changes in visual sensitivity was related to saccadic latency, with the competition between the chosen and nonchosen targets resolved faster before short-latency saccades. On error trials, we observed an increased competition between the chosen and nonchosen targets. Moreover, oculomotor selection and visual sensitivity were influenced by top-down and bottom-up factors as well as by selection history and predicted the direction of saccades. Our findings demonstrate that saccade decisions have direct visual consequences and show that decision making can be traced in the human oculomotor system well before choices are made. Our results also indicate a strong association between decision making, saccade target selection, and visual sensitivity.NEW & NOTEWORTHY We show that saccadic decisions can be tracked by measuring spatial attention. Spatial attention is allocated in parallel to the two competing saccade targets, and the time course of spatial attention differs for fast-slow and for correct-erroneous decisions. Saccade decisions take the form of a competition between potential saccade goals, which is associated with spatial attention allocation to those locations.

Evidence Accumulation and Change Rate Inference in Dynamic Environments

In a constantly changing world, animals must account for environmental volatility when making decisions. To appropriately discount older, irrelevant information, they need to learn the rate at which the environment changes. We develop an ideal observer model capable of inferring the present state of the environment along with its rate of change. Key to this computation is an update of the posterior probability of all possible change point counts. This computation can be challenging, as the number of possibilities grows rapidly with time. However, we show how the computations can be simplified in the continuum limit by a moment closure approximation. The resulting low-dimensional system can be used to infer the environmental state and change rate with accuracy comparable to the ideal observer. The approximate computations can be performed by a neural network model via a rate-correlation-based plasticity rule. We thus show how optimal observers accumulate evidence in changing environments and map this computation to reduced models that perform inference using plausible neural mechanisms.

What to Choose Next? A Paradigm for Testing Human Sequential Decision Making

Many of the decisions we make in our everyday lives are sequential and entail sparse rewards. While sequential decision-making has been extensively investigated in theory (e.g., by reinforcement learning models) there is no systematic experimental paradigm to test it. Here, we developed such a paradigm and investigated key components of reinforcement learning models: the eligibility trace (i.e., the memory trace of previous decision steps), the external reward, and the ability to exploit the statistics of the environment's structure (model-free vs. model-based mechanisms). We show that the eligibility trace decays not with sheer time, but rather with the number of discrete decision steps made by the participants. We further show that, unexpectedly, neither monetary rewards nor the environment's spatial regularity significantly modulate behavioral performance. Finally, we found that model-free learning algorithms describe human performance better than model-based algorithms.

Coupled Decision Processes Update and Maintain Saccadic Priors in a Dynamic Environment

Much of what we know about how the brain forms decisions comes from studies of saccadic eye movements. However, saccadic decisions are often studied in isolation, which limits the insights that they can provide about real-world decisions with complex interdependencies. Here, we used a serial reaction time (RT) task to show that prior expectations affect RTs via interdependent, normative decision processes that operate within and across saccades. We found that human subjects performing the task generated saccades that were governed by a rise-to-threshold decision process with a starting point that reflected expected state-dependent transition probabilities. These probabilities depended on decisions about the current state (the correct target) that, under some conditions, required the accumulation of information across saccades. Without additional feedback, this information was provided by each saccadic decision threshold, which represented the total evidence in favor of the chosen target. Therefore, the output of the within-saccade process was used, not only to generate the saccade, but also to provide input to the across-saccade process. This across-saccade process, in turn, helped to set the starting point of the next within-saccade process. These results imply a novel role for functional information-processing loops in optimizing saccade generation in dynamic environments.SIGNIFICANCE STATEMENT Saccades are the rapid, ballistic eye movements that we make approximately three times every second to scan the visual scene for interesting things to look at. The apparent ease with which we produce saccades belies their computational sophistication, which can be studied quantitatively in the laboratory to provide insights into how our brain manages the interplay between sensory input and motor output. The present work is important because we show for the first time how this interplay operates both within and across saccades to ensure that these eye movements are guided effectively by learned expectations in dynamic environments. More generally, this study shows how sensory-motor decision processes, typically studied in isolation, interact via functional information-processing loops in the brain to produce complex, adaptive behaviors.

Monkeys choose as if maximizing utility compatible with basic principles of revealed preference theory

Revealed preference theory provides axiomatic tools for assessing whether individuals make observable choices "as if" they are maximizing an underlying utility function. The theory evokes a tradeoff between goods whereby individuals improve themselves by trading one good for another good to obtain the best combination. Preferences revealed in these choices are modeled as curves of equal choice (indifference curves) and reflect an underlying process of optimization. These notions have far-reaching applications in consumer choice theory and impact the welfare of human and animal populations. However, they lack the empirical implementation in animals that would be required to establish a common biological basis. In a design using basic features of revealed preference theory, we measured in rhesus monkeys the frequency of repeated choices between bundles of two liquids. For various liquids, the animals' choices were compatible with the notion of giving up a quantity of one good to gain one unit of another good while maintaining choice indifference, thereby implementing the concept of marginal rate of substitution. The indifference maps consisted of nonoverlapping, linear, convex, and occasionally concave curves with typically negative, but also sometimes positive, slopes depending on bundle composition. Out-of-sample predictions using homothetic polynomials validated the indifference curves. The animals' preferences were internally consistent in satisfying transitivity. Change of option set size demonstrated choice optimality and satisfied the Weak Axiom of Revealed Preference (WARP). These data are consistent with a version of revealed preference theory in which preferences are stochastic; the monkeys behaved "as if" they had well-structured preferences and maximized utility.

Temporal coding of reward-guided choice in the posterior parietal cortex

Making a decision involves computations across distributed cortical and subcortical networks. How such distributed processing is performed remains unclear. We test how the encoding of choice in a key decision-making node, the posterior parietal cortex (PPC), depends on the temporal structure of the surrounding population activity. We recorded spiking and local field potential (LFP) activity in the PPC while two rhesus macaques performed a decision-making task. We quantified the mutual information that neurons carried about an upcoming choice and its dependence on LFP activity. The spiking of PPC neurons was correlated with LFP phases at three distinct time scales in the theta, beta, and gamma frequency bands. Importantly, activity at these time scales encoded upcoming decisions differently. Choice information contained in neural firing varied with the phase of beta and gamma activity. For gamma activity, maximum choice information occurred at the same phase as the maximum spike count. However, for beta activity, choice information and spike count were greatest at different phases. In contrast, theta activity did not modulate the encoding properties of PPC units directly but was correlated with beta and gamma activity through cross-frequency coupling. We propose that the relative timing of local spiking and choice information reveals temporal reference frames for computations in either local or large-scale decision networks. Differences between the timing of task information and activity patterns may be a general signature of distributed processing across large-scale networks.

Posterior resting state EEG asymmetries are associated with hedonic valuation of food

Research on the hedonic value of food has been important in understanding the motivational and emotional correlates of normal and abnormal eating behaviour. The aim of the present study was to explore associations between hemispheric asymmetries recorded during resting state electroencephalogram (EEG) and hedonic valuation of food. Healthy adult volunteers were recruited and four minutes of resting state EEG were recorded from the scalp. Hedonic food valuation and reward sensitivity were assessed with the hedonic attitude to food and behavioural activation scale. Results showed that parieto-occipital resting state EEG asymmetries in the alpha (8-12Hz) and beta (13-30Hz) frequency range correlate with the hedonic valuation of food. Our findings suggest that self-reported sensory-related attitude towards food is associated with interhemispheric asymmetries in resting state oscillatory activity. Our findings contribute to understanding the electrophysiological correlates of hedonic valuation, and may provide an opportunity to modulate the cortical imbalance by using non-invasive brain stimulation methods to change food consumption.

Neuroanatomical Substrates for Risk Behavior

Individuals vary substantially in their tendency to take risks. In the past two decades, a large number of neuroimaging studies in humans have explored the neural mechanisms of several cognitive processes that contribute to risk taking. In this article, I focus on functional and structural MRI studies that investigated uncertainty processing, one of the main features of risk behavior. Using decision-making and learning paradigms, these studies implicated a network of brain areas, including posterior parietal cortex, anterior insula, anterior cingulate cortex, and ventrolateral prefrontal cortex, in various aspects of uncertainty processing. Individual differences in behavior under uncertainty are reflected in the function and structure of some of these areas and are integrated into value representations in ventromedial prefrontal cortex and ventral striatum, reinforcing the potential contribution of all of these brain structures to individual tendencies to take risks.

History-dependent variability in population dynamics during evidence accumulation in cortex

We studied how the posterior parietal cortex combined new information with ongoing activity dynamics as mice accumulated evidence during a virtual-navigation task. Using new methods to analyze population activity on single trials, we found that activity transitioned rapidly between different sets of active neurons. Each event in a trial — whether an evidence cue or a behavioral choice — caused seconds-long modifications to the probabilities that govern how one activity pattern transitions to the next, forming a short-term memory. A sequence of evidence cues triggered a chain of these modifications resulting in a signal for accumulated evidence. Multiple distinguishable activity patterns were possible for the same accumulated evidence because representations of ongoing events were influenced by previous within and across trial events. Therefore, evidence accumulation need not require the explicit competition between groups of neurons, as in winner-take-all models, but could instead emerge implicitly from general dynamical properties that instantiate short-term memory.

A dynamic code for economic object valuation in prefrontal cortex neurons

Neuronal reward valuations provide the physiological basis for economic behaviour. Yet, how such valuations are converted to economic decisions remains unclear. Here we show that the dorsolateral prefrontal cortex (DLPFC) implements a flexible value code based on object-specific valuations by single neurons. As monkeys perform a reward-based foraging task, individual DLPFC neurons signal the value of specific choice objects derived from recent experience. These neuronal object values satisfy principles of competitive choice mechanisms, track performance fluctuations and follow predictions of a classical behavioural model (Herrnstein’s matching law). Individual neurons dynamically encode both, the updating of object values from recently experienced rewards, and their subsequent conversion to object choices during decision-making. Decoding from unselected populations enables a read-out of motivational and decision variables not emphasized by individual neurons. These findings suggest a dynamic single-neuron and population value code in DLPFC that advances from reward experiences to economic object values and future choices.

Economic decisions are based on perceived reward value but it is unclear how individual neurons encode value estimates as input for decision mechanisms. Here authors show that dorsolateral prefrontal cortex uses a dynamic value code based on object-specific valuations by single neurons.

Effects of reward on oculomotor control

The present study examines the extent to which distractors that signal the availability of monetary reward on a given trial affect eye movements. We used a novel eye movement task in which observers had to follow a target around the screen while ignoring distractors presented at varying locations. We examined the effects of reward magnitude and distractor location on a host of oculomotor properties, including saccade latency, amplitude, landing position, curvature, and erroneous saccades toward the distractor. We found consistent effects of reward magnitude on classic oculomotor phenomena such as the remote distractor effect, the global effect, and oculomotor capture by the distractor. We also show that a distractor in the visual hemifield opposite to the target had a larger effect on oculomotor control than an equidistant distractor in the same hemifield as the target. Bayesian hierarchical drift diffusion modeling revealed large differences in drift rate depending on the reward value, location, and visual hemifield of the distractor stimulus. Our findings suggest that high reward distractors not only capture the eyes but also affect a multitude of oculomotor properties associated with oculomotor inhibition and control.

Learning the opportunity cost of time in a patch-foraging task

Although most decision research concerns choice between simultaneously presented options, in many situations options are encountered serially, and the decision is whether to exploit an option or search for a better one. Such problems have a rich history in animal foraging, but we know little about the psychological processes involved. In particular, it is unknown whether learning in these problems is supported by the well-studied neurocomputational mechanisms involved in more conventional tasks. We investigated how humans learn in a foraging task, which requires deciding whether to harvest a depleting resource or switch to a replenished one. The optimal choice (given by the marginal value theorem; MVT) requires comparing the immediate return from harvesting to the opportunity cost of time, which is given by the long-run average reward. In two experiments, we varied opportunity cost across blocks, and subjects adjusted their behavior to blockwise changes in environmental characteristics. We examined how subjects learned their choice strategies by comparing choice adjustments to a learning rule suggested by the MVT (in which the opportunity cost threshold is estimated as an average over previous rewards) and to the predominant incremental-learning theory in neuroscience, temporal-difference learning (TD). Trial-by-trial decisions were explained better by the MVT threshold-learning rule. These findings expand on the foraging literature, which has focused on steady-state behavior, by elucidating a computational mechanism for learning in switching tasks that is distinct from those used in traditional tasks, and suggest connections to research on average reward rates in other domains of neuroscience.

Auditory-induced neural dynamics in sensory-motor circuitry predict learned temporal and sequential statistics of birdsong

Predicting future events is a critical computation for both perception and behavior. Despite the essential nature of this computation, there are few studies demonstrating neural activity that predicts specific events in learned, probabilistic sequences. Here, we test the hypotheses that the dynamics of internally generated neural activity are predictive of future events and are structured by the learned temporal-sequential statistics of those events. We recorded neural activity in Bengalese finch sensory-motor area HVC in response to playback of sequences from individuals' songs, and examined the neural activity that continued after stimulus offset. We found that the strength of response to a syllable in the sequence depended on the delay at which that syllable was played, with a maximal response when the delay matched the intersyllable gap normally present for that specific syllable during song production. Furthermore, poststimulus neural activity induced by sequence playback resembled the neural response to the next syllable in the sequence when that syllable was predictable, but not when the next syllable was uncertain. Our results demonstrate that the dynamics of internally generated HVC neural activity are predictive of the learned temporal-sequential structure of produced song and that the strength of this prediction is modulated by uncertainty.

Reward-based plasticity of spatial priority maps: Exploiting inter-subject variability to probe the underlying neurobiology

Recent evidence indicates that the attentional priority of objects and locations is altered by the controlled delivery of reward, reflecting reward-based attentional learning. Here, we take an approach hinging on intersubject variability to probe the neurobiological bases of the reward-driven plasticity of spatial priority maps. Specifically, we ask whether an individual's susceptibility to the reward-based treatment can be accounted for by specific predictors, notably personality traits that are linked to reward processing (along with more general personality traits), but also gender. Using a visual search protocol, we show that when different target locations are associated with unequal reward probability, different priorities are acquired by the more rewarded relative to the less rewarded locations. However, while males exhibit the expected pattern of results, with greater priority for locations associated with higher reward, females show an opposite trend. Critically, both the extent and the direction of reward-based adjustments are further predicted by personality traits indexing reward sensitivity, indicating that not only male and female brains are differentially sensitive to reward, but also that specific personality traits further contribute to shaping their learning-dependent attentional plasticity. These results contribute to a better understanding of the neurobiology underlying reward-dependent attentional learning and cross-subject variability in this domain.

Valor del Reforzador: ¿Cómo se Usa y Para qué se Usa el Concepto?

<p>La reseña muestra un panorama general de qué es el valor del reforzador, cómo se ha conceptualizado en la literatura y qué investigaciones han utilizado el concepto.  En un sentido general el concepto se utiliza para calificar un reforzador como más o menos efectivo: entre mayor valor del reforzador mayor su eficacia.  Primero se mostrará cómo se ha medido históricamente el valor del reforzador y cómo se ha definido a partir de la literatura sobre economía conductual.  Luego se mostrarán dos usos diferentes del concepto: 1). constructo hipotético, medible indirectamente a través de diferentes procedimientos, 2). variable interviniente, reúne los efectos de un conjunto de operaciones experimentales.  En el segundo conjunto también se incluyen definiciones operacionales, donde no se define exhaustivamente todas las variables dependientes e independientes asociadas, por lo que no es variable interviniente, pero tampoco agrega significado más allá del nivel de observación, por lo que no es constructo hipotético.  Luego se explora la relación entre demora del reforzador y descuento temporal, un tema de gran relevancia en la investigación contemporánea.  En las consideraciones finales se retoma la discusión sobre el valor heurístico y la conveniencia de usar el concepto de valor de reforzador.</p>

Decision Neuroscience

Investigations of decision making have historically been undertaken by different disciplines, each using different techniques and assumptions, and few unifying efforts have been made. Economists have focused on precise mathematical models of normative decision making, psychologists have examined how decisions are actually made based on cognitive constraints, and neuroscientists have concentrated on the detailed operation of neural systems in simple choices. In recent years, however, researchers in these separate fields have joined forces in an attempt to better specify the foundations of decision making. This interdisciplinary effort has begun to use decision theory to guide the search for the neural bases of reward value and predictability. Concurrently, these formal models are beginning to incorporate processes such as social reward and emotion. The combination of these diverse theoretical approaches and methodologies is already yielding significant progress in the construction of more comprehensive decision-making models.

Decoding subjective decisions from orbitofrontal cortex

When making a subjective choice, the brain must compute a value for each option and compare those values to make a decision. The orbitofrontal cortex (OFC) is critically involved in this process, but the neural mechanisms remain obscure, in part due to limitations in our ability to measure and control the internal deliberations that can alter the dynamics of the decision process. Here, we tracked the dynamics by recovering temporally precise neural states from multi-dimensional data in OFC. During individual choices, OFC alternated between states associated with the value of two available options, with dynamics that predicted whether a subject would decide quickly or vacillate between the two alternatives. Ensembles of value-encoding neurons contributed to these states, with individual neurons shifting activity patterns as the network evaluated each option. Thus, the mechanism of subjective decision-making involves the dynamic activation of OFC states associated with each choice alternative.

Free Will and Neuroscience: From Explaining Freedom Away to New Ways of Operationalizing and Measuring It

The concept of free will is hard to define, but crucial to both individual and social life. For centuries people have wondered how freedom is possible in a world ruled by physical determinism; however, reflections on free will have been confined to philosophy until half a century ago, when the topic was also addressed by neuroscience. The first relevant, and now well-known, strand of research on the brain correlates of free will was that pioneered by Libet et al. (1983), which focused on the allegedly unconscious intentions taking place in decisions regarded as free and voluntary. Libet’s interpretation of the so-called readiness potential (RP) seems to favor a sort of deflation of freedom (Soon et al., 2008). However, recent studies seem to point to a different interpretation of the RP, namely that the apparent build-up of the brain activity preceding subjectively spontaneous voluntary movements (SVM) may reflect the ebb and flow of the background neuronal noise, which is triggered by many factors (Schurger et al., 2016). This interpretation seems to bridge the gap between the neuroscientific perspective on free will and the intuitive, commonsensical view of it (Roskies, 2010b), but many problems remain to be solved and other theoretical paths can be hypothesized. The article therefore, proposes to start from an operationalizable concept of free will (Lavazza and Inglese, 2015) to find a connection between higher order descriptions (useful for practical life) and neural bases. This new way to conceptualize free will should be linked to the idea of “capacity”: that is, the availability of a repertoire of general skills that can be manifested and used without moment by moment conscious control. The capacity index, which is also able to take into account the differences of time scales in decisions, includes reasons-responsiveness and is related to internal control, understood as the agent’s ownership of the mechanisms that trigger the relevant behavior. Cognitive abilities, needed for one to have capacity, might be firstly operationalized as a set of neuropsychological tests, which can be used to operationalize and measure specific executive functions, as they are strongly linked to the concept of control. Subsequently, a free will index would allow for the search of the underlying neural correlates of the capacity exhibited by people and the limits in capacity exhibited by each individual.

Playing the electric light orchestra--how electrical stimulation of visual cortex elucidates the neural basis of perception

Vision research has the potential to reveal fundamental mechanisms underlying sensory experience. Causal experimental approaches, such as electrical microstimulation, provide a unique opportunity to test the direct contributions of visual cortical neurons to perception and behaviour. But in spite of their importance, causal methods constitute a minority of the experiments used to investigate the visual cortex to date. We reconsider the function and organization of visual cortex according to results obtained from stimulation techniques, with a special emphasis on electrical stimulation of small groups of cells in awake subjects who can report their visual experience. We compare findings from humans and monkeys, striate and extrastriate cortex, and superficial versus deep cortical layers, and identify a number of revealing gaps in the ‘causal map′ of visual cortex. Integrating results from different methods and species, we provide a critical overview of the ways in which causal approaches have been used to further our understanding of circuitry, plasticity and information integration in visual cortex. Electrical stimulation not only elucidates the contributions of different visual areas to perception, but also contributes to our understanding of neuronal mechanisms underlying memory, attention and decision-making.

The Cognitive Neuroscience of Human Decision Making: A Review and Conceptual Framework

Decision making, the process of choosing between options, is a fundamental human behavior that has been studied intensively by disciplines ranging from cognitive psychology to economics. Despite the importance of this behavior, the neural substrates of decision making are only beginning to be understood. Impaired decision making is recognized in neuropsychiatric conditions such as dementia and drug addiction, and the inconsistencies and biases of healthy decision makers have been intensively studied. However, the tools of cognitive neuroscience have only recently been applied to understanding the brain basis of this complex behavior. This article reviews the literature on the cognitive neuroscience of human decision making, focusing on the roles of the frontal lobes, and provides a conceptual framework for organizing this disparate body of work.

Auditory Processing in the Posterior Parietal Cortex

Goal-directed behavior can be characterized as a dynamic link between a sensory stimulus and a motor act. Neural correlates of many of the intermediate events of goal-directed behavior are found in the posterior parietal cortex. Although the parietal cortex’s role in guiding visual behaviors has received considerable attention, relatively little is known about its role in mediating auditory behaviors. Here, the authors review recent studies that have focused on how neurons in the lateral intraparietal area (area LIP) differentially process auditory and visual stimuli. These studies suggest that area LIP contains a modality-dependent representation that is highly dependent on behavioral context.

Neural Mechanisms Underlying Compensatory and Noncompensatory Strategies in Risky Choice

Individuals may differ systematically in their applied decision strategies, which has critical implications for decision neuroscience but is yet scarcely studied. Our study's main focus was therefore to investigate the neural mechanisms underlying compensatory versus noncompensatory strategies in risky choice. Here, we compared people using a compensatory expected value maximization with people using a simplified noncompensatory loss-minimizing choice strategy. To this end, we used a two-choice paradigm including a set of "simple" items (e.g., simple condition), in which one option was superior on all attributes, and a set of "conflict" items, in which one option was superior on one attribute but inferior on other attributes. A binomial mixture analysis of the decisions elicited by these items differentiated between decision-makers using either a compensatory or a noncompensatory strategy. Behavioral differences were particularly pronounced in the conflict condition, and these were paralleled by neural results. That is, we expected compensatory decision-makers to use an integrated value comparison during choice in the conflict condition. Accordingly, the compensatory group tracked the difference in expected value between choice options reflected in neural activation in the parietal cortex. Furthermore, we expected noncompensatory, compared with compensatory, decision-makers to experience increased conflict when attributes provided conflicting information. Accordingly, the noncompensatory group showed greater dorsomedial PFC activation only in the conflict condition. These pronounced behavioral and neural differences indicate the need for decision neuroscience to account for individual differences in risky choice strategies and to broaden its scope to noncompensatory risky choice strategies.

Emotor control: computations underlying bodily resource allocation, emotions, and confidence

Emotional processes are central to behavior, yet their deeply subjective nature has been a challenge for neuroscientific study as well as for psychiatric diagnosis. Here we explore the relationships between subjective feelings and their underlying brain circuits from a computational perspective. We apply recent insights from systems neuroscience—approaching subjective behavior as the result of mental computations instantiated in the brain—to the study of emotions. We develop the hypothesis that emotions are the product of neural computations whose motor role is to reallocate bodily resources mostly gated by smooth muscles. This “emotor” control system is analagous to the more familiar motor control computations that coordinate skeletal muscle movements. To illustrate this framework, we review recent research on “confidence.” Although familiar as a feeling, confidence is also an objective statistical quantity: an estimate of the probability that a hypothesis is correct. This model-based approach helped reveal the neural basis of decision confidence in mammals and provides a bridge to the subjective feeling of confidence in humans. These results have important implications for psychiatry, since disorders of confidence computations appear to contribute to a number of psychopathologies. More broadly, this computational approach to emotions resonates with the emerging view that psychiatric nosology may be best parameterized in terms of disorders of the cognitive computations underlying complex behavior.

Coherent neuronal ensembles are rapidly recruited when making a look-reach decision

Selecting and planning actions recruits neurons across many areas of the brain but how ensembles of neurons work together to make decisions is unknown. Temporally-coherent neural activity may provide a mechanism by which neurons coordinate their activity in order to make decisions. If so, neurons that are part of coherent ensembles may predict movement choices before other ensembles of neurons. We recorded neuronal activity in the lateral and medial banks of the intraparietal sulcus (IPS) of the posterior parietal cortex, while monkeys made choices about where to look and reach and decoded the activity to predict the choices. Ensembles of neurons that displayed coherent patterns of spiking activity extending across the IPS, “dual coherent” ensembles, predicted movement choices substantially earlier than other neuronal ensembles. We propose that dual-coherent spike timing reflects interactions between groups of neurons that play an important role in how we make decisions.

Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system

Recent experiments have shown that animals and humans have a remarkable ability to adapt their learning rate according to the volatility of the environment. Yet the neural mechanism responsible for such adaptive learning has remained unclear. To fill this gap, we investigated a biophysically inspired, metaplastic synaptic model within the context of a well-studied decision-making network, in which synapses can change their rate of plasticity in addition to their efficacy according to a reward-based learning rule. We found that our model, which assumes that synaptic plasticity is guided by a novel surprise detection system, captures a wide range of key experimental findings and performs as well as a Bayes optimal model, with remarkably little parameter tuning. Our results further demonstrate the computational power of synaptic plasticity, and provide insights into the circuit-level computation which underlies adaptive decision-making.

Mesolimbic dopamine signals the value of work

Dopamine cell firing can encode errors in reward prediction, providing a learning signal to guide future behavior. Yet dopamine is also a key modulator of motivation, invigorating current behavior. Existing theories propose that fast (phasic) dopamine fluctuations support learning, whereas much slower (tonic) dopamine changes are involved in motivation. We examined dopamine release in the nucleus accumbens across multiple time scales, using complementary microdialysis and voltammetric methods during adaptive decision-making. We found that minute-by-minute dopamine levels covaried with reward rate and motivational vigor. Second-by-second dopamine release encoded an estimate of temporally discounted future reward (a value function). Changing dopamine immediately altered willingness to work and reinforced preceding action choices by encoding temporal-difference reward prediction errors. Our results indicate that dopamine conveys a single, rapidly evolving decision variable, the available reward for investment of effort, which is employed for both learning and motivational functions.

Reward alters the perception of time

Recent findings indicate that monetary rewards have a powerful effect on cognitive performance. In order to maximize overall gain, the prospect of earning reward biases visual attention to specific locations or stimulus features improving perceptual sensitivity and processing. The question we addressed in this study is whether the prospect of reward also affects the subjective perception of time. Here, participants performed a prospective timing task using temporal oddballs. The results show that temporal oddballs, displayed for varying durations, presented in a sequence of standard stimuli were perceived to last longer when they signaled a relatively high reward compared to when they signaled no or low reward. When instead of the oddball the standards signaled reward, the perception of the temporal oddball remained unaffected. We argue that by signaling reward, a stimulus becomes subjectively more salient thereby modulating its attentional deployment and distorting how it is perceived in time.

Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats

Decision-making behavior is often characterized by substantial variability, but its source remains unclear. We developed a visual accumulation of evidence task designed to quantify sources of noise and to be performed during voluntary head restraint, enabling cellular resolution imaging in future studies. Rats accumulated discrete numbers of flashes presented to the left and right visual hemifields and indicated the side that had the greater number of flashes. Using a signal-detection theory-based model, we found that the standard deviation in their internal estimate of flash number scaled linearly with the number of flashes. This indicates a major source of noise that, surprisingly, is not consistent with the widely used 'drift-diffusion modeling' (DDM) approach but is instead closely related to proposed models of numerical cognition and counting. We speculate that this form of noise could be important in accumulation of evidence tasks generally.

DOI:http://dx.doi.org/10.7554/eLife.11308.001

Perceptual decision-making, i.e. making choices based on observed evidence, is rarely perfect. Humans and other animals tend to respond correctly on some trials and incorrectly on others. For over a century, this variability has been used to study the basis of decision-making. Most behavioral models assume that random fluctuations or 'noise' in the decision-making process is the primary source of variability and errors. However, the nature of this noise is unclear and the subject of intense scrutiny.

To investigate the sources of the behavioral variability during decision-making, Scott, Constantinople et al. trained rats to perform a visual 'accumulation of evidence' task. The animals counted flashes of light that appeared on either their left or their right. Up to 15 flashes occurred on each side, in a random order, and the rats then received a reward if they selected the side that the greatest number of flashes had occurred on. The rats chose correctly on many occasions but not on every single one.

Using a computer-controlled rat training facility or 'rat academy', Scott, Constantinople et al. collected hundreds of thousands of behavioral trials from over a dozen rats. This large dataset provided the statistical power necessary to test the assumptions of leading models of behavioral variability during decision-making, and revealed that noise grew more rapidly with the number of flashes than previously predicted. This finding explained patterns of behavior that previous models struggled with, most notably the fact that individuals make errors even on the easiest trials. The analysis also revealed that animals maintain two separate running totals – one of stimuli on the left and another of stimuli on the right – rather than a single tally of the difference between the two.

Scott, Constantinople et al. further demonstrated that rats could be trained to perform this task using a new system that enables functional brain imaging. The next step is to repeat these experiments while simultaneously recording brain activity to study the neural circuits that underlie decision-making and its variability.

DOI:http://dx.doi.org/10.7554/eLife.11308.002

The Role of the Parietal Cortex in the Representation of Task-Reward Associations

Rewards obtained from specific behaviors can and do change across time. To adapt to such conditions, humans need to represent and update associations between behaviors and their outcomes. Much previous work focused on how rewards affect the processing of specific tasks. However, abstract associations between multiple potential behaviors and multiple rewards are an important basis for adaptation as well. In this experiment, we directly investigated which brain areas represent associations between multiple tasks and rewards, using time-resolved multivariate pattern analysis of functional magnetic resonance imaging data. Importantly, we were able to dissociate neural signals reflecting task-reward associations from those related to task preparation and reward expectation processes, variables that were often correlated in previous research. We hypothesized that brain regions involved in processing tasks and/or rewards will be involved in processing associations between them. Candidate areas included the dorsal anterior cingulate cortex, which is involved in associating simple actions and rewards, and the parietal cortex, which has been shown to represent task rules and action values. Our results indicate that local spatial activation patterns in the inferior parietal cortex indeed represent task-reward associations. Interestingly, the parietal cortex flexibly changes its content of representation within trials. It first represents task-reward associations, later switching to process tasks and rewards directly. These findings highlight the importance of the inferior parietal cortex in associating behaviors with their outcomes and further show that it can flexibly reconfigure its function within single trials. Significance statement: Rewards obtained from specific behaviors rarely remain constant over time. To adapt to changing conditions, humans need to continuously update and represent the current association between behavior and its outcomes. However, little is known about the neural representation of behavior-outcome associations. Here, we used multivariate pattern analysis of functional magnetic resonance imaging data to investigate the neural correlates of such associations. Our results demonstrate that the parietal cortex plays a central role in representing associations between multiple behaviors and their outcomes. They further highlight the flexibility of the parietal cortex, because we find it to adapt its function to changing task demands within trials on relatively short timescales.

Sequential selection of economic good and action in medial frontal cortex of macaques during value-based decisions

Value-based decisions could rely either on the selection of desired economic goods or on the selection of the actions that will obtain the goods. We investigated this question by recording from the supplementary eye field (SEF) of monkeys during a gambling task that allowed us to distinguish chosen good from chosen action signals. Analysis of the individual neuron activity, as well as of the population state-space dynamic, showed that SEF encodes first the chosen gamble option (the desired economic good) and only ~100 ms later the saccade that will obtain it (the chosen action). The action selection is likely driven by inhibitory interactions between different SEF neurons. Our results suggest that during value-based decisions, the selection of economic goods precedes and guides the selection of actions. The two selection steps serve different functions and can therefore not compensate for each other, even when information guiding both processes is given simultaneously.

DOI:http://dx.doi.org/10.7554/eLife.09418.001

Much of our decision making seems to involve selecting the best option from among those currently available, and then working out how to attain that particular outcome. However, while this might sound straightforward in principle, exactly how this process is organized within the brain is not entirely clear.

One possibility is that the brain compares all the possible outcomes of a decision with each other before constructing a plan of action to achieve the most desirable of these. This is known as the 'goods-based' model of decision making. However, an alternative possibility is that the brain instead considers all the possible actions that could be performed at any given time. One specific action is then chosen based on a range of factors, including the potential outcomes that might result from each. This is an 'action-based' model of decision making.

Chen and Stuphorn have now distinguished between these possibilities by training two monkeys to perform a gambling task. The animals learned to make eye movements to one of two targets on a screen to earn a reward. The identity of the targets varied between trials, with some associated with larger rewards or a higher likelihood of receiving a reward than others. The location of the targets also changed in different trials, which meant that the choice of 'action' (moving the eyes to the left or right) could be distinguished from the choice of 'goods' (the reward).

By using electrodes to record from a region of the brain called the supplementary eye field, which helps to control eye movements, Chen and Stuphorn showed that the activity of neurons in this region predicted the monkeys’ decision-making behavior. Crucially, it did so in two stages: neurons first encoded the reward chosen by the monkey, before subsequently encoding the action that the monkey selected to obtain that outcome.

These data argue against an action-based model of decision making because outcomes are encoded before actions. However, they also argue against a purely goods-based model. This is because all possible actions are encoded by the brain (including those that are subsequently rejected), with the highest levels of activity seen for the action that is ultimately selected. The data instead support a new model of decision making, in which outcomes and actions are selected sequentially via two independent brain circuits.

DOI:http://dx.doi.org/10.7554/eLife.09418.002

I Plan Therefore I Choose: Free-Choice Bias Due to Prior Action-Probability but Not Action-Value

According to an emerging view, decision-making, and motor planning are tightly entangled at the level of neural processing. Choice is influenced not only by the values associated with different options, but also biased by other factors. Here we test the hypothesis that preliminary action planning can induce choice biases gradually and independently of objective value when planning overlaps with one of the potential action alternatives. Subjects performed center-out reaches obeying either a clockwise or counterclockwise cue-response rule in two tasks. In the probabilistic task, a pre-cue indicated the probability of each of the two potential rules to become valid. When the subsequent rule-cue unambiguously indicated which of the pre-cued rules was actually valid (instructed trials), subjects responded faster to rules pre-cued with higher probability. When subjects were allowed to choose freely between two equally rewarded rules (choice trials) they chose the originally more likely rule more often and faster, despite the lack of an objective advantage in selecting this target. In the amount task, the pre-cue indicated the amount of potential reward associated with each rule. Subjects responded faster to rules pre-cued with higher reward amount in instructed trials of the amount task, equivalent to the more likely rule in the probabilistic task. Yet, in contrast, subjects showed hardly any choice bias and no increase in response speed in favor of the original high-reward target in the choice trials of the amount task. We conclude that free-choice behavior is robustly biased when predictability encourages the planning of one of the potential responses, while prior reward expectations without action planning do not induce such strong bias. Our results provide behavioral evidence for distinct contributions of expected value and action planning in decision-making and a tight interdependence of motor planning and action selection, supporting the idea that the underlying neural mechanisms overlap.

Dynamics of pre- and post-choice behaviour: rats approximate optimal strategy in a discrete-trial decision task

We simulate two types of environments to investigate how closely rats approximate optimal foraging. Rats initiated a trial where they chose between two spouts for sucrose, which was delivered at distinct probabilities. The discrete trial procedure used allowed us to observe the relationship between choice proportions, response latencies and obtained rewards. Our results show that rats approximate the optimal strategy across a range of environments that differ in the average probability of reward as well as the dynamics of the depletion-renewal cycle. We found that the constituent components of a single choice differentially reflect environmental contingencies. Post-choice behaviour, measured as the duration of time rats spent licking at the spouts on unrewarded trials, was the most sensitive index of environmental variables, adjusting most rapidly to changes in the environment. These findings have implications for the role of confidence in choice outcomes for guiding future choices.

Inactivation of Parietal Reach Region Affects Reaching But Not Saccade Choices in Internally Guided Decisions

The posterior parietal cortex (PPC) has traditionally been considered important for awareness, spatial perception, and attention. However, recent findings provide evidence that the PPC also encodes information important for making decisions. These findings have initiated a running argument of whether the PPC is critically involved in decision making. To examine this issue, we reversibly inactivated the parietal reach region (PRR), the area of the PPC that is specialized for reaching movements, while two monkeys performed a memory-guided reaching or saccade task. The task included choices between two equally rewarded targets presented simultaneously in opposite visual fields. Free-choice trials were interleaved with instructed trials, in which a single cue presented in the peripheral visual field defined the reach and saccade target unequivocally. We found that PRR inactivation led to a strong reduction of contralesional choices, but only for reaches. On the other hand, saccade choices were not affected by PRR inactivation. Importantly, reaching and saccade movements to single instructed targets remained largely intact. These results cannot be explained as an effector-nonspecific deficit in spatial attention or awareness, since the temporary "lesion" had an impact only on reach choices. Hence, the PPR is a part of a network for reach decisions and not just reach planning.

SIGNIFICANCE STATEMENT

There has been an ongoing debate on whether the posterior parietal cortex (PPC) represents only spatial awareness, perception, and attention or whether it is also involved in decision making for actions. In this study we explore whether the parietal reach region (PRR), the region of the PPC that is specialized for reaches, is involved in the decision process. We inactivated the PRR while two monkeys performed reach and saccade choices between two targets presented simultaneously in both hemifields. We found that inactivation affected only the reach choices, while leaving saccade choices intact. These results cannot be explained as a deficit in attention, since the temporary lesion affected only the reach choices. Thus, PRR is a part of a network for making reach decisions.

Attentional Selection Can Be Predicted by Reinforcement Learning of Task-relevant Stimulus Features Weighted by Value-independent Stickiness

Attention includes processes that evaluate stimuli relevance, select the most relevant stimulus against less relevant stimuli, and bias choice behavior toward the selected information. It is not clear how these processes interact. Here, we captured these processes in a reinforcement learning framework applied to a feature-based attention task that required macaques to learn and update the value of stimulus features while ignoring nonrelevant sensory features, locations, and action plans. We found that value-based reinforcement learning mechanisms could account for feature-based attentional selection and choice behavior but required a value-independent stickiness selection process to explain selection errors while at asymptotic behavior. By comparing different reinforcement learning schemes, we found that trial-by-trial selections were best predicted by a model that only represents expected values for the task-relevant feature dimension, with nonrelevant stimulus features and action plans having only a marginal influence on covert selections. These findings show that attentional control subprocesses can be described by (1) the reinforcement learning of feature values within a restricted feature space that excludes irrelevant feature dimensions, (2) a stochastic selection process on feature-specific value representations, and (3) value-independent stickiness toward previous feature selections akin to perseveration in the motor domain. We speculate that these three mechanisms are implemented by distinct but interacting brain circuits and that the proposed formal account of feature-based stimulus selection will be important to understand how attentional subprocesses are implemented in primate brain networks.

Reward expectation differentially modulates attentional behavior and activity in visual area V4

Neural activity in visual area V4 is enhanced when attention is directed into neuronal receptive fields. However, the source of this enhancement is unclear, as most physiological studies have manipulated attention by changing the absolute reward associated with a particular location as well as its value relative to other locations. We trained monkeys to discriminate the orientation of two stimuli presented simultaneously in different hemifields while we independently varied the reward magnitude associated with correct discrimination at each location. Behavioral measures of attention were controlled by the relative value of each location. By contrast, neurons in V4 were consistently modulated by absolute reward value, exhibiting increased activity, increased gamma-band power and decreased trial-to-trial variability whenever receptive field locations were associated with large rewards. These data challenge the notion that the perceptual benefits of spatial attention rely on increased signal-to-noise in V4. Instead, these benefits likely derive from downstream selection mechanisms.