Matching behavior and the representation of value in the parietal cortex

Brain activation covaries with reported criminal behaviors when making risky choices: A fuzzy-trace theory approach

Criminal behavior has been associated with abnormal neural activity when people experience risks and rewards or exercise inhibition. However, neural substrates of mental representations that underlie criminal and noncriminal risk-taking in adulthood have received scant attention. We take a new approach, applying fuzzy-trace theory, to examine neural substrates of risk preferences and criminality. We extend ideas about gist (simple meaning) and verbatim (precise risk-reward tradeoffs) representations used to explain adolescent risk-taking to uncover neural correlates of developmentally inappropriate adult risk-taking. We tested predictions using a risky-choice framing task completed in the MRI scanner, and examined neural covariation with self-reported criminal and noncriminal risk-taking. As predicted, risk-taking was correlated with a behavioral pattern of risk preferences called "reverse framing" (preferring sure losses over a risky option and a risky option over sure gains, the opposite of typical framing biases) that has been linked to risky behavior in adolescents and is rarely observed in nondisordered adults. Experimental manipulations confirmed processing interpretations of typical framing (gist-based) and reverse-framing (verbatim-based) risk preferences. In the brain, covariation with criminal and noncriminal risk-taking was observed predominantly when subjects made reverse-framing choices. Noncriminal risk-taking behavior was associated with emotional reactivity (amygdala) and reward motivation (striatal) areas, whereas criminal behavior was associated with greater activation in temporal and parietal cortices, their junction, and insula. When subjects made more developmentally typical framing choices, reflecting nonpreferred gist processing, activation in dorsolateral prefrontal cortex covaried with criminal risk-taking, which may reflect cognitive effort to process gist while inhibiting preferred verbatim processing. (PsycINFO Database Record

Probability Matching on a Simple Simulated Foraging Task: The Effects of Reward Persistence and Accumulation on Choice Behavior

Over a series of decisions between two or more probabilistically rewarded options, humans have a tendency to diversify their choices, even when this will lead to diminished overall reward. In the extreme case of probability matching, this tendency is expressed through allocation of choices in proportion to their likelihood of reward. Research suggests that this behaviour is an instinctive response, driven by heuristics, and that it may be overruled through the application of sufficient deliberation and self-control. However, if this is the case, then how and why did this response become established? The present study explores the hypothesis that diversification of choices, and potentially probability matching, represents an overextension of a historically normative foraging strategy. This is done through examining choice behaviour on a simple simulated foraging task, designed to model the natural process of accumulation of unharvested resources over time. Behaviour was then directly compared with that observed on a standard fixed probability task (cf. Ellerby & Tunney, 2017). Results indicated a convergence of choice patterns on the simulated foraging task, between participants who acted intuitively and those who took a more strategic approach. These findings are also compared with those of another similarly motivated study (Schulze, van Ravenzwaaij, & Newell, 2017).

Individual Neurons in the Cingulate Cortex Encode Action Monitoring, Not Selection, during Adaptive Decision-Making

The cingulate cortex contributes to complex, adaptive behaviors, but the exact nature of its contributions remains unresolved. Proposals from previous studies, including evaluating past actions or selecting future ones, have been difficult to distinguish in part because of an incomplete understanding of the task-relevant variables that are encoded by individual cingulate neurons. In this study, we recorded from individual neurons in parts of both the anterior cingulate cortex (ACC) and posterior cingulate cortex (PCC) in 2 male rhesus monkeys performing a saccadic reward task. The task required them to use adaptive, feedback-driven strategies to infer the spatial location of a rewarded saccade target in the presence of different forms of uncertainty. We found that task-relevant, spatially selective feedback signals were encoded by the activity of individual neurons in both brain regions, with stronger selectivity for spatial choice and reward-target signals in PCC and stronger selectivity for feedback in ACC. Moreover, neurons in both regions were sensitive to sequential effects of feedback that partly reflected sequential behavioral patterns. However, neither brain region exhibited systematic modulations by the blockwise conditions that governed the reliability of the trial-by-trial feedback and drove adaptive behavioral patterns. There was also little evidence that single-neuron responses in either brain region directly predicted the extent to which feedback and contextual information were used to inform choices on the subsequent trial. Thus, certain cingulate neurons encode diverse, evaluative signals needed for adaptive, feedback-driven decision-making, but those signals may be integrated elsewhere in the brain to guide actions.SIGNIFICANCE STATEMENT Effective decision-making in dynamic environments requires adapting to changes in feedback and context. The anterior and posterior cingulate cortex have been implicated in adaptive decision-making, but the exact nature of their respective roles remains unresolved. Here we compare patterns of task-driven activity of subsets of individual neurons from parts of the two brain regions in monkeys performing a saccadic task with dynamically changing reward locations. We find evidence for regional specializations in neural representations of choice and feedback, including task-relevant modulations of activity that could be used for performance monitoring. However, we find little evidence that these neural representations are used directly to adjust choice behavior, which thus likely requires integration of these signals elsewhere in the brain.

Subcortical Substrates of Explore-Exploit Decisions in Primates

The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate learning based on past choices and outcomes, it is unclear whether they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques as they solved a task that required them to balance novelty-driven exploration with exploitation of what they had already learned. Using a partially observable Markov decision process (POMDP) model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding explore-exploit decisions.

Flexibility of Timescales of Evidence Evaluation for Decision Making

To understand the neural mechanisms that support decision making, it is critical to characterize the timescale of evidence evaluation. Recent work has shown that subjects can adaptively adjust the timescale of evidence evaluation across blocks of trials depending on context [1]. However, it's currently unknown if adjustments to evidence evaluation occur online during deliberations based on a single stream of evidence. To examine this question, we employed a change-detection task in which subjects report their level of confidence in judging whether there has been a change in a stochastic auditory stimulus. Using a combination of psychophysical reverse correlation analyses and single-trial behavioral modeling, we compared the time period over which sensory information has leverage on detection report choices versus confidence. We demonstrate that the length of this period differs on separate sets of trials based on what's being reported. Surprisingly, confidence judgments on trials with no detection report are influenced by evidence occurring earlier than the time period of influence for detection reports. Our findings call into question models of decision formation involving static parameters that yield a singular timescale of evidence evaluation and instead suggest that the brain represents and utilizes multiple timescales of evidence evaluation during deliberation.

Dissociable dopamine dynamics for learning and motivation

The dopamine projection from ventral tegmental area (VTA) to nucleus accumbens (NAc) is critical for motivation to work for rewards and reward-driven learning. How dopamine supports both functions is unclear. Dopamine cell spiking can encode prediction errors, which are vital learning signals in computational theories of adaptive behaviour. By contrast, dopamine release ramps up as animals approach rewards, mirroring reward expectation. This mismatch might reflect differences in behavioural tasks, slower changes in dopamine cell spiking or spike-independent modulation of dopamine release. Here we compare spiking of identified VTA dopamine cells with NAc dopamine release in the same decision-making task. Cues that indicate an upcoming reward increased both spiking and release. However, NAc core dopamine release also covaried with dynamically evolving reward expectations, without corresponding changes in VTA dopamine cell spiking. Our results suggest a fundamental difference in how dopamine release is regulated to achieve distinct functions: broadcast burst signals promote learning, whereas local control drives motivation.

Area-Specificity and Plasticity of History-Dependent Value Coding During Learning

Decision making is often driven by the subjective value of available options, a value which is formed through experience. To support this fundamental behavior, the brain must encode and maintain the subjective value. To investigate the area specificity and plasticity of value coding, we trained mice in a value-based decision task and imaged neural activity in 6 cortical areas with cellular resolution. History- and value-related signals were widespread across areas, but their strength and temporal patterns differed. In expert mice, the retrosplenial cortex (RSC) uniquely encoded history- and value-related signals with persistent population activity patterns across trials. This unique encoding of RSC emerged during task learning with a strong increase in more distant history signals. Acute inactivation of RSC selectively impaired the reward-history-based behavioral strategy. Our results indicate that RSC flexibly changes its history coding and persistently encodes value-related signals to support adaptive behaviors.

The Neuro-Computational Architecture of Value-Based Selection in the Human Brain

Current neural models of value-based decision-making consider choices as a 2-stage process, proceeding from the "valuation" of each option under consideration to the "selection" of the best option on the basis of their subjective values. However, little is known about the computational mechanisms at play at the selection stage and its implementation in the human brain. Here, we used drift-diffusion models combined with model-based functional magnetic resonance imaging, effective connectivity, and multivariate pattern analysis to characterize the neuro-computational architecture of value-based decisions. We found that 2 key drift-diffusion computations at the selection stage, namely integration and choice readout, engage distinct brain regions, with the dorsolateral prefrontal cortex integrating a decision value signal computed in the ventromedial prefrontal cortex, and the posterior parietal cortex reading out choice outcomes. Our findings suggest that this prefronto-parietal network acts as a hub implementing behavioral selection through a distributed drift-diffusion process.

Short-Term Influence of Recent Trial History on Perceptual Choice Changes with Stimulus Strength

Perceptual decisions, especially for difficult stimuli, can be influenced by choices and outcomes in previous trials. However, it is not well understood how stimulus strength modulates the temporal characteristics as well as the magnitude of trial history influence. We addressed this question using a contrast detection task in freely moving mice. We found that, at lower as compared to higher stimulus contrast, the current choice of the mice was more influenced by choices and outcomes in the past trials and the influence emerged from a longer history. To examine the neural basis of stimulus strength-dependent history influence, we recorded from the secondary motor cortex (M2), a prefrontal region that plays an important role in cue-guided actions and memory-guided behaviors. We found that more M2 neurons conveyed information about choices on the past two trials at lower than at higher contrast. Furthermore, history-trial activity in M2 was important for decoding upcoming choice at low contrast. Thus, trial history influence of perceptual choice is adaptive to the strength of sensory evidence, which may be important for action selection in a dynamic environment.

Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales

Behavior deviating from our normative expectations often appears irrational. For example, even though behavior following the so-called matching law can maximize reward in a stationary foraging task, actual behavior commonly deviates from matching. Such behavioral deviations are interpreted as a failure of the subject; however, here we instead suggest that they reflect an adaptive strategy, suitable for uncertain, non-stationary environments. To prove it, we analyzed the behavior of primates that perform a dynamic foraging task. In such nonstationary environment, learning on both fast and slow timescales is beneficial: fast learning allows the animal to react to sudden changes, at the price of large fluctuations (variance) in the estimates of task relevant variables. Slow learning reduces the fluctuations but costs a bias that causes systematic behavioral deviations. Our behavioral analysis shows that the animals solved this bias-variance tradeoff by combining learning on both fast and slow timescales, suggesting that learning on multiple timescales can be a biologically plausible mechanism for optimizing decisions under uncertainty.

Recent experience can only provide limited information to guide decisions in a volatile environment. Here, the authors report that the choices made by a monkey in a dynamic foraging task can be better explained by a model that combines learning on both fast and slow timescales.

Responses of Neurons in Lateral Intraparietal Area Depend on Stimulus-Associated Reward During Binocular Flash Suppression

Discovering neural correlates of subjective perception and dissociating them from sensory input has fascinated neuroscientists for a long time. Bistable and multistable perception phenomena have exhibited great experimental potential to address this question. Here, we performed electrophysiological recordings from single neurons in lateral intraparietal area (LIP) of rhesus macaques during stimulus and perceptual transitions induced by binocular flash suppression (BFS). LIP neurons demonstrated transient bursts of activity after stimulus presentation and stimulus or perceptual switches but only a minority of cells demonstrated stimulus and perceptual selectivity. To enhance LIP neural selectivity, we performed a second experiment in which the competing stimuli were associated with asymmetric rewards. We found that transient and sustained activities substantially increased while the proportion of stimulus selective neurons remained approximately the same, albeit with increased selectivity magnitude. In addition, we observed mild increases in the proportion of perceptually selective neurons which also showed increase magnitude of selectivity. Importantly, the increased selectivity of cells after the reward manipulation was not directly reflecting the reward size per se but an enhancement in stimulus differentiation. Based on our results, we conjecture that LIP contributes to perceptual transitions and serves a modulatory role in perceptual selection taking into account the stimulus behavioral value.

Movement Vigor as a Reflection of Subjective Economic Utility

To understand subjective evaluation of an option, various disciplines have quantified the interaction between reward and effort during decision making, producing an estimate of economic utility, namely the subjective 'goodness' of an option. However, variables that affect utility of an option also influence the vigor of movements toward that option. For example, expectation of reward increases speed of saccadic eye movements, whereas expectation of effort decreases this speed. These results imply that vigor may serve as a new, real-time metric with which to quantify subjective utility, and that the control of movements may be an implicit reflection of the brain's economic evaluation of the expected outcome.

Latent goal models for dynamic strategic interaction

Understanding the principles by which agents interact with both complex environments and each other is a key goal of decision neuroscience. However, most previous studies have used experimental paradigms in which choices are discrete (and few), play is static, and optimal solutions are known. Yet in natural environments, interactions between agents typically involve continuous action spaces, ongoing dynamics, and no known optimal solution. Here, we seek to bridge this divide by using a “penalty shot” task in which pairs of monkeys competed against each other in a competitive, real-time video game. We modeled monkeys’ strategies as driven by stochastically evolving goals, onscreen positions that served as set points for a control model that produced observed joystick movements. We fit this goal-based dynamical system model using approximate Bayesian inference methods, using neural networks to parameterize players’ goals as a dynamic mixture of Gaussian components. Our model is conceptually simple, constructed of interpretable components, and capable of generating synthetic data that capture the complexity of real player dynamics. We further characterized players’ strategies using the number of change points on each trial. We found that this complexity varied more across sessions than within sessions, and that more complex strategies benefited offensive players but not defensive players. Together, our experimental paradigm and model offer a powerful combination of tools for the study of realistic social dynamics in the laboratory setting.

Most studies of strategic decision making make use of simple tasks in which agents choose among only a limited number of distinct options. But real-world behavior is complex, requiring ongoing adjustment of strategies. Here, we propose a new model that is capable of reproducing the rich behavior of monkeys playing against each other in a dynamic decision task. Our model quantifies players’ goals at each moment and offers a means of performing controlled experiments via simulation. This makes possible more realistic experimental paradigms for the study of strategic decision making and social interaction.

Modeling Saccadic Action Selection: Cortical and Basal Ganglia Signals Coalesce in the Superior Colliculus

The distributed nature of information processing in the brain creates a complex variety of decision making behavior. Likewise, computational models of saccadic decision making behavior are numerous and diverse. Here we present a generative model of saccadic action selection in the context of competitive decision making in the superior colliculus (SC) in order to investigate how independent neural signals may converge to interact and guide saccade selection, and to test if systematic variations can better replicate the variability in responses that are part of normal human behavior. The model was tasked with performing pro- and anti-saccades in order to replicate specific attributes of healthy human saccade behavior. Participants (ages 18-39) were instructed to either look toward (pro-saccade, well-practiced automated response) or away from (anti-saccade, combination of inhibitory and voluntary responses) a peripheral visual stimulus. They generated express and regular latency saccades in the pro-saccade task. In the anti-saccade task, correct reaction times were longer and participants occasionally looked at the stimulus (direction error) at either express or regular latencies. To gain a better understanding of the underlying neural processes that lead to saccadic action selection and response inhibition, we implemented 8 inputs inspired by systems neuroscience. These inputs reflected known sensory, automated, voluntary, and inhibitory components of cortical and basal ganglia activity that coalesces in the intermediate layers of the SC (SCi). The model produced bimodal reaction time distributions, where express and regular latency saccades had distinct modes, for both correct pro-saccades and direction errors in the anti-saccade task. Importantly, express and regular latency direction errors resulted from interactions of different inputs in the model. Express latency direction errors were due to a lack of pre-emptive fixation and inhibitory activity, which aloud sensory and automated inputs to initiate a stimulus-driven saccade. Regular latency errors occurred when the automated motor signals were stronger than the voluntary motor signals. While previous models have emulated fewer aspects of these behavioral findings, the focus of the simulations here is on the interaction of a wide variety of physiologically-based information integration producing a richer set of natural behavioral variability.

Luring the Motor System: Impact of Performance-Contingent Incentives on Pre-Movement Beta-Band Activity and Motor Performance

It has been shown that when incentives are provided during movement preparation, activity in parieto-frontal regions reflects both expected value and motivational salience. Yet behavioral work suggests that the processing of rewards is faster than for punishments, raising the possibility that expected value and motivational salience manifest at different latencies during movement planning. Given the role of beta oscillations (13-30 Hz) in movement preparation and in communication within the reward circuit, this study investigated how beta activity is modulated by positive and negative monetary incentives during reach planning, and in particular whether it reflects expected value and motivational salience at different latencies. Electroencephalography was recorded while male and female humans performed a reaching task in which reward or punishment delivery depended on movement accuracy. Before a preparatory delay period, participants were informed of the consequences of hitting or missing the target, according to four experimental conditions: Neutral (hit/miss:+0/-0¢), Reward (hit/miss:+5/-0¢), Punish (hit/miss:+0/-5¢) and Mixed (hit/miss:+5/-5¢). Results revealed that beta power over parieto-frontal regions was strongly modulated by incentives during the delay period, with power positively correlating with movement times. Interestingly, beta power was selectively sensitive to potential rewards early in the delay period, after which it came to reflect motivational salience as movement onset neared. These results demonstrate that beta activity reflects expected value and motivational salience on different time scales during reach planning. They also provide support for models that link beta activity with basal ganglia and dopamine for the allocation of neural resources according to behavioral salience.SIGNIFICANCE STATEMENT The present work demonstrates that pre-movement parieto-frontal beta power is modulated by monetary incentives in a goal-directed reaching task. Specifically, beta power transiently scaled with the availability of rewards early in movement planning, before reflecting motivational salience as movement onset neared. Moreover, pre-movement beta activity correlated with the vigor of the upcoming movement. These findings suggest that beta oscillations reflect neural processes that mediate the invigorating effect of incentives on motor performance, possibly through dopamine-mediated interactions with the basal ganglia.

Gravity modulates behaviour control strategy

Human behaviour is a trade-off between exploitation of familiar resources and exploration of new ones. In a challenging environment-such as outer space-making the correct decision is vital. On Earth, gravity is always there, and is an important reference for behaviour. Thus, altered gravitational signals may affect behaviour control strategies. Here, we investigated whether changing the body's orientation to the gravitational vector would modulate the balance between routine and novel behaviour. Participants completed a random number generation task while upright or supine. We found decreased randomness when participants were supine. In particular, the degree of equiprobability of pairs of consecutive responses was reduced in the supine orientation. Online gravitational signals may shape the balance between exploitation and exploration, in favour of more stereotyped and routine responses.

The neural instantiation of a priority map

The term priority map is commonly used to describe a map of the visual scene, in which objects and locations are represented by their attentional priority, which itself is a combination of low-level salience and top-down control. The aim of this review is to examine how such a map may be represented at the neuronal level. We propose that there is not a single, common map in the brain, but that a number of cortical areas work together to generate the resultant behavior. Specifically, we suggest that the lateral intraparietal area (LIP) of posterior parietal cortex provides a simple representation of attentional priority, which remaps across saccades, so that there is an apparent allocentric map in a region with retinocentric encoding scheme. We propose that the frontal eye field (FEF) of prefrontal cortex receives the responses from LIP, but can suppress them to control the flow of eye movement behavior, and that the intermediate layers of the superior colliculus (SCi) reflect the final saccade goal. Together, these areas function to guide eye movements and may play a similar role in allocating covert visual attention.

Predictive coding of the statistical parameters of uncertain rewards by orbitofrontal neurons

Uncertain reward outcomes are characterised by statistical parameters that capture the numerical values of the underlying probability distributions of reward values, including the expected value, risk (variance) and probability. Here we show coding of an integrated expected value signal by single orbitofrontal neurons in response to visual cues predicting uncertain rewards. Separate subpopulations of orbitofrontal neurons predominantly code the prediction of one statistical parameter with few neurons showing combined coding. These signals are likely combined with subjective value signals to inform learning and decision making under conditions of uncertainty.

Decision and navigation in mouse parietal cortex

Posterior parietal cortex (PPC) has been implicated in navigation, in the control of movement, and in visually-guided decisions. To relate these views, we measured activity in PPC while mice performed a virtual navigation task driven by visual decisions. PPC neurons were selective for specific combinations of the animal's spatial position and heading angle. This selectivity closely predicted both the activity of individual PPC neurons, and the arrangement of their collective firing patterns in choice-selective sequences. These sequences reflected PPC encoding of the animal's navigation trajectory. Using decision as a predictor instead of heading yielded worse fits, and using it in addition to heading only slightly improved the fits. Alternative models based on visual or motor variables were inferior. We conclude that when mice use vision to choose their trajectories, a large fraction of parietal cortex activity can be predicted from simple attributes such as spatial position and heading.

Dissociating Reward- and Attention-driven Biasing of Global Feature-based Selection in Human Visual Cortex

Objects that promise rewards are prioritized for visual selection. The way this prioritization shapes sensory processing in visual cortex, however, is debated. It has been suggested that rewards motivate stronger attentional focusing, resulting in a modulation of sensory selection in early visual cortex. An open question is whether those reward-driven modulations would be independent of similar modulations indexing the selection of attended features that are not associated with reward. Here, we use magnetoencephalography in human observers to investigate whether the modulations indexing global color-based selection in visual cortex are separable for target- and (monetary) reward-defining colors. To assess the underlying global color-based activity modulation, we compare the event-related magnetic field response elicited by a color probe in the unattended hemifield drawn either in the target color, the reward color, both colors, or a neutral task-irrelevant color. To test whether target and reward relevance trigger separable modulations, we manipulate attention demands on target selection while keeping reward-defining experimental parameters constant. Replicating previous observations, we find that reward and target relevance produce almost indistinguishable gain modulations in ventral extratriate cortex contralateral to the unattended color probe. Importantly, increasing attention demands on target discrimination increases the response to the target-defining color, whereas the response to the rewarded color remains largely unchanged. These observations indicate that, although task relevance and reward influence the very same feature-selective area in extrastriate visual cortex, the associated modulations are largely independent.

Adaptive Gaze Strategies to Reduce Environmental Uncertainty During a Sequential Visuomotor Behaviour

People must decide where, when, and for how long to allocate gaze to perform different motor behaviours. However, the factors guiding gaze during these ongoing, natural behaviours are poorly understood. Gaze shifts help acquire information, suggesting that people should direct gaze to locations where environmental details most relevant to the task are uncertain. To explore this, human subjects stepped on a series of targets as they walked. We used different levels of target uncertainty, and through instruction, altered the importance of (or subjective value assigned to) foot-placement accuracy. Gaze time on targets increased with greater target uncertainty when precise foot placement was more important, and these longer gaze times associated with reduced foot-placement error. Gaze times as well as the gaze shifts to and from targets relative to stepping differed depending on the target's position in the sequence and uncertainty level. Overall, we show that gaze is allocated to reduce uncertainty about target locations, and this depends on the value of this information gain for successful task performance. Furthermore, we show that the spatial-temporal pattern of gaze to resolve uncertainty changes with the evolution of the motor behaviour, indicating a flexible strategy to plan and control movement.

Bayesian updating during development predicts genotypic differences in plasticity

Interactions between genotypes and environments are central to evolutionary genetics, but such interactions are typically described, rather than predicted from theory. Recent Bayesian models of development generate specific predictions about genotypic differences in developmental plasticity (changes in the value of a given trait as a result of a given experience) based on genotypic differences in the value of the trait that is expressed by naïve subjects. We used these models to make a priori predictions about the effects of an aversive olfactory conditioning regime on the response of Drosophila melanogaster larvae to the odor of ethyl acetate. As predicted, across 116 genotypes initial trait values were related to plasticity. Genotypes most strongly attracted to the odor of ethyl acetate when naïve reduced their attraction scores more as a result of the aversive training regime than those less attracted to the same odor when naïve. Thus, as predicted, the variance across genotypes in attraction scores was higher before than after the shared experience. These results support predictions generated by Bayesian models of development and indicate that such models can be successfully used to investigate how variation across genotypes in information derived from ancestors combines with personal experience to differentially affect developmental plasticity in response to specific types of experience.

Surprise About Sensory Event Timing Drives Cortical Transients in the Beta Frequency Band

Learning the statistical structure of the environment is crucial for adaptive behavior. Humans and nonhuman decision-makers seem to track such structure through a process of probabilistic inference, which enables predictions about behaviorally relevant events. Deviations from such predictions cause surprise, which in turn helps improve inference. Surprise about the timing of behaviorally relevant sensory events drives phasic responses of neuromodulatory brainstem systems, which project to the cerebral cortex. Here, we developed a computational model-based magnetoencephalography (MEG) approach for mapping the resulting cortical transients across space, time, and frequency, in the human brain (N = 28, 17 female). We used a Bayesian ideal observer model to learn the statistics of the timing of changes in a simple visual detection task. This model yielded quantitative trial-by-trial estimates of temporal surprise. The model-based surprise variable predicted trial-by-trial variations in reaction time more strongly than the externally observable interval timings alone. Trial-by-trial variations in surprise were negatively correlated with the power of cortical population activity measured with MEG. This surprise-related power suppression occurred transiently around the behavioral response, specifically in the beta frequency band. It peaked in parietal and prefrontal cortices, remote from the motor cortical suppression of beta power related to overt report (button press) of change detection. Our results indicate that surprise about sensory event timing transiently suppresses ongoing beta-band oscillations in association cortex. This transient suppression of frontal beta-band oscillations might reflect an active reset triggered by surprise, and is in line with the idea that beta-oscillations help maintain cognitive sets.SIGNIFICANCE STATEMENT The brain continuously tracks the statistical structure of the environment to anticipate behaviorally relevant events. Deviations from such predictions cause surprise, which in turn drives neural activity in subcortical brain regions that project to the cerebral cortex. We used magnetoencephalography in humans to map out surprise-related modulations of cortical population activity across space, time, and frequency. Surprise was elicited by variable timing of visual stimulus changes requiring a behavioral response. Surprise was quantified by means of an ideal observer model. Surprise predicted behavior as well as a transient suppression of beta frequency-band oscillations in frontal cortical regions. Our results are in line with conceptual accounts that have linked neural oscillations in the beta-band to the maintenance of cognitive sets.

Comparison of Decision-Related Signals in Sensory and Motor Preparatory Responses of Neurons in Area LIP

Neurons in the lateral intraparietal (LIP) area of Macaques exhibit both sensory and oculomotor preparatory responses. During perceptual decision making, the preparatory responses have been shown to track the state of the evolving evidence leading to the decision. The sensory responses are known to reflect categorical properties of visual stimuli, but it is not known whether these responses also track evolving evidence. We recorded neural responses from lateral intraparietal area of 2 female rhesus monkeys during a direction discrimination task. We compared sensory and oculomotor-preparatory responses in the same neurons when either the discriminandum (random dot motion) or an eye movement choice-target was in the neuron's response field. The neural responses in both configurations reflected the strength and direction of motion and were correlated with the animal's choice, albeit more prominently when the choice-target was in the response field. However, the variance and autocorrelation pattern of only the motor preparatory responses reflected the process of evidence accumulation. Simulations suggest that the task related activity of sensory responses could be inherited through lateral interactions with neurons that are carrying evidence accumulation signals in their motor-preparatory responses. The results are consistent with the proposal that evolving decision processes are supported by persistent neural activity in the service of actions or intentions, as opposed to high-order representations of stimulus properties.SIGNIFICANCE STATEMENT Perceptual decision making is the process of choosing an appropriate motor action based on perceived sensory information. Association areas of the cortex play an important role in this sensory-motor transformation. The neurons in these areas show both sensory- and motor-related activity. We show here that, in the macaque parietal association area LIP, signatures of the process of evidence accumulation that underlies the decisions are predominantly reflected in the motor-related activity. This finding supports the proposal that perceptual decision making is implemented in the brain as a process of choosing between available motor actions rather than as a process of representing the properties of the sensory stimulus.

Applying the matching law as micro-foundation of social phenomena

Social phenomena are suggested to be explained by the matching law - an empirical regularity of individual behaviour. While a considerable amount of psychological research on this law exists, only a few sociological applications can be found. This paper points to the problems that come with its usage as micro-foundation of social behaviour and provides solutions. In particular, a model of melioration learning enables the derivation of social phenomena from the matching law. The proposed approach is illustrated by the application of the learning model to the volunteer's dilemma. In contrast to game-theoretical solutions, the matching law leads to more intuitive results in case of the asymmetric dilemma. The relationship between the matching law and utility maximisation is discussed by its integration into economic consumer theory.

An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals

Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.

Serotonin (5-HT) plays many important roles in reward, punishment, patience and beyond, and optogenetic stimulation of 5-HT neurons has not crisply parsed them. The authors report a novel analysis of a reward-based decision-making experiment, and show that 5-HT stimulation increases the learning rate, but only on a select subset of choices.

Lateral intraparietal area (LIP) is largely effector-specific in free-choice decisions

Despite many years of intense research, there is no strong consensus about the role of the lateral intraparietal area (LIP) in decision making. One view of LIP function is that it guides spatial attention, providing a “saliency map” of the external world. If this were the case, it would contribute to target selection regardless of which action would be performed to implement the choice. On the other hand, LIP inactivation has been shown to influence spatial selection and oculomotor metrics in free-choice decisions, which are made using eye movements, arguing that it contributes to saccade decisions. To dissociate between a more general attention role and a more effector specific saccade role, we reversibly inactivated LIP while non-human primates freely selected between two targets, presented in the two hemifields, with either saccades or reaches. Unilateral LIP inactivation induced a strong choice bias to ipsilesional targets when decisions were made with saccades. Interestingly, the inactivation also caused a reduction of contralesional choices when decisions were made with reaches, albeit the effect was less pronounced. These findings suggest that LIP is part of a network for making oculomotor decisions and is largely effector-specific in free-choice decisions.

Primary Tactile Thalamus Spiking Reflects Cognitive Signals

Little is known about whether information transfer at primary sensory thalamic nuclei is modified by behavioral context. Here we studied the influence of previous decisions/rewards on current choices and preceding spike responses of ventroposterior medial thalamus (VPm; the primary sensory thalamus in the rat whisker-related tactile system). We trained head-fixed rats to detect a ramp-like deflection of one whisker interspersed within ongoing white noise stimulation. Using generative modeling of behavior, we identify two task-related variables that are predictive of actual decisions. The first reflects task engagement on a local scale ("trial history": defined as the decisions and outcomes of a small number of past trials), whereas the other captures behavioral dynamics on a global scale ("satiation": slow dynamics of the response pattern along an entire session). Although satiation brought about a slow drift from Go to NoGo decisions during the session, trial history was related to local (trial-by-trial) patterning of Go and NoGo decisions. A second model that related the same predictors first to VPm spike responses, and from there to decisions, indicated that spiking, in contrast to behavior, is sensitive to trial history but relatively insensitive to satiation. Trial history influences VPm spike rates and regularity such that a history of Go decisions would predict fewer noise-driven spikes (but more regular ones), and more ramp-driven spikes. Neuronal activity in VPm, thus, is sensitive to local behavioral history, and may play an important role in higher-order cognitive signaling.SIGNIFICANCE STATEMENT It is an important question for perceptual and brain functions to find out whether cognitive signals modulate the sensory signal stream and if so, where in the brain this happens. This study provides evidence that decision and reward history can already be reflected in the ascending sensory pathway, on the level of first-order sensory thalamus. Cognitive signals are relayed very selectively such that only local trial history (spanning a few trials) but not global history (spanning an entire session) are reflected.

Spatial eye-hand coordination during bimanual reaching is not systematically coded in either LIP or PRR

When we reach for something, we also look at it. If we reach for two objects at once, one with each hand, we look first at one and then the other. It is not known which brain areas underlie this coordination. We studied two parietal areas known to be involved in eye and arm movements. Neither area was sensitive to the order in which the targets were looked at. This implies that coordinated saccades are driven by downstream areas and not by the parietal cortex as is commonly assumed.

We often orient to where we are about to reach. Spatial and temporal correlations in eye and arm movements may depend on the posterior parietal cortex (PPC). Spatial representations of saccade and reach goals preferentially activate cells in the lateral intraparietal area (LIP) and the parietal reach region (PRR), respectively. With unimanual reaches, eye and arm movement patterns are highly stereotyped. This makes it difficult to study the neural circuits involved in coordination. Here, we employ bimanual reaching to two different targets. Animals naturally make a saccade first to one target and then the other, resulting in different patterns of limb–gaze coordination on different trials. Remarkably, neither LIP nor PRR cells code which target the eyes will move to first. These results suggest that the parietal cortex plays at best only a permissive role in some aspects of eye–hand coordination and makes the role of LIP in saccade generation unclear.

Selection history: How reward modulates selectivity of visual attention

Visual attention enables us to selectively prioritize or suppress information in the environment. Prominent models concerned with the control of visual attention differentiate between goal-directed, top-down and stimulus-driven, bottom-up control, with the former determined by current selection goals and the latter determined by physical salience. In the current review, we discuss recent studies that demonstrate that attentional selection does not need to be the result of top-down or bottom-up processing but, instead, is often driven by lingering biases due to the "history" of former attention deployments. This review mainly focuses on reward-based history effects; yet other types of history effects such as (intertrial) priming, statistical learning and affective conditioning are also discussed. We argue that evidence from behavioral, eye-movement and neuroimaging studies supports the idea that selection history modulates the topographical landscape of spatial "priority" maps, such that attention is biased toward locations having the highest activation on this map.

Extinction as a deficit of the decision-making circuitry in the posterior parietal cortex

Extinction is a common neurologic deficit that often occurs as one of a constellation of symptoms seen with lesions of the posterior parietal cortex (PPC). Although extinction has typically been considered a deficit in the allocation of attention, new findings, particularly from nonhuman primate studies, point to one potential and important source of extinction as damage to decision-making circuits for actions within the PPC. This new understanding provides clues to potential therapies for extinction. Also the finding that the PPC is important for action decisions and action planning has led to new neuroprosthetic applications using PPC recordings as control signals to assist paralyzed patients.

Accurate step-hold tracking of smoothly varying periodic and aperiodic probability

Subjects observing many samples from a Bernoulli distribution are able to perceive an estimate of the generating parameter. A question of fundamental importance is how the current percept-what we think the probability now is-depends on the sequence of observed samples. Answers to this question are strongly constrained by the manner in which the current percept changes in response to changes in the hidden parameter. Subjects do not update their percept trial-by-trial when the hidden probability undergoes unpredictable and unsignaled step changes; instead, they update it only intermittently in a step-hold pattern. It could be that the step-hold pattern is not essential to the perception of probability and is only an artifact of step changes in the hidden parameter. However, we now report that the step-hold pattern obtains even when the parameter varies slowly and smoothly. It obtains even when the smooth variation is periodic (sinusoidal) and perceived as such. We elaborate on a previously published theory that accounts for: (i) the quantitative properties of the step-hold update pattern; (ii) subjects' quick and accurate reporting of changes; (iii) subjects' second thoughts about previously reported changes; (iv) subjects' detection of higher-order structure in patterns of change. We also call attention to the challenges these results pose for trial-by-trial updating theories.

Control of saccadic latency in a dynamic environment: allocation of saccades in time follows the matching law

When exploring the visual environment, one uses saccades to shift gaze and fixation to gather spatially and temporally localized information. We propose that the temporal structure of our environment should constrain the temporal allocation of saccades. Here we probe the possibility of learning to control saccadic latencies in a choice paradigm. Six participants made saccades within 80–300 ms following a target horizontally stepping by 10° between two fixed locations. For each participant we constructed two classes of latencies, “short” and “long,” using the first and last quartiles of the individual baseline distribution (e.g., [80;152] ms and [185;300] ms, respectively). We then concurrently reinforced each class in three blocked conditions across ~60 experimental sessions per participant, using different reinforcement probabilities such that the relative ratio of reinforcement rates for short vs. long latencies was 9/1, 1/9, or 1/1. Latency distributions followed the reinforcement conditions: distributions shifted toward the shorter or longer values or became strongly bimodal. Moreover, the relative rates of short over long latencies matched the relative rates of reinforcers earned for the corresponding latencies (slope up to 0.95), which reveals the ability to choose when to saccade. Our results reveal that learned contingencies considerably affect the allocation of saccades in time and are in line with recent studies on the temporal adjustment of behavior to dynamic environments. This study provides strong evidence for fine operant control of saccadic latency, supporting the hypothesis of a cost-benefit control of saccade latencies.

NEW & NOTEWORTHY Saccades may be regarded as an information-foraging behavior mostly concerned with the spatial localization of objects, yet our world is dynamic and environmental temporal regularities should also affect saccade decisions. We present behavioral data from a choice task establishing that humans can learn to choose their saccadic latencies depending on the reinforcement contingencies. This suggests a cost-benefit-based policy that takes into account the learned temporal properties of the environmental contingencies for controlling saccade triggering.

Adaptive History Biases Result from Confidence-Weighted Accumulation of past Choices

Perceptual decision-making is biased by previous events, including the history of preceding choices: observers tend to repeat (or alternate) their judgments of the sensory environment more often than expected by chance. Computational models postulate that these so-called choice history biases result from the accumulation of internal decision signals across trials. Here, we provide psychophysical evidence for such a mechanism and its adaptive utility. Male and female human observers performed different variants of a challenging visual motion discrimination task near psychophysical threshold. In a first experiment, we decoupled categorical perceptual choices and motor responses on a trial-by-trial basis. Choice history bias was explained by previous perceptual choices, not motor responses, highlighting the importance of internal decision signals in action-independent formats. In a second experiment, observers performed the task in stimulus environments containing different levels of autocorrelation and providing no external feedback about choice correctness. Despite performing under overall high levels of uncertainty, observers adjusted both the strength and the sign of their choice history biases to these environments. When stimulus sequences were dominated by either repetitions or alternations, the individual degree of this adjustment of history bias was about as good a predictor of individual performance as individual perceptual sensitivity. The history bias adjustment scaled with two proxies for observers' confidence about their previous choices (accuracy and reaction time). Together, our results are consistent with the idea that action-independent, confidence-modulated decision variables are accumulated across choices in a flexible manner that depends on decision-makers' model of their environment.

SIGNIFICANCE STATEMENT Decisions based on sensory input are often influenced by the history of one's preceding choices, manifesting as a bias to systematically repeat (or alternate) choices. We here provide support for the idea that such choice history biases arise from the context-dependent accumulation of a quantity referred to as the decision variable: the variable's sign dictates the choice and its magnitude the confidence about choice correctness. We show that choices are accumulated in an action-independent format and a context-dependent manner, weighted by the confidence about their correctness. This confidence-weighted accumulation of choices enables decision-makers to flexibly adjust their behavior to different sensory environments. The bias adjustment can be as important for optimizing performance as one's sensitivity to the momentary sensory input.

Free choice shapes normalized value signals in medial orbitofrontal cortex

Normalization is a common cortical computation widely observed in sensory perception, but its importance in perception of reward value and decision making remains largely unknown. We examined (1) whether normalized value signals occur in the orbitofrontal cortex (OFC) and (2) whether changes in behavioral task context influence the normalized representation of value. We record medial OFC (mOFC) single neuron activity in awake-behaving monkeys during a reward-guided lottery task. mOFC neurons signal the relative values of options via a divisive normalization function when animals freely choose between alternatives. The normalization model, however, performed poorly in a variant of the task where only one of the two possible choice options yields a reward and the other was certain not to yield a reward (so called: “forced choice”). The existence of such context-specific value normalization may suggest that the mOFC contributes valuation signals critical for economic decision making when meaningful alternative options are available.

Neurons in prefrontal areas including the medial orbitofrontal cortex (mOFC) represent the relative reward value of choices. Here the authors report that mOFC neurons implement divisive normalization to encode the relative values of lottery options only when the decision involves free choice.

Feeder Approach between Trials Is Increased by Uncertainty and Affects Subsequent Choices

Animals quickly learn to approach sources of food. Here, we report on a form of approach in which rats made volitional orofacial contact with inactive feeders between trials of a self-paced operant task. This extraneous feeder sampling (EFS) was never reinforced and therefore imposed an opportunity and effort cost. EFS decreased during initial training but persisted thereafter. The relative rate of EFS to operant responding increased with novel changes to the operant chamber, reward devaluation by prefeeding, or lesions to the dorsolateral striatum. We speculate that this may function to increase exploration when the task is uncertain (early in learning or introduction of novel apparatus components), when the opportunity cost is low, or when the learned sensorimotor solution is compromised. Moreover, EFS strongly affected subsequent choices by triggering a lose-shift response away from the sampled feeder, even though it occurred outside of the trial context. This indicates that at least some behaviors occurring between trials impact future behaviors and should be considered in decision-making studies.

Reward Size Informs Repeat-Switch Decisions and Strongly Modulates the Activity of Neurons in Parietal Cortex

Behavior is guided by previous experience. Good, positive outcomes drive a repetition of a previous behavior or choice, whereas poor or bad outcomes lead to an avoidance. How these basic drives are implemented by the brain has been of primary interest to psychology and neuroscience. We engaged animals in a choice task in which the size of a reward outcome strongly governed the animals' subsequent decision whether to repeat or switch the previous choice. We recorded the discharge activity of neurons implicated in reward-based choice in 2 regions of parietal cortex. We found that the tendency to retain previous choice following a large (small) reward was paralleled by a marked decrease (increase) in the activity of parietal neurons. This neural effect is independent of, and of sign opposite to, value-based modulations reported in parietal cortex previously. This effect shares the same basic properties with signals previously reported in the limbic system that detect the size of the recently obtained reward to mediate proper repeat-switch decisions. We conclude that the size of the obtained reward is a decision variable that guides the decision between retaining a choice or switching, and neurons in parietal cortex strongly respond to this novel decision variable.

Exploration Disrupts Choice-Predictive Signals and Alters Dynamics in Prefrontal Cortex

In uncertain environments, decision-makers must balance two goals: they must "exploit" rewarding options but also "explore" in order to discover rewarding alternatives. Exploring and exploiting necessarily change how the brain responds to identical stimuli, but little is known about how these states, and transitions between them, change how the brain transforms sensory information into action. To address this question, we recorded neural activity in a prefrontal sensorimotor area while monkeys naturally switched between exploring and exploiting rewarding options. We found that exploration profoundly reduced spatially selective, choice-predictive activity in single neurons and delayed choice-predictive population dynamics. At the same time, reward learning was increased in brain and behavior. These results indicate that exploration is related to sudden disruptions in prefrontal sensorimotor control and rapid, reward-dependent reorganization of control dynamics. This may facilitate discovery through trial and error.

The determinants of response time in a repeated constant-sum game: A robust Bayesian hierarchical dual-process model

The investigation of response time and behavior has a long tradition in cognitive psychology, particularly for non-strategic decision-making. Recently, experimental economists have also studied response time in strategic interactions, but with an emphasis on either one-shot games or repeated social-dilemmas. I investigate the determinants of response time in a repeated (pure-conflict) game, admitting a unique mixed strategy Nash equilibrium, with fixed partner matching. Response times depend upon the interaction of two decision models embedded in a dual-process framework (Achtziger and Alós-Ferrer, 2014; Alós-Ferrer, 2016). The first decision model is the commonly used win-stay/lose-shift heuristic and the second the pattern-detecting reinforcement learning model in Spiliopoulos (2013b). The former is less complex and can be executed more quickly than the latter. As predicted, conflict between these two models (i.e., each one recommending a different course of action) led to longer response times than cases without conflict. The dual-process framework makes other qualitative response time predictions arising from the interaction between the existence (or not) of conflict and which one of the two decision models the chosen action is consistent with-these were broadly verified by the data. Other determinants of RT were hypothesized on the basis of existing theory and tested empirically. Response times were strongly dependent on the actions chosen by both players in the previous rounds and the resulting outcomes. Specifically, response time was shortest after a win in the previous round where the maximum possible payoff was obtained; response time after losses was significantly longer. Strongly auto-correlated behavior (regardless of its sign) was also associated with longer response times. I conclude that, similar to other tasks, there is a strong coupling in repeated games between behavior and RT, which can be exploited to further our understanding of decision making.

The real deal: Willingness-to-pay and satiety expectations are greater for real foods versus their images

Laboratory studies of human dietary choice have relied on computerized two-dimensional (2D) images as stimuli, whereas in everyday life, consumers make decisions in the context of real foods that have actual caloric content and afford grasping and consumption. Surprisingly, few studies have compared whether real foods are valued more than 2D images of foods, and in the studies that have, differences in the stimuli and testing conditions could have resulted in inflated bids for the real foods. Moreover, although the caloric content of food images has been shown to influence valuation, no studies to date have investigated whether 'real food exposure effects' on valuation reflect greater sensitivity to the caloric content of real foods versus images. Here, we compared willingness-to-pay (WTP) for, and expectations about satiety after consuming, everyday snack foods that were displayed as real foods versus 2D images. Critically, our 2D images were matched closely to the real foods for size, background, illumination, and apparent distance, and trial presentation and stimulus timing were identical across conditions. We used linear mixed effects modeling to determine whether effects of display format were modulated by food preference and the caloric content of the foods. Compared to food images, observers were willing to pay 6.62% more for (Experiment 1) and believed that they would feel more satiated after consuming (Experiment 2), foods displayed as real objects. Moreover, these effects appeared to be consistent across food preference, caloric content, as well as observers' estimates of the caloric content of the foods. Together, our results confirm that consumers' perception and valuation of everyday foods is influenced by the format in which they are displayed. Our findings raise important new insights into the factors that shape dietary choice in real-world contexts and highlight potential avenues for improving public health approaches to diet and obesity.

Neural Signature of Value-Based Sensorimotor Prioritization in Humans

In situations in which impending sensory events demand fast action choices, we must be ready to prioritize higher-value courses of action to avoid missed opportunities. When such a situation first presents itself, stimulus-action contingencies and their relative value must be encoded to establish a value-biased state of preparation for an impending sensorimotor decision. Here, we sought to identify neurophysiological signatures of such processes in the human brain (both female and male). We devised a task requiring fast action choices based on the discrimination of a simple visual cue in which the differently valued sensory alternatives were presented 750-800 ms before as peripheral "targets" that specified the stimulus-action mapping for the upcoming decision. In response to the targets, we identified a discrete, transient, spatially selective signal in the event-related potential (ERP), which scaled with relative value and strongly predicted the degree of behavioral bias in the upcoming decision both across and within subjects. This signal is not compatible with any hitherto known ERP signature of spatial selection and also bears novel distinctions with respect to characterizations of value-sensitive, spatially selective activity found in sensorimotor areas of nonhuman primates. Specifically, a series of follow-up experiments revealed that the signal was reliably invoked regardless of response laterality, response modality, sensory feature, and reward valence. It was absent, however, when the response deadline was relaxed and the strategic need for biasing removed. Therefore, more than passively representing value or salience, the signal appears to play a versatile and active role in adaptive sensorimotor prioritization.SIGNIFICANCE STATEMENT In many situations such as fast-moving sports, we must be ready to act fast in response to sensory events and, in our preparation, prioritize courses of action that lead to greater rewards. Although behavioral effects of value biases in sensorimotor decision making have been widely studied, little is known about the neural processes that set these biases in place beforehand. Here, we report the discovery of a transient, spatially selective neural signal in humans that encodes the relative value of competing decision alternatives and strongly predicts behavioral value biases in decisions made ∼500 ms later. Follow-up manipulations of value differential, reward valence, response modality, sensory features, and time constraints establish that the signal reflects an active, feature- and effector-general preparatory mechanism for value-based prioritization.

History-based action selection bias in posterior parietal cortex

Making decisions based on choice-outcome history is a crucial, adaptive ability in life. However, the neural circuit mechanisms underlying history-dependent decision-making are poorly understood. In particular, history-related signals have been found in many brain areas during various decision-making tasks, but the causal involvement of these signals in guiding behavior is unclear. Here we addressed this issue utilizing behavioral modeling, two-photon calcium imaging, and optogenetic inactivation in mice. We report that a subset of neurons in the posterior parietal cortex (PPC) closely reflect the choice-outcome history and history-dependent decision biases, and PPC inactivation diminishes the history dependency of choice. Specifically, many PPC neurons show history- and bias-tuning during the inter-trial intervals (ITI), and history dependency of choice is affected by PPC inactivation during ITI and not during trial. These results indicate that PPC is a critical region mediating the subjective use of history in biasing action selection.

Working memory accuracy for multiple targets is driven by reward expectation and stimulus contrast with different time-courses

The richness of sensory input dictates that the brain must prioritize and select information for further processing and storage in working memory. Stimulus salience and reward expectations influence this prioritization but their relative contributions and underlying mechanisms are poorly understood. Here we investigate how the quality of working memory for multiple stimuli is determined by priority during encoding and later memory phases. Selective attention could, for instance, act as the primary gating mechanism when stimuli are still visible. Alternatively, observers might still be able to shift priorities across memories during maintenance or retrieval. To distinguish between these possibilities, we investigated how and when reward cues determine working memory accuracy and found that they were only effective during memory encoding. Previously learned, but currently non-predictive, color-reward associations had a similar influence, which gradually weakened without reinforcement. Finally, we show that bottom-up salience, manipulated through varying stimulus contrast, influences memory accuracy during encoding with a fundamentally different time-course than top-down reward cues. While reward-based effects required long stimulus presentation, the influence of contrast was strongest with brief presentations. Our results demonstrate how memory resources are distributed over memory targets and implicates selective attention as a main gating mechanism between sensory and memory systems.

Ventromedial Prefrontal Cortex Damage Is Associated with Decreased Ventral Striatum Volume and Response to Reward

The ventral striatum and ventromedial prefrontal cortex (vmPFC) are two central nodes of the "reward circuit" of the brain. Human neuroimaging studies have demonstrated coincident activation and functional connectivity between these brain regions, and animal studies have demonstrated that the vmPFC modulates ventral striatum activity. However, there have been no comparable data in humans to address whether the vmPFC may be critical for the reward-related response properties of the ventral striatum. In this study, we used fMRI in five neurosurgical patients with focal vmPFC lesions to test the hypothesis that the vmPFC is necessary for enhancing ventral striatum responses to the anticipation of reward. In support of this hypothesis, we found that, compared with age- and gender-matched neurologically healthy subjects, the vmPFC-lesioned patients had reduced ventral striatal activity during the anticipation of reward. Furthermore, we observed that the vmPFC-lesioned patients had decreased volumes of the accumbens subregion of the ventral striatum. Together, these functional and structural neuroimaging data provide novel evidence for a critical role for the vmPFC in contributing to reward-related activity of the ventral striatum. These results offer new insight into the functional and structural interactions between key components of the brain circuitry underlying human affective function and decision-making.

SIGNIFICANCE STATEMENT

Maladaptive decision-making is a common problem across multiple mental health disorders. Developing new pathophysiologically based strategies for diagnosis and treatment thus requires a better understanding of the brain circuits responsible for adaptive decision-making and related psychological subprocesses (e.g., reward valuation, anticipation, and motivation). Animal studies provide evidence that these functions are mediated through direct interactions between two key nodes of a posited "reward circuit," the ventral striatum and the ventromedial prefrontal cortex (vmPFC). For the first time in humans, we demonstrate that damage to the vmPFC results in decreased ventral striatum activity during reward anticipation. These data provide unique evidence on the causal mechanisms by which the vmPFC and ventral striatum interact during the anticipation of rewards.

Object comparison in the lateral intraparietal area

We can search for and locate specific objects in our environment by looking for objects with similar features. Object recognition involves stimulus similarity responses in ventral visual areas and task-related responses in prefrontal cortex. We tested whether neurons in the lateral intraparietal area (LIP) of posterior parietal cortex could form an intermediary representation, collating information from object-specific similarity map representations to allow general decisions about whether a stimulus matches the object being looked for. We hypothesized that responses to stimuli would correlate with how similar they are to a sample stimulus. When animals compared two peripheral stimuli to a sample at their fovea, the response to the matching stimulus was similar, independent of the sample identity, but the response to the nonmatch depended on how similar it was to the sample: the more similar, the greater the response to the nonmatch stimulus. These results could not be explained by task difficulty or confidence. We propose that LIP uses its known mechanistic properties to integrate incoming visual information, including that from the ventral stream about object identity, to create a dynamic representation that is concise, low dimensional, and task relevant and that signifies the choice priorities in mental matching behavior.NEW & NOTEWORTHY Studies in object recognition have focused on the ventral stream, in which neurons respond as a function of how similar a stimulus is to their preferred stimulus, and on prefrontal cortex, where neurons indicate which stimulus is being looked for. We found that parietal area LIP uses its known mechanistic properties to form an intermediary representation in this process. This creates a perceptual similarity map that can be used to guide decisions in prefrontal areas.

Distinct Roles for the Amygdala and Orbitofrontal Cortex in Representing the Relative Amount of Expected Reward

The same reward can possess different motivational meaning depending upon its magnitude relative to other rewards. To study the neurophysiological mechanisms mediating assignment of motivational meaning, we recorded the activity of neurons in the amygdala and orbitofrontal cortex (OFC) of monkeys during a Pavlovian task in which the relative amount of liquid reward associated with one conditioned stimulus (CS) was manipulated by changing the reward amount associated with a second CS. Anticipatory licking tracked relative reward magnitude, implying that monkeys integrated information about recent rewards to adjust the motivational meaning of a CS. Upon changes in relative reward magnitude, neural responses to reward-predictive cues updated more rapidly in OFC than amygdala, and activity in OFC but not the amygdala was modulated by recent reward history. These results highlight a distinction between the amygdala and OFC in assessing reward history to support the flexible assignment of motivational meaning to sensory cues.

Prefrontal Contribution to Decision-Making under Free-Choice Conditions

Executive function is thought to be the coordinated operation of multiple neural processes and allows to accomplish a current goal flexibly. The most important function of the prefrontal cortex is the executive function. Among a variety of executive functions in which the prefrontal cortex participates, decision-making is one of the most important. Although the prefrontal contribution to decision-making has been examined using a variety of behavioral tasks, recent studies using fMRI have shown that the prefrontal cortex participates in decision-making under free-choice conditions. Since decision-making under free-choice conditions represents the very first stage for any kind of decision-making process, it is important that we understand its neural mechanism. Although few studies have examined this issue while a monkey performed a free-choice task, those studies showed that, when the monkey made a decision to subsequently choose one particular option, prefrontal neurons showing selectivity to that option exhibited transient activation just before presentation of the imperative cue. Further studies have suggested that this transient increase is caused by the irregular fluctuation of spontaneous firing just before cue presentation, which enhances the response to the cue and biases the strength of the neuron's selectivity to the option. In addition, this biasing effect was observed only in neurons that exhibited sustained delay-period activity, indicating that this biasing effect not only influences the animal's decision for an upcoming choice, but also is linked to working memory mechanisms in the prefrontal cortex.