Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain

Is the enhancement of memory due to reward driven by value or salience?

Past research using two levels of reward has shown that the higher-value items are remembered better than lower-value items and this enhancement is assumed to be driven by an effect of reward value. In the present study, multiple levels of reward were used to test the influence of reward salience on memory. Using a value-learning procedure, words were associated with reward values, and then memory for these words was later tested with free recall. Critically, multiple reward levels were used, allowing us to test two specific hypotheses whereby rewards can influence memory: (a) higher value items are remembered better than lower value items (reward value hypothesis), and (b) highest and lowest value items are remembered best and intermediate-value items are remembered worst (following a U-shaped relationship between value and memory; reward salience hypothesis). In two experiments we observed a U-shaped relationship between reward value and memory, supporting the notion that memory is enhanced due to reward salience, and not purely through reward value.

Dissociable reward and timing signals in human midbrain and ventral striatum

Reward prediction error (RPE) signals are central to current models of reward-learning. Temporal difference (TD) learning models posit that these signals should be modulated by predictions, not only of magnitude but also timing of reward. Here we show that BOLD activity in the VTA conforms to such TD predictions: responses to unexpected rewards are modulated by a temporal hazard function and activity between a predictive stimulus and reward is depressed in proportion to predicted reward. By contrast, BOLD activity in ventral striatum (VS) does not reflect a TD RPE, but instead encodes a signal on the variable relevant for behavior, here timing but not magnitude of reward. The results have important implications for dopaminergic models of cortico-striatal learning and suggest a modification of the conventional view that VS BOLD necessarily reflects inputs from dopaminergic VTA neurons signaling an RPE.

Considering anger from a cognitive neuroscience perspective

The goal of this paper is to consider anger from a cognitive neuroscience perspective. Five main claims are made: first, reactive aggression is the ultimate behavioral expression of anger and thus we can begin to understand anger by understanding reactive aggression. Second, neural systems implicated in reactive aggression (amygdala, hypothalamus, and periaqueductal gray; the basic threat system) are critically implicated in anger. Factors such as exposure to extreme threat that increase the responsiveness of these systems, should be (and are in the context of posttraumatic stress disorder), associated with increased anger. Third, regions of frontal cortex implicated in regulating the basic threat system, when dysfunctional (e.g., in the context of lesions) should be associated with increased anger. Fourth, frustration occurs when an individual continues to do an action in the expectation of a reward but does not actually receive that reward, and is associated with anger. Individuals who show impairment in the ability to alter behavioral responding when actions no longer receive their expected rewards should be (and are in the context of psychopathy) associated with increased anger. Fifth, someone not doing what another person wants them to do (particularly if this thwarts the person's goal) is frustrating and consequently anger inducing. The response to such a frustrating social event relies on the neural architecture implicated in changing behavioral responses in nonsocial frustrating situations. WIREs Cogn Sci 2012, 3:65-74. doi: 10.1002/wcs.154 This article is categorized under: Psychology > Brain Function and Dysfunction.

A reward prediction error for charitable donations reveals outcome orientation of donators

The motives underlying prosocial behavior, like charitable donations, can be related either to actions or to outcomes. To address the neural basis of outcome orientation in charitable giving, we asked 33 subjects to make choices affecting their own payoffs and payoffs to a charity organization, while being scanned by functional magnetic resonance imaging (fMRI). We experimentally induced a reward prediction error (RPE) by subsequently discarding some of the chosen outcomes. Co-localized to a nucleus accumbens BOLD signal corresponding to the RPE for the subject's own payoff, we observed an equivalent RPE signal for the charity's payoff in those subjects who were willing to donate. This unique demonstration of a neuronal RPE signal for outcomes exclusively affecting unrelated others indicates common brain processes during outcome evaluation for selfish, individual and nonselfish, social rewards and strongly suggests the effectiveness of outcome-oriented motives in charitable giving.

Impact of size and delay on neural activity in the rat limbic corticostriatal system

A number of factors influence an animal’s economic decisions. Two most commonly studied are the magnitude of and delay to reward. To investigate how these factors are represented in the firing rates of single neurons, we devised a behavioral task that independently manipulated the expected delay to and size of reward. Rats perceived the differently delayed and sized rewards as having different values and were more motivated under short delay and big-reward conditions than under long delay and small reward conditions as measured by percent choice, accuracy, and reaction time. Since the creation of this task, we have recorded from several different brain areas including, orbitofrontal cortex, striatum, amygdala, substantia nigra pars reticulata, and midbrain dopamine neurons. Here, we review and compare those data with a substantial focus on those areas that have been shown to be critical for performance on classic time discounting procedures and provide a potential mechanism by which they might interact when animals are deciding between differently delayed rewards. We found that most brain areas in the cortico-limbic circuit encode both the magnitude and delay to reward delivery in one form or another, but only a few encode them together at the single neuron level.

Striatal topography of probability and magnitude information for decisions under uncertainty

Most decisions involve some element of uncertainty. When the outcomes of these decisions have different likelihoods of occurrence, the decision-maker must consider both the magnitude of each outcome and the probability of its occurrence, but how do individual decision makers combine the two dimensions of magnitude and probability? Here, we approach the problem by separating in time the presentation of magnitude and probability information, and focus the analysis of fMRI activations on the first piece of information only. Thus, we are able to identify distinct neural circuits for the two dimensions without the confounding effect of divided attention or the cognitive operation of combining them. We find that magnitude information correlates with the size of the response of the ventral striatum while probability information correlates with the response in the dorsal striatum. The relative responsiveness of these two striatal regions correlates with the behavioral tendency to weight one more than the other. The results are consistent with a second-order process of information aggregation in which individuals make separate judgments for magnitude and probability and then integrate those judgments.

Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment

The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what extent, the striatum is modulated by punishment during an instrumental learning task. Using high-resolution functional magnetic resonance imaging (fMRI) during a reward- and punishment-based probabilistic associative learning task, we observed activity in the ventral putamen for stimuli learned via reward regardless of whether participants were correct or incorrect (i.e., outcome). In contrast, activity in the dorsal caudate was modulated by trials that received feedback--either correct reward or incorrect punishment trials. We also identified an anterior/posterior dissociation reflecting reward and punishment prediction error estimates. Additionally, differences in patterns of activity that correlated with the amount of training were identified along the anterior/posterior axis of the striatum. We suggest that unique subregions of the striatum--separated along both a dorsal/ventral and anterior/posterior axis--differentially participate in the learning of associations through reward and punishment.

A neural signature of hierarchical reinforcement learning

Human behavior displays hierarchical structure: simple actions cohere into subtask sequences, which work together to accomplish overall task goals. Although the neural substrates of such hierarchy have been the target of increasing research, they remain poorly understood. We propose that the computations supporting hierarchical behavior may relate to those in hierarchical reinforcement learning (HRL), a machine-learning framework that extends reinforcement-learning mechanisms into hierarchical domains. To test this, we leveraged a distinctive prediction arising from HRL. In ordinary reinforcement learning, reward prediction errors are computed when there is an unanticipated change in the prospects for accomplishing overall task goals. HRL entails that prediction errors should also occur in relation to task subgoals. In three neuroimaging studies we observed neural responses consistent with such subgoal-related reward prediction errors, within structures previously implicated in reinforcement learning. The results reported support the relevance of HRL to the neural processes underlying hierarchical behavior.

The inherent reward of choice

Research suggests that the exercise of control is desirable and adaptive, but the precise mechanisms underlying the affective value of control are not well understood. The study reported here characterized the affective experience of personal control by examining the neural substrates recruited when individuals anticipate the opportunity to make a choice--in other words, when they anticipate the means for exercising control. We used an experimental paradigm that probed the value of having a choice. Participants reported liking cues that predicted a future opportunity to make a choice more than cues that predicted no choice. The anticipation of choice itself was associated with increased activity in corticostriatal regions, particularly the ventral striatum, involved in affective and motivational processes. This study is the first direct examination of the affective value of having the opportunity to choose. These findings have important implications for understanding the role of perception of control, and choice itself, in self-regulatory processes.

Electroencephalographic topography measures of experienced utility

Economic theory distinguishes two concepts of utility: decision utility, objectively quantifiable by choices, and experienced utility, referring to the satisfaction by an obtainment. To date, experienced utility is typically measured with subjective ratings. This study intended to quantify experienced utility by global levels of neuronal activity. Neuronal activity was measured by means of electroencephalographic (EEG) responses to gain and omission of graded monetary rewards at the level of the EEG topography in human subjects. A novel analysis approach allowed approximating psychophysiological value functions for the experienced utility of monetary rewards. In addition, we identified the time windows of the event-related potentials (ERP) and the respective intracortical sources, in which variations in neuronal activity were significantly related to the value or valence of outcomes. Results indicate that value functions of experienced utility and regret disproportionally increase with monetary value, and thus contradict the compressing value functions of decision utility. The temporal pattern of outcome evaluation suggests an initial (∼250 ms) coarse evaluation regarding the valence, concurrent with a finer-grained evaluation of the value of gained rewards, whereas the evaluation of the value of omitted rewards emerges later. We hypothesize that this temporal double dissociation is explained by reward prediction errors. Finally, a late, yet unreported, reward-sensitive ERP topography (∼500 ms) was identified. The sources of these topographical covariations are estimated in the ventromedial prefrontal cortex, the medial frontal gyrus, the anterior and posterior cingulate cortex and the hippocampus/amygdala. The results provide important new evidence regarding "how," "when," and "where" the brain evaluates outcomes with different hedonic impact.

Componential Granger causality, and its application to identifying the source and mechanisms of the top-down biased activation that controls attention to affective vs sensory processing

We describe a new measure of Granger causality, componential Granger causality, and show how it can be applied to the identification of the directionality of influences between brain areas with functional neuroimaging data. Componential Granger causality measures the effect of y on x, but allows interaction effects between y and x to be measured. In addition, the terms in componential Granger causality sum to 1, allowing causal effects to be directly compared between systems. We show using componential Granger causality analysis applied to an fMRI investigation that there is a top-down attentional effect from the anterior dorsolateral prefrontal cortex to the orbitofrontal cortex when attention is paid to the pleasantness of a taste, and that this effect depends on the activity in the orbitofrontal cortex as shown by the interaction term. Correspondingly there is a top-down attentional effect from the posterior dorsolateral prefrontal cortex to the insular primary taste cortex when attention is paid to the intensity of a taste, and this effect depends on the activity of the insular primary taste cortex as shown by the interaction term. Componential Granger causality thus not only can reveal the directionality of effects between areas (and these can be bidirectional), but also allows the mechanisms to be understood in terms of whether the causal influence of one system on another depends on the state of the system being causally influenced. Componential Granger causality measures the full effects of second order statistics by including variance and covariance effects between each time series, thus allowing interaction effects to be measured, and also provides a systematic framework within which to measure the effects of cross, self, and noise contributions to causality. The findings reveal some of the mechanisms involved in a biased activation theory of selective attention.

The neural correlates of subjective utility of monetary outcome and probability weight in economic and in motor decision under risk

In decision under risk, people choose between lotteries that contain a list of potential outcomes paired with their probabilities of occurrence. We previously developed a method for translating such lotteries to mathematically equivalent "motor lotteries." The probability of each outcome in a motor lottery is determined by the subject's noise in executing a movement. In this study, we used functional magnetic resonance imaging in humans to compare the neural correlates of monetary outcome and probability in classical lottery tasks in which information about probability was explicitly communicated to the subjects and in mathematically equivalent motor lottery tasks in which probability was implicit in the subjects' own motor noise. We found that activity in the medial prefrontal cortex (mPFC) and the posterior cingulate cortex quantitatively represent the subjective utility of monetary outcome in both tasks. For probability, we found that the mPFC significantly tracked the distortion of such information in both tasks. Specifically, activity in mPFC represents probability information but not the physical properties of the stimuli correlated with this information. Together, the results demonstrate that mPFC represents probability from two distinct forms of decision under risk.

Glücksspiel im Gehirn: Neurobiologische Grundlagen pathologischen Glücksspielens

Hintergrund: Pathologisches Glücksspielen (PG) wird in den internationalen Klassifikationssystemen bislang als Impulskontrollstörung klassifiziert. Erst in jüngster Zeit wird aufgrund der Ähnlichkeiten in Phänomenologie, Ätiologie, Verlauf sowie genetischen und neurobiologischen Faktoren mit der Substanzabhängigkeit eine Einordnung des PGs als Verhaltenssucht diskutiert. Insbesondere neurobiologische und neuropsychologische Befunde haben zu dieser veränderten Sichtweise beigetragen. Methode: Im vorliegenden Beitrag werden Befunde zu neurobiologischen Grundlagen PGs vorgestellt. Der Schwerpunkt liegt auf neurokognitiven Prozessen wie Belohnungs- und Bestrafungsverarbeitung, Cue-Reaktivität, Impulsivität und Entscheidungsfindung. Die Befunde werden im Hinblick auf Ähnlichkeiten und Unterschiede PGs zur Substanzabhängigkeit diskutiert. Ergebnisse: Ähnlich wie bei der Substanzabhängigkeit zeigt sich auch bei pathologischen Spielern Veränderungen mesolimbischer-präfrontaler Netzwerke, die sich in einer verminderten Belohnungs- und Bestrafungssensitivität, Impulshemmung und einer erhöhten Cue-Reaktivität auf glücksspielassoziierte Reize äußern können. Jedoch sind die Befunde teilweise nicht eindeutig und eine Vielzahl der Studien unterliegt methodischen Einschränkungen. Schlussfolgerungen: Bisherige Befunde stützen die Einordnung pathologischen Spielverhaltens als Verhaltenssucht.

Imaging genetics for utility of risks over gains and losses

One tenet of behavioral economics is the asymmetry in how decision makers evaluate risks involving gains versus risks involving losses. Correspondingly, an increasingly important question is what neuroanatomical and neurochemical correlates underpin valuation over gains and losses. By employing an imaging genetics strategy, this paper aims at identifying the specific neurotransmitter pathways underlying these decision making processes. We find enhanced striatal activation responding to increases in the magnitude of utility for risks over gains and to increases in the magnitude of disutility for risks over losses, while increased amygdala activation correlates only with the disutility for risks over losses. Stratifying brain activation by genotype, we find that a well-characterized polymorphism in the dopamine transporter (DAT1) contributes to individual differences in striatal response for gain-oriented risks, whereas a polymorphism in the serotonin transporter (STin2) partially accounts for individual differences in amygdala responses for loss-oriented risks. Together, our results suggest the role of the amygdala and corresponding serotonergic pathway in evaluating losses. This further corroborates the hypothesis of serotonin being linked to dopamine in an "opponent partnership".

Differential magnitude coding of gains and omitted rewards in the ventral striatum

Physiologic studies revealed that neurons in the dopaminergic midbrain of non-human primates encode reward prediction errors. It was furthermore shown that reward prediction errors are adaptively scaled with respect to the range of possible outcomes, enabling sensitive encoding for a large range of reward values. Congruently, neuroimaging studies in humans demonstrated that BOLD-responses in the ventral striatum encode reward prediction errors in similar fashion as dopaminergic midbrain neurons, suggesting that these BOLD-responses may be driven by dopaminergic midbrain activity. However, neuroimaging results are ambiguous with respect to the adaptive scaling of reward prediction errors, leading to the conjecture that under certain circumstances other than dopaminergic midbrain input may drive ventral striatal BOLD-responses. The goal of this study was to substantiate whether BOLD-responses in the ventral striatum rather respond to adaptively scaled reward prediction errors or absolute reward magnitude. In addition, we aimed to identify neuronal structures modulating activity in the ventral striatum. Sixteen healthy participants played a wheel of fortune game, where they could win three differently valued rewards while being scanned. BOLD-responses increased after gaining rewards; this gain was however independent of the absolute reward magnitude. In contrast BOLD-responses upon reward omission decreased with reward magnitude. A psychophysiological interaction analysis identified a cluster in the brainstem in proximity of the dorsal raphe nucleus, a cluster in the lateral orbitofrontal cortex, and a cluster in the rostral cingulate zone. These clusters changed their connectivity with the ventral striatum in relation to the absolute reward magnitude in reward omission trials.

Neural correlates of forward planning in a spatial decision task in humans

Although reinforcement learning (RL) theories have been influential in characterizing the mechanisms for reward-guided choice in the brain, the predominant temporal difference (TD) algorithm cannot explain many flexible or goal-directed actions that have been demonstrated behaviorally. We investigate such actions by contrasting an RL algorithm that is model based, in that it relies on learning a map or model of the task and planning within it, to traditional model-free TD learning. To distinguish these approaches in humans, we used functional magnetic resonance imaging in a continuous spatial navigation task, in which frequent changes to the layout of the maze forced subjects continually to relearn their favored routes, thereby exposing the RL mechanisms used. We sought evidence for the neural substrates of such mechanisms by comparing choice behavior and blood oxygen level-dependent (BOLD) signals to decision variables extracted from simulations of either algorithm. Both choices and value-related BOLD signals in striatum, although most often associated with TD learning, were better explained by the model-based theory. Furthermore, predecessor quantities for the model-based value computation were correlated with BOLD signals in the medial temporal lobe and frontal cortex. These results point to a significant extension of both the computational and anatomical substrates for RL in the brain.

Neural systems underlying aversive conditioning in humans with primary and secondary reinforcers

Money is a secondary reinforcer commonly used across a range of disciplines in experimental paradigms investigating reward learning and decision-making. The effectiveness of monetary reinforcers during aversive learning and associated neural basis, however, remains a topic of debate. Specifically, it is unclear if the initial acquisition of aversive representations of monetary losses depends on similar neural systems as more traditional aversive conditioning that involves primary reinforcers. This study contrasts the efficacy of a biologically defined primary reinforcer (shock) and a socially defined secondary reinforcer (money) during aversive learning and its associated neural circuitry. During a two-part experiment, participants first played a gambling game where wins and losses were based on performance to gain an experimental bank. Participants were then exposed to two separate aversive conditioning sessions. In one session, a primary reinforcer (mild shock) served as an unconditioned stimulus (US) and was paired with one of two colored squares, the conditioned stimuli (CS+ and CS−, respectively). In another session, a secondary reinforcer (loss of money) served as the US and was paired with one of two different CS. Skin conductance responses were greater for CS+ compared to CS− trials irrespective of type of reinforcer. Neuroimaging results revealed that the striatum, a region typically linked with reward-related processing, was found to be involved in the acquisition of aversive conditioned response irrespective of reinforcer type. In contrast, the amygdala was involved during aversive conditioning with primary reinforcers, as suggested by both an exploratory fMRI analysis and a follow-up case study with a patient with bilateral amygdala damage. Taken together, these results suggest that learning about potential monetary losses may depend on reinforcement learning related systems, rather than on typical structures involved in more biologically based fears.

An imperfect dopaminergic error signal can drive temporal-difference learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.

What are the physiological changes that take place in the brain when we solve a problem or learn a new skill? It is commonly assumed that behavior adaptations are realized on the microscopic level by changes in synaptic efficacies. However, this is hard to verify experimentally due to the difficulties of identifying the relevant synapses and monitoring them over long periods during a behavioral task. To address this question computationally, we develop a spiking neuronal network model of actor-critic temporal-difference learning, a variant of reinforcement learning for which neural correlates have already been partially established. The network learns a complex task by means of an internally generated reward signal constrained by recent findings on the dopaminergic system. Our model combines top-down and bottom-up modelling approaches to bridge the gap between synaptic plasticity and system-level learning. It paves the way for further investigations of the dopaminergic system in reward learning in the healthy brain and in pathological conditions such as Parkinson's disease, and can be used as a module in functional models based on brain-scale circuitry.

Impulse control disorders in Parkinson's disease: seeking a roadmap toward a better understanding

The development of an impulse control disorder (ICD) is now recognized as a potential nonmotor adverse effect of dopamine replacement therapy in Parkinson's disease (PD). Here, recent epidemiological, neurophysiological and genetic advances are summarized to outline potential mechanisms involved. It is safe to say that dopaminergic drugs, particularly dopamine agonists, are able to induce ICDs only in a minority of patients, while the majority are somehow protected from this adverse effect. While it seems clear that men with early-onset PD are more vulnerable, other predisposing factors, such as various current or pre-PD personality traits, are a matter of debate. In terms of neurophysiological advances, one may find striking analogies to the addiction literature suggesting a causal chain beginning with certain predisposing conditions of striatal dopamine synapses, an "unnatural" increase of dopamine stimulation and a characteristic pattern of resulting functional changes in remote networks of appetitive drive and impulse control. Future prospects include potential add-on medications and the possible identification of genetic predispositions at a genome-wide scale. Functional imaging of pharmacogenetic interactions (imaging pharmacogenomics) may be an important tool on that road.

Differentiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning

To understand how the human amygdala contributes to associative learning, it is necessary to differentiate the contributions of its subregions. However, major limitations in the techniques used for the acquisition and analysis of functional magnetic resonance imaging (fMRI) data have hitherto precluded segregation of function with the amygdala in humans. Here, we used high-resolution fMRI in combination with a region-of-interest-based normalization method to differentiate functionally the contributions of distinct subregions within the human amygdala during two different types of instrumental conditioning: reward and avoidance learning. Through the application of a computational-model-based analysis, we found evidence for a dissociation between the contributions of the basolateral and centromedial complexes in the representation of specific computational signals during learning, with the basolateral complex contributing more to reward learning, and the centromedial complex more to avoidance learning. These results provide unique insights into the computations being implemented within fine-grained amygdala circuits in the human brain.

The integration of negative affect, pain and cognitive control in the cingulate cortex

It has been argued that emotion, pain and cognitive control are functionally segregated in distinct subdivisions of the cingulate cortex. However, recent observations encourage a fundamentally different view. Imaging studies demonstrate that negative affect, pain and cognitive control activate an overlapping region of the dorsal cingulate--the anterior midcingulate cortex (aMCC). Anatomical studies reveal that the aMCC constitutes a hub where information about reinforcers can be linked to motor centres responsible for expressing affect and executing goal-directed behaviour. Computational modelling and other kinds of evidence suggest that this intimacy reflects control processes that are common to all three domains. These observations compel a reconsideration of the dorsal cingulate's contribution to negative affect and pain.

Expected value information improves financial risk taking across the adult life span

When making decisions, individuals must often compensate for cognitive limitations, particularly in the face of advanced age. Recent findings suggest that age-related variability in striatal activity may increase financial risk-taking mistakes in older adults. In two studies, we sought to further characterize neural contributions to optimal financial risk taking and to determine whether decision aids could improve financial risk taking. In Study 1, neuroimaging analyses revealed that individuals whose mesolimbic activation correlated with the expected value estimates of a rational actor made more optimal financial decisions. In Study 2, presentation of expected value information improved decision making in both younger and older adults, but the addition of a distracting secondary task had little impact on decision quality. Remarkably, provision of expected value information improved the performance of older adults to match that of younger adults at baseline. These findings are consistent with the notion that mesolimbic circuits play a critical role in optimal choice, and imply that providing simplified information about expected value may improve financial risk taking across the adult life span.

Social and monetary reward learning engage overlapping neural substrates

Learning to make choices that yield rewarding outcomes requires the computation of three distinct signals: stimulus values that are used to guide choices at the time of decision making, experienced utility signals that are used to evaluate the outcomes of those decisions and prediction errors that are used to update the values assigned to stimuli during reward learning. Here we investigated whether monetary and social rewards involve overlapping neural substrates during these computations. Subjects engaged in two probabilistic reward learning tasks that were identical except that rewards were either social (pictures of smiling or angry people) or monetary (gaining or losing money). We found substantial overlap between the two types of rewards for all components of the learning process: a common area of ventromedial prefrontal cortex (vmPFC) correlated with stimulus value at the time of choice and another common area of vmPFC correlated with reward magnitude and common areas in the striatum correlated with prediction errors. Taken together, the findings support the hypothesis that shared anatomical substrates are involved in the computation of both monetary and social rewards.

Differential effects of dopaminergic therapies on dorsal and ventral striatum in Parkinson's disease: implications for cognitive function

Cognitive abnormalities are a feature of Parkinson's disease (PD). Unlike motor symptoms that are clearly improved by dopaminergic therapy, the effect of dopamine replacement on cognition seems paradoxical. Some cognitive functions are improved whereas others are unaltered or even hindered. Our aim was to understand the effect of dopamine replacement therapy on various aspects of cognition. Whereas dorsal striatum receives dopamine input from the substantia nigra (SN), ventral striatum is innervated by dopamine-producing cells in the ventral tegmental area (VTA). In PD, degeneration of SN is substantially greater than cell loss in VTA and hence dopamine-deficiency is significantly greater in dorsal compared to ventral striatum. We suggest that dopamine supplementation improves functions mediated by dorsal striatum and impairs, or heightens to a pathological degree, operations ascribed to ventral striatum. We consider the extant literature in light of this principle. We also survey the effect of dopamine replacement on functional neuroimaging in PD relating the findings to this framework. This paper highlights the fact that currently, titration of therapy in PD is geared to optimizing dorsal striatum-mediated motor symptoms, at the expense of ventral striatum operations. Increased awareness of contrasting effects of dopamine replacement on dorsal versus ventral striatum functions will lead clinicians to survey a broader range of symptoms in determining optimal therapy, taking into account both those aspects of cognition that will be helped versus those that will be hindered by dopaminergic treatment.

Different gain/loss sensitivity and social adaptation ability in gifted adolescents during a public goods game

Gifted adolescents are considered to have high IQs with advanced mathematical and logical performances, but are often thought to suffer from social isolation or emotional mal-adaptation to the social group. The underlying mechanisms that cause stereotypic portrayals of gifted adolescents are not well known. We aimed to investigate behavioral performance of gifted adolescents during social decision-making tasks to assess their affective and social/non-social cognitive abilities. We examined cooperation behaviors of 22 gifted and 26 average adolescents during an iterative binary public goods (PG) game, a multi-player social interaction game, and analyzed strategic decision processes that include cooperation and free-riding. We found that the gifted adolescents were more cooperative than average adolescents. Particularly, comparing the strategies for the PG game between the two groups, gifted adolescents were less sensitive to loss, yet were more sensitive to gain. Additionally, the behavioral characteristics of average adolescents, such as low trust of the group and herding behavior, were not found in gifted adolescents. These results imply that gifted adolescents have a high cognitive ability but a low ability to process affective information or to adapt in social groups compared with average adolescents. We conclude that gain/loss sensitivity and the ability to adapt in social groups develop to different degrees in average and gifted adolescents.

The affective impact of financial skewness on neural activity and choice

Few finance theories consider the influence of “skewness” (or large and asymmetric but unlikely outcomes) on financial choice. We investigated the impact of skewed gambles on subjects' neural activity, self-reported affective responses, and subsequent preferences using functional magnetic resonance imaging (FMRI). Neurally, skewed gambles elicited more anterior insula activation than symmetric gambles equated for expected value and variance, and positively skewed gambles also specifically elicited more nucleus accumbens (NAcc) activation than negatively skewed gambles. Affectively, positively skewed gambles elicited more positive arousal and negatively skewed gambles elicited more negative arousal than symmetric gambles equated for expected value and variance. Subjects also preferred positively skewed gambles more, but negatively skewed gambles less than symmetric gambles of equal expected value. Individual differences in both NAcc activity and positive arousal predicted preferences for positively skewed gambles. These findings support an anticipatory affect account in which statistical properties of gambles—including skewness—can influence neural activity, affective responses, and ultimately, choice.

Temporal dynamics of reward processing revealed by magnetoencephalography

Monetary gains and losses in gambling situations are associated with a distinct electroencephalographic signature: in the event-related potentials (ERPs), a mediofrontal feedback-related negativity (FRN) is seen for losses, whereas oscillatory activity shows a burst of in the θ-range for losses and in the β-range for gains. We used whole-head magnetoencephalography to pinpoint the magnetic counterparts of these effects in young healthy adults and explore their evolution over time. On each trial, participants bet on one of two visually presented numbers (25 or 5) by button-press. Both numbers changed color: if the chosen number turned green (red), it indicated a gain (loss) of the corresponding sum in Euro cent. For losses, we found the magnetic correlate of the FRN extending between 230 and 465 ms. Source localization with low-resolution electromagnetic tomography indicated a first generator in posterior cingulate cortex with subsequent activity in the anterior cingulate cortex. Importantly, this effect was sensitive to the magnitude of the monetary loss (25 cent > 5 cent). Later activation was also found in the right insula. Time-frequency analysis revealed a number of oscillatory components in the theta, alpha, and high-beta/low-gamma bands associated to gains, and in the high-beta band, associated to the magnitude of the loss. All together, these effects provide a more fine-grained picture of the temporal dynamics of the processing of monetary rewards and losses in the brain.

Relationship between adolescent risk preferences on a laboratory task and behavioral measures of risk-taking

PURPOSE

The goal of the study was to assess individual differences among adolescents regarding risk-taking behavior in the laboratory. The second aim was to evaluate whether the laboratory-based risk-taking behavior is associated with other behavioral and psychological measures associated with risk-taking behavior.

METHODS

A total of 82 adolescents with no personal history of psychiatric disorder completed a computerized decision-making task, the Wheel of Fortune. On the basis of the choices made between clearly defined probabilities and real monetary outcomes, this task assesses risk preferences when participants are confronted with potential rewards and losses. The participants also completed a variety of behavioral and psychological measures associated with risk-taking behavior.

RESULTS

Performance on the task varied on the basis of probability and anticipated outcomes. In the winning sub-task, participants selected low-probability-high-magnitude reward (high-risk choice) less frequently than high-probability-low-magnitude reward (low-risk choice). In the losing sub-task, participants selected low-probability-high-magnitude loss more often than high-probability-low-magnitude loss. On average, the selection of probabilistic rewards was optimal and similar to performance in adults. There were, however, individual differences in performance, and one-third of the adolescents made high-risk choice more frequently than low-risk choice while selecting a reward. After controlling for sociodemographic and psychological variables, high-risk choice on the winning task predicted "real-world" risk-taking behavior and substance-related problems.

CONCLUSIONS

These findings highlight individual differences in risk-taking behavior. Regarding validity of the Wheel of Fortune task, the preliminary data suggest that it might be a valuable laboratory tool for studying behavioral and neurobiological processes associated with risk-taking behavior in adolescents.

The influence of emotion regulation on decision-making under risk

Cognitive strategies typically involved in regulating negative emotions have recently been shown to also be effective with positive emotions associated with monetary rewards. However, it is less clear how these strategies influence behavior, such as preferences expressed during decision-making under risk, and the underlying neural circuitry. That is, can the effective use of emotion regulation strategies during presentation of a reward-conditioned stimulus influence decision-making under risk and neural structures involved in reward processing such as the striatum? To investigate this question, we asked participants to engage in imagery-focused regulation strategies during the presentation of a cue that preceded a financial decision-making phase. During the decision phase, participants then made a choice between a risky and a safe monetary lottery. Participants who successfully used cognitive regulation, as assessed by subjective ratings about perceived success and facility in implementation of strategies, made fewer risky choices in comparison with trials where decisions were made in the absence of cognitive regulation. Additionally, BOLD responses in the striatum were attenuated during decision-making as a function of successful emotion regulation. These findings suggest that exerting cognitive control over emotional responses can modulate neural responses associated with reward processing (e.g., striatum) and promote more goal-directed decision-making (e.g., less risky choices), illustrating the potential importance of cognitive strategies in curbing risk-seeking behaviors before they become maladaptive (e.g., substance abuse).

Common and distinct networks underlying reward valence and processing stages: a meta-analysis of functional neuroimaging studies

To better understand the reward circuitry in human brain, we conducted activation likelihood estimation (ALE) and parametric voxel-based meta-analyses (PVM) on 142 neuroimaging studies that examined brain activation in reward-related tasks in healthy adults. We observed several core brain areas that participated in reward-related decision making, including the nucleus accumbens (NAcc), caudate, putamen, thalamus, orbitofrontal cortex (OFC), bilateral anterior insula, anterior cingulate cortex (ACC) and posterior cingulate cortex (PCC), as well as cognitive control regions in the inferior parietal lobule and prefrontal cortex (PFC). The NAcc was commonly activated by both positive and negative rewards across various stages of reward processing (e.g., anticipation, outcome, and evaluation). In addition, the medial OFC and PCC preferentially responded to positive rewards, whereas the ACC, bilateral anterior insula, and lateral PFC selectively responded to negative rewards. Reward anticipation activated the ACC, bilateral anterior insula, and brain stem, whereas reward outcome more significantly activated the NAcc, medial OFC, and amygdala. Neurobiological theories of reward-related decision making should therefore take distributed and interrelated representations of reward valuation and valence assessment into account.

A behavioral and neural evaluation of prospective decision-making under risk

Making the best choice when faced with a chain of decisions requires a person to judge both anticipated outcomes and future actions. Although economic decision-making models account for both risk and reward in single-choice contexts, there is a dearth of similar knowledge about sequential choice. Classical utility-based models assume that decision-makers select and follow an optimal predetermined strategy, regardless of the particular order in which options are presented. An alternative model involves continuously reevaluating decision utilities, without prescribing a specific future set of choices. Here, using behavioral and functional magnetic resonance imaging (fMRI) data, we studied human subjects in a sequential choice task and use these data to compare alternative decision models of valuation and strategy selection. We provide evidence that subjects adopt a model of reevaluating decision utilities, in which available strategies are continuously updated and combined in assessing action values. We validate this model by using simultaneously acquired fMRI data to show that sequential choice evokes a pattern of neural response consistent with a tracking of anticipated distribution of future reward, as expected in such a model. Thus, brain activity evoked at each decision point reflects the expected mean, variance, and skewness of possible payoffs, consistent with the idea that sequential choice evokes a prospective evaluation of both available strategies and possible outcomes.

How the brain integrates costs and benefits during decision making

When we make decisions, the benefits of an option often need to be weighed against accompanying costs. Little is known, however, about the neural systems underlying such cost-benefit computations. Using functional magnetic resonance imaging and choice modeling, we show that decision making based on cost-benefit comparison can be explained as a stochastic accumulation of cost-benefit difference. Model-driven functional MRI shows that ventromedial and left dorsolateral prefrontal cortex compare costs and benefits by computing the difference between neural signatures of anticipated benefits and costs from the ventral striatum and amygdala, respectively. Moreover, changes in blood oxygen level dependent (BOLD) signal in the bilateral middle intraparietal sulcus reflect the accumulation of the difference signal from ventromedial prefrontal cortex. In sum, we show that a neurophysiological mechanism previously established for perceptual decision making, that is, the difference-based accumulation of evidence, is fundamental also in value-based decisions. The brain, thus, weighs costs against benefits by combining neural benefit and cost signals into a single, difference-based neural representation of net value, which is accumulated over time until the individual decides to accept or reject an option.

Learning from other people's experience: a neuroimaging study of decisional interactive-learning

Decision-making is strongly influenced by the counterfactual anticipation of personal regret and relief, through a learning process involving the ventromedial-prefrontal cortex. We previously reported that observing the regretful outcomes of another's choices reactivates the regret-network. Here we extend those findings by investigating whether this resonant mechanism also underpins interactive-learning from others' previous outcomes. In this functional-Magnetic-Resonance-Imaging study 24 subjects either played a gambling task or observed another player's risky/non-risky choices and resulting outcomes, thus experiencing personal or shared regret/relief for risky/non-risky decisions. Subjects' risk-aptitude in subsequent choices was significantly influenced by both their and the other's previous outcomes. This influence reflected in cerebral regions specifically coding the effect of previously experienced regret/relief, as indexed by the difference between factual and counterfactual outcomes in the last trial, when making a new choice. The subgenual cortex and caudate nucleus tracked the outcomes that increased risk-seeking (relief for a risky choice, and regret for a non-risky choice), while activity in the ventromedial-prefrontal cortex, amygdala and periaqueductal gray-matter reflected those reducing risk-seeking (relief for a non-risky choice, and regret for a risky choice). Crucially, a subset of the involved regions was also activated when subjects chose after observing the other player's outcomes, leading to the same behavioural change as in a first person experience. This resonant neural mechanism at choice may subserve interactive-learning in decision-making.

Separate valuation subsystems for delay and effort decision costs

Decision making consists of choosing among available options on the basis of a valuation of their potential costs and benefits. Most theoretical models of decision making in behavioral economics, psychology, and computer science propose that the desirability of outcomes expected from alternative options can be quantified by utility functions. These utility functions allow a decision maker to assign subjective values to each option under consideration by weighting the likely benefits and costs resulting from an action and to select the one with the highest subjective value. Here, we used model-based neuroimaging to test whether the human brain uses separate valuation systems for rewards (erotic stimuli) associated with different types of costs, namely, delay and effort. We show that humans devalue rewards associated with physical effort in a strikingly similar fashion to those they devalue that are associated with delays, and that a single computational model derived from economics theory can account for the behavior observed in both delay discounting and effort discounting. However, our neuroimaging data reveal that the human brain uses distinct valuation subsystems for different types of costs, reflecting in opposite fashion delayed reward and future energetic expenses. The ventral striatum and the ventromedial prefrontal cortex represent the increasing subjective value of delayed rewards, whereas a distinct network, composed of the anterior cingulate cortex and the anterior insula, represent the decreasing value of the effortful option, coding the expected expense of energy. Together, these data demonstrate that the valuation processes underlying different types of costs can be fractionated at the cerebral level.

Testing the reward prediction error hypothesis with an axiomatic model

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

From bad to worse: striatal coding of the relative value of painful decisions

The majority of decision-related research has focused on how the brain computes decisions over outcomes that are positive in expectation. However, much less is known about how the brain integrates information when all possible outcomes in a decision are negative. To study decision-making over negative outcomes, we used fMRI along with a task in which participants had to accept or reject 50/50 lotteries that could result in more or fewer electric shocks compared to a reference amount. We hypothesized that behaviorally, participants would treat fewer shocks from the reference amount as a gain, and more shocks from the reference amount as a loss. Furthermore, we hypothesized that this would be reflected by a greater BOLD response to the prospect of fewer shocks in regions typically associated with gain, including the ventral striatum and orbitofrontal cortex. The behavioral data suggest that participants in our study viewed all outcomes as losses, despite our attempt to induce a status quo. We find that the ventral striatum showed an increase in BOLD response to better potential gambles (i.e., fewer expected shocks). This lends evidence to the idea that the ventral striatum is not solely responsible for reward processing but that it might also signal the relative value of an expected outcome or action, regardless of whether the outcome is entirely appetitive or aversive. We also find a greater response to worse gambles in regions previously associated with aversive valuation, suggesting an opposing but simultaneous valuation signal to that conveyed by the striatum.

The role of ventral medial prefrontal cortex in social decisions: converging evidence from fMRI and frontotemporal lobar degeneration

The ventral medial prefrontal cortex (vmPFC) has been implicated in social and affectively influenced decision-making. Disease in this region may have clinical consequences for social judgments in patients with frontotemporal lobar degeneration (FTLD). To test this hypothesis, regional cortical activation was monitored with fMRI while healthy adults judged the acceptability of brief social scenarios such as cutting into a movie ticket line or going through a red light at 2 AM. The scenarios described: (i) a socially neutral condition, (ii) a variant of each scenario containing a negatively valenced feature, and (iii) a variant containing a positively valenced feature. Results revealed that healthy adults activated vmPFC during judgments of negatively valenced scenarios relative to positive scenarios and neutral scenarios. In a comparative behavioral study, the same social decision-making paradigm was administered to patients with a social disorder due to FTLD. Patients differed significantly from healthy controls, specifically showing less sensitivity to negatively valenced features. Comparative anatomical analysis revealed considerable overlap of vmPFC activation in healthy adults and vmPFC cortical atrophy in FTLD patients. These converging results support the role of vmPFC in social decision-making where potentially negative consequences must be considered.

The architecture of reward value coding in the human orbitofrontal cortex

To ensure their survival, animals exhibit a number of reward-directed behaviors, such as foraging for food or searching for mates. This suggests that a core set of brain regions may be shared by many species to process different types of rewards. Conversely, many new brain areas have emerged over the course of evolution, suggesting potential specialization of specific brain regions in the processing of more recent rewards such as money. Here, using functional magnetic resonance imaging in humans, we identified the common and distinct brain systems processing the value of erotic stimuli and monetary gains. First, we provide evidence that a set of neural structures, including the ventral striatum, anterior insula, anterior cingulate cortex, and midbrain, encodes the subjective value of rewards regardless of their type, consistent with a general hedonic representation. More importantly, our results reveal reward-specific representations in the orbitofrontal cortex (OFC): whereas the anterior lateral OFC, a phylogenetically recent structure, processes monetary gains, the posterior lateral OFC, phylogenetically and ontogenetically older, processes more basic erotic stimuli. This dissociation between OFC representations of primary and secondary rewards parallels current views on lateral prefrontal cortex organization in cognitive control, suggesting an increasing trend in complexity along a postero-anterior axis according to more abstract representations. Together, our results support a modular view of reward value coding in the brain and propose that a unifying principle of postero-anterior organization can be applied to the OFC.

Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making

An essential feature of choice is the assignment of goal values (GVs) to the different options under consideration at the time of decision making. This computation is done when choosing among appetitive and aversive items. Several groups have studied the location of GV computations for appetitive stimuli, but the problem of valuation in aversive contexts at the time of decision making has been ignored. Thus, although dissociations between appetitive and aversive components of value signals have been shown in other domains such as anticipatory and outcome values, it is not known whether appetitive and aversive GVs are computed in similar brain regions or in separate ones. We investigated this question using two different functional magnetic resonance imaging studies while human subjects placed real bids in an economic auction for the right to eat/avoid eating liked/disliked foods. We found that activity in a common area of the medial orbitofrontal cortex and the dorsolateral prefrontal cortex correlated with both appetitive and aversive GVs. These findings suggest that these regions might form part of a common network.

Attentional Modulation of Affective Versus Sensory Processing: Functional Connectivity and a Top-Down Biased Activation Theory of Selective Attention

Top-down selective attention to the affective properties of taste stimuli increases activation to the taste stimuli in the orbitofrontal cortex (OFC) and pregenual cingulate cortex (PGC), and selective attention to the intensity of the stimuli increases the activation in the insular taste cortex, but the origin of the top-down attentional biases is not known. Using psychophysiological interaction connectivity analyses, we showed that in the anterior lateral prefrontal cortex (LPFC) at Y = 53 mm the correlation with activity in OFC and PGC seed regions was greater when attention was to pleasantness compared with when attention was to intensity. Conversely, we showed that in a more posterior region of the LPFC at Y = 34 the correlation with activity in the anterior insula seed region was greater when attention was to intensity compared with when attention was to pleasantness. We also showed that correlations between areas in these separate processing streams were dependent on selective attention to affective value versus physical intensity of the stimulus. We then propose a biased activation theory of selective attention to account for the findings and contrast this with a biased competition theory of selective attention.

Dissociable responses to punishment in distinct striatal regions during reversal learning

Adaptive behavior depends on the ability to flexibly alter our choices in response to changes in reward and punishment contingencies. One brain region frequently implicated in such behavior is the striatum. However, this region is functionally diverse and there are a number of apparent inconsistencies across previous studies. For instance, how can significant BOLD responses in the ventral striatum during punishment-based reversal learning be reconciled with the frequently demonstrated role of the ventral striatum in reward processing? Here we attempt to address this question by separately examining BOLD responses during reversal learning driven by reward and during reversal learning driven by punishment. We demonstrate simultaneous valence-specific and valence-nonspecific signals in the striatum, with the posterior dorsal striatum responding only to unexpected reward, and the anterior ventral striatum responding to both unexpected punishment as well as unexpected reward. These data help to reconcile conflicting findings from previous studies by showing that distinct regions of the striatum exhibit dissociable responses to punishment during reversal learning.

Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray

A form of aversively motivated learning called fear conditioning occurs when a neutral conditioned stimulus is paired with an aversive unconditioned stimulus (UCS). UCS-evoked depolarization of amygdala neurons may instruct Hebbian plasticity that stores memories of the conditioned stimulus-unconditioned stimulus association, but the origin of UCS inputs to the amygdala is unknown. Theory and evidence suggest that instructive UCS inputs to the amygdala will be inhibited when the UCS is expected, but this has not been found during fear conditioning. We investigated neural pathways that relay information about the UCS to the amygdala by recording neurons in the amygdala and periaqueductal gray (PAG) of rats during fear conditioning. UCS-evoked responses in both amygdala and PAG were inhibited by expectation. Pharmacological inactivation of the PAG attenuated UCS-evoked responses in the amygdala and impaired acquisition of fear conditioning, indicating that PAG may be an important part of the pathway that relays instructive signals to the amygdala.

All that glitters ... dissociating attention and outcome expectancy from prediction errors signals

Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.

The neural correlates of endowment effect without economic transaction

People always concern about what they have and what they might lose even it is just imaginary property. According to Prospect Theory, the losses might be weighted by subjects higher than gain, which would cause the disparity between the willingness to accept (WTA) and willingness to pay (WTP) compensation in economic valuation. Using functional MRI, we investigated neural correlates of this inconsistent value estimation, known as the endowment effect, during a simple pricing task without economic transaction. Brain activation associated with this price discrepancy was observed in the right inferior frontal gyrus (IFG), where voxel-based morphometry of MRI revealed the positive correlation between gray matter concentration and WTA/WTP ratio. These findings suggest the functional relevance of IFG in WTA/WTP discrepancy for pricing without any actual gain and loss, where an integration of loss aversion-related signals from insula and expected value signals may occur.

Effects of category-specific costs on neural systems for perceptual decision-making

Perceptual judgments are often biased by prospective losses, leading to changes in decision criteria. Little is known about how and where sensory evidence and cost information interact in the brain to influence perceptual categorization. Here we show that prospective losses systematically bias the perception of noisy face-house images. Asymmetries in category-specific cost were associated with enhanced blood-oxygen-level-dependent signal in a frontoparietal network. We observed selective activation of parahippocampal gyrus for changes in category-specific cost in keeping with the hypothesis that loss functions enact a particular task set that is communicated to visual regions. Across subjects, greater shifts in decision criteria were associated with greater activation of the anterior cingulate cortex (ACC). Our results support a hypothesis that costs bias an intermediate representation between perception and action, expressed via general effects on frontal cortex, and selective effects on extrastriate cortex. These findings indicate that asymmetric costs may affect a neural implementation of perceptual decision making in a similar manner to changes in category expectation, constituting a step toward accounting for how prospective losses are flexibly integrated with sensory evidence in the brain.