Learning-related human brain activations reflecting individual finances

Behavioral and psychiatric correlates of brain responses to social feedback

Maladaptive responses to peer acceptance and rejection arise in numerous psychiatric disorders in adolescence; yet, homogeneity and heterogeneity across disorders suggest common and unique mechanisms of impaired social function. We tested the hypothesis that social feedback is processed similarly to other forms of feedback (e.g., monetary) by examining the correspondence between the brain's response to social acceptance and rejection and behavioral performance on a separate reward and loss task. We also examined the relationship between these brain responses and depression and social anxiety severity. The sample consisted of one hundred and thirteen 16-21-year olds who received virtual peer acceptance/rejection feedback in an event-related potential (ERP) task. We used temporospatial principal component analysis and identified a component consistent with the reward positivity (RewP) or feedback negativity (FN). RewP to social acceptance was not significantly related to reward bias or the FN to social rejection related to loss avoidance. The relationship between RewP and depression severity, while nonsignificant, was of a similar magnitude to prior studies. Exploratory analyses yielded a significant relationship between lower socioeconomic status (SES) and blunted RewP and between lower SES and heightened loss avoidance and blunted reward bias. These findings build on prior work to improve our understanding of the function of the brain's response to social feedback, while also suggesting a pathway for further study, whereby poverty leads to depression via social and reward learning mechanisms.

Analysis of individual differences in neurofeedback training illuminates successful self-regulation of the dopaminergic midbrain

The dopaminergic midbrain is associated with reinforcement learning, motivation and decision-making - functions often disturbed in neuropsychiatric disorders. Previous research has shown that dopaminergic midbrain activity can be endogenously modulated via neurofeedback. However, the robustness of endogenous modulation, a requirement for clinical translation, is unclear. Here, we examine whether the activation of particular brain regions associates with successful regulation transfer when feedback is no longer available. Moreover, to elucidate mechanisms underlying effective self-regulation, we study the relation of successful transfer with learning (temporal difference coding) outside the midbrain during neurofeedback training and with individual reward sensitivity in a monetary incentive delay (MID) task. Fifty-nine participants underwent neurofeedback training either in standard (Study 1 N = 15, Study 2 N = 28) or control feedback group (Study 1, N = 16). We find that successful self-regulation is associated with prefrontal reward sensitivity in the MID task (N = 25), with a decreasing relation between prefrontal activity and midbrain learning signals during neurofeedback training and with increased activity within cognitive control areas during transfer. The association between midbrain self-regulation and prefrontal temporal difference and reward sensitivity suggests that reinforcement learning contributes to successful self-regulation. Our findings provide insights in the control of midbrain activity and may facilitate individually tailoring neurofeedback training.

Factors Influencing the Intention to Pursue Surgery among Female Pre-Medical Students: A Cross-Sectional Study in Pakistan

BACKGROUND

While gender disparities in surgery are documented worldwide, it is unclear to what extent women consider surgery as a career before embarking on their medical school journey. This study aimed to report the percentage of pre-medical women in Pakistan who intend to eventually specialize in surgery and assess the factors motivating and deterring this decision.

METHODS

An online survey was conducted among female pre-medical (high school) students across Pakistan. Multivariable logistic regression was performed to determine motivating and deterring factors associated with the intention to pursue surgery.

RESULTS

Out of 1219 female high-school students, 764 (62.7) intended to join medical school. Among these 764, only 9.8% reported an exclusive intent to pursue surgery, while just 20.3% reported considering other specialties in addition to surgery. Significant motivators to pursue surgery exclusively were the intellectual satisfaction of pursuing surgery (adjusted odds ratio: 2.302), having opportunities to travel internationally for work (2.300) and use cutting-edge technology (2.203), interest in the specialty of surgery (2.031), the social prestige of becoming a surgeon (1.910), and considering one's personality well-suited to surgery (1.888). Major deterrents included the lack of interest in surgery (adjusted odds ratio: 3.812), surgical education and training being too difficult (2.440) and lengthy (1.404), and the risk of aggressive behavior from patients (2.239).

CONCLUSION

Even before entering medical school, most female pre-medical students have already decided against considering a future surgical career. Deterrents likely stem from women being pressured to conform to deep-seated societal expectations to dedicate their time and energy to domestic responsibilities.

Multiple systems in macaques for tracking prediction errors and other types of surprise

Animals learn from the past to make predictions. These predictions are adjusted after prediction errors, i.e., after surprising events. Generally, most reward prediction errors models learn the average expected amount of reward. However, here we demonstrate the existence of distinct mechanisms for detecting other types of surprising events. Six macaques learned to respond to visual stimuli to receive varying amounts of juice rewards. Most trials ended with the delivery of either 1 or 3 juice drops so that animals learned to expect 2 juice drops on average even though instances of precisely 2 drops were rare. To encourage learning, we also included sessions during which the ratio between 1 and 3 drops changed. Additionally, in all sessions, the stimulus sometimes appeared in an unexpected location. Thus, 3 types of surprising events could occur: reward amount surprise (i.e., a scalar reward prediction error), rare reward surprise, and visuospatial surprise. Importantly, we can dissociate scalar reward prediction errors—rewards that deviated from the average reward amount expected—and rare reward events—rewards that accorded with the average reward expectation but that rarely occurred. We linked each type of surprise to a distinct pattern of neural activity using functional magnetic resonance imaging. Activity in the vicinity of the dopaminergic midbrain only reflected surprise about the amount of reward. Lateral prefrontal cortex had a more general role in detecting surprising events. Posterior lateral orbitofrontal cortex specifically detected rare reward events regardless of whether they followed average reward amount expectations, but only in learnable reward environments.

Animals learn from the past to make predictions, and predictions are adjusted whenever surprising outcomes violate those predictions. This study shows that macaques use multiple systems to detect different types of rare reward events; while activity in the vicinity of the dopaminergic midbrain reflected scalar reward prediction errors (i.e. reward amount), activity in the orbitofrontal cortex reflected a rare reward signal.

Neural Mechanisms of Choice Diversification

When asked to select several options at once, people tend to choose a greater diversity of items than when they are asked to make these selections one at a time. Using functional magnetic resonance imaging (fMRI), we provide novel insight into the neural mechanisms underlying diversification in portfolio choices. We found that, as participants made multiple selections from a menu of different options, the current state of their choice portfolio (i.e., the previously selected options) dynamically modulates activity in the neural valuation system in response to the options under evaluation. More specifically, we found that activity in the ventral striatum (VS) decreases when the option has already been selected ("satiation"), while activity in the ventromedial prefrontal cortex increases when other options have previously been selected ("novelty-seeking"). Our findings reveal two processes that drive diversification in portfolio choices, and suggest that the context of previous selections strongly impacts how the brain evaluates current choice options.

Amygdala and nucleus accumbens involvement in appetitive extinction

Extinction of appetitive conditioning is regarded as an important model for the treatment of psychiatric disorders like addiction. However, very few studies have investigated its neural correlates. Therefore, we investigated neural correlates of appetitive extinction in a large human sample including all genders (N = 76, 40 females) to replicate and extend results from a previous study. During differential appetitive conditioning, one stimulus (CS+) was paired with the chance to win a monetary reward, whereas another stimulus (CS−) was not. During appetitive extinction on the next day, neither the CS+ nor the CS− were reinforced. After successful acquisition of appetitive conditioning, the extinction phase elicited significant reductions of valence and arousal ratings toward the CS+ and a significant reduction in skin conductance responses to the CS+ from early to late extinction. On a neural level, early extinction showed significant differential (CS+ − CS−) activation in dACC and hippocampus, whereas involvement of the vACC and caudate nucleus did not replicate. The differential activation of amygdala and nucleus accumbens during late extinction was replicated, with the amygdala displaying significantly higher differential activation during the late phase of extinction as compared to the early phase of extinction. We show discernible signals for reward learning and extinction in subregions of amygdala and nucleus accumbens after extinction learning. This successful replication underlines the role of nucleus accumbens and amygdala in neural models of appetitive extinction in humans that was previously only based on animal findings.

Effects of familial risk and stimulant drug use on the anticipation of monetary reward: an fMRI study

The association between stimulant drug use and aberrant reward processing is well-documented in the literature, but the nature of these abnormalities remains elusive. The present study aims to disentangle the separate and interacting effects of stimulant drug use and pre-existing familial risk on abnormal reward processing associated with stimulant drug addiction. We used the Monetary Incentive Delay task, a well-validated measure of reward processing, during fMRI scanning in four distinct groups: individuals with familial risk who were either stimulant drug-dependent (N = 41) or had never used stimulant drugs (N = 46); and individuals without familial risk who were either using stimulant drugs (N = 25) or not (N = 48). We first examined task-related whole-brain activation followed by a psychophysiological interaction analysis to further explore brain functional connectivity. For analyses, we used a univariate model with two fixed factors (familial risk and stimulant drug use). Our results showed increased task-related activation in the putamen and motor cortex of stimulant-using participants. We also found altered task-related functional connectivity between the putamen and frontal regions in participants with a familial risk (irrespective of whether they were using stimulant drugs or not). Additionally, we identified an interaction between stimulant drug use and familial risk in task-related functional connectivity between the putamen and motor-related cortical regions in potentially at-risk individuals. Our findings suggest that abnormal task-related activation in motor brain systems is associated with regular stimulant drug use, whereas abnormal task-related functional connectivity in frontostriatal brain systems, in individuals with familial risk, may indicate pre-existing neural vulnerability for developing addiction.

Orbitofrontal Signaling of Future Reward is Associated with Hyperactivity in Attention-Deficit/Hyperactivity Disorder

Alterations in motivated behavior are a hallmark of attention-deficit/hyperactivity disorder (ADHD), one of the most common psychiatric disorders in children and adolescents. The orbitofrontal cortex (OFC) plays a key role in controlling goal-directed behavior, but the link between OFC dysfunction and behavioral deficits in ADHD, particularly in adolescence, remains poorly understood. Here we used advanced high-resolution functional magnetic resonance imaging (fMRI) of the human OFC in adolescents with ADHD and typically developing (TD) controls (N = 39, age 12-16, all male except for one female per group) to study reward-related OFC responses and how they relate to behavioral dysfunction in ADHD. During fMRI data acquisition, participants performed a simple decision-making task, allowing us to image expectation-related responses to small and large monetary outcomes. Across all participants, we observed significant signal increases to large versus small expected rewards in the OFC. These responses were significantly enhanced in ADHD relative to TD participants. Moreover, stronger reward-related activity was correlated with individual differences in hyperactive/impulsive symptoms in the ADHD group, whereas high cognitive ability was associated with normalized OFC responses. These results provide evidence for the importance of OFC dysfunctions in the neuropathology of ADHD, highlighting the role of OFC-dependent goal-directed control mechanisms in this disorder.SIGNIFICANCE STATEMENT Attention-deficit/hyperactivity disorder (ADHD) is characterized by alterations in motivated behavior which can be understood as diminished goal-directed control. The orbitofrontal cortex (OFC) plays a key role in controlling goal-directed behavior, but its potential contribution to ADHD symptomatology remains poorly understood. Using high-resolution fMRI, we show that adolescent ADHD patients display enhanced OFC signaling of future rewards and that these increased reward-related responses are correlated with the severity of hyperactivity/impulsivity. These findings suggest that an inability to adequately evaluate future outcomes may translate into maladaptive behavior in ADHD patients. They also challenge the idea that dysfunctions in dopaminergic brain areas are the sole contributor to reward-related symptoms in ADHD and point to a central contribution of goal-directed control circuits in hyperactivity.

Neural correlates of appetitive extinction in humans

Appetitive extinction receives attention as an important model for the treatment of psychiatric disorders. However, in humans, its underlying neural correlates remain unknown. To close this gap, we investigated appetitive acquisition and extinction with fMRI in a 2-day monetary incentive delay paradigm. During appetitive conditioning, one stimulus (CS+) was paired with monetary reward, while another stimulus (CS−) was never rewarded. Twenty-four hours later, subjects underwent extinction, in which neither CS was reinforced. Appetitive conditioning elicited stronger skin conductance responses to the CS+ as compared with the CS−. Regarding subjective ratings, the CS+ was rated more pleasant and arousing than the CS− after conditioning. Furthermore, fMRI-results (CS+ − CS−) showed activation of the reward circuitry including amygdala, midbrain and striatal areas. During extinction, conditioned responses were successfully extinguished. In the early phase of extinction, we found a significant activation of the caudate, the hippocampus, the dorsal and ventral anterior cingulate cortex (dACC and vACC). In the late phase, we found significant activation of the nucleus accumbens (NAcc) and the amygdala. Correlational analyses with subjective ratings linked extinction success to the vACC and the NAcc, while associating the dACC with reduced extinction. The results reveal neural correlates of appetitive extinction in humans and extend assumptions from models for human extinction learning.

Aging and Neuroeconomics: Insights from Research on Neuromodulation of Reward-based Decision Making

‘Neuroeconomics’ can be broadly defined as the research of how the brain interacts with the environment to make decisions that are functional given individual and contextual constraints. Deciphering such brain-environment transactions requires mechanistic understandings of the neurobiological processes that implement value-dependent decision making. To this end, a common empirical approach is to investigate neural mechanisms of reward-based decision making. Flexible updating of choices and associated expected outcomes in ways that are adaptive for a given task (or a given set of tasks) at hand relies on dynamic neurochemical tuning of the brain’s functional circuitries involved in the representation of tasks, goals and reward prediction. Empirical evidence as well as computational theories indicate that various neurotransmitter systems (e.g., dopamine, norepinephrine, and serotonin) play important roles in reward-based decision making. In light of the apparent aging-related decline in various aspects of the dopaminergic system as well as the effects of neuromodulation on reward-related processes, this article focuses selectively on the literature that highlights the triadic relations between dopaminergic modulation, reward-based decision making, and aging. Directions for future research on aging and neuroeconomoics are discussed.

How learning shapes the empathic brain

Deficits in empathy enhance conflicts and human suffering. Thus, it is crucial to understand how empathy can be learned and how learning experiences shape empathy-related processes in the human brain. As a model of empathy deficits, we used the well-established suppression of empathy-related brain responses for the suffering of out-groups and tested whether and how out-group empathy is boosted by a learning intervention. During this intervention, participants received costly help equally often from an out-group member (experimental group) or an in-group member (control group). We show that receiving help from an out-group member elicits a classical learning signal (prediction error) in the anterior insular cortex. This signal in turn predicts a subsequent increase of empathy for a different out-group member (generalization). The enhancement of empathy-related insula responses by the neural prediction error signal was mediated by an establishment of positive emotions toward the out-group member. Finally, we show that surprisingly few positive learning experiences are sufficient to increase empathy. Our results specify the neural and psychological mechanisms through which learning interacts with empathy, and thus provide a neurobiological account for the plasticity of empathic reactions.

Dopamine reward prediction error responses reflect marginal utility

BACKGROUND

Optimal choices require an accurate neuronal representation of economic value. In economics, utility functions are mathematical representations of subjective value that can be constructed from choices under risk. Utility usually exhibits a nonlinear relationship to physical reward value that corresponds to risk attitudes and reflects the increasing or decreasing marginal utility obtained with each additional unit of reward. Accordingly, neuronal reward responses coding utility should robustly reflect this nonlinearity.

RESULTS

In two monkeys, we measured utility as a function of physical reward value from meaningful choices under risk (that adhered to first- and second-order stochastic dominance). The resulting nonlinear utility functions predicted the certainty equivalents for new gambles, indicating that the functions' shapes were meaningful. The monkeys were risk seeking (convex utility function) for low reward and risk avoiding (concave utility function) with higher amounts. Critically, the dopamine prediction error responses at the time of reward itself reflected the nonlinear utility functions measured at the time of choices. In particular, the reward response magnitude depended on the first derivative of the utility function and thus reflected the marginal utility. Furthermore, dopamine responses recorded outside of the task reflected the marginal utility of unpredicted reward. Accordingly, these responses were sufficient to train reinforcement learning models to predict the behaviorally defined expected utility of gambles.

CONCLUSIONS

These data suggest a neuronal manifestation of marginal utility in dopamine neurons and indicate a common neuronal basis for fundamental explanatory constructs in animal learning theory (prediction error) and economic decision theory (marginal utility).

Surprise beyond prediction error

Surprise drives learning. Various neural “prediction error” signals are believed to underpin surprise‐based reinforcement learning. Here, we report a surprise signal that reflects reinforcement learning but is neither un/signed reward prediction error (RPE) nor un/signed state prediction error (SPE). To exclude these alternatives, we measured surprise responses in the absence of RPE and accounted for a host of potential SPE confounds. This new surprise signal was evident in ventral striatum, primary sensory cortex, frontal poles, and amygdala. We interpret these findings via a normative model of surprise. Hum Brain Mapp 35:4805–4814, 2014. © 2014 The Authors. Human Brain Mapping Published by Wiley Periodicals, Inc.

Endogenous cortisol levels are associated with an imbalanced striatal sensitivity to monetary versus non-monetary cues in pathological gamblers

Pathological gambling is a behavioral addiction characterized by a chronic failure to resist the urge to gamble. It shares many similarities with drug addiction. Glucocorticoid hormones including cortisol are thought to play a key role in the vulnerability to addictive behaviors, by acting on the mesolimbic reward pathway. Based on our previous report of an imbalanced sensitivity to monetary versus non-monetary incentives in the ventral striatum of pathological gamblers (PGs), we investigated whether this imbalance was mediated by individual differences in endogenous cortisol levels. We used functional magnetic resonance imaging (fMRI) and examined the relationship between cortisol levels and the neural responses to monetary versus non-monetary cues, while PGs and healthy controls were engaged in an incentive delay task manipulating both monetary and erotic rewards. We found a positive correlation between cortisol levels and ventral striatal responses to monetary versus erotic cues in PGs, but not in healthy controls. This indicates that the ventral striatum is a key region where cortisol modulates incentive motivation for gambling versus non-gambling related stimuli in PGs. Our results extend the proposed role of glucocorticoid hormones in drug addiction to behavioral addiction, and help understand the impact of cortisol on reward incentive processing in PGs.

Fatal attraction: ventral striatum predicts costly choice errors in humans

Animals approach rewards and cues associated with reward, even when this behavior is irrelevant or detrimental to the attainment of these rewards. Motivated by these findings we study the biology of financially-costly approach behavior in humans. Our subjects passively learned to predict the occurrence of erotic rewards. We show that neuronal responses in ventral striatum during this Pavlovian learning task stably predict an individual's general tendency towards financially-costly approach behavior in an active choice task several months later. Our data suggest that approach behavior may prevent some individuals from acting in their own interests.

Imbalance in the sensitivity to different types of rewards in pathological gambling

Pathological gambling is an addictive disorder characterized by a persistent and compulsive desire to engage in gambling activities. This maladaptive behaviour has been suggested to result from a decreased sensitivity to experienced rewards, regardless of reward type. Alternatively, pathological gambling might reflect an imbalance in the sensitivity to monetary versus non-monetary incentives. To directly test these two hypotheses, we examined how the brain reward circuit of pathological gamblers responds to different types of rewards. Using functional magnetic resonance imaging, we compared the brain responses of 18 pathological gamblers and 20 healthy control subjects while they engaged in a simple incentive task manipulating both monetary and visual erotic rewards. During reward anticipation, the ventral striatum of pathological gamblers showed a differential response to monetary versus erotic cues, essentially driven by a blunted reactivity to cues predicting erotic stimuli. This differential response correlated with the severity of gambling symptoms and was paralleled by a reduced behavioural motivation for erotic rewards. During reward outcome, a posterior orbitofrontal cortex region, responding to erotic rewards in both groups, was further recruited by monetary gains in pathological gamblers but not in control subjects. Moreover, while ventral striatal activity correlated with subjective ratings assigned to monetary and erotic rewards in control subjects, it only correlated with erotic ratings in gamblers. Our results point to a differential sensitivity to monetary versus non-monetary rewards in pathological gambling, both at the motivational and hedonic levels. Such an imbalance might create a bias towards monetary rewards, potentially promoting addictive gambling behaviour.

Shifted risk preferences in pathological gambling

BACKGROUND

Pathological gambling (PG) is an impulse control disorder characterized by excessive monetary risk seeking in the face of negative consequences. We used tools from the field of behavioral economics to refine our description of risk-taking behavior in pathological gamblers. This theoretical framework allowed us to confront two hypotheses: (1) pathological gamblers distort winning probabilities more than controls; and (2) pathological gamblers merely overweight the whole probability range. Method Eighteen pathological gamblers and 20 matched healthy participants performed a decision-making task involving choices between safe amounts of money and risky gambles. The online adjustment of safe amounts, depending on participants' decisions, allowed us to compute 'certainty equivalents' reflecting the subjective probability weight associated with each gamble. The behavioral data were then fitted with a mathematical function known as the 'probability weighting function', allowing us to disentangle our two hypotheses.

RESULTS

The results favored the second hypothesis, suggesting that pathological gamblers' behavior reflects economic preferences globally shifted towards risk, rather than excessively distorted probability weighting. A mathematical parameter (elevation parameter) estimated by our fitting procedure was found to correlate with gambling severity among pathological gamblers, and with gambling affinity among controls.

CONCLUSIONS

PG is associated with a specific pattern of economic preferences, characterized by a global (i.e. probability independent) shift towards risky options. The observed correlation with gambling severity suggests that the present 'certainty equivalent' task may be relevant for clinical use.

Fronto-striatal dysregulation in drug addiction and pathological gambling: Consistent inconsistencies?

Alterations in appetitive processing are central to the major psychological theories of addiction, with differential predictions made by the reward deficiency, incentive salience, and impulsivity hypotheses. Functional MRI has become the chief means of testing these predictions, with experiments reliably highlighting disturbances at the level of the striatum, medial prefrontal cortex, and affiliated regions. However, demonstrations of hypo-reactivity and hyper-reactivity of this circuitry in drug addicted groups are reported in approximately equal measure. Similar findings are echoed in the emergent neuroimaging literature on pathological gambling, which has recently witnessed a coming of age. The first aim of this article is to consider some of the methodological aspects of these experiments that could influence the observed direction of group-level effects, including the baseline condition, trial structure and timing, and the nature of the appetitive cues (drug-related, monetary, or primary rewards). The second aim is to highlight the conceptual traction that is offered by pathological gambling, as a model of a ‘toxicity free’ addiction and an illness where tasks of monetary reinforcement afford a more direct mapping to the abused commodity. Our conclusion is that relatively subtle decisions in task design appear capable of driving group differences in fronto-striatal circuitry in entirely opposing directions, even with tasks and task variants that look ostensibly similar. Differentiation between the psychological theories of addiction will require a greater breadth of experimental designs, with more research needed on processing of primary appetitive cues, aversive processing, and in vulnerable/at-risk groups.

•

We outline the current psychological theories of addiction and their predictions.

•

We review recent fMRI literature of substance addictions and appetitive processing.

•

Reasons for opposing results (hyper- vs hypo-active reward regions) are discussed.

•

Recent fMRI findings of appetitive processing in pathological gambling are reviewed.

•

Pathological gambling is suggested as a prototypical addiction for imaging research.

Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement

The orbitofrontal cortex and adjacent ventromedial prefrontal cortex carry reward representations and mediate flexible behaviour when circumstances change. Here we review how recent experiments in humans and macaques have confirmed the existence of a major difference between the functions of the ventromedial prefrontal cortex and adjacent medial orbitofrontal cortex (mOFC) on the one hand and the lateral orbitofrontal cortex (lOFC) on the other. These differences, however, may not be best accounted for in terms of specializations for reward and error/punishment processing as is commonly assumed. Instead we argue that both lesion and functional magnetic resonance imaging studies reveal that the lOFC is concerned with the assignment of credit for both reward and error outcomes to the choice of specific stimuli and with the linking of specific stimulus representations to representations of specific types of reward outcome. By contrast, we argue that the ventromedial prefrontal cortex/mOFC is concerned with evaluation, value-guided decision-making and maintenance of a choice over successive decisions. Despite the popular view that they cause perseveration of behaviour and inability to inhibit repetition of a previously made choice, we found that lesions in neither orbitofrontal subdivision caused perseveration. On the contrary, lesions in the lOFC made animals switch more rapidly between choices when they were finding it difficult to assign reward values to choices. Lesions in the mOFC caused animals to lose their normal predisposition to repeat previously successful choices, suggesting that the mOFC does not just mediate value comparison in choice but also facilitates maintenance of the same choice if it has been successful.

Effort increases sensitivity to reward and loss magnitude in the human brain

It is ecologically adaptive that the amount of effort invested to achieve a reward increases the relevance of the resulting outcome. Here, we investigated the effect of effort on activity in reward and loss processing brain areas by using functional magnetic resonance imaging. In total, 28 subjects were endowed with monetary rewards of randomly varying magnitude after performing arithmetic calculations that were either difficult (high effort), easy (low effort) or already solved (no effort). Subsequently, a forced donation took place, where a varying part of the endowment was transferred to a charity organization, causing a loss for the subject. Results show that reward magnitude positively modulates activity in reward-processing brain areas (subgenual anterior cingulate cortex and nucleus accumbens) only in the high effort condition. Furthermore, anterior insular activity was positively modulated by loss magnitude only after high effort. The results strongly suggest an increasing relevance of outcomes with increasing previous effort.

Neural responses to advantageous and disadvantageous inequity

In this paper we study neural responses to inequitable distributions of rewards despite equal performance. We specifically focus on differences between advantageous inequity (AI) and disadvantageous inequity (DI). AI and DI were realized in a hyperscanning functional magnetic resonance imaging (fMRI) experiment with pairs of subjects simultaneously performing a task in adjacent scanners and observing both subjects' rewards. Results showed (1) hypoactivation of the ventral striatum (VS) under DI but not under AI; (2) inequity induced activation of the right dorsolateral prefrontal cortex (DLPFC) that was stronger under DI than under AI; (3) correlations between subjective evaluations of AI evaluation and bilateral ventrolateral prefrontal and left insular activity. Our study provides neurophysiological evidence for different cognitive processes that occur when exposed to DI and AI, respectively. One possible interpretation is that any form of inequity represents a norm violation, but that important differences between AI and DI emerge from an asymmetric involvement of status concerns.

BOLD responses in reward regions to hypothetical and imaginary monetary rewards

Monetary rewards are uniquely human. Because money is easy to quantify and present visually, it is the reward of choice for most fMRI studies, even though it cannot be handed over to participants inside the scanner. A typical fMRI study requires hundreds of trials and thus small amounts of monetary rewards per trial (e.g. 5p) if all trials are to be treated equally. However, small payoffs can have detrimental effects on performance due to their limited buying power. Hypothetical monetary rewards can overcome the limitations of smaller monetary rewards but it is less well known whether predictors of hypothetical rewards activate reward regions. In two experiments, visual stimuli were associated with hypothetical monetary rewards. In Experiment 1, we used stimuli predicting either visually presented or imagined hypothetical monetary rewards, together with non-rewarding control pictures. Activations to reward predictive stimuli occurred in reward regions, namely the medial orbitofrontal cortex and midbrain. In Experiment 2, we parametrically varied the amount of visually presented hypothetical monetary reward keeping constant the amount of actually received reward. Graded activation in midbrain was observed to stimuli predicting increasing hypothetical rewards. The results demonstrate the efficacy of using hypothetical monetary rewards in fMRI studies.

A reward prediction error for charitable donations reveals outcome orientation of donators

The motives underlying prosocial behavior, like charitable donations, can be related either to actions or to outcomes. To address the neural basis of outcome orientation in charitable giving, we asked 33 subjects to make choices affecting their own payoffs and payoffs to a charity organization, while being scanned by functional magnetic resonance imaging (fMRI). We experimentally induced a reward prediction error (RPE) by subsequently discarding some of the chosen outcomes. Co-localized to a nucleus accumbens BOLD signal corresponding to the RPE for the subject's own payoff, we observed an equivalent RPE signal for the charity's payoff in those subjects who were willing to donate. This unique demonstration of a neuronal RPE signal for outcomes exclusively affecting unrelated others indicates common brain processes during outcome evaluation for selfish, individual and nonselfish, social rewards and strongly suggests the effectiveness of outcome-oriented motives in charitable giving.

A NEUROECONOMIC MODELING OF ATTENTION-DEFICIT/HYPERACTIVITY DISORDER (ADHD)

In this paper we present a new neuroeconomics model for decision-making applied to the Attention-Deficit/Hyperactivity Disorder (ADHD). The model is based on the hypothesis that decision-making is dependent on the evaluation of expected rewards and risks assessed simultaneously in two decision spaces: the personal (PDS) and the interpersonal emotional spaces (IDS). Motivation to act is triggered by necessities identified in PDS or IDS. The adequacy of an action in fulfilling a given necessity is assumed to be dependent on the expected reward and risk evaluated in the decision spaces. Conflict generated by expected reward and risk influences the easiness (cognitive effort) and the future perspective of the decision-making. Finally, the willingness (not) to act is proposed to be a function of the expected reward (or risk), adequacy, easiness and future perspective. The two most frequent clinical forms are ADHD hyperactive(AD/HDhyp) and ADHD inattentive(AD/HDdin). AD/HDhyp behavior is hypothesized to be a consequence of experiencing high rewarding expectancies for short periods of time, low risk evaluation, and short future perspective for decision-making. AD/HDin is hypothesized to be a consequence of experiencing high rewarding expectancies for long periods of time, low risk evaluation, and long future perspective for decision-making.

Distinct roles of three frontal cortical areas in reward-guided behavior

Functional magnetic resonance imaging was used to measure activity in three frontal cortical areas, the lateral orbitofrontal cortex (lOFC), medial orbitofrontal cortex (mOFC)/ventromedial frontal cortex (vmPFC), and anterior cingulate cortex (ACC), when expectations about type of reward, and not just reward presence or absence, could be learned. Two groups of human subjects learned 12 stimulus-response pairings. In one group (Consistent), correct performances of a given pairing were always reinforced with a specific reward outcome, whereas in the other group (Inconsistent), correct performances were reinforced with randomly selected rewards. The mOFC/vmPFC and lOFC were not distinguished by simple differences in relative preference for positive and negative outcomes. Instead lOFC activity reflected updating of reward-related associations specific to reward type; lOFC was active whenever informative outcomes allowed updating of reward-related associations, regardless of whether the outcomes were positive or negative, and the effects were greater when consistent stimulus-outcome and response-outcome mappings were present. A psychophysiological interaction analysis demonstrated changed coupling between lOFC and brain areas for visual object representation, such as perirhinal cortex, and reward-guided learning, such as the amygdala, ventral striatum, and habenula/mediodorsal thalamus. In contrast, mOFC/vmPFC activity reflected expected values of outcomes and occurrence of positive outcomes, regardless of consistency of outcome mappings. The third frontal cortical region, the ACC, reflected the use of reward type information to guide response selection. ACC activity reflected the probability of selecting the correct response, was greater when consistent outcome mappings were present, and was related to individual differences in propensity to select the correct response.

Gain and loss learning differentially contribute to life financial outcomes

Emerging findings imply that distinct neurobehavioral systems process gains and losses. This study investigated whether individual differences in gain learning and loss learning might contribute to different life financial outcomes (i.e., assets versus debt). In a community sample of healthy adults (n = 75), rapid learners had smaller debt-to-asset ratios overall. More specific analyses, however, revealed that those who learned rapidly about gains had more assets, while those who learned rapidly about losses had less debt. These distinct associations remained strong even after controlling for potential cognitive (e.g., intelligence, memory, and risk preferences) and socioeconomic (e.g., age, sex, ethnicity, income, education) confounds. Self-reported measures of assets and debt were additionally validated with credit report data in a subset of subjects. These findings support the notion that different gain and loss learning systems may exert a cumulative influence on distinct life financial outcomes.

Differential effects of dopaminergic therapies on dorsal and ventral striatum in Parkinson's disease: implications for cognitive function

Cognitive abnormalities are a feature of Parkinson's disease (PD). Unlike motor symptoms that are clearly improved by dopaminergic therapy, the effect of dopamine replacement on cognition seems paradoxical. Some cognitive functions are improved whereas others are unaltered or even hindered. Our aim was to understand the effect of dopamine replacement therapy on various aspects of cognition. Whereas dorsal striatum receives dopamine input from the substantia nigra (SN), ventral striatum is innervated by dopamine-producing cells in the ventral tegmental area (VTA). In PD, degeneration of SN is substantially greater than cell loss in VTA and hence dopamine-deficiency is significantly greater in dorsal compared to ventral striatum. We suggest that dopamine supplementation improves functions mediated by dorsal striatum and impairs, or heightens to a pathological degree, operations ascribed to ventral striatum. We consider the extant literature in light of this principle. We also survey the effect of dopamine replacement on functional neuroimaging in PD relating the findings to this framework. This paper highlights the fact that currently, titration of therapy in PD is geared to optimizing dorsal striatum-mediated motor symptoms, at the expense of ventral striatum operations. Increased awareness of contrasting effects of dopamine replacement on dorsal versus ventral striatum functions will lead clinicians to survey a broader range of symptoms in determining optimal therapy, taking into account both those aspects of cognition that will be helped versus those that will be hindered by dopaminergic treatment.

Art for reward's sake: visual art recruits the ventral striatum

A recent study showed that people evaluate products more positively when they are physically associated with art images than similar non-art images. Neuroimaging studies of visual art have investigated artistic style and esthetic preference but not brain responses attributable specifically to the artistic status of images. Here we tested the hypothesis that the artistic status of images engages reward circuitry, using event-related functional magnetic resonance imaging (fMRI) during viewing of art and non-art images matched for content. Subjects made animacy judgments in response to each image. Relative to non-art images, art images activated, on both subject- and item-wise analyses, reward-related regions: the ventral striatum, hypothalamus and orbitofrontal cortex. Neither response times nor ratings of familiarity or esthetic preference for art images correlated significantly with activity that was selective for art images, suggesting that these variables were not responsible for the art-selective activations. Investigation of effective connectivity, using time-varying, wavelet-based, correlation-purged Granger causality analyses, further showed that the ventral striatum was driven by visual cortical regions when viewing art images but not non-art images, and was not driven by regions that correlated with esthetic preference for either art or non-art images. These findings are consistent with our hypothesis, leading us to propose that the appeal of visual art involves activation of reward circuitry based on artistic status alone and independently of its hedonic value.

Neural basis of the undermining effect of monetary reward on intrinsic motivation

Contrary to the widespread belief that people are positively motivated by reward incentives, some studies have shown that performance-based extrinsic reward can actually undermine a person's intrinsic motivation to engage in a task. This "undermining effect" has timely practical implications, given the burgeoning of performance-based incentive systems in contemporary society. It also presents a theoretical challenge for economic and reinforcement learning theories, which tend to assume that monetary incentives monotonically increase motivation. Despite the practical and theoretical importance of this provocative phenomenon, however, little is known about its neural basis. Herein we induced the behavioral undermining effect using a newly developed task, and we tracked its neural correlates using functional MRI. Our results show that performance-based monetary reward indeed undermines intrinsic motivation, as assessed by the number of voluntary engagements in the task. We found that activity in the anterior striatum and the prefrontal areas decreased along with this behavioral undermining effect. These findings suggest that the corticobasal ganglia valuation system underlies the undermining effect through the integration of extrinsic reward value and intrinsic task value.

How the opinion of others affects our valuation of objects

The opinions of others can easily affect how much we value things. We investigated what happens in our brain when we agree with others about the value of an object and whether or not there is evidence, at the neural level, for social conformity through which we change object valuation. Using functional magnetic resonance imaging we independently modeled (1) learning reviewer opinions about a piece of music, (2) reward value while receiving a token for that music, and (3) their interaction in 28 healthy adults. We show that agreement with two “expert” reviewers on music choice produces activity in a region of ventral striatum that also responds when receiving a valued object. It is known that the magnitude of activity in the ventral striatum reflects the value of reward-predicting stimuli [1–8]. We show that social influence on the value of an object is associated with the magnitude of the ventral striatum response to receiving it. This finding provides clear evidence that social influence mediates very basic value signals in known reinforcement learning circuitry [9–12]. Influence at such a low level could contribute to rapid learning and the swift spread of values throughout a population.

The neural correlates of endowment effect without economic transaction

People always concern about what they have and what they might lose even it is just imaginary property. According to Prospect Theory, the losses might be weighted by subjects higher than gain, which would cause the disparity between the willingness to accept (WTA) and willingness to pay (WTP) compensation in economic valuation. Using functional MRI, we investigated neural correlates of this inconsistent value estimation, known as the endowment effect, during a simple pricing task without economic transaction. Brain activation associated with this price discrepancy was observed in the right inferior frontal gyrus (IFG), where voxel-based morphometry of MRI revealed the positive correlation between gray matter concentration and WTA/WTP ratio. These findings suggest the functional relevance of IFG in WTA/WTP discrepancy for pricing without any actual gain and loss, where an integration of loss aversion-related signals from insula and expected value signals may occur.

Subjective neuronal coding of reward: temporal value discounting and risk

A key question in the neurobiology of reward relates to the nature of coding. Rewards are objects that are advantageous or necessary for the survival of individuals in a variety of environmental situations. Thus reward appears to depend on the individual and its environment. The question arises whether neuronal systems in humans and monkeys code reward in subjective terms, objective terms or both. The present review addresses this issue by dealing with two important reward processes, namely the individual discounting of reward value across temporal delays, and the processing of information about risky rewards that depends on individual risk attitudes. The subjective value of rewards decreases with the temporal distance to the reward. In experiments using neurophysiology and brain imaging, dopamine neurons and striatal systems discount reward value across temporal delays of a few seconds, despite unchanged objective reward value, suggesting subjective value coding. The subjective values of risky outcomes depend on the risk attitude of individual decision makers; these values decrease for risk-avoiders and increase for risk-seekers. The signal for risk and the signal for the value of risky reward covary with individual risk attitudes in regions of the human prefrontal cortex, suggesting subjective rather than objective coding of risk and risky value. These data demonstrate that important parameters of reward are coded in a subjective manner in key reward structures of the brain. However, these data do not rule out that other neurons or brain structures may code reward according to its objective value and risk.

Encoding of marginal utility across time in the human brain

Marginal utility theory prescribes the relationship between the objective property of the magnitude of rewards and their subjective value. Despite its pervasive influence, however, there is remarkably little direct empirical evidence for such a theory of value, let alone of its neurobiological basis. We show that human preferences in an intertemporal choice task are best described by a model that integrates marginally diminishing utility with temporal discounting. Using functional magnetic resonance imaging, we show that activity in the dorsal striatum encodes both the marginal utility of rewards, over and above that which can be described by their magnitude alone, and the discounting associated with increasing time. In addition, our data show that dorsal striatum may be involved in integrating subjective valuation systems inherent to time and magnitude, thereby providing an overall metric of value used to guide choice behavior. Furthermore, during choice, we show that anterior cingulate activity correlates with the degree of difficulty associated with dissonance between value and time. Our data support an integrative architecture for decision making, revealing the neural representation of distinct subcomponents of value that may contribute to impulsivity and decisiveness.

Choking on the money: reward-based performance decrements are associated with midbrain activity

A pernicious paradox in human motivation is the occasional reduced performance associated with tasks and situations that involve larger-than-average rewards. Three broad explanations that might account for such performance decrements are attentional competition (distraction theories), inhibition by conscious processes (explicit-monitoring theories), and excessive drive and arousal (overmotivation theories). Here, we report incentive-dependent performance decrements in humans in a reward-pursuit task; subjects were less successful in capturing a more valuable reward in a computerized maze. Concurrent functional magnetic resonance imaging revealed that increased activity in ventral midbrain, a brain area associated with incentive motivation and basic reward responding, correlated with both reduced number of captures and increased number of near-misses associated with imminent high rewards. These data cast light on the neurobiological basis of choking under pressure and are consistent with overmotivation accounts.

Individual differences in risk preference predict neural responses during financial decision-making

We investigated the neural correlates of subjective valuations during a task involving risky choices about lotteries. Because expected value was held constant across all lotteries, decisions were influenced by subjective preferences, which manifest behaviorally as risk-seeking or risk-averse attitudes. To isolate structures encoding risk preference during choice, we probed for areas showing increased activation as a function of selected risk-level. Such response patterns were obtained in anterior (ACC) and posterior cingulate cortex (PCC), superior frontal gyrus, caudate nucleus, and substantia nigra. Behavioral results revealed the presence of risk-averse and risk-neutral individuals. In parallel, brain signals revealed modulation of activity by risk attitude during choice. Correlations between risk-seeking attitudes and neural activity during risky choice were obtained in superior and inferior frontal gyri, medial and lateral orbitofrontal cortex, and parahippocampal gyrus, while correlations with risk-averse attitudes were found in the caudate. The dynamics of neural responses relevant to each stage of the task (decision, anticipation, outcome) were investigated via timeseries and conjunction analyses. Though the networks engaged in each of the task stages were mostly distinct, regions within ACC, PCC and caudate were consistently activated during each decision-making phase. These results demonstrate (1) that subjective assessments of risk, as well as individual attitudes toward risk, play a significant role in modulating activity within brain regions recruited during decision-making, and (2) that ACC, PCC and caudate are relevant during each phase of a decision-making task requiring subjective valuations, strengthening the role of these regions in self-referential subjective valuations during choice.

Functional imaging of the human dopaminergic midbrain

Invasive recording of dopamine neurons in the substantia nigra and ventral tegmental area (SN/VTA) of behaving animals suggests a role for these neurons in reward learning and novelty processing. In humans, functional magnetic resonance imaging (fMRI) is currently the only non-invasive event-related method to measure SN/VTA activity, but it is debated to what extent fMRI enables inference about dopaminergic responses within the SN/VTA. We consider the anatomical and functional parcellation of the primate SN/VTA and find that its homogeneity suggests little variation in the regional specificity of fMRI signals for reward-related dopaminergic responses. Hence, these responses seem to be well captured by the compound fMRI signal from the SN/VTA, which seems quantitatively related to dopamine release in positron emission tomography (PET). We outline how systematic investigation of the functional parcellation of the SN/VTA in animals, new developments in fMRI analysis and combined PET-fMRI studies can narrow the gap between fMRI and dopaminergic neurotransmission.

The neurobiology of reference-dependent value computation

A key focus of current research in neuroeconomics concerns how the human brain computes value. Although, value has generally been viewed as an absolute measure (e.g., expected value, reward magnitude), much evidence suggests that value is more often computed with respect to a changing reference point, rather than in isolation. Here, we present the results of a study aimed to dissociate brain regions involved in reference-independent (i.e., "absolute") value computations, from those involved in value computations relative to a reference point. During functional magnetic resonance imaging, subjects acted as buyers and sellers during a market exchange of lottery tickets. At a behavioral level, we demonstrate that subjects systematically accorded a higher value to objects they owned relative to those they did not, an effect that results from a shift in reference point (i.e., status quo bias or endowment effect). Our results show that activity in orbitofrontal cortex and dorsal striatum track parameters such as the expected value of lottery tickets indicating the computation of reference-independent value. In contrast, activity in ventral striatum indexed the degree to which stated prices, at a within-subjects and between-subjects level, were distorted with respect to a reference point. The findings speak to the neurobiological underpinnings of reference dependency during real market value computations.

Effort discounting in human nucleus accumbens

A great deal of behavioral and economic research suggests that the value attached to a reward stands in inverse relation to the amount of effort required to obtain it, a principle known as effort discounting. In the present article, we present the first direct evidence for a neural analogue of effort discounting. We used fMRI to measure neural responses to monetary rewards in the human nucleus accumbens (NAcc), a structure previously demonstrated to encode reference-dependent reward information. The magnitude of accumbens activation was found to vary with both reward outcome and the degree of mental effort demanded to obtain individual rewards. For a fixed level of reward, the NAcc was less strongly activated following a high-demand for effort than following a low demand. The magnitude of this effect was noted to correlate with preceding activation in the dorsal anterior cingulate cortex, a region that has been proposed to monitor information-processing demands and to mediate in the subjective experience of effort.

The coordination dynamics of economic decision making: a multilevel approach to social neuroeconomics

The basic reciprocity between individual parts and collective organization constitutes a key scientific question spanning the biological and social sciences. Such reciprocity is accompanied by the absence of direct linkages between levels of description giving rise to what is often referred to as the aggregation or nonequivalence problem between levels of analysis. This issue is encountered both in neuroscience and economics. So far, in spite of being identified and extensively discussed in various (other) scientific fields, the problem of understanding the nature of the interactions and coordination dynamics between individual (neuron approximately agent) and collective (neural networks approximately population of humans) behaviors has received little, if any attention in the growing field of neuroeconomics. The present contribution focuses on bringing a theoretical perspective to the interpretation of experiments recently published in this field and addressing how the concepts and methods of coordination dynamics may impact future research. First, we very briefly discuss the links between biology and economics. Second, we address the nonequivalence problem between different levels of analysis and the concept of reciprocal causality. Third, neuroeconomics studies that investigate the neural underpinnings of social decision making in the context of two economic games (trust and ultimatum) are reviewed to highlight issues that arise when experimental results exist at multiple scales of observation and description. Finally, in the last two sections, we discuss how coordination dynamics might provide novel routes to studying and modelling the relation between brain activity and decision making.

General mechanisms for making decisions?

It has been suggested that many aspects of reward-guided behaviour can be understood within the framework of a computational account of decision making. The account emphasises representation of expectations about decision outcomes and the revision of future expectations in the light of the prediction error-the discrepancy between the actual outcome and prior expectation. Frontal cortex and striatum are implicated in such processes in humans, monkeys, and rats suggesting they are ubiquitous and found in many species. Disagreement remains over the exact contribution made by each brain region. A growing body of work even suggests analogous processes may account for behaviour outside the domain of reward-guided decision making, for example, when people and animals learn about visual and social environments.

Short-term temporal discounting of reward value in human ventral striatum

Delayed rewards lose their value for economic decisions and constitute weaker reinforcers for learning. Temporal discounting of reward value already occurs within a few seconds in animals, which allows investigations of the underlying neurophysiological mechanisms. However, it is difficult to relate these mechanisms to human discounting behavior, which is usually studied over days and months and may engage different brain processes. Our study aimed to bridge the gap by using very short delays and measuring human functional magnetic resonance responses in one of the key reward centers of the brain, the ventral striatum. We used psychometric methods to assess subjective timing and valuation of monetary rewards with delays of 4.0-13.5 s. We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate. Lower reward magnitudes induced steeper behavioral and striatal discounting. By contrast, striatal responses following the delivery of reward reflected the uncertainty in subjective timing associated with delayed rewards rather than value discounting. These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses. Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.

Reinforcement learning: the good, the bad and the ugly

Reinforcement learning provides both qualitative and quantitative frameworks for understanding and modeling adaptive decision-making in the face of rewards and punishments. Here we review the latest dispatches from the forefront of this field, and map out some of the territories where lie monsters.

Individual differences in reinforcement learning: behavioral, electrophysiological, and neuroimaging correlates

During reinforcement learning, phasic modulations of activity in midbrain dopamine neurons are conveyed to the dorsal anterior cingulate cortex (dACC) and basal ganglia (BG) and serve to guide adaptive responding. While the animal literature supports a role for the dACC in integrating reward history over time, most human electrophysiological studies of dACC function have focused on responses to single positive and negative outcomes. The present electrophysiological study investigated the role of the dACC in probabilistic reward learning in healthy subjects using a task that required integration of reinforcement history over time. We recorded the feedback-related negativity (FRN) to reward feedback in subjects who developed a response bias toward a more frequently rewarded ("rich") stimulus ("learners") versus subjects who did not ("non-learners"). Compared to non-learners, learners showed more positive (i.e., smaller) FRNs and greater dACC activation upon receiving reward for correct identification of the rich stimulus. In addition, dACC activation and a bias to select the rich stimulus were positively correlated. The same participants also completed a monetary incentive delay (MID) task administered during functional magnetic resonance imaging. Compared to non-learners, learners displayed stronger BG responses to reward in the MID task. These findings raise the possibility that learners in the probabilistic reinforcement task were characterized by stronger dACC and BG responses to rewarding outcomes. Furthermore, these results highlight the importance of the dACC to probabilistic reward learning in humans.

Fractal connectivity of long-memory networks

Using the multivariate long memory (LM) model and Taylor expansions, we find the conditions for convergence of the wavelet correlations between two LM processes on an asymptotic value at low frequencies. These mathematical results, and a least squares estimator of LM parameters, are validated in simulations and applied to neurophysiological (human brain) and financial market time series. Both brain and market systems had multivariate LM properties including a "fractal connectivity" regime of scales over which wavelet correlations were invariantly close to their asymptotic value. This analysis provides efficient and unbiased estimation of long-term correlations in diverse dynamic networks.