Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain

Examining neural reactivity to gambling cues in the age of online betting

Purpose of Review

The goal of this review is to provide new insights as to how and why functional magnetic resonance imaging (fMRI) research on gambling cue reactivity can contribute to significant progress towards the understanding of gambling disorder. After having offered a detailed description of experimental paradigms and a comprehensive summary of findings related to gambling cue reactivity, the present review suggests methodological avenues for future research.

Recent Findings

The fMRI literature on problem gambling has identified the main neural pathways associated with reactivity to gambling cues. Yet, the current knowledge on the key factors underlying cue reactivity in gambling is still very incomplete. Here, we suggest that the recent expansion of online sports betting calls for a new line of research offering a fine-grained and up-to-date approach of neural cue reactivity in gambling disorder.

Summary

Experimental designs that investigate individual-specific and study-specific factors related to sports betting have the potential to foster progress towards efficient treatment and prevention of gambling disorder.

Investigating the influence of 'losses disguised as wins' on decision making and motivation in rats

Multiline slot machines encourage continued play through 'losses disguised as wins' (LDWs), outcomes in which the money returned is less than that wagered. Individuals with gambling problems may be susceptible to this game feature. The cognitive and neurobiological mechanisms through which LDWs act are unknown. In a novel rat operant task, animals chose between a 'certain' lever, which always delivered two sugar pellets, or an 'uncertain' lever, resulting in four sugar pellets on 50% of trials. LDWs were then introduced as a return of three sugar pellets on 30-40% of uncertain rewarded trials. For half the rats, winning outcomes were paired with audiovisual feedback (cues). In a second study, the basolateral amygdala (BLA) was inactivated during initial presentation of LDWs. While LDWs shifted most rats' choice toward the certain lever, a subgroup of LDW vulnerable rats continued to choose the uncertain option, when the reward rate diminished. This profile of LDW vulnerability was reproduced after inactivating the BLA. Persistent choice of uncertain outcomes despite lower reward rates may reflect impaired functioning within the BLA. Future work using this model may provide insight into the neurobiological mechanisms contributing to the motivational properties of LDWs and their contribution to problematic gambling.

Disentangling the effects of reward value and probability on anticipatory event-related potentials

Optimal decision-making requires humans to predict the value and probability of prospective (rewarding) outcomes. The aim of the present study was to evaluate and dissociate the cortical mechanisms activated by information on an upcoming potentially rewarded target stimulus with varying probabilities. Electro-cortical activity was recorded during a cued Go/NoGo experiment, during which cue letters signaled upcoming target letters to which participants had to respond. The probability of target letter appearance after the cue letter and the amount of money that could be won for correct and fast responses were orthogonally manipulated across four task blocks. As expected, reward availability affected a prefrontally distributed reward-related positivity, and a centrally distributed P300-like event-related potential (ERP). Moreover, a late prefrontally distributed ERP was affected by probability information. These results show that information on value and probability, respectively, activates separate mechanisms in the cortex. These results contribute to a further understanding of the neural underpinnings of normal and abnormal reward processing.

Reward anticipation and punishment anticipation are instantiated in the brain via opponent mechanisms

fMRI investigations have examined the extent to which reward and punishment motivation are associated with common or opponent neural systems, but such investigations have been limited by confounding variables and methodological constraints. The present study aimed to address limitations of earlier approaches and more comprehensively evaluate the extent to which neural activation associated with reward and punishment motivation reflects opponent or shared systems. Participants completed a modified monetary incentive delay task, which involved the presentation of a cue followed by a target to which participants were required to make a speeded button press. Using a factorial design, cues indicated whether monetary reward and/or loss (i.e., cues signaled probability of reward, punishment, both, or neither) could be expected depending upon response speed. Neural analyses evaluated evidence of (a) directionally opposing effects by testing for regions of differential activation for reward and punishment anticipation, (b) mutual inhibition by testing for interactive effects of reward and punishment anticipation within a factorial design, and (c) opposing effects on shared outputs via a psychophysiological interaction analysis. Evidence supporting all three criteria for opponent systems was obtained. Collectively, present findings support conceptualizing reward and punishment motivation as opponent forces influencing brain and behavior and indicate that shared activation does not suggest the operation of a common neural mechanism instantiating reward and punishment motivation.

Altered prefrontal correlates of monetary anticipation and outcome in chronic pain

Chronic pain may alter both affect- and value-related behaviors, which represents a potentially treatable aspect of chronic pain experience. Current understanding of how chronic pain influences the function of brain reward systems, however, is limited. Using a monetary incentive delay task and functional magnetic resonance imaging (fMRI), we measured neural correlates of reward anticipation and outcomes in female participants with the chronic pain condition of fibromyalgia (N = 17) and age-matched, pain-free, female controls (N = 15). We hypothesized that patients would demonstrate lower positive arousal, as well as altered reward anticipation and outcome activity within corticostriatal circuits implicated in reward processing. Patients demonstrated lower arousal ratings as compared with controls, but no group differences were observed for valence, positive arousal, or negative arousal ratings. Group fMRI analyses were conducted to determine predetermined region of interest, nucleus accumbens (NAcc) and medial prefrontal cortex (mPFC), responses to potential gains, potential losses, reward outcomes, and punishment outcomes. Compared with controls, patients demonstrated similar, although slightly reduced, NAcc activity during gain anticipation. Conversely, patients demonstrated dramatically reduced mPFC activity during gain anticipation-possibly related to lower estimated reward probabilities. Further, patients demonstrated normal mPFC activity to reward outcomes, but dramatically heightened mPFC activity to no-loss (nonpunishment) outcomes. In parallel to NAcc and mPFC responses, patients demonstrated slightly reduced activity during reward anticipation in other brain regions, which included the ventral tegmental area, anterior cingulate cortex, and anterior insular cortex. Together, these results implicate altered corticostriatal processing of monetary rewards in chronic pain.

I feel what I do: Relating interoceptive processes and reward-related behavior

Interoceptive signalling has been shown to contribute to action regulation and action experience. Here, we assess whether motor behaviour can be influenced by anticipated homeostatic feeling states induced through different predictable contexts. Participants performed a reward incentive paradigm in which accurate responses increased (gain) or avoided the depletion (averted loss) of a credit score. Across two types of blocks, we varied the predictability of the outcome state. In predictable blocks, a cue signaled a gain, loss or control trial (motor response did not affect the credit score). This allowed participants to anticipate the interoceptive feeling state associated with the outcome. In unpredictable blocks, the cue had no relation to the type of outcome. Thus, participants were unable to anticipate the feeling state it produced. Via EEG, we measured the Heartbeat Evoked Potential (HEP) and the Contingent Negative Variation (CNV) as indices of interoceptive and motor processing respectively. In addition, we measured feedback P3 amplitude following outcome presentation and accuracy and reaction times of the required motor response. We observed higher HEP and CNV amplitudes as well as faster and more accurate motor responses in predictable compared to unpredictable outcome blocks. Similarly, feedback-related P3 amplitudes were significantly lower for predictable relative to unpredictable outcomes. Crucially, HEP amplitudes measured prior to feedback predicted feedback-related P3 amplitudes for anticipated outcome events. Results suggest that accurate anticipation of homeostatic feeling states associated with gain, loss or control outcomes facilitates motor execution and outcome evaluation. Findings are hereby the first to empirically assess the link between interoceptive and motor domains and provide primary evidence for a joint processing structure.

Common and distinct neural substrates of the money illusion in win and loss domains

People often evaluate money based on its face value and overlook its real purchasing power, a phenomenon known as the money illusion. In the present study, using functional magnetic resonance imaging (fMRI) combined with a gambling task, we examined the neural signatures of the money illusion in both win and loss domains. Behavioral results showed that self-reported satisfaction with outcomes was modulated by the face value but not the true value of money in both win and loss domains. At the neural level, activity in the posterior insula was associated with the true value of money in the win domain, but not in the loss domain. Importantly, we found that the ventral striatum, ventromedial prefrontal cortex (vmPFC) and amygdala encoded the money illusion in both domains, indicating a domain-general rather than domain-specific neural signature. Moreover, participants with a larger degree of money illusion at the behavioral level showed stronger functional connectivity between the ventral striatum and ventral anterior cingulate cortex (vACC) in the win domain, but stronger functional connectivity between the ventral striatum and amygdala in the loss domain. Our findings highlight the overlapping and distinct neural substrates underlying the money illusion in the context of wins and losses.

Effects of Positive and Negative Expectations on Human Pain Perception Engage Separate But Interrelated and Dependently Regulated Cerebral Mechanisms

Expectations substantially influence pain perception, but the relationship between positive and negative expectations remains unclear. Recent evidence indicates that the integration between pain-related expectations and prediction errors is crucial for pain perception, which suggests that aversive prediction error-associated regions, such as the anterior insular cortex (aIC) and rostral anterior cingulate cortex (rACC), may play a pivotal role in expectation-induced pain modulation and help to delineate the relationship between positive and negative expectations. In a stimulus expectancy paradigm combining fMRI in healthy volunteers of both sexes, we found that, although positive and negative expectations respectively engaged the right aIC and right rACC to modulate pain, their associated activations and pain rating changes were significantly correlated. When positive and negative expectations modulated pain, the right aIC and rACC exhibited opposite coupling with periaqueductal gray (PAG) and the mismatch between actual and expected pain respectively modulated their coupling with PAG and thalamus across individuals. Participants' certainty about expectations predicted the extent of pain modulation, with positive expectations involving connectivity between aIC and hippocampus, a region regulating anxiety, and negative expectations engaging connectivity between rACC and lateral orbitofrontal cortex, a region reflecting outcome value and certainty. Interestingly, the strength of these certainty-related connectivities was also significantly associated between positive and negative expectations. These findings suggest that aversive prediction-error-related regions interact with pain-processing circuits to underlie stimulus expectancy effects on pain, with positive and negative expectations engaging dissociable but interrelated neural responses that are dependently regulated by individual certainty about expectations.SIGNIFICANCE STATEMENT Positive and negative expectations substantially influence pain perception, but their relationship remains unclear. Using fMRI in a stimulus expectancy paradigm, we found that, although positive and negative expectations engaged separate brain regions encoding the mismatch between actual and expected pain and involved opposite functional connectivities with the descending pain modulatory system, they produced significantly correlated pain rating changes and brain activation. Moreover, participants' certainty about expectations predicted the magnitude of both types of pain modulation, with the underlying functional connectivities significantly correlated between positive and negative expectations. These findings advance current understanding about cognitive modulation of pain, suggesting that both types of pain modulation engage different aversive prediction error signals but are dependently regulated by individual certainty about expectations.

Relationship between extraversion personality and gray matter volume and functional connectivity density in healthy young adults: an fMRI study

Extraversion and neuroticism are two main dimensions of Eysenck's personality. We assessed the relationship between extraversion and neuroticism with brain structure and function by voxel-based morphometry (VBM) and functional connectivity density (FCD). The resting state functional magnetic resonance image and high resolution structural T1 weighted images of 100 young healthy subjects were used in analysis. Our results showed that extraversion was negatively correlated with gray matter volume (GMV) of the bilateral putamen, and it was negatively correlated with FCD in the precuneus. No associations between neuroticism and brain structure and function changes. Overall, our results suggested that several brain regions involved in shaping of extraversion traits among young individuals, which may provide a neurobiological basis of extraversion.

Hedonic processing in humans is mediated by an opioidergic mechanism in a mesocorticolimbic system

It has been hypothesized that the pleasure of a reward in humans is mediated by an opioidergic system involving the hypothalamus, nucleus accumbens and the amygdala. Importantly, enjoying the pleasure of a reward is distinct from incentive salience induced by cues predicting the reward. We investigated this issue using a within subject, pharmacological challenge design with the opioid receptor antagonist naloxone and fMRI. Our data show that blocking opioid receptors reduced pleasure associated with viewing erotic pictures more than viewing symbols of reward such as money. This was paralleled by a reduction of activation in the ventral striatum, lateral orbitofrontal cortex, amygdala, hypothalamus and medial prefrontal cortex. Crucially, the naloxone induced activation decrease was observed at reward delivery, but not during reward anticipation, indicating that blocking opioid receptors decreases the pleasure of rewards in humans.

Editorial note: This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).

A neurobiological pathway to smoking in adolescence: TTC12-ANKK1-DRD2 variants and reward response

The TTC12-ANKK1-DRD2 gene-cluster has been implicated in adult smoking. Here, we investigated the contribution of individual genes in the TTC12-ANKK1-DRD2 cluster in smoking and their association with smoking-associated reward processing in adolescence. A meta-analysis of TTC12-ANKK1-DRD2 variants and self-reported smoking behaviours was performed in four European adolescent cohorts (N = 14,084). The minor G-allele of rs2236709, mapping TTC12, was associated with self-reported smoking (p = 5.0 × 10^-4) and higher plasma cotinine levels (p = 7.0 × 10^-5). This risk allele was linked to an increased ventral-striatal blood-oxygen level-dependent (BOLD) response during reward anticipation (n = 1,263) and with higher DRD2 gene expression in the striatum (p = 0.013), but not with TTC12 or ANKK gene expression. These data suggest a role for the TTC12-ANKK1-DRD2 gene-cluster in adolescent smoking behaviours, provide evidence for the involvement of DRD2 in the early stages of addiction and support the notion that genetically-driven inter-individual differences in dopaminergic transmission mediate reward sensitivity and risk to smoking.

Differential neural reward mechanisms in treatment-responsive and treatment-resistant schizophrenia

BACKGROUND

The significant proportion of schizophrenia patients refractory to treatment, primarily directed at the dopamine system, suggests that multiple mechanisms may underlie psychotic symptoms. Reinforcement learning tasks have been employed in schizophrenia to assess dopaminergic functioning and reward processing, but these have not directly compared groups of treatment-refractory and non-refractory patients.

METHODS

In the current functional magnetic resonance imaging study, 21 patients with treatment-resistant schizophrenia (TRS), 21 patients with non-treatment-resistant schizophrenia (NTR), and 24 healthy controls (HC) performed a probabilistic reinforcement learning task, utilizing emotionally valenced face stimuli which elicit a social bias toward happy faces. Behavior was characterized with a reinforcement learning model. Trial-wise reward prediction error (RPE)-related neural activation and the differential impact of emotional bias on these reward signals were compared between groups.

RESULTS

Patients showed impaired reinforcement learning relative to controls, while all groups demonstrated an emotional bias favoring happy faces. The pattern of RPE signaling was similar in the HC and TRS groups, whereas NTR patients showed significant attenuation of RPE-related activation in striatal, thalamic, precentral, parietal, and cerebellar regions. TRS patients, but not NTR patients, showed a positive relationship between emotional bias and RPE signal during negative feedback in bilateral thalamus and caudate.

CONCLUSION

TRS can be dissociated from NTR on the basis of a different neural mechanism underlying reinforcement learning. The data support the hypothesis that a favorable response to antipsychotic treatment is contingent on dopaminergic dysfunction, characterized by aberrant RPE signaling, whereas treatment resistance may be characterized by an abnormality of a non-dopaminergic mechanism - a glutamatergic mechanism would be a possible candidate.

Mapping adolescent reward anticipation, receipt, and prediction error during the monetary incentive delay task

The functional neuroanatomy and connectivity of reward processing in adults are well documented, with relatively less research on adolescents, a notable gap given this developmental period's association with altered reward sensitivity. Here, a large sample (n = 1,510) of adolescents performed the monetary incentive delay (MID) task during functional magnetic resonance imaging. Probabilistic maps identified brain regions that were reliably responsive to reward anticipation and receipt, and to prediction errors derived from a computational model. Psychophysiological interactions analyses were used to examine functional connections throughout reward processing. Bilateral ventral striatum, pallidum, insula, thalamus, hippocampus, cingulate cortex, midbrain, motor area, and occipital areas were reliably activated during reward anticipation. Bilateral ventromedial prefrontal cortex and bilateral thalamus exhibited positive and negative activation, respectively, during reward receipt. Bilateral ventral striatum was reliably active following prediction errors. Previously, individual differences in the personality trait of sensation seeking were shown to be related to individual differences in sensitivity to reward outcome. Here, we found that sensation seeking scores were negatively correlated with right inferior frontal gyrus activity following reward prediction errors estimated using a computational model. Psychophysiological interactions demonstrated widespread cortical and subcortical connectivity during reward processing, including connectivity between reward-related regions with motor areas and the salience network. Males had more activation in left putamen, right precuneus, and middle temporal gyrus during reward anticipation. In summary, we found that, in adolescents, different reward processing stages during the MID task were robustly associated with distinctive patterns of activation and of connectivity.

Acute tramadol enhances brain activity associated with reward anticipation in the nucleus accumbens

BACKGROUND

Tramadol is an analgesic with monoamine reuptake inhibition and μ-opioid receptor activation. Although tramadol has been widely used for treatment of various pain conditions, there is controversy over the risk of abuse potential. We examined the effects of tramadol on the reward system in humans using functional magnetic resonance imaging (fMRI) to assess the potential of tramadol for drug abuse or dependence.

METHODS

A randomized, double-blind, placebo-controlled, crossover study was conducted for 19 healthy adults under tramadol or placebo. In association with subjective mood questionnaires, monetary incentive delay (MID) task was performed to assess the neural response to reward anticipation during fMRI. Subjective mood measures and blood oxygenation level-dependent (BOLD) signal during gain and loss anticipation were compared between tramadol and placebo.

RESULTS

Tramadol significantly reduced anxiety (Z = - 2.513, p = 0.012) and enhanced vigor (Z = - 2.725, p = 0.006) compared with placebo. By Mood Rating Scale, tramadol provoked contented (Z = - 2.316, p = 0.021), relaxed (Z = - 2.236, p = 0.025), and amicable feelings (Z = - 2.015, p = 0.044) as well as increased alertness (Z = - 1.972, p = 0.049) and contentedness domains (Z = - 2.174, p = 0.030) compared with placebo. Several brain regions including nucleus accumbens (NAc) were activated during gain anticipation in the MID task under both tramadol and placebo. Tramadol increased the %BOLD signal change in NAc at +¥500 cue significantly more than the placebo (Z = - 2.295, p = 0.022).

CONCLUSION

Tramadol enhances the reward system and thereby may have abuse potential or precipitate drug abuse in human.

Risky economic choices and frontal EEG asymmetry in the context of Reinforcer-Sensitivity-Theory-5

This study investigated individual risky choice behavior in a gambling task and its relation with traits proposed by the Reinforcer-Sensitivity-Theory-5 (RST-5; Corr & McNaughton in Neuroscience and Biobehavioral Reviews, 36(10), 2339-2354, 2012) as well as with frontal EEG asymmetry. As assumed by the RST-5, the results showed independent influences of approach/avoidance and gain/loss sensitivities on participants' behavior in risky choices. Individual approach/avoidance sensitivity was predicted by trait measures of the behavioral approach system (BAS) and the behavioral inhibition system (BIS), while no such correlation was present for gain/loss sensitivity. EEG recordings revealed relatively stronger left-frontal cortical activity for trials with approach motivation compared to conflict and avoidance motivation. On the individual level, relatively stronger left-frontal cortical activity was associated with trait BAS. In addition, activity changes in frontal EEG asymmetry were associated relatively higher behavioral approach sensitivity. We conclude that frontal EEG asymmetry is an especially useful neuronal marker of BAS sensitivity and that the traits proposed by the RST-5 (measured by frontal EEG asymmetry and self-report) can be used to explain individual differences in risky choice behavior.

Enhanced transformation of incidentally learned knowledge into explicit memory by dopaminergic modulation

During incidental learning statistical regularities are extracted from the environment without the intention to learn. Acquired implicit memory of these regularities can affect behavior in the absence of awareness. However, conscious insight in the underlying regularities can also develop during learning. Such emergence of explicit memory is an important learning mechanism that is assumed to involve prediction errors in the striatum and to be dopamine-dependent. Here we directly tested this hypothesis by manipulating dopamine levels during incidental learning in a modified serial reaction time task (SRTT) featuring a hidden regular sequence of motor responses in a placebo-controlled between-group study. Awareness for the sequential regularity was subsequently assessed using cued generation and additionally verified using free recall. The results demonstrated that dopaminergic modulation nearly doubled the amount of explicit sequence knowledge emerged during learning in comparison to the placebo group. This strong effect clearly argues for a causal role of dopamine-dependent processing for the development of awareness for sequential regularities during learning.

Methylation of OPRL1 mediates the effect of psychosocial stress on binge drinking in adolescents

BACKGROUND

Nociceptin is a key regulator linking environmental stress and alcohol drinking. In a genome-wide methylation analysis, we recently identified an association of a methylated region in the OPRL1 gene with alcohol-use disorders.

METHODS

Here, we investigate the biological basis of this observation by analysing psychosocial stressors, methylation of the OPRL1 gene, brain response during reward anticipation and alcohol drinking in 660 fourteen-year-old adolescents of the IMAGEN study. We validate our findings in marchigian sardinian (msP) alcohol-preferring rats that are genetically selected for increased alcohol drinking and stress sensitivity.

RESULTS

We found that low methylation levels in intron 1 of OPRL1 are associated with higher psychosocial stress and higher frequency of binge drinking, an effect mediated by OPRL1 methylation. In individuals with low methylation of OPRL1, frequency of binge drinking is associated with stronger BOLD response in the ventral striatum during reward anticipation. In msP rats, we found that stress results in increased alcohol intake and decreased methylation of OPRL1 in the nucleus accumbens.

CONCLUSIONS

Our findings describe an epigenetic mechanism that helps to explain how psychosocial stress influences risky alcohol consumption and reward processing, thus contributing to the elucidation of biological mechanisms underlying risk for substance abuse.

Diet matters: Glucocorticoid-related neuroadaptations associated with calorie intake in female rhesus monkeys

Exposure to psychosocial stressors increases consumption of palatable, calorically dense diets (CDD) and the risk for obesity, especially in females. While consumption of an obesogenic diet and chronic stress have both been shown to decrease dopamine 2 receptor (D2R) binding and alter functional connectivity (FC) within the prefrontal cortex (PFC) and the nucleus accumbens (NAcc), it remains uncertain how social experience and dietary environment interact to affect reward pathways critical for the regulation of motivated behavior. Using positron emission tomography (PET) and resting state functional connectivity magnetic resonance neuroimaging (rs-fMRI), in female rhesus monkeys maintained in a low calorie chow (n = 18) or a dietary choice condition (chow and a CDD; n = 16) for 12 months, the current study tested the overarching hypothesis that the adverse social experience resulting from subordinate social status would interact with consumption of an obesogenic diet to increase caloric intake that would be predicted by greater cortisol, lower prefrontal D2R binding potential (D2R-BP) and lower PFC-NAcc FC. Results showed that the consequences of adverse social experience imposed by chronic social subordination vary significantly depending on the dietary environment and are associated with alterations in prefrontal D2R-BP and FC in NAcc-PFC sub-regions that predict differences in caloric intake, body weight gain, and fat accumulation. Higher levels of cortisol in the chow-only condition were associated with mild inappetence, as well as increased orbitofrontal (OFC) D2R-BP and greater FC between the NAcc and the dorsolateral PFC (dlPFC) and ventromedial PFC (vmPFC). However, increased cortisol release in females in the dietary choice condition was associated with reduced prefrontal D2R-BP, and opposite FC between the NAcc and the vmPFC and dlPFC observed in the chow-only females. Importantly, the degree of these glucocorticoid-related neuroadaptations predicted significantly more total calorie intake as well as more consumption of the CDD for females having a dietary choice, but had no relation to calorie intake in the chow-only condition. Overall, the current findings suggest that dietary environment modifies the consequences of adverse social experience on reward pathways and appetite regulation and, in an obesogenic dietary environment, may reflect impaired cognitive control of food intake.

The Protective Action Encoding of Serotonin Transients in the Human Brain

The role of serotonin in human brain function remains elusive due, at least in part, to our inability to measure rapidly the local concentration of this neurotransmitter. We used fast-scan cyclic voltammetry to infer serotonergic signaling from the striatum of 14 brains of human patients with Parkinson's disease. Here we report these novel measurements and show that they correlate with outcomes and decisions in a sequential investment game. We find that serotonergic concentrations transiently increase as a whole following negative reward prediction errors, while reversing when counterfactual losses predominate. This provides initial evidence that the serotonergic system acts as an opponent to dopamine signaling, as anticipated by theoretical models. Serotonin transients on one trial were also associated with actions on the next trial in a manner that correlated with decreased exposure to poor outcomes. Thus, the fluctuations observed for serotonin appear to correlate with the inhibition of over-reactions and promote persistence of ongoing strategies in the face of short-term environmental changes. Together these findings elucidate a role for serotonin in the striatum, suggesting it encodes a protective action strategy that mitigates risk and modulates choice selection particularly following negative environmental events.

The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task

The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta‐analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.

Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex

There is general consensus that dopaminergic midbrain neurons signal reward prediction errors, computed as the difference between expected and received reward value. However, recent work in rodents shows that these neurons also respond to errors related to inferred value and sensory features, indicating an expanded role for dopamine beyond learning cached values. Here we utilize a transreinforcer reversal learning task and functional magnetic resonance imaging (fMRI) to test whether prediction error signals in the human midbrain are evoked when the expected identity of an appetitive food odor reward is violated, while leaving value matched. We found that midbrain fMRI responses to identity and value errors are correlated, suggesting a common neural origin for these error signals. Moreover, changes in reward-identity expectations, encoded in the orbitofrontal cortex (OFC), are directly related to midbrain activity, demonstrating that identity-based error signals in the midbrain support the formation of outcome identity expectations in OFC.

Responses in the dopaminergic midbrain are known to signal prediction errors for reward value. Here, the authors show that the human midbrain also encodes errors in predicted reward identity, and that these signals update expectations of reward identity in the orbitofrontal cortex.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Neural Mechanisms for Adaptive Learned Avoidance of Mental Effort

Humans tend to avoid mental effort. Previous studies have demonstrated this tendency using various demand-selection tasks; participants generally avoid options associated with higher cognitive demand. However, it remains unclear whether humans avoid mental effort adaptively in uncertain and nonstationary environments. If so, it also remains unclear what neural mechanisms underlie such learned avoidance and whether they remain the same regardless of cognitive-demand types. We addressed these issues by developing novel demand-selection tasks where associations between choice options and cognitive-demand levels change over time, with two variations using mental arithmetic and spatial reasoning problems (males/females: 29:4 and 18:2). Most participants showed avoidance, and their choices depended on the demand experienced on multiple preceding trials. We assumed that participants updated the expected cost of mental effort through experience, and fitted their choices by reinforcement learning models, comparing several possibilities. Model-based fMRI analyses revealed that activity in the dorsomedial and lateral frontal cortices was positively correlated with the trial-by-trial expected cost for the chosen option commonly across the different types of cognitive demand. Analyses also revealed a trend of negative correlation in the ventromedial prefrontal cortex. We further identified correlates of cost-prediction error at time of problem presentation or answering the problem, the latter of which partially overlapped with or were proximal to the correlates of expected cost at time of choice cue in the dorsomedial frontal cortex. These results suggest that humans adaptively learn to avoid mental effort, having neural mechanisms to represent expected cost and cost-prediction error, and the same mechanisms operate for various types of cognitive demand.SIGNIFICANCE STATEMENT In daily life, humans encounter various cognitive demands and tend to avoid high-demand options. However, it remains unclear whether humans avoid mental effort adaptively under dynamically changing environments. If so, it also remains unclear what the underlying neural mechanisms are and whether they operate regardless of cognitive-demand types. To address these issues, we developed novel tasks where participants could learn to avoid high-demand options under uncertain and nonstationary environments. Through model-based fMRI analyses, we found regions whose activity was correlated with the expected mental effort cost, or cost-prediction error, regardless of demand type. These regions overlap, or are adjacent with each other, in the dorsomedial frontal cortex. This finding helps clarify the mechanisms for cognitive-demand avoidance, and provides empirical building blocks for the emerging computational theory of mental effort.

Decision-making impairments following insular and medial temporal lobe resection for drug-resistant epilepsy

Besides the prefrontal cortex, the insula and medial structures of the temporal lobe are thought to be involved in risky decision-making. However, their respective contributions to decision processes remain unclear due to the lack of studies involving patients with isolated insular damage. We assessed adult patients who underwent resection of the insula (n = 13) or of the anterior temporal lobe (including medial structures) (n = 13) as part of their epilepsy surgery, and a group of healthy volunteers (n = 20), on the Iowa Gambling Task (IGT) and on the Cups Task. Groups were matched on sociodemographic, estimated-IQ and surgery-related factors. On the IGT, patients with temporal lobe resection performed significantly worse than both the insular and healthy control groups, as they failed to learn which decks were advantageous on the long-term. On the Cups Task, the insular and temporal groups both showed impaired sensitivity to expected value in the loss domain, when compared with healthy controls. These findings provide clinical evidence that the insula and mesiotemporal structures are specifically involved in risky decision-making when facing a potential loss, and that temporal structures are also involved in learning the association between behavior and consequences in the long-term.

Why we stay with our social partners: Neural mechanisms of stay/leave decision-making

How do we decide to keep interacting (e.g., stay) with a social partner or to switch (e.g., leave) to another? This paper investigated the neural mechanisms of stay/leave decision-making. We hypothesized that these decisions fit within a framework of value-based decision-making, and explored four potential mechanisms underlying a hypothesized bias to stay. Twenty-six participants underwent functional Magnetic Resonance Imaging (fMRI) while completing social and nonsocial versions of a stay/leave decision-making task. On each trial, participants chose between four alternative options, after which they received a monetary reward. Crucially, in the social condition, reward magnitude was ostensibly determined by the generosity of social partners, whereas in the nonsocial condition, reward amounts were ostensibly determined in a pre-programmed manner. Results demonstrated that participants were more likely to stay with options of relatively high expected value, with these values updated through Reinforcement Learning mechanisms and represented neurally within ventromedial prefrontal cortex. Moreover, we demonstrated that greater brain activity in ventromedial prefrontal cortex, caudate nucleus, and septo-hypothalamic regions for social versus nonsocial decisions to stay may underlie a bias towards staying with social partners in particular. These findings complement existing social psychological theories by investigating the neural mechanisms of actual stay/leave decisions.

The Attraction Effect Modulates Reward Prediction Errors and Intertemporal Choices

Classical economic theory contends that the utility of a choice option should be independent of other options. This view is challenged by the attraction effect, in which the relative preference between two options is altered by the addition of a third, asymmetrically dominated option. Here, we leveraged the attraction effect in the context of intertemporal choices to test whether both decisions and reward prediction errors (RPE) in the absence of choice violate the independence of irrelevant alternatives principle. We first demonstrate that intertemporal decision making is prone to the attraction effect in humans. In an independent group of participants, we then investigated how this affects the neural and behavioral valuation of outcomes using a novel intertemporal lottery task and fMRI. Participants' behavioral responses (i.e., satisfaction ratings) were modulated systematically by the attraction effect and this modulation was correlated across participants with the respective change of the RPE signal in the nucleus accumbens. Furthermore, we show that, because exponential and hyperbolic discounting models are unable to account for the attraction effect, recently proposed sequential sampling models might be more appropriate to describe intertemporal choices. Our findings demonstrate for the first time that the attraction effect modulates subjective valuation even in the absence of choice. The findings also challenge the prospect of using neuroscientific methods to measure utility in a context-free manner and have important implications for theories of reinforcement learning and delay discounting.

SIGNIFICANCE STATEMENT

Many theories of value-based decision making assume that people first assess the attractiveness of each option independently of each other and then pick the option with the highest subjective value. The attraction effect, however, shows that adding a new option to a choice set can change the relative value of the existing options, which is a violation of the independence principle. Using an intertemporal choice framework, we tested whether such violations also occur when the brain encodes the difference between expected and received rewards (i.e., the reward prediction error). Our results suggest that neither intertemporal choice nor valuation without choice adhere to the independence principle.

Valence-Dependent Belief Updating: Computational Validation

People tend to update beliefs about their future outcomes in a valence-dependent way: they are likely to incorporate good news and to neglect bad news. However, belief formation is a complex process which depends not only on motivational factors such as the desire for favorable conclusions, but also on multiple cognitive variables such as prior beliefs, knowledge about personal vulnerabilities and resources, and the size of the probabilities and estimation errors. Thus, we applied computational modeling in order to test for valence-induced biases in updating while formally controlling for relevant cognitive factors. We compared biased and unbiased Bayesian models of belief updating, and specified alternative models based on reinforcement learning. The experiment consisted of 80 trials with 80 different adverse future life events. In each trial, participants estimated the base rate of one of these events and estimated their own risk of experiencing the event before and after being confronted with the actual base rate. Belief updates corresponded to the difference between the two self-risk estimates. Valence-dependent updating was assessed by comparing trials with good news (better-than-expected base rates) with trials with bad news (worse-than-expected base rates). After receiving bad relative to good news, participants' updates were smaller and deviated more strongly from rational Bayesian predictions, indicating a valence-induced bias. Model comparison revealed that the biased (i.e., optimistic) Bayesian model of belief updating better accounted for data than the unbiased (i.e., rational) Bayesian model, confirming that the valence of the new information influenced the amount of updating. Moreover, alternative computational modeling based on reinforcement learning demonstrated higher learning rates for good than for bad news, as well as a moderating role of personal knowledge. Finally, in this specific experimental context, the approach based on reinforcement learning was superior to the Bayesian approach. The computational validation of valence-dependent belief updating represents a novel support for a genuine optimism bias in human belief formation. Moreover, the precise control of relevant cognitive variables justifies the conclusion that the motivation to adopt the most favorable self-referential conclusions biases human judgments.

Functional dissociation of stimulus intensity encoding and predictive coding of pain in the insula

The computational principles by which the brain creates a painful experience from nociception are still unknown. Classic theories suggest that cortical regions either reflect stimulus intensity or additive effects of intensity and expectations, respectively. By contrast, predictive coding theories provide a unified framework explaining how perception is shaped by the integration of beliefs about the world with mismatches resulting from the comparison of these beliefs against sensory input. Using functional magnetic resonance imaging during a probabilistic heat pain paradigm, we investigated which computations underlie pain perception. Skin conductance, pupil dilation, and anterior insula responses to cued pain stimuli strictly followed the response patterns hypothesized by the predictive coding model, whereas posterior insula encoded stimulus intensity. This novel functional dissociation of pain processing within the insula together with previously observed alterations in chronic pain offer a novel interpretation of aberrant pain processing as disturbed weighting of predictions and prediction errors.

Dorsolateral prefrontal cortex contributes to the impaired behavioral adaptation in alcohol dependence

Substance-dependent individuals often lack the ability to adjust decisions flexibly in response to the changes in reward contingencies. Prediction errors (PEs) are thought to mediate flexible decision-making by updating the reward values associated with available actions. In this study, we explored whether the neurobiological correlates of PEs are altered in alcohol dependence. Behavioral, and functional magnetic resonance imaging (fMRI) data were simultaneously acquired from 34 abstinent alcohol-dependent patients (ADP) and 26 healthy controls (HC) during a probabilistic reward-guided decision-making task with dynamically changing reinforcement contingencies. A hierarchical Bayesian inference method was used to fit and compare learning models with different assumptions about the amount of task-related information subjects may have inferred during the experiment. Here, we observed that the best-fitting model was a modified Rescorla-Wagner type model, the “double-update” model, which assumes that subjects infer the knowledge that reward contingencies are anti-correlated, and integrate both actual and hypothetical outcomes into their decisions. Moreover, comparison of the best-fitting model's parameters showed that ADP were less sensitive to punishments compared to HC. Hence, decisions of ADP after punishments were loosely coupled with the expected reward values assigned to them. A correlation analysis between the model-generated PEs and the fMRI data revealed a reduced association between these PEs and the BOLD activity in the dorsolateral prefrontal cortex (DLPFC) of ADP. A hemispheric asymmetry was observed in the DLPFC when positive and negative PE signals were analyzed separately. The right DLPFC activity in ADP showed a reduced correlation with positive PEs. On the other hand, ADP, particularly the patients with high dependence severity, recruited the left DLPFC to a lesser extent than HC for processing negative PE signals. These results suggest that the DLPFC, which has been linked to adaptive control of action selection, may play an important role in cognitive inflexibility observed in alcohol dependence when reinforcement contingencies change. Particularly, the left DLPFC may contribute to this impaired behavioral adaptation, possibly by impeding the extinction of the actions that no longer lead to a reward.

•

Alcohol-dependent patients (ADP) had difficulty adapting to the reversals.

•

The impaired adaptation was associated with a decrease in punishment sensitivity.

•

The dorsolateral prefrontal cortex (DLPFC) of ADP failed to track prediction errors.

•

A reduced tracking of the negative prediction error was present in the left DLPFC.

•

The clinical severity of dependence was correlated with abnormal DLPFC activity.

Cognitive Control Modulates Effects of Episodic Simulation on Delay Discounting in Aging

Enhancing prospective thinking by tagging the future with specific episodic events has been shown to reduce delay discounting in young age (“tag-effect”). So far, it is unclear whether such beneficial effect extends to old adulthood. Since the general ability of future thinking and cognitive control are crucial modulators of temporal discounting in young age, potential age-related decline in these functions might impact on the effect. We focused on this issue by combining functional magnetic resonance imaging (fMRI) with an established intertemporal choice task including episodic “tags” in healthy older participants. Future thinking ability was assessed using autobiographical interviews for future event simulations and a visual search task was applied to assess participants’ cognitive control ability. In contrast to previous data in young adults, the group of older participants did not benefit from tagging the future with episodic events. Older participants’ cognitive control function was directly associated with discounting rates in the episodic conditions: the less the older adults were able to focus their attention the less they benefited from the inclusion of episodic events. Consistent with this, imaging results revealed that: (a) subjective value (SV) signals in the hippocampus and the anterior cingulate cortex (ACC) as well as; (b) hippocampal-striatal coupling during the episodic condition were positively related to participants’ control capacity. Our findings highlight the critical role of executive functioning for the simultaneous integration of episodic information with future value computation in aging. Boosting delay gratification by including episodic tags might hence be limited in older individuals with pronounced decline in distraction control.

Strategies for exploration in the domain of losses

Many decisions in everyday life involve a choice between exploring options that are currently unknown and exploiting options that are already known to be rewarding. Previous work has suggested that humans solve such “explore-exploit” dilemmas using a mixture of two strategies: directed exploration, in which information seeking drives exploration by choice, and random exploration, in which behavioral variability drives exploration by chance. One limitation of this previous work was that, like most studies on explore-exploit decision making, it focused exclusively on the domain of gains, where the goal was to maximize reward. In many real-world decisions, however, the goal is to minimize losses and it is well known from Prospect Theory that behavior can be quite different in this domain. In this study, we compared explore-exploit behavior of human subjects under conditions of gain and loss. We found that people use both directed and random exploration regardless of whether they are exploring to maximize gains or minimize losses and that there is quantitative agreement between the exploration parameters across domains. Our results also revealed an overall bias towards the more uncertain option in the domain of losses. While this bias towards uncertainty was qualitatively consistent with the predictions of Prospect Theory, quantitatively we found that the bias was better described by a Bayesian account, in which subjects had a prior that was optimistic for losses and pessimistic for gains. Taken together, our results suggest that explore-exploit decisions are driven by three independent processes: directed and random exploration, and a baseline uncertainty seeking that is driven by a prior.

Blunted ventral striatal responses to anticipated rewards foreshadow problematic drug use in novelty-seeking adolescents

Novelty-seeking tendencies in adolescents may promote innovation as well as problematic impulsive behaviour, including drug abuse. Previous research has not clarified whether neural hyper- or hypo-responsiveness to anticipated rewards promotes vulnerability in these individuals. Here we use a longitudinal design to track 144 novelty-seeking adolescents at age 14 and 16 to determine whether neural activity in response to anticipated rewards predicts problematic drug use. We find that diminished BOLD activity in mesolimbic (ventral striatal and midbrain) and prefrontal cortical (dorsolateral prefrontal cortex) regions during reward anticipation at age 14 predicts problematic drug use at age 16. Lower psychometric conscientiousness and steeper discounting of future rewards at age 14 also predicts problematic drug use at age 16, but the neural responses independently predict more variance than psychometric measures. Together, these findings suggest that diminished neural responses to anticipated rewards in novelty-seeking adolescents may increase vulnerability to future problematic drug use.

Some adolescents seek novelty, but it is unknown whether the brain circuits underlying this behaviour can be used to predict later, problematic behaviour. Here, authors show that diminished ventral striatal and prefrontal activity in response to anticipated rewards at age 14 in these individuals predicts problematic drug use at age 16.

Striatum on the anxiety map: Small detours into adolescence

Adolescence is the most sensitive period for the development of pathological anxiety. Moreover, specific neural changes associated with the striatum might be related to adolescent vulnerability to anxiety. Up to now, the study of anxiety has primarily focused on the amygdala, bed nucleus of the stria terminalis (BNST), hippocampus and ventromedial prefrontal cortex (vmPFC), while the striatum has typically not been considered as part of the anxiety system. This review proposes the addition of the striatum, a complex, multi-component structure, to the anxiety network by underscoring two lines of research. First, the co-occurrence of the adolescent striatal development with the peak vulnerability of adolescents to anxiety disorders might potentially reflect a causal relationship. Second, the recognition of the role of the striatum in fundamental behavioral processes that do affect anxiety supports the putative importance of the striatum in anxiety. These behavioral processes include (1) attention, (2) conditioning/prediction error, and (3) motivation. This review proposes a simplistic schematic representation of the anxiety circuitry that includes the striatum, and aims to promote further work in this direction, as the role of the striatum in shaping an anxiety phenotype during adolescence could have critical implications for understanding and preventing the peak onset of anxiety disorders during this period. This article is part of a Special Issue entitled SI: Adolescent plasticity.

Oscillatory responses to reward processing in borderline personality disorder

OBJECTIVES

Previous electrophysiological studies have confirmed impaired reward processing in patients with BPD. However, it is not clear which aspects of reward processing are affected and which brain regions are involved. The present study investigated both evoked and induced event-related oscillations (EROs) to feedback events (thought to represent different aspects of feedback processing), and used source localization (sLORETA) to assess activity in two areas known to contribute to reward processing, the dorsomedial prefrontal/anterior cingulate cortex (dmPFC/ACC) and the orbitofrontal cortex (OFC).

METHODS

Eighteen patients with BPD and 22 healthy controls performed a gambling task, while 64-channel electroencephalographic activity was recorded. Evoked and induced theta and high-beta band EROs as well as activity in the two regions of interest were investigated depending on the valence and magnitude of feedback events.

RESULTS

Theta-band responses to negative feedback were reduced in BPD, an effect that involved only evoked responses and the dmPFC/ ACC region, and was associated with trait impulsivity in patients. sLORETA analyses revealed disturbed evoked responses depending on feedback magnitude in the theta (OFC) and high-beta (dmPFC/ACC and OFC) frequency range.

CONCLUSIONS

The results indicate multiple dysfunctions of feedback processing in patients with BPD, implicating several distinct subsets of reward-processing mechanisms.

Impact of Short Social Training on Prosocial Behaviors: An fMRI Study

Efficient brain-computer interfaces (BCIs) are in need of knowledge about the human brain and how it interacts, plays games, and socializes with other brains. A breakthrough can be achieved by revealing the microfoundations of sociality, an additional component of the utility function reflecting the value of contributing to group success derived from social identity. Building upon our previous behavioral work, we conduct a series of functional magnetic resonance imaging (fMRI) experiments (N = 10 in the Pilot Study and N = 15 in the Main Study) to measure whether and how sociality alters the functional activation of and connectivity between specific systems in the brain. The overarching hypothesis of this study is that sociality, even in a minimal form, serves as a natural mechanism of sustainable cooperation by fostering interaction between brain regions associated with social cognition and those related to value calculation. We use group-based manipulations to induce varying levels of sociality and compare behavior in two social dilemmas: Prisoner's Dilemma and variations of Ultimatum Game. We find that activation of the right inferior frontal gyrus, a region previously associated with cognitive control and modulation of the valuation system, is correlated with activity in the medial prefrontal cortex (mPFC) to a greater degree when participants make economic decisions in a game with an acquaintance, high sociality condition, compared to a game with a random individual, low sociality condition. These initial results suggest a specific biological mechanism through which sociality facilitates cooperation, fairness and provision of public goods at the cost of individual gain. Future research should examine neural dynamics in the brain during the computation of utility in the context of strategic games that involve social interaction for a larger sample of subjects.

From Trust in Automation to Decision Neuroscience: Applying Cognitive Neuroscience Methods to Understand and Improve Interaction Decisions Involved in Human Automation Interaction

Human automation interaction (HAI) systems have thus far failed to live up to expectations mainly because human users do not always interact with the automation appropriately. Trust in automation (TiA) has been considered a central influence on the way a human user interacts with an automation; if TiA is too high there will be overuse, if TiA is too low there will be disuse. However, even though extensive research into TiA has identified specific HAI behaviors, or trust outcomes, a unique mapping between trust states and trust outcomes has yet to be clearly identified. Interaction behaviors have been intensely studied in the domain of HAI and TiA and this has led to a reframing of the issues of problems with HAI in terms of reliance and compliance. We find the behaviorally defined terms reliance and compliance to be useful in their functionality for application in real-world situations. However, we note that once an inappropriate interaction behavior has occurred it is too late to mitigate it. We therefore take a step back and look at the interaction decision that precedes the behavior. We note that the decision neuroscience community has revealed that decisions are fairly stereotyped processes accompanied by measurable psychophysiological correlates. Two literatures were therefore reviewed. TiA literature was extensively reviewed in order to understand the relationship between TiA and trust outcomes, as well as to identify gaps in current knowledge. We note that an interaction decision precedes an interaction behavior and believe that we can leverage knowledge of the psychophysiological correlates of decisions to improve joint system performance. As we believe that understanding the interaction decision will be critical to the eventual mitigation of inappropriate interaction behavior, we reviewed the decision making literature and provide a synopsis of the state of the art understanding of the decision process from a decision neuroscience perspective. We forward hypotheses based on this understanding that could shape a research path toward the ability to mitigate interaction behavior in the real world.

A Review of Anticipatory Pleasure in Schizophrenia

PURPOSE OF REVIEW

Anhedonia, traditionally defined as a diminished capacity to experience pleasure, has long been considered a core symptom of schizophrenia. However, recent research calls into question whether individuals with schizophrenia are truly anhedonic, suggesting intact subjective and neurophysiological response to rewarding stimuli in-the-moment. Despite a presumably intact capacity to experience pleasure, people with schizophrenia still engage in fewer reward-seeking behaviors. This discrepancy has been explained as a dissociation between "liking" and "wanting", with dopaminergic and prefrontal influences on incentive salience leading hedonic responses to not effectively translate into motivated behavior. In the current review, the literature on a key aspect of the wanting deficit is reviewed, anticipatory pleasure.

RECENT FINDINGS

Results provide consistent evidence for impairment in some aspects of anticipatory pleasure (e.g., prospection, associative learning between reward predictive cues and outcomes), and inconsistent evidence for others (e.g., anticipatory affect and affective forecasting).

SUMMARY

Mechanisms underlying anticipatory pleasure abnormalities in schizophrenia are discussed and a new model of anticipatory pleasure deficits is proposed. Findings suggest that anticipatory pleasure may be a critical component of impairments in wanting that impact motivated behavior in schizophrenia.

Striatal structure and function predict individual biases in learning to avoid pain

Pain is an elemental inducer of avoidance. Here, we demonstrate that people differ in how they learn to avoid pain, with some individuals refraining from actions that resulted in painful outcomes, whereas others favor actions that helped prevent pain. These individual biases were best explained by differences in learning from outcome prediction errors and were associated with distinct forms of striatal responses to painful outcomes. Specifically, striatal responses to pain were modulated in a manner consistent with an aversive prediction error in individuals who learned predominantly from pain, whereas in individuals who learned predominantly from success in preventing pain, modulation was consistent with an appetitive prediction error. In contrast, striatal responses to success in preventing pain were consistent with an appetitive prediction error in both groups. Furthermore, variation in striatal structure, encompassing the region where pain prediction errors were expressed, predicted participants' predominant mode of learning, suggesting the observed learning biases may reflect stable individual traits. These results reveal functional and structural neural components underlying individual differences in avoidance learning, which may be important contributors to psychiatric disorders involving pathological harm avoidance behavior.

Binary sensitivity of theta activity for gain and loss when monitoring parametric prediction errors

Several theories have been proposed to account for the medial frontal activity that is elicited during the evaluation of outcomes. Respectively, these theories claim that the medial frontal response reflects (i) the absolute deviation between the value of an outcome and its expected value (i.e. an absolute prediction error); (ii) the deviation between actual and expected outcomes, with a specific sensitivity to outcomes that are worse than expected (i.e. a negative prediction error); (iii) a binary evaluation in terms of good and bad outcomes. In the current electroencephalography study, participants were presented with cues that induced specific predictions for the values of trial outcomes (a gain or loss of points). The actual outcomes occasionally deviated from the predicted values, producing prediction errors with parametrically varying size. Analysis of the medial frontal theta activity in response to the outcomes demonstrated a specific sensitivity to the occurrence of a loss of points when a gain had been predicted. However, the absolute deviation with respect to the predicted value did not modulate the theta response. This finding is consistent with the idea that outcome monitoring activity measured over medial frontal cortex is sensitive to the binary distinction between good and bad outcomes.

Dissociating Value Representation and Inhibition of Inappropriate Affective Response during Reversal Learning in the Ventromedial Prefrontal Cortex

Decision-making studies have implicated the ventromedial prefrontal cortex (vmPFC) in tracking the value of rewards and punishments. At the same time, fear-learning studies have pointed to a role of the same area in updating previously learned cue-outcome associations. To disentangle these accounts, we used a reward reversal-learning paradigm in a functional magnetic resonance imaging study in 18 human participants. Participants first learned that one of two colored squares (color A) was associated with monetary reward, whereas the other (color B) was not, and then had to learn that these contingencies reversed. Consistent with value representation, activity of a dorsal region of vmPFC was positively correlated with reward magnitude. Conversely, a more ventral region of vmPFC responded more to color A than to color B after contingency reversal, compatible with a role of inhibiting the previously learned response that was no longer appropriate. Moreover, the response strength was correlated with subjects' behavioral learning strength. Our findings provide direct evidence for the spatial dissociation of value representation and affective response inhibition in the vmPFC.

Neural reward processing in individuals remitted from major depression

BACKGROUND

Dysfunctional behavioural and neural processing of reward has been found in currently depressed individuals. However, little is known about altered reward processing in remitted depressed individuals.

METHOD

A total of 23 medication-free individuals with remitted major depressive disorder (rMDD) and 23 matched healthy controls (HCs) performed a reward task during functional magnetic resonance imaging. We also investigated reward dependence, novelty seeking and harm avoidance using the Tridimensional Personality Questionnaire and their association with neural responses of reward processing.

RESULTS

Compared to HCs, individuals with rMDD exhibited enhanced responses to reward-predicting cues in the hippocampus, amygdala and superior frontal gyrus. When reward was delivered, rMDD subjects did not significantly differ from HCs. In both groups neural activity during reward anticipation was negatively correlated with harm avoidance.

CONCLUSIONS

Our results show that rMDD is characterized by hyperactivation in fronto-limbic regions during reward anticipation. Alterations in neural activation during reward processing might reflect an increased effort in remitted depressed individuals to allocate neural activity for executive and evaluative processes during anticipatory reward processing.

Role of emotional processing in depressive responses to sex-hormone manipulation: a pharmacological fMRI study

Sex-hormone fluctuations may increase risk for developing depressive symptoms and alter emotional processing as supported by observations in menopausal and pre- to postpartum transition. In this double-blinded, placebo-controlled study, we used blood-oxygen level dependent functional magnetic resonance imaging (fMRI) to investigate if sex-steroid hormone manipulation with a gonadotropin-releasing hormone agonist (GnRHa) influences emotional processing. Fifty-six healthy women were investigated twice: at baseline (follicular phase of menstrual cycle) and 16 ± 3 days post intervention. At both sessions, fMRI-scans during exposure to faces expressing fear, anger, happiness or no emotion, depressive symptom scores and estradiol levels were acquired. The fMRI analyses focused on regions of interest for emotional processing. As expected, GnRHa initially increased and subsequently reduced estradiol to menopausal levels, which was accompanied by an increase in subclinical depressive symptoms relative to placebo. Women who displayed larger GnRHa-induced increase in depressive symptoms had a larger increase in both negative and positive emotion-elicited activity in the anterior insula. When considering the post-GnRHa scan only, depressive responses were associated with emotion-elicited activity in the anterior insula and amygdala. The effect on regional activity in anterior insula was not associated with the estradiol net decline, only by the GnRHa-induced changes in mood. Our data implicate enhanced insula recruitment during emotional processing in the emergence of depressive symptoms following sex-hormone fluctuations. This may correspond to the emotional hypersensitivity frequently experienced by women postpartum.

The good, the bad and the brain: Neural correlates of appetitive and aversive values underlying decision making

Approaching rewards and avoiding punishments could be considered as core principles governing behavior. Experiments from behavioral economics have shown that choices involving gains and losses follow different policy rules, suggesting that appetitive and aversive processes might rely on different brain systems. Here we contrast this hypothesis with recent neuroscience studies exploring the human brain from brainstem nuclei to cortical areas. Although some circuits show rigid specialization, many others appear to process both appetitive and aversive stimuli, in a flexible manner that depends on a context-wise subjective reference point. Moreover, appetitive and aversive aspects are often integrated into net values that are signaled with enhanced activity in 'positive regions', and suppressed activity in 'negative regions'. This dichotomy might explain why drugs or lesions can produce valence-specific effects, biasing decisions towards approaching a reward or avoiding a punishment.

Studying the effects of dietary body weight-adjusted acute tryptophan depletion on punishment-related behavioral inhibition

Background

Alterations in serotonergic (5-HT) neurotransmission are thought to play a decisive role in affective disorders and impulse control.

Objective

This study aims to reproduce and extend previous findings on the effects of acute tryptophan depletion (ATD) and subsequently diminished central 5-HT synthesis in a reinforced categorization task using a refined body weight–adjusted depletion protocol.

Design

Twenty-four young healthy adults (12 females, mean age [SD]=25.3 [2.1] years) were subjected to a double-blind within-subject crossover design. Each subject was administered both an ATD challenge and a balanced amino acid load (BAL) in two separate sessions in randomized order. Punishment-related behavioral inhibition was assessed using a forced choice go/no-go task that incorporated a variable payoff schedule.

Results

Administration of ATD resulted in significant reductions in TRP measured in peripheral blood samples, indicating reductions of TRP influx across the blood–brain barrier and related brain 5-HT synthesis. Overall accuracy and response time performance were improved after ATD administration. The ability to adjust behavioral responses to aversive outcome magnitudes and behavioral adjustments following error contingent punishment remained intact after decreased brain 5-HT synthesis. A previously observed dissociation effect of ATD on punishment-induced inhibition was not observed.

Conclusions

Our results suggest that neurodietary challenges with ATD Moja–De have no detrimental effects on task performance and punishment-related inhibition in healthy adults.

Effects of prospective thinking on intertemporal choice: The role of familiarity

Imagining future events while performing an intertemporal choice task can attenuate the devaluation of future rewards. Here, we investigated whether this effect and its neural basis depend on the degree of personal prior experience associated with the simulated future scenarios. Functional magnetic resonance imaging was combined with a modified intertemporal choice task in which the delayed options were either purely monetary, or linked with a social event. Subject-specific events differed regarding familiarity, that is, meeting a close, familiar person or a celebrity in a café. In line with recent hypotheses on episodic construction, the simulation of future familiar and unfamiliar events equally attenuated delay discounting behavior in comparison with the control condition and both were imagined with similar richness. Imaging data, however, indicate that these results rely on differential neural activation patterns. The hippocampus was particularly involved in the simulation of unfamiliar future scenarios, probably reflecting enhanced construction processes when personal experience with similar past events is lacking. Consequently, functional coupling of the hippocampus with neural valuation signals in the anterior cingulate cortex predicted the subjective value only of rewards offered in the unfamiliar context. In contrast, valuation of rewards in a familiar context was predicted by activation in key nodes of emotional and autobiographical memory retrieval and dynamically modulated by frontal-striatal connectivity. The present data emphasize that the mechanisms underlying neural valuation of prospective rewards largely depend on the pre-experience with the context in which they are offered.