The good, the bad and the brain: Neural correlates of appetitive and aversive values underlying decision making

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

From compulsivity to compulsion: the neural basis of compulsive disorders

Compulsive behaviour, an apparently irrational perseveration in often maladaptive acts, is a potential transdiagnostic symptom of several neuropsychiatric disorders, including obsessive-compulsive disorder and addiction, and may reflect the severe manifestation of a dimensional trait termed compulsivity. In this Review, we examine the psychological basis of compulsions and compulsivity and their underlying neural circuitry using evidence from human neuroimaging and animal models. Several main elements of this circuitry are identified, focused on fronto-striatal systems implicated in goal-directed behaviour and habits. These systems include the orbitofrontal, prefrontal, anterior cingulate and insular cortices and their connections with the basal ganglia as well as sensoriomotor and parietal cortices and cerebellum. We also consider the implications for future classification of impulsive-compulsive disorders and their treatment.

Contextual valence influences the neural dynamics of time and magnitude representation during feedback evaluation

Contextual valence is an important dimension during value-based decision-making. Previous research has revealed behavioral and neural asymmetries between the gain context and the loss context. The present event-related potential study investigated the effects of contextual valence on neural dynamics underlying magnitude and time, two important reward dimensions, during feedback evaluation. Forty-two participants performed a simple guessing task in which they experienced both a gain context wherein high or low rewards were delivered immediately or six months later, and a loss context wherein high or low losses were delivered in the same way. Results showed that in the gain context, time and magnitude information were processed in a parallel way during the time windows of the reward positivity (RewP) and the P3. In the loss context, however, time and magnitude information were processed in a serial way such that time information was encoded during the RewP and P3 periods, whereas magnitude information was not tracked until the time window of the late positive potential. Our findings suggest that the neural dynamics underlying time and magnitude information are distinct between the gain and loss contexts, thus providing a novel perspective for the well-known gain-loss asymmetry.

Appetitive and aversive cue reactivities differentiate neural subtypes of alcohol drinkers

Craving reflects the subjective urge to use drugs and can be triggered by both positive and negative emotional states. No studies have systematically investigated the relative roles of these mechanisms in the pathophysiology of substance misuse. Here, we performed meta-analyses of drug cue-elicited reactivity and win and loss processing in the monetary incentive delay task to identify distinct neural correlates of appetitive and aversive responses to drug cues. We then characterized the appetitive and aversive cue responses in seventy-six alcohol drinkers performing a cue craving task during fMRI. Imaging data were processed according to published routines. The appetitive circuit involved medial cortical regions and the ventral striatum, and the aversive circuit involved the insula, caudate and mid-cingulate cortex. We observed a significant correlation of cue-elicited activity (β estimates) of the appetitive and aversive circuit. However, individuals varied in appetitive and aversive cue responses. From the regression of appetitive (y) vs. aversive (x) β, we identified participants in the top 1/3 each of those with positive and negative residuals as "approach" (n = 15) and "avoidance" (n = 11) and the others as the "mixed" (n = 50) subtype. In clinical characteristics, the avoidance subtype showed higher sensitivity to punishment and, in contrast, the approach subtype showed higher levels of sensation seeking and alcohol expectancy for social and physical pressure. The findings highlighted distinct neural underpinnings of appetitive and aversive components of cue-elicited reactivity and provided evidence for potential subtypes of alcohol drinkers.

RISK aversion in Italian forensic and non-forensic patients with schizophrenia spectrum disorders

BACKGROUND

Goal-directed decision-making is a central component of the broader reward and motivation system, and requires the ability to dynamically integrate both positive and negative feedback from the environment in order to maximize rewards and minimize losses over time. Altered decision-making processes, in which individuals fail to consider the negative consequences of their decisions on both themselves and others, may play a role in driving antisocial behaviour.

AIM

The main study aim was to investigate possible differences in loss and risk aversion across matched patients, all with a schizophrenia spectrum disorder (SSD), but who varied according to whether they had a history of serious interpersonal violence or not, and a sample of healthy controls with no history of violence.

RESULTS

The sample included 14 forensic and 21 non-forensic patients with SSD, and 41 healthy controls. Among the three decision-making variables under investigation, risk aversion was the only significant predictor of membership of the three groups, with greater risk aversion among non-forensic patients with SSD compared to healthy controls. No differences were observed across groups in loss aversion and choice consistency.

CONCLUSIONS

This evidence suggests a new potential treatment target for rehabilitative measures aimed at achieving functional improvements in patients with SSD by selectively leveraging the neuro-cognitive processing of reward.

Threat and Reward Imminence Processing in the Human Brain

In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. In addition, the extent to which aversive-related and appetitive-related processing engage distinct or overlapping circuits remains poorly understood. Here, we sought to investigate the dynamics of aversive and appetitive processing while male and female participants engaged in comparable trials involving threat avoidance or reward seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence. For example, in the aversive domain, we predicted that the bed nucleus of the stria terminalis (BST), but not the amygdala, would exhibit anticipatory responses given the role of the former in anxious apprehension. We also predicted that the periaqueductal gray (PAG) would exhibit threat-proximity responses based on its involvement in proximal-threat processes, and that the ventral striatum would exhibit threat-imminence responses given its role in threat escape in rodents. Overall, we uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the BST, PAG, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Whereas the ventral striatum generated anticipatory responses in the proximity of reward as expected, it also exhibited threat-related imminence responses. In fact, across multiple brain regions, we observed a main effect of arousal. In other words, we uncovered extensive temporally evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information regardless of valence, findings further supported by network analysis.SIGNIFICANCE STATEMENT In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. Here, we sought to investigate the dynamics of aversive/appetitive processing while participants engaged in trials involving threat avoidance or reward seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence. We uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the bed nucleus of the stria terminalis, periaqueductal gray, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Overall, we uncovered extensive temporally evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information regardless of valence.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

Valence-partitioned learning signals drive choice behavior and phenomenal subjective experience in humans

How the human brain generates conscious phenomenal experience is a fundamental problem. In particular, it is unknown how variable and dynamic changes in subjective affect are driven by interactions with objective phenomena. We hypothesize a neurocomputational mechanism that generates valence-specific learning signals associated with 'what it is like' to be rewarded or punished. Our hypothesized model maintains a partition between appetitive and aversive information while generating independent and parallel reward and punishment learning signals. This valence-partitioned reinforcement learning (VPRL) model and its associated learning signals are shown to predict dynamic changes in 1) human choice behavior, 2) phenomenal subjective experience, and 3) BOLD-imaging responses that implicate a network of regions that process appetitive and aversive information that converge on the ventral striatum and ventromedial prefrontal cortex during moments of introspection. Our results demonstrate the utility of valence-partitioned reinforcement learning as a neurocomputational basis for investigating mechanisms that may drive conscious experience.

Anhedonia and sensitivity to punishment in schizophrenia, depression and opiate use disorder

BACKGROUND

From a behavioural perspective anhedonia is defined as diminished interest in the engagement of pleasurable activities. Despite its presence across a range of psychiatric disorders, the cognitive processes that give rise to anhedonia remain unclear.

METHODS

Here we examine whether anhedonia is associated with learning from positive and negative outcomes in patients diagnosed with major depression, schizophrenia and opiate use disorder alongside a healthy control group. Responses in the Wisconsin Card Sorting Test - a task associated with healthy prefrontal cortex function - were fitted to the Attentional Learning Model (ALM) which separates learning from positive and negative feedback.

RESULTS

Learning from punishment, but not from reward, was negatively associated with anhedonia beyond other socio-demographic, cognitive and clinical variables. This impairment in punishment sensitivity was also associated with faster responses following negative feedback, independently of the degree of surprise.

LIMITATIONS

Future studies should test the longitudinal association between punishment sensitivity and anhedonia also in other clinical populations controlling for the effect of specific medications.

CONCLUSIONS

Together the results reveal that anhedonic subjects, because of their negative expectations, are less sensitive to negative feedbacks; this might lead them to persist in actions leading to negative outcomes.

Threat and reward imminence processing in the human brain

In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. In addition, the extent to which aversive- and appetitive-related processing engage distinct or overlapping circuits remains poorly understood. Here, we sought to investigate the dynamics of aversive and appetitive processing while male and female participants engaged in comparable trials involving threat-avoidance or reward-seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence . For example, in the aversive domain, we predicted that the bed nucleus of the stria terminalis (BST), but not the amygdala, would exhibit anticipatory responses given the role of the former in anxious apprehension. We also predicted that the periaqueductal gray (PAG) would exhibit threat-proximity responses based on its involvement in proximal-threat processes, and that the ventral striatum would exhibit threat-imminence responses given its role in threat escape in rodents. Overall, we uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the BST, PAG, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Whereas the ventral striatum generated anticipatory responses in the proximity of reward as expected, it also exhibited threat-related imminence responses. In fact, across multiple brain regions, we observed a main effect of arousal. In other words, we uncovered extensive temporally-evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information irrespective of valence, findings further supported by network analysis.

Significance Statement

In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. Here, we sought to investigate the dynamics of aversive/appetitive processing while participants engaged in trials involving threat-avoidance or reward-seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence . We uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the bed nucleus of the stria terminalis, periaqueductal gray, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Overall, we uncovered extensive temporally-evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information irrespective of valence.

Laser stimulation of the skin for quantitative study of decision-making and motivation

Neuroeconomics studies how decision-making is guided by the value of rewards and punishments. But to date, little is known about how noxious experiences impact decisions. A challenge is the lack of an aversive stimulus that is dynamically adjustable in intensity and location, readily usable over many trials in a single experimental session, and compatible with multiple ways to measure neuronal activity. We show that skin laser stimulation used in human studies of aversion can be used for this purpose in several key animal models. We then use laser stimulation to study how neurons in the orbitofrontal cortex (OFC), an area whose many roles include guiding decisions among different rewards, encode the value of rewards and punishments. We show that some OFC neurons integrated the positive value of rewards with the negative value of aversive laser stimulation, suggesting that the OFC can play a role in more complex choices than previously appreciated.

Intracerebral mechanisms explaining the impact of incidental feedback on mood state and risky choice

Identifying factors whose fluctuations are associated with choice inconsistency is a major issue for rational decision theory. Here, we investigated the neuro-computational mechanisms through which mood fluctuations may bias human choice behavior. Intracerebral EEG data were collected in a large group of subjects (n=30) while they were performing interleaved quiz and choice tasks that were designed to examine how a series of unrelated feedbacks affect decisions between safe and risky options. Neural baseline activity preceding choice onset was confronted first to mood level, estimated by a computational model integrating the feedbacks received in the quiz task, and then to the weighting of option attributes, in a computational model predicting risk attitude in the choice task. Results showed that (1) elevated broadband gamma activity (BGA) in the ventromedial prefrontal cortex (vmPFC) and dorsal anterior insula (daIns) was respectively signaling periods of high and low mood, (2) increased vmPFC and daIns BGA respectively promoted and tempered risk taking by overweighting gain vs. loss prospects. Thus, incidental feedbacks induce brain states that correspond to different moods and bias the evaluation of risky options. More generally, these findings might explain why people experiencing positive (or negative) outcome in some part of their life tend to expect success (or failure) in any other.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Dissociable influences of reward and punishment on adaptive cognitive control

To invest effort into any cognitive task, people must be sufficiently motivated. Whereas prior research has focused primarily on how the cognitive control required to complete these tasks is motivated by the potential rewards for success, it is also known that control investment can be equally motivated by the potential negative consequence for failure. Previous theoretical and experimental work has yet to examine how positive and negative incentives differentially influence the manner and intensity with which people allocate control. Here, we develop and test a normative model of control allocation under conditions of varying positive and negative performance incentives. Our model predicts, and our empirical findings confirm, that rewards for success and punishment for failure should differentially influence adjustments to the evidence accumulation rate versus response threshold, respectively. This dissociation further enabled us to infer how motivated a given person was by the consequences of success versus failure.

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation

BACKGROUND

Drug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.

METHODS

We investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.

RESULTS

Computational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.

CONCLUSION

Our findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.

Absence of covert face valuation in Autism

Autism is a neurodevelopmental condition defined on clinical criteria related to diminished social reciprocity and stereotyped behavior. An influential view explains autism as a social motivation disorder characterized by less attention paid to the social environment and less pleasure experienced with social rewards. However, experimental attempts to validate this theory, by testing the impact of social reward on behavioral choice and brain activity, has yielded mixed results, possibly due to variations in how explicit instructions were about task goals. Here, we specified the putative motivation deficit as an absence of spontaneous valuation in the social domain, unexplained by inattention and correctible by explicit instruction. Since such deficit cannot be assessed with behavioral measures, we used functional neuroimaging (fMRI) to readout covert subjective values, assigned to social and nonsocial stimuli (faces and objects), either explicitly asked to participants (during a likeability judgment task) or not (during age or size estimation tasks). Value-related neural activity observed for objects, or for faces under explicit instructions, was very similar in autistic and control participants, with an activation peak in the ventromedial prefrontal cortex (vmPFC), known as a key node of the brain valuation system. The only difference observed in autistic participants was an absence of the spontaneous valuation normally triggered by faces, even when they were attended for age estimation. Our findings, therefore, suggest that in autism, social stimuli might fail to trigger the automatic activation of the brain valuation system.

Anatomical dissociation of intracerebral signals for reward and punishment prediction errors in humans

Whether maximizing rewards and minimizing punishments rely on distinct brain systems remains debated, given inconsistent results coming from human neuroimaging and animal electrophysiology studies. Bridging the gap across techniques, we recorded intracerebral activity from twenty participants while they performed an instrumental learning task. We found that both reward and punishment prediction errors (PE), estimated from computational modeling of choice behavior, correlate positively with broadband gamma activity (BGA) in several brain regions. In all cases, BGA scaled positively with the outcome (reward or punishment versus nothing) and negatively with the expectation (predictability of reward or punishment). However, reward PE were better signaled in some regions (such as the ventromedial prefrontal and lateral orbitofrontal cortex), and punishment PE in other regions (such as the anterior insula and dorsolateral prefrontal cortex). These regions might therefore belong to brain systems that differentially contribute to the repetition of rewarded choices and the avoidance of punished choices.

Whether maximizing rewards and minimizing punishments rely on distinct brain learning systems remains debated. Here, using intracerebral recordings in humans, the authors provide evidence for brain regions differentially engaged in signaling reward and punishment prediction errors that prescribe repetition versus avoidance of past choices.

The description-experience gap: a challenge for the neuroeconomics of decision-making under uncertainty

The experimental investigation of decision-making in humans relies on two distinct types of paradigms, involving either description- or experience-based choices. In description-based paradigms, decision variables (i.e. payoffs and probabilities) are explicitly communicated by means of symbols. In experience-based paradigms decision variables are learnt from trial-by-trial feedback. In the decision-making literature, 'description-experience gap' refers to the fact that different biases are observed in the two experimental paradigms. Remarkably, well-documented biases of description-based choices, such as under-weighting of rare events and loss aversion, do not apply to experience-based decisions. Here, we argue that the description-experience gap represents a major challenge, not only to current decision theories, but also to the neuroeconomics research framework, which relies heavily on the translation of neurophysiological findings between human and non-human primate research. In fact, most non-human primate neurophysiological research relies on behavioural designs that share features of both description- and experience-based choices. As a consequence, it is unclear whether the neural mechanisms built from non-human primate electrophysiology should be linked to description-based or experience-based decision-making processes. The picture is further complicated by additional methodological gaps between human and non-human primate neuroscience research. After analysing these methodological challenges, we conclude proposing new lines of research to address them. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

A molecular insight into the dissociable regulation of associative learning and motivation by the synaptic protein neuroligin-1

Background

In a changing environment, a challenge for the brain is to flexibly guide adaptive behavior towards survival. Complex behavior and the underlying neural computations emerge from the structural components of the brain across many levels: circuits, cells, and ultimately the signaling complex of proteins at synapses. In line with this logic, dynamic modification of synaptic strength or synaptic plasticity is widely considered the cellular level implementation for adaptive behavior such as learning and memory. Predominantly expressed at excitatory synapses, the postsynaptic cell-adhesion molecule neuroligin-1 (Nlgn1) forms trans-synaptic complexes with presynaptic neurexins. Extensive evidence supports that Nlgn1 is essential for NMDA receptor transmission and long-term potentiation (LTP), both of which are putative synaptic mechanisms underlying learning and memory. Here, employing a comprehensive battery of touchscreen-based cognitive assays, we asked whether impaired NMDA receptor transmission and LTP in mice lacking Nlgn1 does in fact disrupt decision-making. To this end, we addressed two key decision problems: (i) the ability to learn and exploit the associative structure of the environment and (ii) balancing the trade-off between potential rewards and costs, or positive and negative utilities of available actions.

Results

We found that the capacity to acquire complex associative structures and adjust learned associations was intact. However, loss of Nlgn1 alters motivation leading to a reduced willingness to overcome effort cost for reward and an increased willingness to exert effort to escape an aversive situation. We suggest Nlgn1 may be important for balancing the weighting on positive and negative utilities in reward-cost trade-off.

Conclusions

Our findings update canonical views of this key synaptic molecule in behavior and suggest Nlgn1 may be essential for regulating distinct cognitive processes underlying action selection. Our data demonstrate that learning and motivational computations can be dissociated within the same animal model, from a detailed behavioral dissection. Further, these results highlight the complexities in mapping synaptic mechanisms to their behavioral consequences, and the future challenge to elucidate how complex behavior emerges through different levels of neural hardware.

The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

The term "psychological well-being" is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one's environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

Subjective value then confidence in human ventromedial prefrontal cortex

Two fundamental goals of decision making are to select actions that maximize rewards while minimizing costs and to have strong confidence in the accuracy of a judgment. Neural signatures of these two forms of value: the subjective value (SV) of choice alternatives and the value of the judgment (confidence), have both been observed in ventromedial prefrontal cortex (vmPFC). However, the relationship between these dual value signals and their relative time courses are unknown. Twenty-eight men and women underwent fMRI while performing a two-phase approach-avoidance (Ap-Av) task with mixed-outcomes of monetary rewards paired with painful shock stimuli. Neural responses were measured during offer valuation (offer phase) and choice valuation (commit phase) and analyzed with respect to observed decision outcomes, model-estimated SV and confidence. During the offer phase, vmPFC tracked SV and the decision but not confidence. During the commit phase, vmPFC tracked confidence, computed as the quadratic extension of SV, but not the offer valuation nor the decision. In fact, vmPFC responses from the commit phase were selective for confidence even for reject decisions wherein confidence and SV are inversely related. Conversely, activation of the cognitive control network, including within lateral prefrontal cortex (lPFC) and dorsal anterior cingulate cortex (dACC) was associated with ambivalence, during both the offer and commit phases. Taken together, our results reveal complementary representations in vmPFC during value-based decision making that temporally dissociate such that offer valuation (SV) emerges before decision valuation (confidence).

Goal congruency dominates reward value in accounting for behavioral and neural correlates of value-based decision-making

When choosing between options, whether menu items or career paths, we can evaluate how rewarding each one will be, or how congruent it is with our current choice goal (e.g., to point out the best option or the worst one.). Past decision-making research interpreted findings through the former lens, but in these experiments the most rewarding option was always most congruent with the task goal (choosing the best option). It is therefore unclear to what extent expected reward vs. goal congruency can account for choice value findings. To deconfound these two variables, we performed three behavioral studies and an fMRI study in which the task goal varied between identifying the best vs. the worst option. Contrary to prevailing accounts, we find that goal congruency dominates choice behavior and neural activity. We separately identify dissociable signals of expected reward. Our findings call for a reinterpretation of previous research on value-based choice.

Decision-making research has confounded the reward value of options with their goal-congruency, as the task goal was always to pick the most rewarding option. Here, authors separately asked participants to select the least rewarding of a set of options, revealing a dominant role for goal congruency.

Strive to Win or Not to Lose? Age-Related Differences in Framing Effects on Effort-Based Decision-Making

Objectives

This study sought to assess how framing effects modulate age-related differences in effort-based decision-making. Consistent with the selection, optimization, and compensation (SOC) model’s loss prevention account of aging, we predicted that older adults would be more willing to select high-effort options in loss contexts than gain contexts.

Method

Older and younger adults completed the effort expenditure for rewards task (EEfRT) in either a gain or loss context. The EEfRT is an effort-based decision-making paradigm in which participants choose between a low-effort, “easy” option and a high-effort, “hard” option for several trials. The probability and value of an outcome varies on a trial-by-trial basis.

Results

The results supported our prediction and the SOC model. Older adults chose more high-effort, difficult options in loss frames than gain frames. Older adults also chose more low-effort, easy options than younger adults in gain contexts, but did not differ from younger adults in loss contexts.

Discussion

These findings demonstrate that framing effects impact older adults’ effort-based decisions. Older adults appear willing to incur a greater “cost” in the form of effort to prevent a loss than to attain a reward.

Reward anticipation and punishment anticipation are instantiated in the brain via opponent mechanisms

fMRI investigations have examined the extent to which reward and punishment motivation are associated with common or opponent neural systems, but such investigations have been limited by confounding variables and methodological constraints. The present study aimed to address limitations of earlier approaches and more comprehensively evaluate the extent to which neural activation associated with reward and punishment motivation reflects opponent or shared systems. Participants completed a modified monetary incentive delay task, which involved the presentation of a cue followed by a target to which participants were required to make a speeded button press. Using a factorial design, cues indicated whether monetary reward and/or loss (i.e., cues signaled probability of reward, punishment, both, or neither) could be expected depending upon response speed. Neural analyses evaluated evidence of (a) directionally opposing effects by testing for regions of differential activation for reward and punishment anticipation, (b) mutual inhibition by testing for interactive effects of reward and punishment anticipation within a factorial design, and (c) opposing effects on shared outputs via a psychophysiological interaction analysis. Evidence supporting all three criteria for opponent systems was obtained. Collectively, present findings support conceptualizing reward and punishment motivation as opponent forces influencing brain and behavior and indicate that shared activation does not suggest the operation of a common neural mechanism instantiating reward and punishment motivation.

Positive-blank versus negative-blank feedback learning in children and adults

In positive-blank feedback learning, positive feedback is given to a correct response whereas blank feedback is given to an incorrect response. Conversely, in negative-blank feedback learning, blank feedback is given to a correct response and negative feedback to an incorrect response. As blank feedback might be subjectively interpreted as signalling a correct response, negative-blank feedback might be more informative than positive-blank feedback, and thus may result in better performance. However, positive-blank feedback might also be superior as it motivates the learner in lengthy tasks. These "information" and "motivation" accounts were tested in a two-block feedback learning paradigm. In the first block, that is, when the task duration was still short, children but not adults profited more from negative than from positive feedback. The results in children thus support the information account. In the second block, that is, when the task duration had become longer, children and adults profited more from positive feedback, thereby supporting the motivation account. Results are discussed in light of behavioural and neuroscientific theories on feedback learning.

In Cocaine Dependence, Neural Prediction Errors During Loss Avoidance Are Increased With Cocaine Deprivation and Predict Drug Use

BACKGROUND

In substance-dependent individuals, drug deprivation and drug use trigger divergent behavioral responses to environmental cues. These divergent responses are consonant with data showing that short- and long-term adaptations in dopamine signaling are similarly sensitive to state of drug use. The literature suggests a drug state-dependent role of learning in maintaining substance use; evidence linking dopamine to both reinforcement learning and addiction provides a framework to test this possibility.

METHODS

In a randomized crossover design, 22 participants with current cocaine use disorder completed a probabilistic loss-learning task during functional magnetic resonance imaging while on and off cocaine (44 sessions). Another 54 participants without Axis I psychopathology served as a secondary reference group. Within-drug state and paired-subjects' learning effects were assessed with computational model-derived individual learning parameters. Model-based neuroimaging analyses evaluated effects of drug use state on neural learning signals. Relationships among model-derived behavioral learning rates (α+, α-), neural prediction error signals (δ+, δ-), cocaine use, and desire to use were assessed.

RESULTS

During cocaine deprivation, cocaine-dependent individuals exhibited heightened positive learning rates (α+), heightened neural positive prediction error (δ+) responses, and heightened association of α+ with neural δ+ responses. The deprivation-enhanced neural learning signals were specific to successful loss avoidance, comparable to participants without psychiatric conditions, and mediated a relationship between chronicity of drug use and desire to use cocaine.

CONCLUSIONS

Neurocomputational learning signals are sensitive to drug use status and suggest that heightened reinforcement by successful avoidance of negative outcomes may contribute to drug seeking during deprivation. More generally, attention to drug use state is important for delineating substrates of addiction.

Cue awareness in avoiding effortful control

Based on a recent metacognitive account, cognitive effort is the result of an inferential evaluation made over explicitly available cues. Following from this account, we present here a pre-registered experiment that tested the specific hypothesis that explicit awareness of cues that are aligned with cognitive demand is a prerequisite in avoiding effortful lines of action. We attempted to modulate levels of effort avoidance behavior by introducing an incentive (between-subjects) to monitor two lines of action that, unbeknownst to individuals, varied in the probability of a task switch. Importantly, previous research has demonstrated that the difference in these probabilities is relatively opaque to individuals. We did not find strong evidence for our incentive manipulation having an effect on demand avoidance as indexed by individuals' choices in a block of the task where avoiding effort was instructed. However, we do find that being aware of the task-switching cue appears to increase the likelihood of demand avoidance. We consider these results within the context of the metacognition of cognitive effort.

Lay conceptions of norm-based approach and avoidance motivation: Implications for the performance-approach and performance-avoidance goal relation

OBJECTIVE

Performance-approach goals and performance-avoidance goals are conceptually distinct, but they are often moderately or even highly positively correlated. The present research examines lay conceptions of approach and avoidance motivation as a moderator of this intergoal relation.

METHOD

Study 1 (N = 281) assessed whether participants considered norm-based approach motivation as being the same or different from norm-based avoidance motivation and tested these conceptions as a moderator of the performance goal correlation. Study 2 (N = 990) measured and experimentally manipulated lay conceptions.

RESULTS

In both studies, individuals who viewed approach and avoidance motivation as different exhibited a smaller performance goal correlation and lower performance-based goal adoption than those who viewed approach and avoidance goals as the same. Findings from experimentally manipulated conceptions provided further clarity regarding the precise nature of the relations and mean differences observed. Specifically, moderation was driven by the different condition (where the differences between approach and avoidance were highlighted).

CONCLUSIONS

This research sheds light on the nature and magnitude of the focal performance-based goal correlation and highlights the value of attending to lay conceptions of approach and avoidance motivation as well as lay conceptions of ability.

Asymmetric neural tracking of gain and loss magnitude during adolescence

Adolescence has been characterized as a developmental period of heightened reward seeking and attenuated aversive processing. However, it remains unclear how the neural bases of distinct outcome valuation processes shift during this stage of the lifespan. A total of 74 participants ranging in age from 13 to 20 years completed a value-modulated functional magnetic resonance imaging (fMRI) task in which participants earn low and high magnitude monetary outcomes to test whether gain and loss magnitude tracking-the neural representation of relative value in context-change differentially over this age span. Results revealed that gain and loss magnitude tracking follow asymmetric developmental trajectories. Gain magnitude tracking is elevated in the striatum during early adolescence and then decreases with age. By contrast, loss magnitude tracking in the anterior insula follows a quadratic pattern, undergoing a temporary attenuation during mid-late adolescence. A typical comparison of gain vs loss outcomes (collapsing over magnitude effects) showed robust activity across a suite of brain regions sensitive to value based on prior work including the ventral striatum, but they exhibited no changes with age. These findings suggest that value coding subprocesses follow divergent developmental paths across adolescence, which may contribute to normative shifts in adolescent motivated behavior.

Neuro-computational account of how mood fluctuations arise and affect decision making

The influence of mood on choices is a well-established but poorly understood phenomenon. Here, we suggest a three-fold neuro-computational account: (1) the integration of positive and negative events over time induce mood fluctuations, (2) which are underpinned by variations in the baseline activities of critical brain valuation regions, (3) which in turn modulate the relative weights assigned to key dimensions of choice options. We validate this model in healthy participants, using feedback in a quiz task to induce mood fluctuations, and a choice task (accepting vs. declining a motor challenge) to reveal their effects. Using fMRI, we demonstrate the pivotal role of the ventromedial prefrontal cortex and anterior insula, in which baseline activities respectively increase and decrease with theoretical mood level and respectively enhance the weighting of potential gains and losses during decision making. The same mechanisms might explain how decisions are biased in mood disorders at longer timescales.

Interactions of Motivation and Cognitive Control

There is general agreement that both motivation and cognitive control play critical roles in shaping goal-directed behavior, but only recently has scientific interest focused around the question of motivation-control interactions. Here we briefly survey this literature, organizing contemporary findings around three issues: 1) whether motivation preferentially impacts cognitive control processes, 2) the neural mechanisms that underlie motivation-cognition interactions, and 3) why motivation might be relevant for overcoming the costs of control. Dopamine (DA) is discussed as a key neuromodulator in these motivation-cognition interactions. We conclude by highlighting open issues, specifically Pavlovian versus instrumental control distinctions and effects of motivational valence and conflict, which could benefit from future research attention.

The neuroscience of morality and social decision-making

Across cultures humans care deeply about morality and create institutions, such as criminal courts, to enforce social norms. In such contexts, judges and juries engage in complex social decision-making to ascertain a defendant's capacity, blameworthiness, and culpability. Cognitive neuroscience investigations have begun to reveal the distributed neural networks which interact to implement moral judgment and social decision-making, including systems for reward learning, valuation, mental state understanding, and salience processing. These processes are fundamental to morality, and their underlying neural mechanisms are influenced by individual differences in empathy, caring and justice sensitivity. This new knowledge has important implication in legal settings for understanding how triers of fact reason. Moreover, recent work demonstrates how disruptions within the social decision-making network facilitate immoral behavior, as in the case of psychopathy. Incorporating neuroscientific methods with psychology and clinical neuroscience has the potential to improve predictions of recidivism, future dangerousness, and responsivity to particular forms of rehabilitation.

Premonitory urges and tics in Tourette syndrome: computational mechanisms and neural correlates

Tourette syndrome is characterized by open motor behaviors - tics - but another crucial aspect of the disorder is the presence of premonitory urges: uncomfortable sensations that typically precede tics and are temporarily alleviated by tics. We review the evidence implicating the somatosensory cortices and the insula in premonitory urges and the motor cortico-basal ganglia-thalamo-cortical loop in tics. We consider how these regions interact during tic execution, suggesting that the insula plays an important role as a nexus linking the sensory and emotional character of premonitory urges with their translation into tics. We also consider how these regions interact during tic learning, integrating the neural evidence with a computational perspective on how premonitory-urge alleviation reinforces tics.

Behavioral Inefficiency on a Risky Decision-Making Task in Adulthood after Adolescent Intermittent Ethanol Exposure in Rats

Adolescence is a period of development in neural circuits that are critical for adult functioning. There is a relationship between alcohol exposure and risky decision-making, though the enduring effects of adolescent ethanol exposure on risky decision-making in adulthood have not been fully explored. Studies using positive reinforcement have shown that adolescent intermittent ethanol (AIE) exposure results in higher levels of risky decision-making in adulthood, but the effects of AIE on punishment-mediated decision-making have not been explored. Adolescent rats were exposed to AIE or saline vehicle across a 16-day period, and then allowed to mature into adulthood. They were then trained to lever press for food reward and were assessed for risky decision-making by pairing increased levels of food reward with the probability of footshock punishment. AIE did not alter punishment-mediated risky decision-making. However, it did result in a significant increase in the delay to lever pressing. This finding is consistent with previous reports, using other behavioral tasks, which show decreased behavioral efficiency in adulthood after AIE. These findings indicate that AIE increases behavioral inefficiency, but not punishment-mediated risk-taking, in adulthood. Thus they contribute to a more nuanced understanding of the long-term effects of AIE on adult behavior.

Stress and Decision Making: Effects on Valuation, Learning, and Risk-taking

A wide range of stressful experiences can influence human decision making in complex ways beyond the simple predictions of a fight-or-flight model. Recent advances may provide insight into this complicated interaction, potentially in directions that could result in translational applications. Early research suggests that stress exposure influences basic neural circuits involved in reward processing and learning, while also biasing decisions towards habit and modulating our propensity to engage in risk-taking. That said, a substantial array of theoretical and methodological considerations in research on the topic challenge strong cross study comparisons necessary for the field to move forward. In this review we examine the multifaceted stress construct in the context of human decision making, emphasizing stress' effect on valuation, learning, and risk-taking.

Higher Heart-Rate Variability Is Associated with Ventromedial Prefrontal Cortex Activity and Increased Resistance to Temptation in Dietary Self-Control Challenges

Higher levels of self-control in decision making have been linked to better psychosocial and physical health. A similar link to health outcomes has been reported for heart-rate variability (HRV), a marker of physiological flexibility. Here, we sought to link these two, largely separate, research domains by testing the hypothesis that greater HRV would be associated with better dietary self-control in humans. Specifically, we examined whether total HRV at sedentary rest (measured as the SD of normal-to-normal intervals) can serve as a biomarker for the neurophysiological adaptability that putatively underlies self-controlled behavior. We found that HRV explained a significant portion of the individual variability in dietary self-control, with individuals having higher HRV being better able to downregulate their cravings in the face of taste temptations. Furthermore, HRV was associated with activity patterns in the ventromedial prefrontal cortex (vmPFC), a key node in the brain's valuation and decision circuitry. Specifically, individuals with higher HRV showed both higher overall vmPFC blood-oxygen-level-dependent activity and attenuated taste representations when presented with a dietary self-control challenge. Last, the behavioral and neural associations with HRV were consistent across both our stress induction and control experimental conditions. The stability of this association across experimental conditions suggests that HRV may serve as both a readily obtainable and robust biomarker for self-control ability across environmental contexts.

SIGNIFICANCE STATEMENT

Self-control is associated with better health, but behavioral and psychometric self-control measures allow only indirect associations with health outcomes and may be distorted by reporting bias. We tested whether resting heart-rate variability (HRV), a physiological indicator of psychological and physical health, can predict individual differences in dietary self-control in humans. We found that higher HRV was associated with better self-control and improved predictions of choice behavior. Specifically, higher HRV was associated with more effective downregulation of taste temptations, and with a diminished neural representation of taste temptations during self-control challenges. Our results suggest that HRV may serve as an easily acquired, noninvasive, and low-cost biomarker for self-control ability.

Divergence and Convergence of Risky Decision Making Across Prospective Gains and Losses: Preferences and Strategies

People choose differently when facing potential gains than when facing potential losses. Clear gross differences in decision making between gains and losses have been empirically demonstrated in numerous studies (e.g., framing effect, risk preference, loss aversion). However, theories maintain that there are strong underlying connections (e.g., reflection effect). We investigated the relationship between gains and losses decision making, examining risk preferences, and choice strategies (the reliance on option information) using a monetary gamble task with interleaved trials. For risk preferences, participants were on average risk averse in the gains domain and risk neutral/seeking in the losses domain. We specifically tested for a theoretically hypothesized correlation between individual risk preferences across the gains and losses domains (the reflection effect), but found no significant relationship in the predicted direction. Interestingly, despite the lack of reflected risk preferences, cross-domain risk preferences were still informative of individual choice behavior. For choice strategies, in both domains participants relied more heavily on the maximizing strategy than the satisficing strategy, with increased reliance on the maximizing strategy in the losses domain. Additionally, while there is no mathematical reliance between the risk preference and strategy metrics, within both domains there were significant relationships between risk preferences and strategies-the more participants relied upon the maximizing strategy the more risk neutral they were (equating value and utility maximization). These results demonstrate the complexity of gains and losses decision making, indicating the apparent contradiction that their underlying cognitive/neural processes are both dissociable and overlapping.