Neural systems underlying aversive conditioning in humans with primary and secondary reinforcers

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Reinforcement learning is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g., money, points) that are later exchanged for primary reinforcers (e.g., food, drink). Although symbolic reinforcers are ubiquitous in our daily lives, widely used in laboratory tasks because they can be motivating, mechanisms by which they become motivating are less understood. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g., the current number of accumulated tokens, choice options, task epoch, trials since the last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to then correlate with the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n = 5 monkeys, three males and two females). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in the state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.

The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Examining the Association between Punishment and Reward Sensitivity and Response Inhibition to Previously-Incentivized Cues across Development

Processing and learning from affective cues to guide goal-directed behavior may be particularly important during adolescence; yet the factors that promote and/or disrupt the ability to integrate value in order to guide decision making across development remain unclear. The present study (N = 1046) assessed individual difference factors (self-reported punishment and reward sensitivity) related to whether previously-rewarded and previously-punished cues differentially impact goal-directed behavior (response inhibition) in a large developmental sample. Participants were between the ages of 8-21 years (Mage = 14.29, SD = 3.97, 50.38% female). Previously-rewarded cues improved response inhibition among participants age 14 and older. Further, punishment sensitivity predicted overall improved response inhibition among participants aged 10 to 18. The results highlight two main factors that are associated with improvements in the ability to integrate value to guide goal-directed behaviour - cues in the environment (e.g., reward-laden cues) and individual differences in punishment sensitivity. These findings have implications for both educational and social policies aimed at characterizing the ways in which youth integrate value to guide decision making.

Computational mechanisms underlying motivation to earn symbolic reinforcers

Reinforcement learning (RL) is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g. money, points) that can later be exchanged for primary reinforcers (e.g. food, drink). Although symbolic reinforcers are motivating, little is understood about the neural or computational mechanisms underlying the motivation to earn them. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g. current number of accumulated tokens, choice options, task epoch, trials since last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to capture the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n=5 monkeys). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.

Emotion, motivation, decision-making, the orbitofrontal cortex, anterior cingulate cortex, and the amygdala

The orbitofrontal cortex and amygdala are involved in emotion and in motivation, but the relationship between these functions performed by these brain structures is not clear. To address this, a unified theory of emotion and motivation is described in which motivational states are states in which instrumental goal-directed actions are performed to obtain rewards or avoid punishers, and emotional states are states that are elicited when the reward or punisher is or is not received. This greatly simplifies our understanding of emotion and motivation, for the same set of genes and associated brain systems can define the primary or unlearned rewards and punishers such as sweet taste or pain. Recent evidence on the connectivity of human brain systems involved in emotion and motivation indicates that the orbitofrontal cortex is involved in reward value and experienced emotion with outputs to cortical regions including those involved in language, and is a key brain region involved in depression and the associated changes in motivation. The amygdala has weak effective connectivity back to the cortex in humans, and is implicated in brainstem-mediated responses to stimuli such as freezing and autonomic activity, rather than in declarative emotion. The anterior cingulate cortex is involved in learning actions to obtain rewards, and with the orbitofrontal cortex and ventromedial prefrontal cortex in providing the goals for navigation and in reward-related effects on memory consolidation mediated partly via the cholinergic system.

Primary rewards and aversive outcomes have comparable effects on attentional bias

Attention is biased toward stimuli previously associated with reward. The same is true for aversive conditioning; stimuli previously associated with an aversive outcome also bias attention, suggesting that motivational salience guides attention. Most research that supports this conclusion has manipulated monetary gain-a secondary reinforcer-for reward learning, and electric shocks-a primary punisher-for aversive conditioning, making it difficult to directly compare their influence on attention. Therefore, in the present study, we matched for reinforcer dimensions by using primary taste as reinforcers/punishers and assessed their influence on attention. In a training phase, participants learned to associate three colors with sweet juice (reward), salt water (aversive), and no outcome (neutral), respectively. The two primary reinforcers were equated for valence based on choices made in a prior decision-making task. In a later test phase, these three colors were used for targets and distractors in a task in which participants oriented to a shape-defined target. An attentional bias in favor of the aversively conditioned and reward-associated colors was evident when comparing to the neutral color. Importantly, a direct comparison of rewarded and aversive stimuli revealed no significant differences. These results suggest that when matched for reinforcer dimensions and valence, reward and aversive outcomes bias attention in a similar manner and their effects are comparable, providing further evidence in support of the motivational salience account of learning-dependent attention. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Dissociable feedback valence effects on frontal midline theta during reward gain versus threat avoidance learning

While frontal midline theta (FMθ) has been associated with threat processing, with cognitive control in the context of anxiety, and with reinforcement learning, most reinforcement learning studies on FMθ have used reward rather than threat-related stimuli as reinforcer. Accordingly, the role of FMθ in threat-related reinforcement learning is largely unknown. Here, n = 23 human participants underwent one reward-, and one punishment-, based reversal learning task, which differed only with regard to the kind of reinforcers that feedback was tied to (i.e., monetary gain vs. loud noise burst, respectively). In addition to single-trial EEG, we assessed single-trial feedback expectations based on both a reinforcement learning computational model and trial-by-trial subjective feedback expectation ratings. While participants' performance and feedback expectations were comparable between the reward and punishment tasks, FMθ was more reliably amplified to negative vs. positive feedback in the reward vs. punishment task. Regressions with feedback valence, computationally derived, and self-reported expectations as predictors and FMθ as criterion further revealed that trial-by-trial variations in FMθ specifically relate to reward-related feedback-valence and not to threat-related feedback or to violated expectations/prediction errors. These findings suggest that FMθ as measured in reinforcement learning tasks may be less sensitive to the processing of events with direct relevance for fear and anxiety.

Stress effects on memory retrieval of aversive and appetitive instrumental counterconditioning in men

Extinction training creates a second inhibitory memory trace and effectively reduces conditioned responding. However, acute stress inhibits the retrieval of this extinction memory trace. It is not known whether this also applies to other forms of associative learning such as instrumental counterconditioning, where previously learned associations are reversed and paired with the opposite valence. Therefore, the current preregistered study investigates whether stress decreases the retrieval of instrumental counterconditioning memories with aversive and appetitive consequences. Fifty-two healthy men were randomly assigned to either a stress or control group and took part in a two-day instrumental learning paradigm. During a first phase, participants learned that pressing specific buttons in response to the presentation of four neutral stimuli either leads to gaining or losing money. During a second phase, two stimuli reversed their contingencies (counterconditioning). One day later, participants were exposed to acute stress or a control condition prior to the same task, which no longer included feedback about gains or losses. Stressed participants showed more approach behavior towards appetitive and less avoidance behavior towards aversive stimuli as compared to non-stressed participants. Our findings indicate that stress effects on memory retrieval differ depending on the associative learning approach in men. These differences might be related to stress effects on decision making and different motivational systems involved.

Human threat circuits: Threats of pain, aggressive conspecific, and predator elicit distinct BOLD activations in the amygdala and hypothalamus

INTRODUCTION

Threat processing, enabled by threat circuits, is supported by a remarkably conserved neural architecture across mammals. Threatening stimuli relevant for most species include the threat of being attacked by a predator or an aggressive conspecific and the threat of pain. Extensive studies in rodents have associated the threats of pain, predator attack and aggressive conspecific attack with distinct neural circuits in subregions of the amygdala, the hypothalamus and the periaqueductal gray. Bearing in mind the considerable conservation of both the anatomy of these regions and defensive behaviors across mammalian species, we hypothesized that distinct brain activity corresponding to the threats of pain, predator attack and aggressive conspecific attack would also exist in human subcortical brain regions.

METHODS

Forty healthy female subjects underwent fMRI scanning during aversive classical conditioning. In close analogy to rodent studies, threat stimuli consisted of painful electric shocks, a short video clip of an attacking bear and a short video clip of an attacking man. Threat processing was conceptualized as the expectation of the aversive stimulus during the presentation of the conditioned stimulus.

RESULTS

Our results demonstrate differential brain activations in the left and right amygdala as well as in the left hypothalamus for the threats of pain, predator attack and aggressive conspecific attack, for the first time showing distinct threat-related brain activity within the human subcortical brain. Specifically, the threat of pain showed an increase of activity in the left and right amygdala and the left hypothalamus compared to the threat of conspecific attack (pain > conspecific), and increased activity in the left amygdala compared to the threat of predator attack (pain > predator). Threat of conspecific attack revealed heightened activity in the right amygdala, both in comparison to threat of pain (conspecific > pain) and threat of predator attack (conspecific > predator). Finally, for the condition threat of predator attack we found increased activity in the bilateral amygdala and the hypothalamus when compared to threat of conspecific attack (predator > conspecific). No significant clusters were found for the contrast predator attack > pain.

CONCLUSION

Results suggest that threat type-specific circuits identified in rodents might be conserved in the human brain.

The skin conductance response indicating pain relief is independent of self or social influence on pain

Pain relief is defined as the ease of pain and is thus highly relevant for clinical applications and everyday life. Given that pain relief is based on the cessation of an aversive pain experience, it is reasonable to assume that pain relief learning would also be shaped by factors that alter subjective and physiological pain responses, such as social presence or a feeling of control. To date, it remains unclear whether and how factors that shape autonomic pain responses might affect pain relief learning. Here, we investigated how pain relief learning is shaped by two important factors known to modulate pain responses, i.e. social influence and controllability of pain. Skin conductance responses (SCRs) were recorded while participants learned to associate a formerly neutral stimulus with pain relief under three different pain conditions. In the social‐influence condition (N = 34), the pain stimulation could be influenced by another person’s decisions. In the self‐influence condition (N = 31), the participants themselves could influence the pain stimulation. Finally, in the no‐influence condition (N = 32), pain stimulation was simply delivered without any influence. According to our results, the SCRs elicited by the stimulus that was associated with pain relief were significantly smaller compared to the SCRs elicited by a neutral control stimulus, indicating pain relief learning. However, there was no significant difference in the pain relief learning effect across the groups. These results suggest that physiological pain relief learning in humans is not significantly influenced by social influence and pain controllability.

The contents of this page will be shown on the eTOC on the online version only. It will not be published as part of the article PDF. Subjective and physiological pain responses can be altered by social support or pain controllability. Pain relief strongly depends on the perceived pain, but we found that it is less influenceable. Neither social nor self‐influence changed the magnitude of physiological pain‐relief responses. We conclude that pain relief is a basic mechanism not easily influenced by circumstantial manipulations.

Effective Connectivity during an Avoidance-Based Pavlovian-to-Instrumental Transfer Task

Pavlovian-to-instrumental transfer (PIT) refers to a phenomenon whereby a classically conditioned stimulus (CS) impacts the motivational salience of instrumental behavior. We examined behavioral response patterns and functional magnetic resonance imaging (fMRI) based effective connectivity during an avoidance-based PIT task. Eleven participants (8 females; Mage = 28.2, SD = 2.8, range = 25-32 years) completed the task. Effective connectivity between a priori brain regions engaged during the task was determined using hemodynamic response function group iterative multiple model estimation (HRF-GIMME). Participants exhibited behavior that was suggestive of specific PIT, a CS previously associated with a reinforcing outcome increased instrumental responding directed at the same outcome. We did not find evidence for general PIT; a CS did not significantly increase instrumental responding towards a different but related outcome. Using HRF-GIMME, we recovered effective connectivity maps among corticostriatal circuits engaged during the task. Group-level paths revealed directional effects from left putamen to right insula and from right putamen to right cingulate. Importantly, a direct effect of specific PIT stimuli on blood-oxygen-level-dependent (BOLD) activity in the left putamen was found. Results provide initial evidence of effective connectivity in key brain regions in an avoidance-based PIT task network. This study adds to the literature studying PIT effects in humans and employing GIMME models to understand how psychological phenomena are supported in the brain.

Using Primary Reinforcement to Enhance Translatability of a Human Affect and Decision-Making Judgment Bias Task

Good translatability of behavioral measures of affect (emotion) between human and nonhuman animals is core to comparative studies. The judgment bias (JB) task, which measures "optimistic" and "pessimistic" decision-making under ambiguity as indicators of positive and negative affective valence, has been used in both human and nonhuman animals. However, one key disparity between human and nonhuman studies is that the former typically use secondary reinforcers (e.g., money) whereas the latter typically use primary reinforcers (e.g., food). To address this deficiency and shed further light on JB as a measure of affect, we developed a novel version of a JB task for humans using primary reinforcers. Data on decision-making and reported affective state during the JB task were analyzed using computational modeling. Overall, participants grasped the task well, and as anticipated, their reported affective valence correlated with trial-by-trial variation in offered volume of juice. In addition, previous findings from monetary versions of the task were replicated: More positive prediction errors were associated with more positive affective valence, a higher lapse rate was associated with lower affective arousal, and affective arousal decreased as a function of number of trials completed. There was no evidence that more positive valence was associated with greater "optimism," but instead, there was evidence that affective valence influenced the participants' decision stochasticity, whereas affective arousal tended to influence their propensity for errors. This novel version of the JB task provides a useful tool for investigation of the links between primary reward and punisher experience, affect, and decision-making, especially from a comparative perspective.

Naloxone precipitated withdrawal increases dopamine release in the dorsal striatum of opioid dependent men

Dopamine (DA) neurotransmission is critical in the neurobiology of reward and aversion, but its contribution to the aversive state of opioid withdrawal remains unknown in humans. To address this, we used updated voxelwise methods and retrospectively analyzed a [¹¹C]raclopride-PET dataset to measure D_2/3 receptor availability and relative cerebral blood flow (R1) in male opioid use disorder (OUD) participants (n = 10) during placebo and acute opioid withdrawal conditions. We found that acute withdrawal precipitated by the opioid antagonist naloxone significantly increased dorsal striatal DA release in OUD participants (p_FWE < 0.05). Net changes in striatal DA were significantly correlated with a subjective index of withdrawal aversion such that greater DA increases were associated with more aversive responses (r(8) = 0.82, p < 0.005). Withdrawal also affected brain function, as indexed by increases in relative cerebral blood flow in the insula and putamen (p_FWE < 0.05). Our findings are different from preclinical studies that have primarily reported decreases in ventral striatal DA during naloxone precipitated withdrawal, whereas this effect was not significant in OUD participants (p = 0.79). In sum, we provide evidence for the contribution of increases in dorsal striatal DA to the aversive state of naloxone precipitated withdrawal in humans.

The neuroscience of emotional disorders

Emotions can be defined as states elicited by rewards or punishments, and indeed the neurology of emotional disorders can be understood in terms of this foundation. The orbitofrontal cortex in humans and other primates is a critical area in emotion processing, determining the value of stimuli and whether they are rewarding or nonrewarding. The cortical processing that occurs before the orbitofrontal cortex primarily involves defining the identity of stimuli, i.e., "what" is present and not reward value. There is evidence that this holds true for taste, visual, somatosensory, and olfactory stimuli. The human medial orbitofrontal cortex is important in processing many different types of reward, and the lateral orbitofrontal cortex in processing nonreward and punishment. Humans with damage to the orbitofrontal cortex have an impaired ability to identify facial and voice expressions of emotions, and impaired subjective experience of emotion. They can have an altered personality and be impulsive because they are impaired at processing failures to receive expected rewards and at processing punishments. In humans, the role of the amygdala in the processing of emotions is reduced because of the great evolutionary development of the orbitofrontal cortex: amygdala damage has much less effect on emotion than does orbitofrontal cortex damage. The orbitofrontal cortex projects reward value information to the anterior cingulate cortex, which is involved in learning those actions required to obtain rewards and avoid punishments. The cingulate cortex thus provides an output route for emotional behavior. In depression, the medial orbitofrontal cortex has decreased connectivity and sensitivity to reward, and the lateral orbitofrontal cortex has increased connectivity and sensitivity to nonreward. The orbitofrontal cortex has major projections to the anterior cingulate cortex, including its subcommissural region, and the anterior cingulate cortex is also implicated in depression.

Cardiac response in aversive and appetitive olfactory conditioning: Evidence for a valence-independent CS-elicited bradycardia

While the examination of conditioned cardiac responses is well established in human fear conditioning research, comparable studies using less-aversive or rather appetitive unconditioned stimuli (UCS) are sparse and results are mixed. Therefore, the aim of this study was a systematic analysis of cardiac reactions in aversive and appetitive conditioning. Olfactory stimuli were used as unconditioned stimuli as they are suitable reinforcers in both an aversive and an appetitive conditioning offering the opportunity for a comparison between conditioned responses. In total, n = 86 participants took part in both an aversive and an appetitive differential conditioning task with a counterbalanced order across participants. Aversive or appetitive odors, respectively, served as UCS and neutral geometrical figures as CS. Subjective ratings, skin conductance response (SCRs), and evoked cardiac reactions were analyzed and compared between tasks. Conditioned responses in subjective ratings could be observed in both aversive conditioning and appetitive conditioning, while SCRs discriminated between CS+ and CS- in aversive conditioning only. Regarding conditioned cardiac responses, the deceleration for the CS+ was longer than for the CS- in both tasks. In addition, a higher deceleration magnitude and a shorter acceleration for the CS+ as compared to the CS- were found in aversive but not in appetitive conditioning. There were medium-size correlations between aversive and appetitive CRs for subjective ratings and none for physiological responses. The results suggest similarities between cardiac response patterns in aversive and appetitive conditioning, which implies that bradycardia in conditioning might not be fear-specific but presents a valence-independent CS-elicited bradycardia.

Defensive freezing and its relation to approach-avoidance decision-making under threat

Successful responding to acutely threatening situations requires adequate approach-avoidance decisions. However, it is unclear how threat-induced states-like freezing-related bradycardia-impact the weighing of the potential outcomes of such value-based decisions. Insight into the underlying computations is essential, not only to improve our models of decision-making but also to improve interventions for maladaptive decisions, for instance in anxiety patients and first-responders who frequently have to make decisions under acute threat. Forty-two participants made passive and active approach-avoidance decisions under threat-of-shock when confronted with mixed outcome-prospects (i.e., varying money and shock amounts). Choice behavior was best predicted by a model including individual action-tendencies and bradycardia, beyond the subjective value of the outcome. Moreover, threat-related bradycardia (high-vs-low threat) interacted with subjective value, depending on the action-context (passive-vs-active). Specifically, in action-contexts incongruent with participants' intrinsic action-tendencies, stronger bradycardia related to diminished effects of subjective value on choice across participants. These findings illustrate the relevance of testing approach-avoidance decisions in relatively ecologically valid conditions of acute and primarily reinforced threat. These mechanistic insights into approach-avoidance conflict-resolution may inspire biofeedback-related techniques to optimize decision-making under threat. Critically, the findings demonstrate the relevance of incorporating internal psychophysiological states and external action-contexts into models of approach-avoidance decision-making.

Pavlovian Learning Processes in Pediatric Anxiety Disorders: A Critical Review

Deficits in associative and Pavlovian learning are thought to lie at the center of anxiety-related disorders. However, the majority of studies have been carried out in adult populations. The aim of this review was to critically examine the behavioral and neuroimaging literature on Pavlovian learning in pediatric anxiety disorders. We conclude that although there is evidence for deficits in Pavlovian processes (e.g., heightened reactivity to safety cues in anxious samples), the extant literature suffers from key methodological and theoretical issues. We conclude with theoretical and methodological recommendations for future research in order to further elucidate the role of Pavlovian learning in the etiology, maintenance, and treatment of pediatric anxiety disorders.

Associations with monetary values do not influence access to awareness for faces

Human faces can convey socially relevant information in various ways. Since the early detection of such information is crucial in social contexts, socially meaningful information might also have privileged access to awareness. This is indeed suggested by previous research using faces with emotional expressions. However, the social relevance of emotional faces is confounded with their physical stimulus characteristics. Here, we sought to overcome this problem by manipulating the relevance of face stimuli through classical conditioning: Participants had to learn the association between different face exemplars and high or low amounts of positive and negative monetary outcomes. Before and after the conditioning procedure, the time these faces needed to enter awareness was probed using continuous flash suppression, a variant of binocular rivalry. While participants successfully learned the association between the face stimuli and the respective monetary outcomes, faces with a high monetary value did not enter visual awareness faster than faces with a low monetary value after conditioning, neither for rewarding nor for aversive outcomes. Our results tentatively suggest that behaviorally relevant faces do not have privileged access to awareness when the assessment of the faces' relevance is dependent on the processing of face identity, as this requires complex stimulus processing that is likely limited at pre-conscious stages.

The Importance of Common Currency Tasks in Translational Psychiatry

Purpose of Review

Common currency tasks are tasks that investigate the same phenomenon in different species. In this review, we discuss how to ensure the translational validity of common currency tasks, summarise their benefits, present recent research in this area and offer future directions and recommendations.

Recent Findings

We discuss the strengths and limitations of three specific examples where common currency tasks have added to our understanding of psychiatric constructs—affective bias, reversal learning and goal-based decision making.

Summary

Overall, common currency tasks offer the potential to improve drug discovery in psychiatry. We recommend that researchers prioritise construct validity above face validity when designing common currency tasks and suggest that the evidence for construct validity is summarised in papers presenting research in this area.

A voxel-wise meta-analysis of task-based functional MRI studies on impaired gain and loss processing in adults with addiction

BACKGROUND

Disturbances in gain and loss processing have been extensively reported in adults with addiction, a brain disorder characterized by obsession with addictive substances or behaviours. Previous studies have provided conflicting results with respect to neural abnormalities in gain processing in addiction, and few investigations into loss processing.

METHODS

We conducted voxel-wise meta-analyses of abnormal task-evoked regional activities in adults with substance dependence and gambling addiction during the processing of gains and losses not related to their addiction (mainly monetary). We identified 24 studies, including 465 participants with substance dependence, 81 with gambling addiction and 490 healthy controls.

RESULTS

Compared with healthy controls, all participants with addictions showed hypoactivations in the prefrontal cortex, striatum and insula and hyperactivations in the default mode network during gain anticipation; hyperactivations in the prefrontal cortex and both hyper- and hypoactivations in the striatum during loss anticipation; and hyperactivations in the occipital lobe during gain outcome. In the substance dependence subgroup, activity in the occipital lobe was increased during gain anticipation but decreased during loss anticipation.

LIMITATIONS

We were unable to conduct meta-analyses in the gambling addiction subgroup because of a limited data set. We did not investigate the effects of clinical variables because of limited information.

CONCLUSION

The current study identified altered brain activity associated with higher- and lower-level function during gain and loss processing for non-addiction (mainly monetary) stimuli in adults with substance dependence and gambling addiction. Adults with addiction were more sensitive to anticipatory gains than losses at higher- and lower-level brain areas. These results may help us to better understand the pathology of gain and loss processing in addiction.

Roles of the bed nucleus of the stria terminalis and amygdala in fear reactions

The bed nucleus of the stria terminalis (BNST) plays a critical modulatory role in driving fear responses. Part of the so-called extended amygdala, this region shares many functions and connections with the substantially more investigated amygdala proper. In this chapter, we review contributions of the BNST and amygdala to subjective, behavioral, and physiological aspects of fear. Despite the fact that both regions are together involved in each of these aspects of fear, they appear complimentary in their contributions. Specifically, the basolateral amygdala (BLA), through its connections to sensory and orbitofrontal regions, is ideally poised for fast learning and controlling fear reactions in a variety of situations. The central amygdala (CeA) relies on BLA input and is particularly important for adjusting physiological and behavioral responses under acute threat. In contrast, the BNST may profit from more extensive striatal and dorsomedial prefrontal connections to drive anticipatory responses under more ambiguous conditions that allow more time for planning. Thus current evidence suggests that the BNST is ideally suited to play a critical role responding to distant or ambiguous threats and could thereby facilitate goal-directed defensive action.

The orbitofrontal cortex: reward, emotion and depression

The orbitofrontal cortex in primates including humans is the key brain area in emotion, and in the representation of reward value and in non-reward, that is not obtaining an expected reward. Cortical processing before the orbitofrontal cortex is about the identity of stimuli, i.e. 'what' is present, and not about reward value. There is evidence that this holds for taste, visual, somatosensory and olfactory stimuli. The human medial orbitofrontal cortex represents many different types of reward, and the lateral orbitofrontal cortex represents non-reward and punishment. Not obtaining an expected reward can lead to sadness, and feeling depressed. The concept is advanced that an important brain region in depression is the orbitofrontal cortex, with depression related to over-responsiveness and over-connectedness of the non-reward-related lateral orbitofrontal cortex, and to under-responsiveness and under-connectivity of the reward-related medial orbitofrontal cortex. Evidence from large-scale voxel-level studies and supported by an activation study is described that provides support for this hypothesis. Increased functional connectivity of the lateral orbitofrontal cortex with brain areas that include the precuneus, posterior cingulate cortex and angular gyrus is found in patients with depression and is reduced towards the levels in controls when treated with medication. Decreased functional connectivity of the medial orbitofrontal cortex with medial temporal lobe areas involved in memory is found in patients with depression. Some treatments for depression may act by reducing activity or connectivity of the lateral orbitofrontal cortex. New treatments that increase the activity or connectivity of the medial orbitofrontal cortex may be useful for depression. These concepts, and that of increased activity in non-reward attractor networks, have potential for advancing our understanding and treatment of depression. The focus is on the orbitofrontal cortex in primates including humans, because of differences of operation of the orbitofrontal cortex, and indeed of reward systems, in rodents. Finally, the hypothesis is developed that the orbitofrontal cortex has a special role in emotion and decision-making in part because as a cortical area it can implement attractor networks useful in maintaining reward and emotional states online, and in decision-making.

Associative Learning Processes in the Formation of Intergroup Anxiety and Avoidance in Society

Laboratory-based aversive conditioning studies have reliably induced fear toward an image of an outgroup member by pairing the image with a fear-inducing, aversive stimulus. However, laboratory-based studies have been criticized for being simplistic in comparison to the complexities of the real world. The current study is the first to apply an aversive conditioning framework to explain the formation of intergroup fear and subsequent anxiety toward, and avoidance of, the outgroup outside the laboratory. Two samples recalled details of their first negative encounter with an African American (N = 554) or Muslim (N = 613) individual, respectively. Congruent with learning theory, participants who reported an unpleasant event with an outgroup member reported more fear during the encounter than did those who did not report experiencing an unpleasant event. Additionally, the intensity of unpleasantness during the first encounter indirectly predicted outgroup avoidance, via retrospectively recalled fear and current levels of intergroup anxiety.

The affective neuroscience of socioeconomic status: implications for mental health

We review basic science research on neural mechanisms underlying emotional processing in individuals of differing socioeconomic status (SES). We summarise SES differences in response to positive and negative stimuli in limbic and cortical regions associated with emotion and emotion regulation. We discuss the possible relevance of neuroscience to understanding the link between mental health and SES. We hope to provide insights into future neuroscience research on the etiology and pathophysiology of mental disorders relating to SES.

Neural Pattern Similarity Unveils the Integration of Social Information and Aversive Learning

Attributing intentions to others' actions is important for learning to avoid their potentially harmful consequences. Here, we used functional magnetic resonance imaging multivariate pattern analysis to investigate how the brain integrates information about others' intentions with the aversive outcome of their actions. In an interactive aversive learning task, participants (n = 33) were scanned while watching two alleged coparticipants (confederates)-one making choices intentionally and the other unintentionally-leading to aversive (a mild shock) or safe (no shock) outcomes to the participant. We assessed the trial-by-trial changes in participants' neural activation patterns related to observing the coparticipants and experiencing the outcome of their choices. Participants reported a higher number of shocks, more discomfort, and more anger to shocks given by the intentional player. Intentionality enhanced responses to aversive actions in the insula, anterior cingulate cortex, inferior frontal gyrus, dorsal medial prefrontal cortex, and the anterior superior temporal sulcus. Our findings indicate that neural pattern similarities index the integration of social and threat information across the cortex.

Translating concepts of risk and loss in rodent models of gambling and the limitations for clinical applications

Gambling involves placing something of value at risk in exchange for the opportunity to potentially gain something of greater value in return. A variety of gambling paradigms have been designed to study the maladaptive decision-making that underlies problematic gambling. Central to these gambling models are the definitions of "risk" and "loss", especially when translating the results from rodent studies to clinical applications. Risk and loss are not mutually exclusive but rather share some overlap. With careful interpretation and consideration of the limitations of these behavioral paradigms, results from rodent models may provide insights into the neurobiology of risky decision-making that leads to problematic gambling in humans.

From sign-tracking to attentional bias: Implications for gambling and substance use disorders

Sign-tracking behavior in Pavlovian autoshaping is known to be a relevant index of the incentive salience attributed to reward-related cues. Evidence has accumulated to suggest that animals that exhibit a sign-tracker phenotype are especially vulnerable to addiction and relapse due to their proneness to attribute incentive salience to drug cues, and their relatively weak cognitive and attentional control over their behavior. Interestingly, sign-tracking is also influenced by reward uncertainty in a way that may promote gambling disorder. Research indicates that reward uncertainty sensitizes sign-tracking responses and favors the development of a sign-tracker phenotype, compatible with the conditioned attractiveness of lights and sounds in casinos for problem gamblers. The study of attentional biases in humans (an effect akin to sign-tracking in animals) leads to similar observations, notably that the propensity to develop attraction for conditioned stimuli (CSs) is predictive of addictive behavior. Here we review the literature on drug addiction and gambling disorder, highlighting the similarities between studies of sign-tracking and attentional biases.

Effective connectivity modulations related to win and loss outcomes

Previous studies have characterized the brain regions involved in encoding monetary reward and punishment outcomes. The question of how this information is integrated across brain regions has received less attention. Here, we investigated changes in effective connectivity related to the processing of positive and negative monetary outcomes using functional magnetic resonance imaging data from the Human Connectome Project. Specifically, subjects engaged in a card guessing game which could yield win, loss, or neutral outcomes. A general linear model was used to define a network of regions involved in win and loss outcome processing, including anterior insula, anterior cingulate cortex, and ventral striatum. Dynamic causal modelling (DCM) was implemented to study between-region couplings and outcome-related modulations thereof within this network. In addition, we explored the relation between effective connectivity patterns and choice behavior in the gambling task. Parametric empirical Bayesian modelling was conducted for group-level inferences of both DCM and the choice behavior. Behaviorally, both win and loss outcomes increased the probability of choice switches in subsequent gambles. In terms of connectivity, win outcomes were associated with increased extrinsic connectivity across the network, while loss outcomes featured a balance between increased and decreased extrinsic connectivity. Moreover, self-inhibitory connections tended to decrease for both win and loss outcomes. Interestingly, a substantial discrepancy was observed for occipital cortex connectivity, which was characterized by intrinsic disinhibition in loss but not in win trials. The observed differences in effective connectivity during the processing of positive and negative outcomes, despite similarities in average regional activity and choice behavior, highlight the value of exploring network dynamics in the context of incentive manipulations.

Fear in infancy: Lessons from snakes, spiders, heights, and strangers

This review challenges the traditional interpretation of infants' and young children's responses to three types of potentially "fear-inducing" stimuli-snakes and spiders, heights, and strangers. The traditional account is that these stimuli are the objects of infants' earliest developing fears. We present evidence against the traditional account, and provide an alternative explanation of infants' behaviors toward each stimulus. Specifically, we propose that behaviors typically interpreted as "fearful" really reflect an array of stimulus-specific responses that are highly dependent on context, learning, and the perceptual features of the stimuli. We speculate about why researchers so commonly misinterpret these behaviors, and conclude with future directions for studying the development of fear in infants and young children. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Generalization of appetitive conditioned responses

A stimulus (conditioned stimulus, CS) associated with an appetitive unconditioned stimulus (US) acquires positive properties and elicits appetitive conditioned responses (CR). Such associative learning has been examined extensively in animals with food as the US, and results are used to explain psychopathologies (e.g., substance-related disorders or obesity). Human studies on appetitive conditioning exist, too, but we still know little about generalization processes. Understanding these processes may explain why stimuli not associated with a drug, for instance, can elicit craving. Forty-seven hungry participants underwent an appetitive conditioning protocol during which one of two circles with different diameters (CS+) became associated with an appetitive US (chocolate or salty pretzel, according to participants' preference) but never the other circle (CS-). During generalization, US were delivered twice and the two CS were presented again plus four circles (generalization stimuli, GS) with gradually increasing diameters from CS- to CS+. We found successful appetitive conditioning as reflected in appetitive subjective ratings (positive valence, higher contingency) and physiological responses (startle attenuation and larger skin conductance responses) to CS+ versus CS-, and, importantly, both measures confirmed generalization as indicated by generalization gradients. Small changes in CS-US contingency during generalization may have weakened generalization processes on the physiological level. Considering that appetitive conditioned responses can be generalized to non-US-associated stimuli, a next important step would be to investigate risk factors that mediate overgeneralization.

Pupil dilation as an implicit measure of appetitive Pavlovian learning

Appetitive Pavlovian conditioning is a learning mechanism of fundamental biological and pathophysiological significance. Nonetheless, its exploration in humans remains sparse, which is partly attributed to the lack of an established psychophysiological parameter that aptly represents conditioned responding. This study evaluated pupil diameter and other ocular response measures (gaze dwelling time, blink duration and count) as indices of conditioning. Additionally, a learning model was used to infer participants' learning progress on the basis of their pupil dilation. Twenty-nine healthy volunteers completed an appetitive differential delay conditioning paradigm with a primary reward, while the ocular response measures along with other psychophysiological (heart rate, electrodermal activity, postauricular and eyeblink reflex) and behavioral (ratings, contingency awareness) parameters were obtained to examine the relation among different measures. A significantly stronger increase in pupil diameter, longer gaze duration and shorter eyeblink duration was observed in response to the reward-predicting cue compared to the control cue. The Pearce-Hall attention model best predicted the trial-by-trial pupil diameter. This conditioned response was corroborated by a pronounced heart rate deceleration to the reward-predicting cue, while no conditioning effect was observed in the electrodermal activity or startle responses. There was no discernible correlation between the psychophysiological response measures. These results highlight the potential value of ocular response measures as sensitive indices for representing appetitive conditioning.

Counterconditioning following memory retrieval diminishes the reinstatement of appetitive memories in humans

Appetitive memories play a crucial role in learning and behavior, but under certain circumstances, such memories become maladaptive and play a vital role in addiction and other psychopathologies. Recent scientific research has demonstrated that memories can be modified following their reactivation through memory retrieval in a process termed memory reconsolidation. Several nonpharmacological behavioral manipulations yielded mixed results in their capacity to alter maladaptive memories in humans. Here, we aimed to translate the promising findings observed in rodents to humans. We constructed a novel three-day procedure using aversive counterconditioning to alter appetitive memories after short memory retrieval. On the first day, we used appetitive conditioning to form appetitive memories. On the second day, we retrieved these appetitive memories in one group (Retrieval group) but not in a second group. Subsequently, all participants underwent counterconditioning. On the third day, we attempted to reinstate the appetitive memories from day one. We observed a significant reduction in the reinstatement of the original appetitive memory when counterconditioning was induced following memory retrieval. Here, we provide a novel human paradigm that models several memory processes and demonstrate memory attenuation when counterconditioned after its retrieval. This paradigm can be used to study complex appetitive memory dynamics, e.g., memory reconsolidation and its underlying brain mechanisms.

Oxytocin Effects on Pain Perception and Pain Anticipation

There is an ongoing debate whether the neuropeptide oxytocin (OT) modulates pain processing in humans. This study differentiates behavioral and neuronal OT effects on pain perception and pain anticipation by using a Pavlovian conditioning paradigm. Forty-six males received intranasally administered OT in a randomized, double-blind, placebo-controlled group design. Although OT exerted no direct effect on perceived pain, OT was found to modulate the blood oxygen level-dependent response in the ventral striatum for painful versus warm unconditioned stimuli and to decrease activity in the anterior insula (IS) with repeated thermal pain stimuli. Regarding pain anticipation, OT increased responses to CS_pain versus CS_minus in the nucleus accumbens. Furthermore, in the OT condition increased correct expectations, particularly for the most certain conditioned stimuli (CS)-unconditioned stimuli associations (CS_minus and CS_pain) were found, as well as greatest deactivations in the right posterior IS in response to the least certain condition (CS_warm) with posterior IS activity and correct expectancies being positively correlated. In conclusion, OT seems to have both a direct effect on pain processing via the ventral striatum and by inducing habituation in the anterior IS as well as on pain anticipation by boostering associative learning in general and the neuronal conditioned fear of pain response in particular. PERSPECTIVE: The neuropeptide OT has recently raised the hope to offer a novel avenue for modulating pain experience. This study found OT to modulate pain processing and to facilitate the anticipation of pain, inspiring further research on OT effects on the affective dimension of the pain experience.

Sketching the Power of Machine Learning to Decrypt a Neural Systems Model of Behavior

Uncovering brain-behavior mechanisms is the ultimate goal of neuroscience. A formidable amount of discoveries has been made in the past 50 years, but the very essence of brain-behavior mechanisms still escapes us. The recent exploitation of machine learning (ML) tools in neuroscience opens new avenues for illuminating these mechanisms. A key advantage of ML is to enable the treatment of large data, combing highly complex processes. This essay provides a glimpse of how ML tools could test a heuristic neural systems model of motivated behavior, the triadic neural systems model, which was designed to understand behavioral transitions in adolescence. This essay previews analytic strategies, using fictitious examples, to demonstrate the potential power of ML to decrypt the neural networks of motivated behavior, generically and across development. Of note, our intent is not to provide a tutorial for these analyses nor a pipeline. The ultimate objective is to relate, as simply as possible, how complex neuroscience constructs can benefit from ML methods for validation and further discovery. By extension, the present work provides a guide that can serve to query the mechanisms underlying the contributions of prefrontal circuits to emotion regulation. The target audience concerns mainly clinical neuroscientists. As a caveat, this broad approach leaves gaps, for which references to comprehensive publications are provided.

Impaired generalization of reward but not loss in obsessive-compulsive disorder

BACKGROUND

Generalizing from past experiences can be adaptive by allowing those experiences to guide behavior in new situations. Generalizing too much, however, can be maladaptive. For example, individuals with pathological anxiety are believed to overgeneralize emotional responses from past threats, broadening their scope of fears. Whether individuals with pathological anxiety overgeneralize in other situations remains unclear.

METHODS

The present study (N = 57) used a monetary sensory preconditioning paradigm with rewards and losses to address this question in individuals with obsessive-compulsive disorder (OCD) and social anxiety disorder (SAD), comparing them to healthy comparison subjects (HC). In all groups, we tested direct learning of associations between cues and reward vs. loss outcomes, as well as generalization of learning to novel choice options.

RESULTS

We found no differences between the three groups in the direct learning of stimuli with their outcomes: all subjects demonstrated intact stimulus-response learning by choosing rewarding options and avoiding negative ones. However, OCD subjects were less likely to generalize from rewards than either the SAD or HC groups, and this impairment was not found for losses. Additionally, greater deficits in reward generalization were correlated with severity of threat estimation, as measured by a subscale of the Obsessive Beliefs Questionnaire, both within OCD and across all groups.

CONCLUSIONS

These findings suggest that a compromised ability to generalize from rewarding events may impede adaptive behavior in OCD and in those susceptible to high estimation of threat.

A Neural Circuit Mechanism for Encoding Aversive Stimuli in the Mesolimbic Dopamine System

Ventral tegmental area (VTA) dopamine (DA) neurons play a central role in mediating motivated behaviors, but the circuitry through which they signal positive and negative motivational stimuli is incompletely understood. Using in vivo fiber photometry, we simultaneously recorded activity in DA terminals in different nucleus accumbens (NAc) subnuclei during an aversive and reward conditioning task. We find that DA terminals in the ventral NAc medial shell (vNAcMed) are excited by unexpected aversive outcomes and to cues that predict them, whereas DA terminals in other NAc subregions are persistently depressed. Excitation to reward-predictive cues dominated in the NAc lateral shell and was largely absent in the vNAcMed. Moreover, we demonstrate that glutamatergic (VGLUT2-expressing) neurons in the lateral hypothalamus represent a key afferent input for providing information about aversive outcomes to vNAcMed-projecting DA neurons. Collectively, we reveal the distinct functional contributions of separate mesolimbic DA subsystems and their afferent pathways underlying motivated behaviors. VIDEO ABSTRACT.

Financial gain- and loss-related BOLD signals in the human ventral tegmental area and substantia nigra pars compacta

Neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNC) play central roles in reward-related behaviours. Nonhuman animal studies suggest that these neurons also process aversive events. However, our understanding of how the human VTA and SNC responds to such events is limited and has been hindered by the technical challenge of using functional magnetic resonance imaging (fMRI) to investigate a small structure where the signal is particularly vulnerable to physiological noise. Here we show, using methods optimized specifically for the midbrain (including high-resolution imaging, a novel registration protocol, and physiological noise modelling), a BOLD (blood-oxygen-level dependent) signal to both financial gain and loss in the VTA and SNC, along with a response to nil outcomes that are better or worse than expected in the VTA. Taken together, these findings suggest that the human VTA and SNC are involved in the processing of both appetitive and aversive financial outcomes in humans.

Time course of memory formation for an appetitive, one-trial, water-reward task in day-old chicks

Researchers studying learning and memory in the day-old chick have developed a one-trial taste avoidance learning task, in which a chick pecks a bead coated with an aversive-tasting liquid such as methylanthranilate. The current study examined, in two experiments, whether chicks could demonstrate learning of an appetitive version of the one-trial task, and how long this memory might last. The results demonstrate that chicks show significantly decreased latencies to peck a test bead similar to that pecked at training coated with water, and that this memory lasted for at least 6 h after training. These results demonstrate that this appetitive task produces significant learning.

Contrasting temporal difference and opportunity cost reinforcement learning in an empirical money-emergence paradigm

Money is a fundamental and ubiquitous institution in modern economies. However, the question of its emergence remains a central one for economists. The monetary search-theoretic approach studies the conditions under which commodity money emerges as a solution to override frictions inherent to interindividual exchanges in a decentralized economy. Although among these conditions, agents' rationality is classically essential and a prerequisite to any theoretical monetary equilibrium, human subjects often fail to adopt optimal strategies in tasks implementing a search-theoretic paradigm when these strategies are speculative, i.e., involve the use of a costly medium of exchange to increase the probability of subsequent and successful trades. In the present work, we hypothesize that implementing such speculative behaviors relies on reinforcement learning instead of lifetime utility calculations, as supposed by classical economic theory. To test this hypothesis, we operationalized the Kiyotaki and Wright paradigm of money emergence in a multistep exchange task and fitted behavioral data regarding human subjects performing this task with two reinforcement learning models. Each of them implements a distinct cognitive hypothesis regarding the weight of future or counterfactual rewards in current decisions. We found that both models outperformed theoretical predictions about subjects' behaviors regarding the implementation of speculative strategies and that the latter relies on the degree of the opportunity costs consideration in the learning process. Speculating about the marketability advantage of money thus seems to depend on mental simulations of counterfactual events that agents are performing in exchange situations.

Neural hypersensitivity to pleasant touch in women remitted from anorexia nervosa

Interoception, or the sensing and integration of bodily state signals, has been implicated in anorexia nervosa (AN), given that the hallmark symptoms involve food restriction and body image disturbance. Here we focus on brain response to the anticipation and experience of affective interoceptive stimuli. Women remitted from AN (RAN; N = 18) and healthy comparison women (CW; N = 26) underwent a pleasant affective touch paradigm consisting of gentle strokes with a soft brush administered to the forearm or palm during functional neuroimaging. RAN had a lower brain response relative to CW during anticipation of touch, but a greater response when experiencing touch in the right ventral mid-insula. In RAN, this reduced anticipatory response was associated with higher levels of harm avoidance. Exploratory analyses in RAN also suggested that lower response during touch anticipation was associated with greater body dissatisfaction and higher perceived touch intensity ratings. This reduced responsivity to the anticipation of pleasant affective interoceptive stimuli in association with higher harm avoidance, along with an elevated response to the experience of touch, suggests an impaired ability in AN to predict and interpret incoming physiological stimuli. Impaired interoception may thus impact one's sense of self, thereby supporting observations of disturbed body image and avoidance of affective and social stimuli. Therapeutic approaches that help AN to better anticipate and interpret salient affective stimuli or improve tolerance of interoceptive experiences may be an important addition to current interventions.

Behavioral and neurobiological mechanisms of punishment: implications for psychiatric disorders

Punishment involves learning about the relationship between behavior and its adverse consequences. Punishment is fundamental to reinforcement learning, decision-making and choice, and is disrupted in psychiatric disorders such as addiction, depression, and psychopathy. However, little is known about the brain mechanisms of punishment and much of what is known is derived from study of superficially similar, but fundamentally distinct, forms of aversive learning such as fear conditioning and avoidance learning. Here we outline the unique conditions that support punishment, the contents of its learning, and its behavioral consequences. We consider evidence implicating GABA and monoamine neurotransmitter systems, as well as corticostriatal, amygdala, and dopamine circuits in punishment. We show how maladaptive punishment processes are implicated in addictions, impulse control disorders, psychopathy, anxiety, and depression and argue that a better understanding of the cellular, circuit, and cognitive mechanisms of punishment will make important contributions to next generation therapeutic approaches.

Context and Time Matter: Effects of Emotion and Motivation on Episodic Memory Overtime

Previous studies have shown that compared with neutral cues, stimuli with positive and negative/stressful contexts or reward and punishment cues are remembered better. However, it is unclear whether the enhanced effect differs in emotion or motivation dimensions and the passage of time. We addressed these issues by manipulating different contextual cues for neutral words at different time intervals. In experiment 1, subjects were asked to learn words with picture contexts in positive, negative/stressful, and neutral valences and were tested by old/new word recognition and contextual judgment 10 min, 1 day, and 1 week later. In experiment 2, the reward and punishment motivations were manipulated by monetary cues during learning. Word recognition and contextual judgment were assessed 10 min, 1 day, 1 week, and 1 month after the study. Compared with negative and punishment conditions, the words in positive and reward contexts were recognized better at shorter intervals, which was associated with recollection process. In contrast, the words in negative and punishment contexts were recognized better at longer intervals, which was mainly associated with familiarity process. These results clarified how different dimensions of emotional and motivational contexts influence memory at short and long intervals and highlighted the role of contextual features in memory formation and consolidation.

The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task

The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta‐analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.

Differentiating aversive conditioning in bistable perception: Avoidance of a percept vs. salience of a stimulus

Alternating conscious visual perception of bistable stimuli is influenced by several factors. In order to understand the effect of negative valence, we tested the effect of two types of aversive conditioning on dominance durations in binocular rivalry. Participants received either aversive classical conditioning of the stimuli shown alone between rivalry blocks, or aversive percept conditioning of one of the two possible perceptual choices during rivalry. Both groups showed successful aversive conditioning according to skin conductance responses and affective valence ratings. However, while classical conditioning led to an immediate but short-lived increase in dominance durations of the conditioned stimulus, percept conditioning yielded no significant immediate effect but tended to decrease durations of the conditioned percept during extinction. These results show dissociable effects of value learning on perceptual inference in situations of perceptual conflict, depending on whether learning relates to the decision between conflicting perceptual choices or the sensory stimuli per se.

Autonomic conditioning to monetary and social stimuli and aggression in children

Poor conditioning to punishment, such as loud tones or electric shock, has been proposed as an important factor involved in the etiology of aggressive and psychopathic behavior. However, it is not known whether the association holds when monetary or social stimulus is used as the unconditioned stimulus, and if aggressive individuals also have impaired conditioning to rewards. In this study, skin conductance responses in a conditioning task involving both monetary/social reward and punishment as unconditioned stimuli were assessed in 340 male and female 8- to 9-year-old children from the community. Children reported their reactive and proactive aggression using the Reactive and Proactive Aggression Questionnaire (RPQ; Raine et al., 2006). Results showed that monetary/social reward and punishment were effective in eliciting physiological classical conditioning in children, and that reduced reward conditioning was associated with high levels of proactive aggression in particular. Findings highlight the importance of distinguishing between reactive and proactive aggression when examining antisocial behavior in children, and suggest that reward-oriented treatment programs may not be effective for children with more proactive, instrumental aggressive behavior.

Measuring appetitive conditioned responses in humans

Clinical and preclinical findings suggest that individuals with abnormal responses to reward cues (stimuli associated with reward) may be at risk for maladaptive behaviors including obesity, addiction and depression. Our objective was to develop a new paradigm for producing appetitive conditioning using primary (food) rewards in humans, and investigate the equivalency of several outcomes previously used to measure appetitive responses to conditioned cues. We used an individualized food reward, and multimodal subjective, psychophysiological and behavioral measures of appetitive responses to a conditioned stimulus (CS) that predicted delivery of that food. We tested convergence among these measures of appetitive response, and relationships between these measures and action impulsivity, a putative correlate of appetitive conditioning. 90 healthy young adults participated. Although the paradigm produced robust appetitive conditioning in some measures, particularly psychophysiological ones, there were not strong correlations among measures of appetitive responses to the CS, as would be expected if they indexed a single underlying process. In addition, there was only one measure that related to impulsivity. These results provide important information for translational researchers interested in appetitive conditioning, suggesting that various measures of appetitive conditioning cannot be treated interchangeably.