The neurobiology of punishment

Anterior cingulate is a source of valence-specific information about value and uncertainty

Anterior cingulate cortex (ACC) is thought to control a wide range of reward, punishment, and uncertainty-related behaviors. However, how it does so is unclear. Here, in a Pavlovian procedure in which monkeys displayed a diverse repertoire of reward-related, punishment-related, and uncertainty-related behaviors, we show that many ACC-neurons represent expected value and uncertainty in a valence-specific manner, signaling value or uncertainty predictions about either rewards or punishments. Other ACC-neurons signal prediction information about rewards and punishments by displaying excitation to both (rather than excitation to one and inhibition to the other). This diversity in valence representations may support the role of ACC in many behavioral states that are either enhanced by reward and punishment (e.g., vigilance) or specific to either reward or punishment (e.g., approach and avoidance). Also, this first demonstration of punishment-uncertainty signals in the brain suggests that ACC could be a target for the treatment of uncertainty-related disorders of mood.

Rewards or punishments elicit diverse behavioral responses; however, the neural circuits underlying such flexibility are unclear. Here Monosov shows that this diversity could be supported by neurons in the anterior cingulate that represent expected value and uncertainty in a valence-specific manner.

Avoidant Responses to Interpersonal Provocation Are Associated with Increased Amygdala and Decreased Mentalizing Network Activity

When intentionally pushed or insulted, one can either flee from the provoker or retaliate. The implementation of such fight-or-flight decisions is a central aspect in the genesis and evolution of aggression episodes, yet it is usually investigated only indirectly or in nonsocial situations. In the present fMRI study, we aimed to distinguish brain regions associated with aggressive and avoidant responses to interpersonal provocation in humans. Participants (thirty-six healthy young women) could either avoid or face a highly (HP) and a lowly (LP) provoking opponent in a competitive reaction time task: the fight-or-escape (FOE) paradigm. Subjects avoided the HP more often, but retaliated when facing her. Moreover, they chose to fight the HP more quickly, and showed increased heart rate (HR) right before confronting her. Orbitofrontal cortex (OFC) and sensorimotor cortex were more active when participants decided to fight, whereas the mentalizing network was engaged when deciding to avoid. Importantly, avoiding the HP relative to the LP was associated with both higher activation in the right basolateral amygdala and lower relative activity in several mentalizing regions [e.g., medial and inferior frontal gyrus (IFG), temporal-parietal junction (TPJ)]. These results suggest that avoidant responses to provocation might result from heightened threat anticipation and are associated with reduced perspective taking. Furthermore, our study helps to reconcile conflicting findings on the role of the mentalizing network, the amygdala, and the OFC in aggression.

Moral learning: Psychological and philosophical perspectives

The past 15years occasioned an extraordinary blossoming of research into the cognitive and affective mechanisms that support moral judgment and behavior. This growth in our understanding of moral mechanisms overshadowed a crucial and complementary question, however: How are they learned? As this special issue of the journal Cognition attests, a new crop of research into moral learning has now firmly taken root. This new literature draws on recent advances in formal methods developed in other domains, such as Bayesian inference, reinforcement learning and other machine learning techniques. Meanwhile, it also demonstrates how learning and deciding in a social domain-and especially in the moral domain-sometimes involves specialized cognitive systems. We review the contributions to this special issue and situate them within the broader contemporary literature. Our review focuses on how we learn moral values and moral rules, how we learn about personal moral character and relationships, and the philosophical implications of these emerging models.

Genetic variation of dopamine and serotonin function modulates the feedback-related negativity during altruistic punishment

Why do humans cooperate and often punish norm violations of others? In the present study, we sought to investigate the genetic bases of altruistic punishment (AP), which refers to the costly punishment of norm violations with potential benefit for other individuals. Recent evidence suggests that norm violations and unfairness are indexed by the feedback-related negativity (FRN), an anterior cingulate cortex (ACC) generated neural response to expectancy violations. Given evidence on the role of serotonin and dopamine in AP as well as in FRN-generation, we explored the impact of genetic variation of serotonin and dopamine function on FRN and AP behavior in response to unfair vs. fair monetary offers in a Dictator Game (DG) with punishment option. In a sample of 45 healthy participants we observed larger FRN amplitudes to unfair DG assignments both for 7-repeat allele carriers of the dopamine D4 receptor (DRD4) exon III polymorphism and for l/l-genotype carriers of the serotonin transporter gene-linked polymorphic region (5-HTTLRP). Moreover, 5-HTTLPR l/l-genotype carriers punished unfair offers more strongly. These findings support the role of serotonin and dopamine in AP, potentially via their influence on neural mechanisms implicated in the monitoring of expectancy violations and their relation to impulsive and punishment behavior.

Endogenous opioids regulate social threat learning in humans

Many fearful expectations are shaped by observation of aversive outcomes to others. Yet, the neurochemistry regulating social learning is unknown. Previous research has shown that during direct (Pavlovian) threat learning, information about personally experienced outcomes is regulated by the release of endogenous opioids, and activity within the amygdala and periaqueductal gray (PAG). Here we report that blockade of this opioidergic circuit enhances social threat learning through observation in humans involving activity within the amygdala, midline thalamus and the PAG. In particular, anticipatory responses to learned threat cues (CS) were associated with temporal dynamics in the PAG, coding the observed aversive outcomes to other (observational US). In addition, pharmacological challenge of the opioid receptor function is classified by distinct brain activity patterns during the expression of conditioned threats. Our results reveal an opioidergic circuit that codes the observed aversive outcomes to others into threat responses and long-term memory in the observer.

Though humans often learn about negative outcomes from observing the response of others, the neurochemistry underlying this learning is unknown. Here, authors show that blocking opioid receptors enhances social threat learning and describe the brain regions underlying this effect.

Common microbehavioral "footprint" of two distinct classes of conditioned aversion

Avoiding unfavorable situations is a vital skill and a constant task for any animal. Situations can be unfavorable because they feature something that the animal wants to escape from, or because they do not feature something that it seeks to obtain. We investigate whether the microbehavioral mechanisms by which these two classes of aversion come about are shared or distinct. We find that larval Drosophila avoid odors either previously associated with a punishment, or previously associated with the lack of a reward. These two classes of conditioned aversion are found to be strikingly alike at the microbehavioral level. In both cases larvae show more head casts when oriented toward the odor source than when oriented away, and direct fewer of their head casts toward the odor than away when oriented obliquely to it. Thus, conditioned aversion serving two qualitatively different functions—escape from a punishment or search for a reward—is implemented by the modulation of the same microbehavioral features. These features also underlie conditioned approach, albeit with opposite sign. That is, the larvae show conditioned approach toward odors previously associated with a reward, or with the lack of a punishment. In order to accomplish both these classes of conditioned approach the larvae show fewer head casts when oriented toward an odor, and direct more of their head casts toward it when they are headed obliquely. Given that the Drosophila larva is a genetically tractable model organism that is well suited to study simple circuits at the single-cell level, these analyses can guide future research into the neuronal circuits underlying conditioned approach and aversion, and the computational principles of conditioned search and escape.

Winning is not enough: ventral striatum connectivity during physical aggression

Social neuroscience studies have shown that the ventral striatum (VS), a highly reward-sensitive brain area, is activated when participants win competitive tasks. However, in these settings winning often entails both avoiding punishment and punishing the opponent. It is thus unclear whether the rewarding properties of winning are mainly associated to punishment avoidance, or if punishing the opponent can be additionally gratifying. In the present paper we explored the neurophysiological correlates of each outcome, aiming to better understand the development of aggression episodes. We previously introduced a competitive reaction time task that separates both effects: in half of the won trials, participants can physically punish their opponent (active trials), whereas in the other half they can only avoid a punishment (passive trials). We performed functional connectivity analysis seeded in the VS to test for differential network interactions in active compared to passive trials. The VS showed greater connectivity with areas involved in reward valuation (orbitofrontal cortex), arousal (dorsal thalamus and posterior insula), attention (inferior occipital gyrus), and motor control (supplementary motor area) in active compared to passive trials, whereas connectivity between the VS and the inferior frontal gyrus decreased. Interindividual variability in connectivity strength between VS and posterior insula was related to aggressive behavior, whereas connectivity between VS and supplementary motor area was related to faster reaction times in active trials. Our results suggest that punishing a provoking opponent when winning might adaptively favor a "competitive state" in the course of an aggressive interaction.

Group Membership Modulates the Neural Circuitry Underlying Third Party Punishment

This research aims to explore the neural correlates involved in altruistic punishment, parochial altruism and anti-social punishment, using the Third-Party Punishment (TPP) game. In particular, this study considered these punishment behaviors in in-group vs. out-group game settings, to compare how people behave with members of their own national group and with members of another national group. The results showed that participants act altruistically to protect in-group members. This study indicates that norm violation in in-group (but not in out-group) settings results in increased activity in the medial prefrontal cortex and temporo-parietal junction, brain regions involved in the mentalizing network, as the third-party attempts to understand or justify in-group members' behavior. Finally, exploratory analysis during anti-social punishment behavior showed brain activation recruitment of the ventromedial prefrontal cortex, an area associated with altered regulation of emotions.

Phasic dopamine release in the rat nucleus accumbens predicts approach and avoidance performance

Dopamine (DA) is critical for reward processing, but significantly less is known about its role in punishment avoidance. Using a combined approach-avoidance task, we measured phasic DA release in the nucleus accumbens (NAc) of rats during presentation of cues that predicted reward, punishment or neutral outcomes and investigated individual differences based on avoidance performance. Here we show that DA release within a single microenvironment is higher for reward and avoidance cues compared with neutral cues and positively correlated with poor avoidance behaviour. We found that DA release delineates trial-type during sessions with good avoidance but is non-selective during poor avoidance, with high release correlating with poor performance. These data demonstrate that phasic DA is released during cued approach and avoidance within the same microenvironment and abnormal processing of value signals is correlated with poor performance.

Large-scale functional brain connectivity during emotional engagement as revealed by beta-series correlation analysis

It has been hypothesized that the medial prefrontal cortex (mPFC) is a hub in the network that mediates appetitive responses whereas the amygdala is thought to mediate both aversive and appetitive processing. Both structures may facilitate adaptive responses to emotional challenge by linking perception, attention, memory, and motor circuits. We provide an initial exploration of these hypotheses by recording simultaneous EEG-fMRI in eleven participants viewing affective pictures. MPFC- and amygdala-seeded functional connectivity maps were generated by applying the beta-series correlation method. The mPFC-seeded correlation map encompassed visual regions, sensorimotor areas, prefrontal cortex, and medial temporal lobe structures, exclusively for pleasant content. For the amygdala-seeded correlation map, a similar set of distributed brain areas appeared in the unpleasant-neutral contrast, with the addition of structures such as the insula and thalamus. A substantially sparser network was recruited for the pleasant-neutral contrast. Using the late positive potential (LPP) to index the intensity of emotional engagement, functional connectivity was found to be stronger in trials with larger LPP. These results demonstrate that mPFC-mediated functional interactions are engaged specifically during appetitive processing, whereas the amygdala is coupled to distinct sets of brain regions during both aversive and appetitive processing. The strength of these interactions varies as a function of the intensity of emotional engagement.

Aversive Learning and Trait Aggression Influence Retaliatory Behavior

In two experiments (n = 35, n = 34), we used a modified fear-conditioning paradigm to investigate the role of aversive learning in retaliatory behavior in social context. Participants first completed an initial aversive learning phase in which the pairing of a neutral conditioned stimulus (CS; i.e., neutral face) with a naturally aversive unconditioned stimulus (US; electric shock) was learned. Then they were given an opportunity to interact (i.e., administer 0–2 shocks) with the same faces again, during a Test phase. In Experiment 2, we used the same paradigm with the addition of online trial-by-trial ratings (e.g., US expectancy and anger) to examine the role of aversive learning, anger, and the learned expectancy of receiving punishment more closely. Our results indicate that learned aversions influenced future retaliation in a social context. In both experiments, participants showed largest skin conductance responses (SCRs) to the faces paired with one or two shocks, demonstrating successful aversive learning. Importantly, participants administered more shocks to the faces paired with the most number of shocks when the opportunity was given during test. Also, our results revealed that aggressive traits (Buss and Perry Aggression scale) were associated with retaliation only toward CSs associated with aversive experiences. These two experiments show that aggressive traits, when paired with aversive learning experiences enhance the likelihood to act anti-socially toward others.

Safety out of control: dopamine and defence

We enjoy a sophisticated understanding of how animals learn to predict appetitive outcomes and direct their behaviour accordingly. This encompasses well-defined learning algorithms and details of how these might be implemented in the brain. Dopamine has played an important part in this unfolding story, appearing to embody a learning signal for predicting rewards and stamping in useful actions, while also being a modulator of behavioural vigour. By contrast, although choosing correct actions and executing them vigorously in the face of adversity is at least as important, our understanding of learning and behaviour in aversive settings is less well developed. We examine aversive processing through the medium of the role of dopamine and targets such as D2 receptors in the striatum. We consider critical factors such as the degree of control that an animal believes it exerts over key aspects of its environment, the distinction between 'better' and 'good' actual or predicted future states, and the potential requirement for a particular form of opponent to dopamine to ensure proper calibration of state values.

Effective connectivity of brain regions underlying third-party punishment: Functional MRI and Granger causality evidence

Third-party punishment (TPP) for norm violations is an essential deterrent in large-scale human societies, and builds on two essential cognitive functions: evaluating legal responsibility and determining appropriate punishment. Despite converging evidence that TPP is mediated by a specific set of brain regions, little is known about their effective connectivity (direction and strength of connections). Applying parametric event-related functional MRI in conjunction with multivariate Granger causality analysis, we asked healthy participants to estimate how much punishment a hypothetical perpetrator deserves for intentionally committing criminal offenses varying in levels of harm. Our results confirmed that TPP legal decisions are based on two domain-general networks: the mentalizing network for evaluating legal responsibility and the central-executive network for determining appropriate punishment. Further, temporal pole (TP) and dorsomedial prefrontal cortex (PFC) emerged as hubs of the mentalizing network, uniquely generating converging output connections to ventromedial PFC, temporo-parietal junction, and posterior cingulate. In particular, dorsomedial PFC received inputs only from TP and both its activation and its connectivity to dorsolateral PFC correlated with degree of punishment. This supports the hypothesis that dorsomedial PFC acts as the driver of the TPP activation pattern, leading to the decision on the appropriate punishment. In conclusion, these results advance our understanding of the organizational elements of the TPP brain networks and provide better insights into the mental states of judges and jurors tasked with blaming and punishing legal wrongs.

Is Brain Activity during Action Observation Modulated by the Perceived Fairness of the Actor?

Perceiving other people's actions triggers activity in premotor and parietal areas, brain areas also involved in executing and sensing our own actions. Paralleling this phenomenon, observing emotional states (including pain) in others is associated with activity in the same brain areas as activated when experiencing similar emotions directly. This emotion perception associated activity has been shown to be affected by the perceived fairness of the actor, and in-group membership more generally. Here, we examine whether action observation associated brain activity is also affected by the perceived social fairness of the actors. Perceived fairness was manipulated using an alternating iterated Prisoner's Dilemma game between the participant and two confederates, one of whom played fairly and the other unfairly. During fMRI scanning the participants watched movies of the confederates performing object-directed hand actions, and then performed hand actions themselves. Mass-univariate analysis showed that observing the actions triggered robust activation in regions associated with action execution, but failed to identify a strong modulation of this activation based on perceived fairness. Multivariate pattern analysis, however, identified clusters potentially carrying information about the perceived fairness of the actor in the middle temporal gyrus, left postcentral gyrus, right inferior parietal lobule, right middle cingulate cortex, right angular gyrus, and right superioroccipital gyrus. Despite being identified by a whole-brain searchlight analysis (and so without anatomical restriction), these clusters fall into areas frequently associated with action observation. We conclude that brain activity during action observation may be modulated by perceived fairness, but such modulation is subtle; robust activity is associated with observing the actions of both fair and unfair individuals.

A Conclusion, Yet an Opening to Enriching the Normative Approach of Culture

We compile in this article the target article authors’ thoughtful responses to the commentaries. Their responses identify some common threads across the rich contents of the commentary pieces, interlink the observation and theoretical propositions in the commentaries with broader streams of research, present new perspectives inspired by the commentary contributors, and pose provocative questions to further ignite research efforts on the normative analysis of culture.

Contextual modulation of value signals in reward and punishment learning

Compared with reward seeking, punishment avoidance learning is less clearly understood at both the computational and neurobiological levels. Here we demonstrate, using computational modelling and fMRI in humans, that learning option values in a relative—context-dependent—scale offers a simple computational solution for avoidance learning. The context (or state) value sets the reference point to which an outcome should be compared before updating the option value. Consequently, in contexts with an overall negative expected value, successful punishment avoidance acquires a positive value, thus reinforcing the response. As revealed by post-learning assessment of options values, contextual influences are enhanced when subjects are informed about the result of the forgone alternative (counterfactual information). This is mirrored at the neural level by a shift in negative outcome encoding from the anterior insula to the ventral striatum, suggesting that value contextualization also limits the need to mobilize an opponent punishment learning system.

In contrast to predictions from learning theory, humans learn to seek rewards and avoid punishments equally well. Here the authors offer an elegant solution to this problem by demonstrating that humans learn option values relative to a reference point subserved by a common neural substrate.

A general approach-avoidance hypothesis of oxytocin: accounting for social and non-social effects of oxytocin

BACKGROUND

We critically reexamine extant theory and empirical study of Oxytocin. We question whether OT is, in fact, a "social neuropeptide" as argued in dominant theories of OT.

METHOD

We critically review human and animal research on the social and non-social effects of Oxytocin, including behavioral, psychophysiological, neurobiological, and neuroimaging studies.

RESULTS

We find that extant (social) theories of Oxytocin do not account for well-documented non-social effects of Oxytocin. Furthermore, we find a range of evidence that social and non-social effects of Oxytocin may be mediated by core approach-avoidance motivational processes.

CONCLUSIONS

We propose a General Approach-avoidance Hypothesis of Oxytocin (GAAO). We argue that the GAAO may provide a parsimonious account of established social and non-social effects of Oxytocin. We thus re-conceptualize the basic function(s) and mechanism(s) of action of Oxytocin. Finally, we highlight implications of the GAAO for basic and clinical research in humans

Social status modulates the neural response to unfairness

In human society, which is organized by social hierarchies, resources are usually allocated unequally and based on social status. In this study, we analyze how being endowed with different social statuses in a math competition affects the perception of fairness during asset allocation in a subsequent Ultimatum Game (UG). Behavioral data showed that when participants were in high status, they were more likely to reject unfair UG offers than in low status. This effect of social status correlated with activity in the right anterior insula (rAI) and with the functional connectivity between the rAI and a region in the anterior middle cingulate cortex, indicating that these two brain regions are crucial for integrating contextual factors and social norms during fairness perception. Additionally, there was an interaction between social status and UG offer fairness in the amygdala and thalamus, implicating the role of these regions in the modulation of social status on fairness perception. These results demonstrate the effect of social status on fairness perception and the potential neural underpinnings for this effect.

Neural substrates of intention--consequence integration and its impact on reactive punishment in interpersonal transgression

When evaluating interpersonal transgressions, people take into account both the consequential damage and the intention of the agent. The intention and consequence, however, do not always match, as is the case with accidents and failed attempts. We combined an interactive game and functional MRI to investigate the neural substrates underlying the processing of intention and consequence, and its bearing on reactive punishment. The participant interacted with anonymous partners, who decided to deliver pain stimulation either to himself/herself or to the participant to earn a monetary reward. In some cases, the decision was reversed by the computer. After pain delivery, the partner's intention was revealed. Unbeknownst to the partner, the participant was then allowed to punish the partner by reducing his/her monetary reward. Behaviorally, the punishment was lower in the accidental condition (unintended harm relative to intended harm) but higher in the failed-attempt condition (unintended no-harm relative to intended no-harm). Neurally, the left amygdala/hippocampus was activated in the conditions with blameworthy intention (i.e., intentional harm and failed attempt). The accidental (relative to intentional) harm activated the right temporoparietal junction (TPJ) and the anterior inferior frontal gyrus (IFG), while the failed attempt (relative to genuine no-harm) activated the anterior insula (AI) and the posterior IFG. Effective connectivity analysis revealed that in the unintentional conditions (i.e., accidental and failed attempt) the IFG received input from the TPJ and AI, and sent regulatory signals to the amygdala. These findings demonstrate that the processing of intention may gate the emotional responses to transgression and regulate subsequent reactive punishment.

Motivation and affective judgments differentially recruit neurons in the primate dorsolateral prefrontal and anterior cingulate cortex

The judgment of whether to accept or to reject an offer is determined by positive and negative affect related to the offer, but affect also induces motivational responses. Rewarding and aversive cues influence the firing rates of many neurons in primate prefrontal and cingulate neocortical regions, but it still is unclear whether neurons in these regions are related to affective judgment or to motivation. To address this issue, we recorded simultaneously the neuronal spike activities of single units in the dorsolateral prefrontal cortex (dlPFC) and the anterior cingulate cortex (ACC) of macaque monkeys as they performed approach-avoidance (Ap-Av) and approach-approach (Ap-Ap) decision-making tasks that can behaviorally dissociate affective judgment and motivation. Notably, neurons having activity correlated with motivational condition could be distinguished from neurons having activity related to affective judgment, especially in the Ap-Av task. Although many neurons in both regions exhibited similar, selective patterns of task-related activity, we found a larger proportion of neurons activated in low motivational conditions in the dlPFC than in the ACC, and the onset of this activity was significantly earlier in the dlPFC than in the ACC. Furthermore, the temporal onsets of affective judgment represented by neuronal activities were significantly slower in the low motivational conditions than in the other conditions. These findings suggest that motivation and affective judgment both recruit dlPFC and ACC neurons but with differential degrees of involvement and timing.

Neural components of altruistic punishment

Altruistic punishment, which occurs when an individual incurs a cost to punish in response to unfairness or a norm violation, may play a role in perpetuating cooperation. The neural correlates underlying costly punishment have only recently begun to be explored. Here we review the current state of research on the neural basis of altruism from the perspectives of costly punishment, emphasizing the importance of characterizing elementary neural processes underlying a decision to punish. In particular, we emphasize three cognitive processes that contribute to the decision to altruistically punish in most scenarios: inequity aversion, cost-benefit calculation, and social reference frame to distinguish self from others. Overall, we argue for the importance of understanding the neural correlates of altruistic punishment with respect to the core computations necessary to achieve a decision to punish.

Reward processing in neurodegenerative disease

Representation of reward value involves a distributed network including cortical and subcortical structures. Because neurodegenerative illnesses target specific anatomic networks that partially overlap with the reward circuit, they would be predicted to have distinct impairments in reward processing. This review presents the existing evidence of reward processing changes in neurodegenerative diseases including mild cognitive impairment (MCI), Alzheimer's disease, frontotemporal dementia, amyotrophic lateral sclerosis (ALS), Parkinson's disease, and Huntington's disease, as well as in healthy aging. Carefully distinguishing the different aspects of reward processing (primary rewards, secondary rewards, reward-based learning, and reward-based decision-making) and using tasks that differentiate the stages of processing reward will lead to improved understanding of this fundamental process and clarify a contributing cause of behavioral change in these illnesses.

Neural basis of increased costly norm enforcement under adversity

Humans are willing to punish norm violations even at a substantial personal cost. Using fMRI and a variant of the ultimatum game and functional magnetic resonance imaging, we investigated how the brain differentially responds to fairness in loss and gain domains. Participants (responders) received offers from anonymous partners indicating a division of an amount of monetary gain or loss. If they accept, both get their shares according to the division; if they reject, both get nothing or lose the entire stake. We used a computational model to derive perceived fairness of offers and participant-specific inequity aversion. Behaviorally, participants were more likely to reject unfair offers in the loss (vs gain) domain. Neurally, the positive correlation between fairness and activation in ventral striatum was reduced, whereas the negative correlations between fairness and activations in dorsolateral prefrontal cortex were enhanced in the loss domain. Moreover, rejection-related dorsal striatum activation was higher in the loss domain. Furthermore, the gain-loss domain modulates costly punishment only when unfair behavior was directed toward the participants and not when it was directed toward others. These findings provide neural and computational accounts of increased costly norm enforcement under adversity and advanced our understanding of the context-dependent nature of fairness preference.

Reward processing by the lateral habenula in normal and depressive behaviors

The brain reward circuit has a central role in reinforcing behaviors that are rewarding and preventing behaviors that lead to punishment. Recent work has shown that the lateral habenula is an important part of the reward circuit by providing 'negative value' signals to the dopaminergic and serotonergic systems. Studies have also suggested that dysfunction of the lateral habenula is associated with psychiatric disorders, including major depression. Here, we discuss insights gained from neuronal recordings in monkeys regarding how the lateral habenula processes reward-related information. We then highlight recent optogenetic experiments in rodents addressing normal and abnormal functions of the habenula. Finally, we discuss how deregulation of the lateral habenula may be involved in depressive behaviors.

Moral reasoning in schizophrenia: an explorative study into economic decision making

INTRODUCTION

Previous research has suggested that patients with schizophrenia are impaired in appreciating moral values and social norms. Here, we tested the hypothesis--using economic games--that patients with schizophrenia are similar to normals in their ability to recognise unfairness, but perhaps more tolerant towards the acceptance of unfair offers depending on the severity of negative symptoms.

METHODS

Twenty-five patients with schizophrenia played an Ultimatum Game and a Dictator Game with punishment option to examine their ability to appreciate fairness rules and to reinforce equity in comparison to a healthy control group.

RESULTS

As hypothesised, patients accepted significantly more unfair offers than controls. However, rejection rates increased in relation to the unfairness of proposals in both groups. Patients did not differ significantly from controls in the likelihood of punishing unfairness in the Dictator Game or in punishment investment itself.

CONCLUSIONS

Patients with schizophrenia seem to be less sensitive towards unfairness to their own disadvantage, but punish unfairness at a comparable level to controls, which opposes the common view of a general lack of moral value appreciation in schizophrenia.

Neural mechanisms of gain-loss asymmetry in temporal discounting

Humans typically discount future gains more than losses. This phenomenon is referred to as the "sign effect" in experimental and behavioral economics. Although recent studies have reported associations between the sign effect and important social problems, such as obesity and incurring multiple debts, the biological basis for this phenomenon remains poorly understood. Here, we hypothesized that enhanced loss-related neural processing in magnitude and/or delay representation are causes of the sign effect. We examined participants performing intertemporal choice tasks involving future gains or losses and compared the brain activity of those who exhibited the sign effect and those who did not. When predicting future losses, significant differences were apparent between the two participant groups in terms of striatal activity representing delay length and in insular activity representing sensitivity to magnitude. Furthermore, participants with the sign effect exhibited a greater insular response to the magnitude of loss than to that of gain, and also a greater striatal response to the delay of loss than to that of gain. These findings may provide a new biological perspective for the development of novel treatments and preventive measures for social problems associated with the sign effect.

Learning biases underlying individual differences in sensitivity to social rejection

People vary greatly in their dispositions to anxiously expect, readily perceive, and strongly react to social rejection (rejection sensitivity [RS]) with implications for social functioning and health. Here, we examined how RS influences learning about social threat. Using a classical fear conditioning task, we established that high compared to low individuals displayed a resistance to extinction of the conditioned response to angry faces, but not to neutral faces or nonsocial stimuli. Our findings suggest that RS biases the flexible updating of acquired expectations for threat, which helps to explain how RS operates as a self-fulfilling prophecy.

Anatomical correlates of reward-seeking behaviours in behavioural variant frontotemporal dementia

Behavioural variant frontotemporal dementia is characterized by abnormal responses to primary reward stimuli such as food, sex and intoxicants, suggesting abnormal functioning of brain circuitry mediating reward processing. The goal of this analysis was to determine whether abnormalities in reward-seeking behaviour in behavioural variant frontotemporal dementia are correlated with atrophy in regions known to mediate reward processing. Review of case histories in 103 patients with behavioural variant frontotemporal dementia identified overeating or increased sweet food preference in 80 (78%), new or increased alcohol or drug use in 27 (26%), and hypersexuality in 17 (17%). For each patient, a primary reward-seeking score of 0-3 was created with 1 point given for each target behaviour (increased seeking of food, drugs, or sex). Voxel-based morphometry performed in 91 patients with available imaging revealed that right ventral putamen and pallidum atrophy correlated with higher reward-seeking scores. Each of the reward-related behaviours involved partially overlapping right hemisphere reward circuit regions including putamen, globus pallidus, insula and thalamus. These findings indicate that in some patients with behavioural variant frontotemporal dementia, low volume of subcortical reward-related structures is associated with increased pursuit of primary rewards, which may be a product of increased thalamocortical feedback.

Where there's smoke, there's fire: the brain reactivity of chronic smokers when exposed to the negative value of smoking

RATIONAL

The addictive nature of smoking is characterized by responses to cigarette stimuli that significantly impede smoking cessation efforts. Studies have shown that smokers are roused by appetitive smoking-related stimuli, and their consumption tends to be unaffected by the negative value of smoking.

PURPOSE

Using functional magnetic resonance imaging, the goal of this study was two-fold: to examine the brain reactivity of chronic smokers when processing the negative value of smoking using aversive smoking-related cues; to further characterize this response by comparing the latter to the processing of aversive nonsmoking-related and appetitive smoking-related cues.

METHOD

Thirty chronic smokers passively viewed aversive smoking-related, aversive nonsmoking-related, appetitive smoking-related and neutral images presented in a block design while being scanned.

RESULTS

Aversive smoking-related stimuli elicited significantly greater activation in the medial prefrontal cortex, amygdala, inferior frontal gyrus and lateral orbitofrontal cortex than neutral stimuli. Aversive smoking-related stimuli elicited lower activation in the parahippocampal gyrus, insula and inferior frontal gyrus compared to the aversive nonsmoking-related condition, as well as lower activation in the posterior cingulate, precuneus and medial prefrontal cortices compared to appetitive smoking-related cues.

CONCLUSION

The brain activation pattern observed suggests that chronic smokers experience an aversive response when processing aversive smoking-related stimuli, however we argue that the latter triggers a weaker negative emotional and driving response than the aversive non-smoking-related and appetitive smoking-related cues respectively. These fMRI results highlight potentially important processes underlying the insensitivity to the negative value of smoking, an important characteristic of addiction.

Medial-lateral organization of the orbitofrontal cortex

Emerging evidence suggests that specific cognitive functions localize to different subregions of OFC, but the nature of these functional distinctions remains unclear. One prominent theory, derived from human neuroimaging, proposes that different stimulus valences are processed in separate orbital regions, with medial and lateral OFC processing positive and negative stimuli, respectively. Thus far, neurophysiology data have not supported this theory. We attempted to reconcile these accounts by recording neural activity from the full medial-lateral extent of the orbital surface in monkeys receiving rewards and punishments via gain or loss of secondary reinforcement. We found no convincing evidence for valence selectivity in any orbital region. Instead, we report differences between neurons in central OFC and those on the inferior-lateral orbital convexity, in that they encoded different sources of value information provided by the behavioral task. Neurons in inferior convexity encoded the value of external stimuli, whereas those in OFC encoded value information derived from the structure of the behavioral task. We interpret these results in light of recent theories of OFC function and propose that these distinctions, not valence selectivity, may shed light on a fundamental organizing principle for value processing in orbital cortex.

Cost-benefit analysis: the first real rule of fight club?

Competition is ubiquitous among social animals. Vying against a conspecific to achieve a particular outcome often requires one to act aggressively, but this is a costly and inherently risky behavior. So why do we aggressively compete, or at the extreme, fight against others? Early work suggested that competitive aggression might stem from an innate aggressive tendency, emanating from subcortical structures. Later work highlighted key cortical regions that contribute toward an instrumental aggression network, one that is recruited or suppressed as needed to achieve a goal. Recent neuroimaging work hints that competitive aggression is upmost a cost-benefit decision, in that it appears to recruit many components of traditional, non-social decision-making networks. This review provides a historical glimpse into the neuroscience of competitive aggression, and proposes a conceptual advancement for studying competitive behavior by outlining how utility calculations of contested-for resources are skewed, pre- and post-competition. A basic multi-factorial model of utility assessment is proposed to account for competitive endowment effects that stem from the presence of peers, peer salience and disposition, and the tactical effort required for victory. In part, competitive aggression is a learned behavior that should only be repeated if positive outcomes are achieved. However, due to skewed utility assessments, deviations of associative learning occur. Hence truly careful cost-benefit analysis is warranted before choosing to vie against another.

Understanding decision neuroscience: a multidisciplinary perspective and neural substrates

The neuroscience of decision making is a rapidly evolving multidisciplinary research area that employs neuroscientific techniques to explain various parameters associated with decision making behavior. In this chapter, we emphasize the role of multiple disciplines such as psychology, economics, neuroscience, and computational approaches in understanding the phenomenon of decision making. Further, we present a theoretical approach that suggests understanding the building blocks of decision making as bottom-up processes and integrate these with top-down modulatory factors. Relevant neurophysiological and neuroimaging findings that have used the building-block approach are reviewed. A unifying framework emphasizing multidisciplinary views would bring further insights into the active research area of decision making. Pointing to future directions for research, we focus on the role of computational approaches in such a unifying framework.

The nucleus accumbens is involved in both the pursuit of social reward and the avoidance of social punishment

Human social motivation is characterized by the pursuit of social reward and the avoidance of social punishment. The ventral striatum/nucleus accumbens (VS/Nacc), in particular, has been implicated in the reward component of social motivation, i.e., the 'wanting' of social incentives like approval. However, it is unclear to what extent the VS/Nacc is involved in avoiding social punishment like disapproval, an intrinsically pleasant outcome. Thus, we conducted an event-related functional magnetic resonance imaging (fMRI) study using a social incentive delay task with dynamic video stimuli instead of static pictures as social incentives in order to examine participants' motivation for social reward gain and social punishment avoidance. As predicted, the anticipation of avoidable social punishment (i.e., disapproval) recruited the VS/Nacc in a manner that was similar to VS/Nacc activation observed during the anticipation of social reward gain (i.e., approval). Stronger VS/Nacc activity was accompanied by faster reaction times of the participants to obtain those desired outcomes. This data support the assumption that dynamic social incentives elicit robust VS/Nacc activity, which likely reflects motivation to obtain social reward and to avoid social punishment. Clinical implications regarding the involvement of the VS/Nacc in social motivation dysfunction in autism and social phobia are discussed.

Electrical stimulation of lateral habenula during learning: frequency-dependent effects on acquisition but not retrieval of a two-way active avoidance response

The lateral habenula (LHb) is an epithalamic structure involved in signaling reward omission and aversive stimuli, and it inhibits dopaminergic neurons during motivated behavior. Less is known about LHb involvement in the acquisition and retrieval of avoidance learning. Our previous studies indicated that brief electrical stimulation of the LHb, time-locked to the avoidance of aversive footshock (presumably during the positive affective “relief” state that occurs when an aversive outcome is averted), inhibited the acquisition of avoidance learning. In the present study, we used the same paradigm to investigate different frequencies of LHb stimulation. The effect of 20 Hz vs. 50 Hz vs. 100 Hz stimulation was investigated during two phases, either during acquisition or retrieval in Mongolian gerbils. The results indicated that 50 Hz, but not 20 Hz, was sufficient to produce a long-term impairment in avoidance learning, and was somewhat more effective than 100 Hz in this regard. None of the stimulation parameters led to any effects on retrieval of avoidance learning, nor did they affect general motor activity. This suggests that, at frequencies in excess of the observed tonic firing rates of LHb neurons (>1–20 Hz), LHb stimulation may serve to interrupt the consolidation of new avoidance memories. However, these stimulation parameters are not capable of modifying avoidance memories that have already undergone extensive consolidation.

Distinct encoding of risk and value in economic choice between multiple risky options

Neural encoding of value-based stimuli is suggested to involve representations of summary statistics, including risk and expected value (EV). A more complex, but ecologically more common, context is when multiple risky options are evaluated together. However, it is unknown whether encoding related to option evaluation in these situations involves similar principles. Here we employed fMRI during a task that parametrically manipulated EV and risk in two simultaneously presented lotteries, both of which contained either gains or losses. We found representations of EV in medial prefrontal cortex and anterior insula, an encoding that was dependent on which option was chosen (i.e. chosen and unchosen EV) and whether the choice was over gains or losses. Parietal activity reflected whether the riskier or surer option was selected, whilst activity in a network of regions that also included parietal cortex reflected both combined risk and difference in risk for the two options. Our findings provide support for the idea that summary statistics underpin a representation of value-based stimuli, and further that these summary statistics undergo distinct forms of encoding.

Temporal characteristics of the influence of punishment on perceptual decision making in the human brain

Perceptual decision making is the process by which information from sensory systems is combined and used to influence our behavior. In addition to the sensory input, this process can be affected by other factors, such as reward and punishment for correct and incorrect responses. To investigate the temporal dynamics of how monetary punishment influences perceptual decision making in humans, we collected electroencephalography (EEG) data during a perceptual categorization task whereby the punishment level for incorrect responses was parametrically manipulated across blocks of trials. Behaviorally, we observed improved accuracy for high relative to low punishment levels. Using multivariate linear discriminant analysis of the EEG, we identified multiple punishment-induced discriminating components with spatially distinct scalp topographies. Compared with components related to sensory evidence, components discriminating punishment levels appeared later in the trial, suggesting that punishment affects primarily late postsensory, decision-related processing. Crucially, the amplitude of these punishment components across participants was predictive of the size of the behavioral improvements induced by punishment. Finally, trial-by-trial changes in prestimulus oscillatory activity in the alpha and gamma bands were good predictors of the amplitude of these components. We discuss these findings in the context of increased motivation/attention, resulting from increases in punishment, which in turn yields improved decision-related processing.

Cognitive systems for revenge and forgiveness

Minimizing the costs that others impose upon oneself and upon those in whom one has a fitness stake, such as kin and allies, is a key adaptive problem for many organisms. Our ancestors regularly faced such adaptive problems (including homicide, bodily harm, theft, mate poaching, cuckoldry, reputational damage, sexual aggression, and the infliction of these costs on one's offspring, mates, coalition partners, or friends). One solution to this problem is to impose retaliatory costs on an aggressor so that the aggressor and other observers will lower their estimates of the net benefits to be gained from exploiting the retaliator in the future. We posit that humans have an evolved cognitive system that implements this strategy – deterrence – which we conceptualize as a revenge system. The revenge system produces a second adaptive problem: losing downstream gains from the individual on whom retaliatory costs have been imposed. We posit, consequently, a subsidiary computational system designed to restore particular relationships after cost-imposing interactions by inhibiting revenge and motivating behaviors that signal benevolence for the harmdoer. The operation of these systems depends on estimating the risk of future exploitation by the harmdoer and the expected future value of the relationship with the harmdoer. We review empirical evidence regarding the operation of these systems, discuss the causes of cultural and individual differences in their outputs, and sketch their computational architecture.

Differential reward coding in the subdivisions of the primate caudate during an oculomotor task

The basal ganglia play a pivotal role in reward-oriented behavior. The striatum, an input channel of the basal ganglia, is composed of subdivisions that are topographically connected with different cortical and subcortical areas. To test whether reward information is differentially processed in the different parts of the striatum, we compared reward-related neuronal activity along the dorsolateral-ventromedial axis in the caudate nucleus of monkeys performing an asymmetrically rewarded oculomotor task. In a given block, a target in one position was associated with a large reward, whereas the other target was associated with a small reward. The target position-reward value contingency was switched between blocks. We found the following: (1) activity that reflected the block-wise reward contingency emerged before the appearance of a visual target, and it was more prevalent in the dorsal, rather than central and ventral, caudate; (2) activity that was positively related to the reward size of the current trial was evident, especially after reward delivery, and it was more prevalent in the ventral and central, rather than dorsal, caudate; and (3) activity that was modulated by the memory of the outcomes of the previous trials was evident in the dorsal and central caudate. This multiple reward information, together with the target-direction information, was represented primarily by individual caudate neurons, and the different reward information was represented in caudate subpopulations with distinct electrophysiological properties, e.g., baseline firing and spike width. These results suggest parallel processing of different reward information by the basal ganglia subdivisions defined by extrinsic connections and intrinsic properties.

Remembering with gains and losses: effects of monetary reward and punishment on successful encoding activation of source memories

The motivation of getting rewards or avoiding punishments reinforces learning behaviors. Although the neural mechanisms underlying the effect of rewards on episodic memory have been demonstrated, there is little evidence of the effect of punishments on this memory. Our functional magnetic resonance imaging (fMRI) study investigated the effects of monetary rewards and punishments on activation during the encoding of source memories. During encoding, participants memorized words (item) and locations of presented words (source) under 3 conditions (Reward, Punishment, and Control). During retrieval, participants retrieved item and source memories of the words and were rewarded or penalized according to their performance. Source memories encoded with rewards or punishments were remembered better than those without such encoding. fMRI data demonstrated that the ventral tegmental area and substantia nigra and nucleus accumbens activations reflected both the processes of reward and punishment, whereas insular activation increased as a linear function of punishment. Activation in the hippocampus and parahippocampal cortex predicted subsequent retrieval success of source memories. Additionally, correlations between these reward/punishment-related regions and the hippocampus were significant. The successful encoding of source memories could be enhanced by punishments and rewards, and interactions between reward/punishment-related regions and memory-related regions could contribute to memory enhancement by reward and/or punishment.

Neural coding of computational factors affecting decision making

We constantly need to make decisions that can result in rewards of different amounts with different probabilities and at different timing. To characterize the neural coding of such computational factors affecting value-based decision making, we have investigated how reward information processing is influenced by parameters such as reward magnitude, probability, delay, effort, and uncertainty using either fMRI in healthy humans or intracranial recordings in patients with epilepsy. We decomposed brain signals modulated by these computational factors, showing that prediction error (PE), salient PE, and uncertainty signals are computed in partially overlapping brain circuits and that both transient and sustained uncertainty signals coexist in the brain. When investigating the neural representation of primary and secondary rewards, we found both a common brain network, including the ventromedial prefrontal cortex and ventral striatum, and a functional organization of the orbitofrontal cortex according to reward type. Moreover, separate valuation systems were engaged for delay and effort costs when deciding between options. Finally, genetic variations in dopamine-related genes influenced the response of the reward system and may contribute to individual differences in reward-seeking behavior and in predisposition to neuropsychiatric disorders.

Amygdala dysfunction attenuates frustration-induced aggression in psychopathic individuals in a non-criminal population

BACKGROUND

Individuals with psychopathy have an increased tendency toward certain types of aggression. We hypothesized that successful psychopaths, who have no criminal convictions but can be diagnosed with psychopathy in terms of personality characteristics, are skilled at regulating aggressive impulses, compared to incarcerated unsuccessful psychopaths.

METHODS

In this block-designed functional magnetic resonance imaging (fMRI) study, we sought to clarify the neural mechanisms underlying differences in frustration-induced aggression as a function of psychopathy in non-criminal populations. Twenty male undergraduate students who completed a self-report psychopathy questionnaire were scanned while they completed a task in which they either could or could not punish other individuals who made unfair offers of monetary distribution.

RESULTS

Individuals with high psychopathic tendencies were less likely to make a decision to inflict costly punishment on people proposing unfair offers. During this decision-making, psychopathy was associated with less amygdala activity in response to the unfairness of offers. Moreover, the amygdala dysfunction in psychopathic individuals was associated with reduced functional connectivity with dopaminergic-related areas, including the striatum, when punishment was available compared to when it was unavailable.

LIMITATIONS

The possibility that levels of psychopathic traits in a regular population were milder than in incarcerated populations cannot be ruled out.

CONCLUSIONS

The findings indicate that amygdala dysfunction underlies affective deficits of psychopathy. We propose that the insensitivity of the amygdala to the affective significance of social stimuli contributes to an increased risk of violation of social norms, but enhances the ability to attenuate impulses toward maladaptive aggression in successful psychopaths.

Complexity and competition in appetitive and aversive neural circuits

Decision-making often involves using sensory cues to predict possible rewarding or punishing reinforcement outcomes before selecting a course of action. Recent work has revealed complexity in how the brain learns to predict rewards and punishments. Analysis of neural signaling during and after learning in the amygdala and orbitofrontal cortex, two brain areas that process appetitive and aversive stimuli, reveals a dynamic relationship between appetitive and aversive circuits. Specifically, the relationship between signaling in appetitive and aversive circuits in these areas shifts as a function of learning. Furthermore, although appetitive and aversive circuits may often drive opposite behaviors – approaching or avoiding reinforcement depending upon its valence – these circuits can also drive similar behaviors, such as enhanced arousal or attention; these processes also may influence choice behavior. These data highlight the formidable challenges ahead in dissecting how appetitive and aversive neural circuits interact to produce a complex and nuanced range of behaviors.

The role of dopamine in the context of aversive stimuli with particular reference to acoustically signaled avoidance learning

Learning from punishment is a powerful means for behavioral adaptation with high relevance for various mechanisms of self-protection. Several studies have explored the contribution of released dopamine (DA) or responses of DA neurons on reward seeking using rewards such as food, water, and sex. Phasic DA signals evoked by rewards or conditioned reward predictors are well documented, as are modulations of these signals by such parameters as reward magnitude, probability, and deviation of actually occurring from expected rewards. Less attention has been paid to DA neuron firing and DA release in response to aversive stimuli, and the prediction and avoidance of punishment. In this review, we first focus on DA changes in response to aversive stimuli as measured by microdialysis and voltammetry followed by the change in electrophysiological signatures by aversive stimuli and fearful events. We subsequently focus on the role of DA and effect of DA manipulations on signaled avoidance learning, which consists of learning the significance of a warning cue through Pavlovian associations and the execution of an instrumental avoidance response. We present a coherent framework utilizing the data on microdialysis, voltammetry, electrophysiological recording, electrical brain stimulation, and behavioral analysis. We end by outlining current gaps in the literature and proposing future directions aimed at incorporating technical and conceptual progress to understand the involvement of reward circuit on punishment based decisions.

Empathy moderates the effect of repetitive transcranial magnetic stimulation of the right dorsolateral prefrontal cortex on costly punishment

Humans incur considerable costs to punish unfairness directed towards themselves or others. Recent studies using repetitive transcranial magnetic stimulation (rTMS) suggest that the right dorsolateral prefrontal cortex (DLPFC) is causally involved in such strategic decisions. Presently, two partly divergent hypotheses are discussed, suggesting either that the right DLPFC is necessary to control selfish motives by implementing culturally transmitted social norms, or is involved in suppressing emotion-driven prepotent responses to perceived unfairness. Accordingly, we studied the role of the DLPFC in costly (i.e. third party) punishment by applying rTMS to the left and right DLPFC before playing a Dictator Game with the option to punish observed unfair behavior (DG-P). In addition, sham stimulation took place. Individual differences in empathy were assessed with the German version of the Interpersonal Reactivity Index. Costly punishment increased (non-significantly) upon disruption of the right--but not the left--DLPFC as compared to sham stimulation. However, empathy emerged as a highly significant moderator variable of the effect of rTMS over the right, but not left, DLPFC, suggesting that the right DLPFC is involved in controlling prepotent emotional responses to observed unfairness, depending on individual differences in empathy.

Can, and should, behavioural neuroscience influence public policy?

Recent years have seen enormous demand amongst policy makers for new insights from the behavioural sciences, especially neuroscience. This demand is matched by an increasing willingness on behalf of behavioural scientists to translate the policy implications of their work. But can neuroscience really help shape the governance of a nation? Or does this represent growing misuse of neuroscience to attach scientific authority to policy, plus a clutch of neuroscientists trying to overstate their findings for a taste of power?