Behavioral and neurobiological mechanisms of punishment: implications for psychiatric disorders

Orbitofrontal and Prelimbic Cortices Serve Complementary Roles in Adapting Reward Seeking to Learned Anxiety

BACKGROUND

Anxiety is a common symptom of several mental health disorders and adversely affects motivated behaviors. Anxiety can emerge from associating risk of future harm while engaged in goal-guided actions. Using a recently developed behavioral paradigm to model this aspect of anxiety, we investigated the role of 2 cortical subregions, the prelimbic medial frontal cortex (PL) and lateral orbitofrontal cortex (lOFC), which have been implicated in anxiety and outcome expectation, in flexible representation of actions associated with harm risk.

METHODS

A seek-take reward-guided instrumental task design was used to train animals (N = 8) to associate the seek action with a variable risk of punishment. After learning, animals underwent extinction training for this association. Fiber photometry was used to measure and compare neuronal activity in the PL and lOFC during learning and extinction.

RESULTS

Animals increased action suppression in response to punishment contingencies. This increase dissipated after extinction training. These behavioral changes were associated with region-specific changes in neuronal activity. PL neuronal activity preferentially adapted to the threat of punishment, whereas lOFC activity adapted to safe aspects of the task. Moreover, correlated activity between these regions was suppressed during actions associated with harm risk, suggesting that these regions may guide behavior independently under anxiety.

CONCLUSIONS

These findings suggest that the PL and lOFC serve distinct but complementary roles in the representation of learned anxiety. This dissociation may provide a mechanism to explain how overlapping cortical systems are implicated in reward-guided action execution during anxiety.

Nutritional Modulation of the Gut-Brain Axis: A Comprehensive Review of Dietary Interventions in Depression and Anxiety Management

Mental health is an increasing topic of focus since more than 500 million people in the world suffer from depression and anxiety. In this multifactorial disorder, parameters such as inflammation, the state of the microbiota and, therefore, the patient's nutrition are receiving more attention. In addition, food products are the source of many essential ingredients involved in the regulation of mental processes, including amino acids, neurotransmitters, vitamins, and others. For this reason, this narrative review was carried out with the aim of analyzing the role of nutrition in depression and anxiety disorders. To reach the review aim, a critical review was conducted utilizing both primary sources, such as scientific publications and secondary sources, such as bibliographic indexes, web pages, and databases. The search was conducted in PsychINFO, MedLine (Pubmed), Cochrane (Wiley), Embase, and CinAhl. The results show a direct relationship between what we eat and the state of our nervous system. The gut-brain axis is a complex system in which the intestinal microbiota communicates directly with our nervous system and provides it with neurotransmitters for its proper functioning. An imbalance in our microbiota due to poor nutrition will cause an inflammatory response that, if sustained over time and together with other factors, can lead to disorders such as anxiety and depression. Changes in the functions of the microbiota-gut-brain axis have been linked to several mental disorders. It is believed that the modulation of the microbiome composition may be an effective strategy for a new treatment of these disorders. Modifications in nutritional behaviors and the use of ergogenic components are presented as important non-pharmacological interventions in anxiety and depression prevention and treatment. It is desirable that the choice of nutritional and probiotic treatment in individual patients be based on the results of appropriate biochemical and microbiological tests.

Role of serotonin neurons in the dorsal raphe nucleus in heroin self-administration and punishment

One hallmark of substance use disorder is continued drug use despite negative consequences. When drug-taking behavior is punished with aversive stimuli, i.e. footshock, rats can also be categorized into punishment-resistant or compulsive vs. punishment-sensitive or non-compulsive phenotypes. The serotonin (5-hydroxytryptamine, 5-HT) system modulates responses to both reward and punishment. The goal of the current study was to examine punishment phenotypes in heroin self-administration and to determine the role of dorsal raphe nucleus (DRN) 5-HT neurons in both basal and punished heroin self-administration. First, rats were exposed to punished heroin self-administration and neuronal excitability of DRN 5-HT neurons was compared between punishment-resistant and punishment-sensitive phenotypes using ex vivo electrophysiology. Second, DRN 5-HT neuronal activity was manipulated in vivo during basal and punished heroin self-administration using chemogenetic tools in a Tph2-iCre rat line. While rats separated into punishment-resistant and punishment-sensitive phenotypes for punished heroin self-administration, DRN 5-HT neuronal excitability did not differ between the phenotypes. While chemogenetic inhibition of DRN 5-HT neurons was without effect, chemogenetic activation of DRN 5-HT neurons increased both basal and punished heroin self-administration selectively in punishment-resistant animals. Additionally, the responsiveness to chemogenetic activation of DRN 5-HT neurons in basal self-administration and motivation for heroin in progressive ratio each predicted resistance to punishment. Therefore, our data support the role for the DRN 5-HT system in compulsive heroin self-administration.

The multiple faces of footshock punishment in animal research on addiction

Continued drug use despite negative consequences is a hallmark of addiction commonly modelled in rodents using punished drug intake. Over the years, addiction research highlighted two subpopulations of punishment sensitive and resistant animals. While helpful to interrogate the neurobiology of drug-related behaviors, these procedures carry some weaknesses that need to be recognized and eventually defused. Mainly focusing on footshock-related work, we will first discuss the criteria used to define punishment-resistant animals and how their relative arbitrariness may impact our findings. With the overarching goal of improving our interpretation of the punishment-resistant phenotype, we will evaluate how tailored punishment protocols may better apprehend resistance to punishment, and how testing the robustness of punishment resistance could yield new results and strengthen interpretations. Second, we will question whether and to what extent punishment sensitivity, as currently defined, is reflective of abstinence and suggest that punishment resistance is, in fact, a prerequisite to model abstinence from addiction. Again, we will examine how challenging the robustness of the punishment-sensitive phenotype may help to better characterize it. Finally, we will evaluate whether diminished relapse-like behavior after repeated punishment-induced abstinence could not only contribute to better understand the mechanisms of abstinence, but also uniquely model progressive recovery (i.e., after repeated failed attempts at recovery) which is the norm in people with addiction. Altogether, by questioning the strengths and weaknesses of our models, we would like to open discussions on the different ways we interpret punishment sensitivity and resistance and the aspects that remain to be explored.

Understanding sex differences and the translational value of models of persistent substance use despite negative consequences

Persistent substance use despite negative consequences is a key facet of substance use disorder. The last decade has seen the preclinical field adopt the use of punishment to model adverse consequences associated with substance use. This has largely involved the pairing of drug use with either electric foot shock or quinine, a bitter tastant. Whilst at face value, these punishers may model aspects of the physical and psychological consequences of substance use, such models are yet to assist the development of approved medications for treatment. This review discusses progress made with animal models of punishment to understand the behavioral consequences of persistent substance use despite negative consequences. We highlight the importance of examining sex differences, especially when the behavioral response to punishment changes following drug exposure. Finally, we critique the translational value these models provide for the substance use disorder field.

A rat model of operant negative reinforcement in opioid-dependent males and females

RATIONALE AND OBJECTIVE

Avoidance of opioid withdrawal plays a key role in human opioid addiction. Here, we present a procedure for studying operant negative reinforcement in rats that was inspired by primate procedures where opioid-dependent subjects lever-press to prevent naloxone infusions.

METHODS

In Experiment 1, we trained rats (n = 30, 15 females) to lever-press to escape and then avoid mild footshocks (0.13-0.27 mA) for 35 days (30 trials/d). Next, we catheterized them and implanted minipumps containing methadone (10 mg/kg/day) or saline. We then paired (4 times, single session) a light cue (20-s) with a naloxone infusion (20 µg/kg, i.v) that precipitated opioid withdrawal. Next, we trained the rats to escape naloxone injections for 10 days (30 trials/d). Each trial started with the onset of the opioid-withdrawal cue. After 20-s, the lever extended, and an infusion of naloxone (1 to 2.2 µg/kg/infusion) began; a lever-press during an 11-s window terminated the withdrawal-paired cue and the infusion. In Experiment 2, we trained rats (n = 34, 17 females) on the same procedure but decreased the footshock escape/avoidance training to 20 days.

RESULTS

All rats learned to lever-press to escape or avoid mild footshocks. In both experiments, a subset, 56% (10/18) and 33% (8/24) of methadone-dependent rats learned to lever-press to escape naloxone infusions.

CONCLUSIONS

We introduce an operant negative reinforcement procedure where a subset of opioid-dependent rats learned to lever-press to escape withdrawal-inducing naloxone infusions. The procedure can be used to study mechanisms of individual differences in opioid negative reinforcement-related behaviors in opioid-dependent rats.

Effects of psilocybin on uncertain punishment learning

Psilocybin may provide a useful treatment for mood disorders including anxiety and depression but its mechanisms of action for these effects are not well understood. While recent preclinical work has begun to assess psilocybin's role in affective behaviors through innate anxiety or fear conditioning, there is scant evidence for its role in conflict between reward and punishment. The current study was designed to determine the impact of psilocybin on the learning of reward-punishment conflict associations, as well as its effects after learning, in male and female rats. We utilized a chained schedule of reinforcement that involved execution of safe and risky reward-guided actions under uncertain punishment. Different patterns of behavioral suppression by psilocybin emerged during learning versus after learning of risky action-reward associations. Psilocybin increased behavioral suppression in female rats as punishment associations were learned. After learning, psilocybin decreased behavioral suppression in both sexes. Thus, psilocybin produces divergent effects on action suppression during approach-avoidance conflict depending on when the conflict is experienced. This observation may have implications for its therapeutic mechanism of action.

Learning and memory processes in behavioural addiction: A systematic review

Similar to addictive substances, addictive behaviours such as gambling and gaming are associated with maladaptive modulation of key brain areas and functional networks implicated in learning and memory. Therefore, this review sought to understand how different learning and memory processes relate to behavioural addictions and to unravel their underlying neural mechanisms. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we systematically searched four databases - PsycINFO, PubMed, Scopus, and Web of Science using the agreed-upon search string. Findings suggest altered executive function-dependent learning processes and enhanced habit learning in behavioural addiction. Whereas the relationship between working memory and behavioural addiction is influenced by addiction type, working memory aspect, and task nature. Additionally, long-term memory is incoherent in individuals with addictive behaviours. Consistently, neurophysiological evidence indicates alterations in brain areas and networks implicated in learning and memory processes in behavioural addictions. Overall, the present review argues that, like substance use disorders, alteration in learning and memory processes may underlie the development and maintenance of behavioural addictions.

Medial prefrontal cortex suppresses reward-seeking behavior with risk of punishment by reducing sensitivity to reward

Reward-seeking behavior is frequently associated with risk of punishment. There are two types of punishment: positive punishment, which is defined as addition of an aversive stimulus, and negative punishment, involves the omission of a rewarding outcome. Although the medial prefrontal cortex (mPFC) is important in avoiding punishment, whether it is important for avoiding both positive and negative punishment and how it contributes to such avoidance are not clear. In this study, we trained male mice to perform decision-making tasks under the risks of positive (air-puff stimulus) and negative (reward omission) punishment, and modeled their behavior with reinforcement learning. Following the training, we pharmacologically inhibited the mPFC. We found that pharmacological inactivation of mPFC enhanced the reward-seeking choice under the risk of positive, but not negative, punishment. In reinforcement learning models, this behavioral change was well-explained as an increase in sensitivity to reward, rather than a decrease in the strength of aversion to punishment. Our results suggest that mPFC suppresses reward-seeking behavior by reducing sensitivity to reward under the risk of positive punishment.

Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning

In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration. We then exposed them to four days of punishment testing in which footshock was delivered randomly on one-third of trials. Before and after punishment testing (four days pre-punishment and ≥ four days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance for cocaine was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment in males, although it was associated with habitual responding both pre- and post-punishment in females, indicating that punishment resistance was predicted by habitual responding in food-seeking females. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Functional diversity of dopamine axons in prefrontal cortex during classical conditioning

Midbrain dopamine neurons impact neural processing in the prefrontal cortex (PFC) through mesocortical projections. However, the signals conveyed by dopamine projections to the PFC remain unclear, particularly at the single-axon level. Here, we investigated dopaminergic axonal activity in the medial PFC (mPFC) during reward and aversive processing. By optimizing microprism-mediated two-photon calcium imaging of dopamine axon terminals, we found diverse activity in dopamine axons responsive to both reward and aversive stimuli. Some axons exhibited a preference for reward, while others favored aversive stimuli, and there was a strong bias for the latter at the population level. Long-term longitudinal imaging revealed that the preference was maintained in reward- and aversive-preferring axons throughout classical conditioning in which rewarding and aversive stimuli were paired with preceding auditory cues. However, as mice learned to discriminate reward or aversive cues, a cue activity preference gradually developed only in aversive-preferring axons. We inferred the trial-by-trial cue discrimination based on machine learning using anticipatory licking or facial expressions, and found that successful discrimination was accompanied by sharper selectivity for the aversive cue in aversive-preferring axons. Our findings indicate that a group of mesocortical dopamine axons encodes aversive-related signals, which are modulated by both classical conditioning across days and trial-by-trial discrimination within a day.

Multivariate association between psychosocial environment, behaviors, and brain functional networks in adolescent depression

BACKGROUND

Adolescent depression shows high clinical heterogeneity. Brain functional networks serve as a powerful tool for investigating neural mechanisms underlying depression profiles. A key challenge is to characterize how variation in brain functional organization links to behavioral features and psychosocial environmental influences.

METHODS

We recruited 80 adolescents with major depressive disorder (MDD) and 42 healthy controls (HCs). First, we estimated the differences in functional connectivity of resting-state networks (RSN) between the two groups. Then, we used sparse canonical correlation analysis to characterize patterns of associations between RSN connectivity and symptoms, cognition, and psychosocial environmental factors in MDD adolescents. Clustering analysis was applied to stratify patients into homogenous subtypes according to these brain-behavior-environment associations.

RESULTS

MDD adolescents showed significantly hyperconnectivity between the ventral attention and cingulo-opercular networks compared with HCs. We identified one reliable pattern of covariation between RSN connectivity and clinical/environmental features in MDD adolescents. In this pattern, psychosocial factors, especially the interpersonal and family relationships, were major contributors to variation in connectivity of salience, cingulo-opercular, ventral attention, subcortical and somatosensory-motor networks. Based on this association, we categorized patients into two subgroups which showed different environment and symptoms characteristics, and distinct connectivity alterations. These differences were covered up when the patients were taken as a whole group.

CONCLUSION

This study identified the environmental exposures associated with specific functional networks in MDD youths. Our findings emphasize the importance of the psychosocial context in assessing brain function alterations in adolescent depression and have the potential to promote targeted treatment and precise prevention.

Updating functional brain units: Insights far beyond Luria

This paper reviews Luria's model of the three functional units of the brain. To meet this objective, several issues were reviewed: the theory of functional systems and the contributions of phylogenesis and embryogenesis to the brain's functional organization. This review revealed several facts. In the first place, the relationship/integration of basic homeostatic needs with complex forms of behavior. Secondly, the multi-scale hierarchical and distributed organization of the brain and interactions between cells and systems. Thirdly, the phylogenetic role of exaptation, especially in basal ganglia and cerebellum expansion. Finally, the tripartite embryogenetic organization of the brain: rhinic, limbic/paralimbic, and supralimbic zones. Obviously, these principles of brain organization are in contradiction with attempts to establish separate functional brain units. The proposed new model is made up of two large integrated complexes: a primordial-limbic complex (Luria's Unit I) and a telencephalic-cortical complex (Luria's Units II and III). As a result, five functional units were delineated: Unit I. Primordial or preferential (brainstem), for life-support, behavioral modulation, and waking regulation; Unit II. Limbic and paralimbic systems, for emotions and hedonic evaluation (danger and relevance detection and contribution to reward/motivational processing) and the creation of cognitive maps (contextual memory, navigation, and generativity [imagination]); Unit III. Telencephalic-cortical, for sensorimotor and cognitive processing (gnosis, praxis, language, calculation, etc.), semantic and episodic (contextual) memory processing, and multimodal conscious agency; Unit IV. Basal ganglia systems, for behavior selection and reinforcement (reward-oriented behavior); Unit V. Cerebellar systems, for the prediction/anticipation (orthometric supervision) of the outcome of an action. The proposed brain units are nothing more than abstractions within the brain's simultaneous and distributed physiological processes. As function transcends anatomy, the model necessarily involves transition and overlap between structures. Beyond the classic approaches, this review includes information on recent systemic perspectives on functional brain organization. The limitations of this review are discussed.

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

Midfrontal theta reactivity to conflict and error are linked to externalizing and internalizing respectively

Dimensional psychopathology scores measure symptom severity; cutting across disorder categories. Their clinical utility is high given comorbidity, but their neural basis is unclear. We used scalp electroencephalography (EEG) to concurrently assess neural activity across internalizing and externalizing traits. "Theta rhythm" (4-7 Hz) spectral power at the frontal midline site Fz in specific goal conflict and action error phases within a trial of a Stop-Signal Task was extracted using process-specific contrasts. A final sample of 146 community participants (63 males, 83 females; mean age = 36; SD = 9; range = 18 - 56), oversampled for externalizing disorder (49% diagnosed with a DSM-5 externalizing disorder), also supplied psychopathology and personality data. We used the Minnesota Multiphasic Personality Inventory-3 (MMPI-3) to measure symptoms and traits of psychopathology. An MMPI-3 measure of the higher-order internalizing psychopathology spectrum was positively correlated with action error theta. An MMPI-3 measure of the higher-order spectrum of externalizing psychopathology was negatively correlated with goal-conflict theta. We showed that goal-conflict and error theta activity are higher-order processes that index psychopathology severity. The associations extend into the nominally healthy range, and so reflect theta-related factors that apply to the general population as well as patients with sub-threshold diagnoses.

Motivational context and neurocomputation of stop expectation moderate early attention responses supporting proactive inhibitory control

Alterations in attention to cues signaling the need for inhibitory control play a significant role in a wide range of psychopathology. However, the degree to which motivational and attentional factors shape the neurocomputations of proactive inhibitory control remains poorly understood. The present study investigated how variation in monetary incentive valence and stake modulate the neurocomputational signatures of proactive inhibitory control. Adults (N = 46) completed a Stop-Signal Task (SST) with concurrent EEG recording under four conditions associated with stop performance feedback: low and high punishment (following unsuccessful stops) and low and high reward (following successful stops). A Bayesian learning model was used to infer individual's probabilistic expectations of the need to stop on each trial: P(stop). Linear mixed effects models were used to examine whether interactions between motivational valence, stake, and P(stop) parameters predicted P1 and N1 attention-related event-related potentials (ERPs) time-locked to the go-onset stimulus. We found that P1 amplitudes increased at higher levels of P(stop) in punished but not rewarded conditions, although P1 amplitude differences between punished and rewarded blocks were maximal on trials when the need to inhibit was least expected. N1 amplitudes were positively related to P(stop) in the high punishment condition (low N1 amplitude), but negatively related to P(stop) in the high reward condition (high N1 amplitude). Critically, high P(stop)-related N1 amplitude to the go-stimulus predicted behavioral stop success during the high reward block, providing evidence for the role of motivationally relevant context and inhibitory control expectations in modulating the proactive allocation of attentional resources that affect inhibitory control. These findings provide novel insights into the neurocomputational mechanisms underlying proactive inhibitory control under valence-dependent motivational contexts, setting the stage for developing motivation-based interventions that boost inhibitory control.

Dopaminergic modulation of sensitivity to immediate and delayed punishment during decision-making

Effective decision-making involves careful consideration of all rewarding and aversive outcomes. Importantly, negative outcomes often occur later in time, leading to underestimation, or "discounting," of these consequences. Despite the frequent occurrence of delayed outcomes, little is known about the neurobiology underlying sensitivity to delayed punishment during decision-making. The Delayed Punishment Decision-making Task (DPDT) addresses this by assessing sensitivity to delayed versus immediate punishment in rats. Rats initially avoid punished reinforcers, then select this option more frequently when delay precedes punishment. We used DPDT to examine effects of acute systemic administration of catecholaminergic drugs on sensitivity to delayed punishment in male and female adult rats. Cocaine did not affect choice of rewards with immediate punishment but caused a dose-dependent reduction in choice of delayed punishment. Neither activation nor blockade of D1-like dopamine receptor affected decision-making, but activation of D2-like dopamine receptors reduced choice of delayed punishment. D2 blockade did not attenuate cocaine's effects on decision-making, suggesting that cocaine's effects are not dependent on D2 receptor activation. Increasing synaptic norepinephrine via atomoxetine also reduced choice of delayed (but not immediate) punishment. Notably, when DPDT was modified from ascending to descending pre-punishment delays, these drugs did not affect choice of delayed or immediate punishment, although high-dose quinpirole impaired behavioral flexibility. In summary, sensitivity to delayed punishment is regulated by both dopamine and norepinephrine transmission in task-specific fashion. Understanding the neurochemical modulation of decision-making with delayed punishment is a critical step toward treating disorders characterized by aberrant sensitivity to negative consequences.

Delay of punishment highlights differential vulnerability to developing addiction-like behavior toward sweet food

Resistance to punishment is commonly used to measure the difficulty in refraining from rewarding activities when negative consequences ensue, which is a hallmark of addictive behavior. We recently developed a progressive shock strength (PSS) procedure in which individual rats can titrate the amount of punishment that they are willing to tolerate to obtain food rewards. Here, we investigated the effects of a range of delays (0-12 s) on resistance to punishment measured by PSS break points. As expected from delay discounting principles, we found that delayed shock was less effective as a punisher, as revealed by higher PSS breakpoints. However, this discounting effect was not equally distributed in the population of rats, and the introduction of a delay highlighted the existence of two populations: rats that were sensitive to immediate punishment were also sensitive to delayed shock, whereas rats that were resistant to immediate punishment showed strong temporal discounting of delayed punishment. Importantly, shock-sensitive rats suppressed responding even in subsequent non-punishment sessions, and they differed from shock-resistant rats in anxiety-like behavior, but not in sensitivity to pain. These results show that manipulation of temporal contingencies of punishment in the PSS procedure provides a valuable tool to identify individuals with a double vulnerability to addiction: low sensitivity to aversion and excessive discounting of negative future consequences. Conversely, the shock-sensitive population may provide a model of humans who are vulnerable to opportunity loss due to excessive anxiety.

Functional Diversity of Dopamine Axons in Prefrontal Cortex During Classical Conditioning

Midbrain dopamine neurons impact neural processing in the prefrontal cortex (PFC) through mesocortical projections. However, the signals conveyed by dopamine projections to the PFC remain unclear, particularly at the single-axon level. Here, we investigated dopaminergic axonal activity in the medial PFC (mPFC) during reward and aversive processing. By optimizing microprism-mediated two-photon calcium imaging of dopamine axon terminals, we found diverse activity in dopamine axons responsive to both reward and aversive stimuli. Some axons exhibited a preference for reward, while others favored aversive stimuli, and there was a strong bias for the latter at the population level. Long-term longitudinal imaging revealed that the preference was maintained in reward- and aversive-preferring axons throughout classical conditioning in which rewarding and aversive stimuli were paired with preceding auditory cues. However, as mice learned to discriminate reward or aversive cues, a cue activity preference gradually developed only in aversive-preferring axons. We inferred the trial-by-trial cue discrimination based on machine learning using anticipatory licking or facial expressions, and found that successful discrimination was accompanied by sharper selectivity for the aversive cue in aversive-preferring axons. Our findings indicate that a group of mesocortical dopamine axons encodes aversive-related signals, which are modulated by both classical conditioning across days and trial-by-trial discrimination within a day.

Pumping the brakes: rostromedial tegmental inhibition of compulsive cocaine seeking

Addiction is marked by aberrant decision-making and an inability to suppress inappropriate and often dangerous behaviors. We previously demonstrated that inactivation of the rostromedial tegmental nucleus (RMTg) in rats causes persistent food seeking despite impending aversive footshock, an effect strikingly similar to the punishment resistance observed in people with a history of protracted drug use [1]. Here, we extend these studies to demonstrate chemogenetic silencing of RMTg axonal projections to the ventral tegmental area (VTA) (RMTg→VTA pathway) causes rats to endure significantly more footshock to receive cocaine infusions. To further test whether activation of this circuit is sufficient to suppress reward seeking in the absence of an overtly aversive stimulus, we used temporally specific optogenetic stimulation of the RMTg→VTA pathway as a "punisher" in place of footshock following lever pressing for either food or cocaine reward. While optical stimulation of the RMTg→VTA pathway robustly suppressed lever pressing for food, we found that stimulation of this circuit had only modest effects on suppressing responding for cocaine infusions. Even though optical RMTg→VTA stimulation was not particularly effective at reducing ongoing cocaine use, this experience nevertheless had long-lasting consequences, as reinstatement of drug seeking in response to cocaine-associated cues was profoundly suppressed when tested nearly two weeks later. These results suggest the RMTg may serve as a useful target for producing enduring reductions in drug craving, particularly during periods of abstinence from drug use.

The FAAH inhibitor URB597 reduces cocaine intake during conditioned punishment and mitigates cocaine seeking during withdrawal

The endocannabinoid system is prominently implicated in the control of cocaine reinforcement due to its relevant role in synaptic plasticity and neurotransmitter modulation in the mesocorticolimbic system. The inhibition of fatty acid amide hydrolase (FAAH), and the resulting increase in anandamide and other N-acylethanolamines, represents a promising strategy for reducing drug seeking. In the present study, we aimed to assess the effects of the FAAH inhibitor URB597 (1 mg/kg) on crucial features of cocaine addictive-like behaviour in mice. Therefore, we tested the effects of URB597 on acquisition of cocaine (0.6 mg/kg/inf) self-administration, compulsive-like cocaine intake and cue-induced drug-seeking behaviour during withdrawal. URB597 reduced cocaine intake under conditioned punishment while having no impact on acquisition. This result was associated to increased cannabinoid receptor 1 gene expression in the ventral striatum and medium spiny neurons activation in the nucleus accumbens shell. Moreover, URB597 mitigated cue-induced drug-seeking behaviour during prolonged abstinence and prevented the withdrawal-induced increase in FAAH gene expression in the ventral striatum. In this case, URB597 decreased activation of medium spiny neurons in the nucleus accumbens core. Our findings evidence the prominent role of endocannabinoids in the development of cocaine addictive-like behaviours and support the potential of FAAH inhibition as a therapeutical target for the treatment of cocaine addiction.

A comparison of renewal, spontaneous recovery, and reacquisition after punishment and extinction

Punishment and extinction are both effective methods of reducing instrumental responding and may involve similar learning mechanisms. To characterize the similarities and differences between them, we examined three well-established recovery or "relapse" effects -renewal, spontaneous recovery, and reacquisition - following either punishment or extinction of an instrumental response. In Experiment 1a, both punished and extinguished responses renewed to similar degrees following a context change at test (ABA renewal). In Experiment 1b, responding spontaneously recovered to similar degrees following punishment or extinction. In Experiment 2, responding was rapidly reacquired when the response was reinforced again following extinction but not following punishment, as predicted by the idea that the reinforcer delivered in reacquisition is part of the context of punishment, but not extinction. The results collectively suggest that both punishment and extinction produce similar context-dependent retroactive interference effects. More broadly, they also suggest that punished and extinguished responses may be equally likely to return following a change of context despite the intuition that punishment might provide a more extreme and effective means of suppressing behavior. To our knowledge, this is the first direct behavioral comparison of response recovery after punishment and extinction within individual experiments.

Response-specific effects of punishment of a discriminated operant response

To determine whether the punishment of a discriminated operant behavior has effects that are specific to the punished response, rats were reinforced for performing two different instrumental responses (lever pressing and chain pulling) in the presence of a single discriminative stimulus (S). They were then either punished with mild footshock for performing one of the responses (R1) in S, or they received the same shocks in a noncontingent manner while performing R1 in S (i.e., a yoked control). In final tests of both R1 and R2 in S, the punished rats were more suppressed to R1 than R2, but the yoked rats were not. The results extend previous results with extinction rather than punishment learning (Bouton, Trask, & Carranza-Jasso, 2016) and support a larger parallel between extinction and punishment of both free-operant and discriminated-operant responding. Punishment is like extinction in creating a response-specific inhibition of either free or discriminated operant behavior.

Working hard to avoid: Fixed-ratio response effort and maladaptive avoidance in humans

Maladaptive avoidance of safe stimuli is a defining feature of anxiety and related disorders. Avoidance may involve physical effort or the completion of a fixed series of responses to prevent occurrence of, or cues associated with, the aversive event. Understanding the role of response effort in the acquisition and extinction of avoidance may facilitate the development of new clinical treatments for maladaptive avoidance. Despite this, little is known about the impact of response effort on extinction-resistant avoidance in humans. Here, we describe findings from two laboratory-based treatment studies designed to investigate the impact of high and low response effort on the extinction (Experiment 1) and return (Experiment 2) of avoidance. Response effort was operationalised as completion of fixed-ratio (FR) reinforcement schedules for both danger and safety cues in a multi-cue avoidance paradigm with behavioural, self-report, and physiology measures. Completion of the FR response requirements cancelled upcoming shock presentations following danger cues and had no impact on the consequences that followed safety cues. Both experiments found persistence of high response-effort avoidance across danger and safety cues and sustained (Experiment 1) and reinstated (Experiment 2) levels of fear and threat expectancy. Skin conductance responses evoked by all cues were similar across experiments. The present findings and paradigm have implications for translational research on maladaptive anxious coping and treatment development.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Corticostriatal Projections Relying on GABA Levels Mediate Exercise-Induced Functional Recovery in Cerebral Ischemic Mice

Stroke is a neurological disorder characterized by high disability and death worldwide. The occlusion of the middle cerebral artery (MCAO) supplying the cortical motor regions and its projection pathway regions can either kill the cortical neurons or block their projections to the spinal cord and subcortical structure. The cerebral cortex is the primary striatal afferent, and the medium spiny neurons of the striatum have been identified as the major output neurons projecting to the substantia nigra and pallidum. Thus, disconnection of the corticostriatal circuit often occurs in the model of MCAO. In this study, we hypothesize that striatal network dysfunction in cerebral ischemic mice ultimately modulates the activity of striatal projections from cortical neurons to improve dysfunction during exercise training. In this study, we observed that the corticostriatal circuit originating from glutamatergic neurons could partially medicate the improvement of motor and anxiety-like behavior in mice with exercise. Furthermore, exercising or activating a single optogenetic corticostriatal circuit can increase the striatal gamma-aminobutyric acid (GABA) level. Using the GABA-A receptor antagonist, bicuculline, we further identified that the striatal glutamatergic projection from the cortical neurons relies on the GABAergic synapse's activity to modulate exercise-induced functional recovery. Overall, those results reveal that the dorsal striatum-projecting subpopulation of cortical glutamatergic neurons can influence GABA levels in the striatum, playing a critical role in modulating exercise-induced improvement of motor and anxiety-like behavior.

Beyond Fear, Extinction, and Freezing: Strategies for Improving the Translational Value of Animal Conditioning Research

Translational neuroscience for anxiety has had limited success despite great progress in understanding the neurobiology of Pavlovian fear conditioning and extinction. This chapter explores the idea that conditioning paradigms have had a modest impact on translation because studies in animals and humans are misaligned in important ways. For instance, animal conditioning studies typically use imminent threats to assess short-duration fear states with single behavioral measures (e.g., freezing), whereas human studies typically assess weaker or more prolonged anxiety states with physiological (e.g., skin conductance) and self-report measures. A path forward may be more animal research on conditioned anxiety phenomena measuring dynamic behavioral and physiological responses in more complex environments. Exploring transitions between defensive brain states during extinction, looming threats, and post-threat recovery may be particularly informative. If care is taken to align paradigms, threat levels, and measures, this strategy may reveal stable patterns of non-conscious defense in animals and humans that correlate better with conscious anxiety. This shift in focus is also warranted because anxiety is a bigger problem than fear, even in disorders defined by dysfunctional fear or panic reactions.

Shared and distinct neural activity during anticipation and outcome of win and loss: A meta-analysis of the monetary incentive delay task

Reward and punishment motivate decision making and behavioral changes. Numerous studies have examined regional activities during anticipation and outcome of win and loss in the monetary incentive delay task (MIDT). However, the great majority of studies reported findings of anticipation or outcome and of win or loss alone. It remains unclear how the neural correlates share and differentiate amongst these processes. We conducted an Activation Likelihood Estimation meta-analysis of 81 studies of the MIDT (5,864 subjects), including 24 published since the most recent meta-analysis, to identify and, with conjunction and subtraction, contrast regional responses to win anticipation, loss anticipation, win outcome, and loss outcome. Win and loss anticipation engaged a shared network of bilateral anterior insula (AI), striatum, thalamus, supplementary motor area (SMA), and precentral gyrus. Win and loss outcomes did not share regional activities. Win and loss outcome each engaged higher activity in medial orbitofrontal cortex (mOFC) and dorsal anterior cingulate cortex. Bilateral striatum and right occipital cortex responded to both anticipation and outcome of win, and right AI to both phases of loss. Win anticipation vs. outcome engaged higher activity in bilateral AI, striatum, SMA and precentral gyrus and right thalamus, and lower activity in bilateral mOFC and posterior cingulate cortex as well as right inferior frontal and angular gyri. Loss anticipation relative to outcome involved higher activity in bilateral striatum and left AI. These findings collectively suggest shared and distinct regional responses during monetary wins and losses. Delineating the neural correlates of these component processes may facilitate empirical research of motivated behaviors and dysfunctional approach and avoidance in psychopathology.

Alcohol Withdrawal and the Associated Mood Disorders-A Review

Recreational use of alcohol is a social norm in many communities worldwide. Alcohol use in moderation brings pleasure and may protect the cardiovascular system. However, excessive alcohol consumption or alcohol abuse are detrimental to one's health. Three million deaths due to excessive alcohol consumption were reported by the World Health Organization. Emerging evidence also revealed the danger of moderate consumption, which includes the increased risk to cancer. Alcohol abuse and periods of withdrawal have been linked to depression and anxiety. Here, we present the effects of alcohol consumption (acute and chronic) on important brain structures-the frontal lobe, the temporal lobe, the limbic system, and the cerebellum. Apart from this, we also present the link between alcohol abuse and withdrawal and mood disorders in this review, thus drawing a link to oxidative stress. In addition, we also discuss the positive impacts of some pharmacotherapies used. Due to the ever-rising demands of life, the cycle between alcohol abuse, withdrawal, and mood disorders may be a never-ending cycle of destruction. Hence, through this review, we hope that we can emphasise the importance and urgency of managing this issue with the appropriate approaches.

Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders

In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.

Spontaneous instrumental avoidance learning in social contexts

Adaptation to our social environment requires learning how to avoid potentially harmful situations, such as encounters with aggressive individuals. Threatening facial expressions can evoke automatic stimulus-driven reactions, but whether their aversive motivational value suffices to drive instrumental active avoidance remains unclear. When asked to freely choose between different action alternatives, participants spontaneously-without instruction or monetary reward-developed a preference for choices that maximized the probability of avoiding angry individuals (sitting away from them in a waiting room). Most participants showed clear behavioral signs of instrumental learning, even in the absence of an explicit avoidance strategy. Inter-individual variability in learning depended on participants' subjective evaluations and sensitivity to threat approach feedback. Counterfactual learning best accounted for avoidance behaviors, especially in participants who developed an explicit avoidance strategy. Our results demonstrate that implicit defensive behaviors in social contexts are likely the product of several learning processes, including instrumental learning.

Mirror neurons and empathy-related regions in psychopathy: systematic review, meta-analysis, and a working model

Mirror neurons have been associated with empathy. People with psychopathic traits present low levels of empathy. To analyze this, a systematic review of fMRI studies of people with psychopathic traits during an emotional facial expression processing task was performed. The regions of interest were structures associated with the mirror neuron system: ventromedial prefrontal cortex (vmPFC), inferior parietal lobe (IPL), inferior frontal gyrus and superior temporal sulcus. The analysis was also extended to structures related to affective empathy (insula, amygdala and anterior cingulate cortex) and to two more emotional processing areas (orbitofrontal cortex and fusiform gyrus). Hypoactivation was more frequently observed in regions of the mirror neuron system from people with high psychopathic traits, as well as in the emotional processing structures, and those associated with affective empathy, except for the insula, where it presented higher activity. Differences were observed for all types of emotions. The results suggest that the mirror neuron system is altered in psychopathy and their relationship with affective empathy deficits is discussed.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-making

In real-world decision-making scenarios, negative consequences do not always occur immediately after a choice. This delay between action and outcome drives the underestimation, or "delay discounting", of punishment. While the neural substrates underlying sensitivity to immediate punishment have been well-studied, there has been minimal investigation of delayed consequences. Here, we assessed the role of lateral orbitofrontal cortex (LOFC) and basolateral amygdala (BLA), two regions implicated in cost/benefit decision-making, in sensitivity to delayed vs immediate punishment. The delayed punishment decision-making task (DPDT) was used to measure delay discounting of punishment in rodents. During DPDT, rats choose between a small, single pellet reward and a large, three pellet reward accompanied by a mild foot shock. As the task progresses, the shock is preceded by a delay that systematically increases or decreases throughout the session. We observed that rats avoid choices associated with immediate punishment, then shift preference toward these options when punishment is delayed. LOFC inactivation did not influence choice of rewards with immediate punishment, but decreased choice of delayed punishment. We also observed that BLA inactivation reduced choice of delayed punishment for ascending but not descending delays. Inactivation of either brain region produced comparable effects on decision-making in males and females, but there were sex differences observed in omissions and latency to make a choice. In summary, both LOFC and BLA contribute to the delay discounting of punishment and may serve as promising therapeutic targets to improve sensitivity to delayed punishment during decision-making.Significance StatementNegative consequences occurring after a delay are often underestimated, which can lead to maladaptive decision-making. While sensitivity to immediate punishment during reward-seeking has been well-studied, the neural substrates underlying sensitivity to delayed punishment remain unclear. Here, we used the Delayed Punishment Decision-making Task to determine that lateral orbitofrontal cortex and basolateral amygdala both regulate the discounting of delayed punishment, suggesting that these regions may be potential targets to improve decision-making in psychopathology.

Sex Differences In Avoidance Extinction After Contextual Fear Conditioning: Anxioescapic Behavior In Female Rats

Fear memories are important for survival and are implicated in the etiology of fear disorders such as Post Traumatic Stress Disorder (PTSD). Fear memories are well studied pre-clinically and sex differences in rodent fear expression have been reported: females tend to freeze less than males. Whether this is a difference in fear learning or expression is debated. We aimed to differentiate between these possibilities with a task that allowed female rats to express fear memory by moving, rather than freezing. We assessed fear extinction after contextual fear conditioning in the isolated Shock Arm of a Y-maze in female and male rats by either placing them back in the isolated Shock Arm (Fear Extinction in the Shock Context) or allowing them to move freely in the Y-maze during extinction training and enter/avoid the Shock Arm (Avoidance Extinction). We confirmed that female rats freeze less than males during fear extinction in both settings. During Avoidance Extinction, however, both sexes had similar avoidance of the Shock Context, showing comparable fear memory and extinction. Additionally, female rats made more entries into the non-shock arms. Thus, female and male rats have similar fear learning but females express it with an active motor response. Furthermore, female rats also exhibited an active motor response under other anxiogenic conditions (Elevated Plus Maze) and had higher reactivity (Acoustic Startle Response) but not when fear-eliciting stimuli were present: cat hair and foot-shock. In summary, female rats have an active motor response to anxiogenic stimuli which we termed 'Anxioescapic' behavior strategy.

Whole-Brain Monosynaptic Afferents to Rostromedial Tegmental Nucleus Gamma-Aminobutyric Acid-Releasing Neurons in Mice

Increasing evidence has revealed that the rostromedial tegmental area (RMTg) mediates many behaviors, including sleep and addiction. However, presynaptic patterns governing the activity of γ-aminobutyric acid-releasing (GABAergic) neurons, the main neuronal type in the RMTg, have not been defined. Here, we used cell-type-specific retrograde trans-synaptic rabies viruses to map and quantify the monosynaptic afferents to RMTg GABAergic neurons in mouse whole brains. We identified 71 ascending projection brain regions. Sixty-eight percent of the input neurons arise from the ipsilateral and 32% from the contralateral areas of the brain. The first three strongest projection regions were the ipsilateral lateral hypothalamus, zone incerta, and contralateral pontine reticular nucleus. Immunohistochemistry imaging showed that the input neurons in the dorsal raphe, laterodorsal tegmentum, and dorsal part of zone incerta were colocalized with serotoninergic, cholinergic, and neuronal nitric oxide synthetase-expressing neurons, respectively. However, in the lateral hypothalamus, a few input neurons innervating RMTg GABAergic neurons colocalized orexinergic neurons but lacked colocalization of melanin-concentrating hormone neurons. Our findings provide anatomical evidence to understand how RMTg GABAergic neurons integrate diverse information to exert varied functions.

Instrumental aversion coding in the basolateral amygdala and its reversion by a benzodiazepine

Punishment involves learning the relationship between actions and their adverse consequences. Both the acquisition and expression of punishment learning depend on the basolateral amygdala (BLA), but how BLA supports punishment remains poorly understood. To address this, we measured calcium (Ca²⁺) transients in BLA principal neurons during punishment. Male rats were trained to press two individually presented levers for food; when one of these levers also yielded aversive footshock, responding on this punished lever decreased relative to the other, unpunished lever. In rats with the Ca²⁺ indicator GCaMP6f targeted to BLA principal neurons, we observed excitatory activity transients to the footshock punisher and inhibitory transients to lever-presses earning a reward. Critically, as rats learned punishment, activity around the punished response transformed from inhibitory to excitatory and similarity analyses showed that these punished lever-press transients resembled BLA transients to the punisher itself. Systemically administered benzodiazepine (midazolam) selectively alleviated punishment. Moreover, the degree to which midazolam alleviated punishment was associated with how much punished response-related BLA transients reverted to their pre-punishment state. Together, these findings show that punishment learning is supported by aversion-coding of instrumental responses in the BLA and that the anti-punishment effects of benzodiazepines are associated with a reversion of this aversion coding.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Increased responsiveness to punishment of cocaine self-administration after experience with high punishment

One behavioral feature of drug addiction is continued drug use despite awareness that this causes negative consequences. Attempts to model this feature in animals typically involve punishing drug self-administration with electrical footshock to identify individuals whose drug use is differently suppressed by punishment. Here we sought to further study individual responsiveness of drug use to punishment in rats self-administering intravenous cocaine. Rats were first trained during several weeks to self-administer cocaine under a fixed-ratio 3 schedule of reinforcement. Then, their self-administration behavior was punished with increasing intensity of footshock (i.e., from 0.1 mA to 0.9 mA, every 30 min). With increasing intensity of punishment, rats first continued to self-administer cocaine before eventually stopping near completely. When retested, however, drug use became more responsive to punishment and was suppressed by a low and initially ineffective footshock intensity (i.e., 0.1 mA). This increase in responsiveness to punishment was seen in all individuals tested, albeit with varying degrees, and was acquired after one single experience with an intensity of punishment that near completely suppressed drug self-administration. Mere passive, non-contingent exposure to the same intensity, however, had no such effect. Once acquired, increased responsiveness to punishment persisted during at least one month when rats were tested every week, but not every day. Finally, increased responsiveness to punishment was not observed after exposure to a non-painful form of punishment (i.e., histamine). Overall, this study reveals that initial responsiveness of drug use to punishment can change rapidly and persistently with experience. We discuss several possible mechanisms that may account for this change in punishment responsiveness and also draw some of the implications and future perspectives for research on animal models of compulsion-like behavior.

Engaging endogenous opioid circuits in pain affective processes

The pervasive use of opioid compounds for pain relief is rooted in their utility as one of the most effective therapeutic strategies for providing analgesia. While the detrimental side effects of these compounds have significantly contributed to the current opioid epidemic, opioids still provide millions of patients with reprieve from the relentless and agonizing experience of pain. The human experience of pain has long recognized the perceived unpleasantness entangled with a unique sensation that is immediate and identifiable from the first-person subjective vantage point as "painful." From this phenomenological perspective, how is it that opioids interfere with pain perception? Evidence from human lesion, neuroimaging, and preclinical functional neuroanatomy approaches is sculpting the view that opioids predominately alleviate the affective or inferential appraisal of nociceptive neural information. Thus, opioids weaken pain-associated unpleasantness rather than modulate perceived sensory qualities. Here, we discuss the historical theories of pain to demonstrate how modern neuroscience is revisiting these ideas to deconstruct the brain mechanisms driving the emergence of aversive pain perceptions. We further detail how targeting opioidergic signaling within affective or emotional brain circuits remains a strong avenue for developing targeted pharmacological and gene-therapy analgesic treatments that might reduce the dependence on current clinical opioid options.

Deconstructing and reconstructing behaviour relevant to mental health disorders: The benefits of a psychological approach, with a focus on addiction

RUTHERFORD, L.G. and Milton, A.L. Deconstructing and reconstructing behaviour relevant to mental health disorders: what can psychology offer? NEUROSCI BIOBEHAV REV XX(X)XXX-XXX, 2021. - Current treatments for mental health disorders are successful only for some patients, and there is an unmet clinical need for new treatment development. One challenge for treatment development has been how best to model complex human conditions in animals, where mechanism can be more readily studied with a range of neuroscientific techniques. We suggest that an approach to modelling based on associative animal learning theory provides a good framework for deconstructing complex mental health disorders such that they can be studied in animals. These individual simple models can subsequently be used in combination to 'reconstruct' a more complex model of the mental health disorder of interest. Using examples primarily from the field of drug addiction, we explore the 'psychological approach' and suggest that in addition to facilitating translation and backtranslation of tasks between animal models and patients, it is also highly concordant with the concept of triangulation.

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation

BACKGROUND

Drug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.

METHODS

We investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.

RESULTS

Computational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.

CONCLUSION

Our findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.

The Aversion Function of the Limbic Dopaminergic Neurons and Their Roles in Functional Neurological Disorders

The Freudian theory of conversion suggested that the major symptoms of functional neurological disorders (FNDs) are due to internal conflicts at motivation, especially at the sex drive or libido. FND patients might behave properly at rewarding situations, but they do not know how to behave at aversive situations. Sex drive is the major source of dopamine (DA) release in the limbic area; however, the neural mechanism involved in FND is not clear. Dopaminergic (DAergic) neurons have been shown to play a key role in processing motivation-related information. Recently, DAergic neurons are found to be involved in reward-related prediction error, as well as the prediction of aversive information. Therefore, it is suggested that DA might change the rewarding reactions to aversive reactions at internal conflicts of FND. So DAergic neurons in the limbic areas might induce two major motivational functions: reward and aversion at internal conflicts. This article reviewed the recent advances on studies about DAergic neurons involved in aversive stimulus processing at internal conflicts and summarizes several neural pathways, including four limbic system brain regions, which are involved in the processing of aversion. Then the article discussed the vital function of these neural circuits in addictive behavior, depression treatment, and FNDs. In all, this review provided a prospect for future research on the aversion function of limbic system DA neurons and the therapy of FNDs.

Punishments and rewards both modestly impair visuomotor memory retention

While the effects of rewards on memory appear well documented, the effects of punishments remain uncertain. Based on neuroimaging data, this study tested the hypothesis that, as compared to a neutral condition, a context allowing successful punishment avoidance would enhance memory to a similar extent as rewards. In a fully within-subject and counter-balanced design, participants (n = 18) took part in 3 distinct learning sessions during which the delivery of performance-contingent monetary punishments and rewards was manipulated. Specifically, participants had to reach towards visual targets while compensating for a gradually introduced visual deviation. Accuracy at achieving targets was either punished (Hit: "+0$"; Miss: "-0.5$), rewarded (Hit: "+0.5$"; Miss: "-0$"), or associated with neutral binary feedback (Hit: "Hit"; Miss: "Miss"). Retention was assessed through reach aftereffects both immediately and 24 h after initial acquisition. The results disconfirmed the hypothesis by showing that the punishment and reward learning sessions both impaired retention as compared to the neutral session, suggesting that both types of incentives similarly impaired memory formation and consolidation. Two alternative but complementary interpretations are discussed. One interpretation is that the presence of punishments and rewards induced a negative learning context, which - based on neurobiological data - could have been sufficient to interfere with memory formation and consolidation. Another interpretation is that punishments and rewards emphasized the disrupting effects of target hits on implicit learning processes, therefore yielding retention impairments. Altogether, these results suggest that incentives can have counter-productive effects on memory.

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

The lesser evil: Pavlovian-instrumental transfer & aversive motivation

While our understanding of appetitive motivation includes accounts of rich cognitive phenomena, such as choice, sensory-specificity and outcome valuation, the same is not true in aversive processes. A highly sophisticated picture has emerged of Pavlovian fear conditioning and extinction, but progress in aversive motivation has been somewhat limited to these fundamental behaviors. Many differences between appetitive and aversive stimuli permit different kinds of analyses; a widely used procedure in appetitive studies that can expand the scope of aversive motivation is Pavlovian-instrumental transfer (PIT). Recently, this motivational transfer effect has been used to examine issues pertaining to sensory-specificity and the nature of defensive control in avoidance learning. Given enduring controversies and unresolved criticisms surrounding avoidance research, PIT offers a valuable, well-controlled procedure with which to similarly probe this form of motivation. Furthermore, while avoidance itself can be criticized as artificial, PIT can be an effective model for how skills learned through avoidance can be practically applied to encounters with threatening or fearful stimuli and stress. Despite sensory-related challenges presented by the limited aversive unconditioned stimuli typically used in research, transfer testing can nevertheless provide valuable information on the psychological nature of this historically controversial phenomenon.

Punishment insensitivity in humans is due to failures in instrumental contingency learning

Punishment maximises the probability of our individual survival by reducing behaviours that cause us harm, and also sustains trust and fairness in groups essential for social cohesion. However, some individuals are more sensitive to punishment than others and these differences in punishment sensitivity have been linked to a variety of decision-making deficits and psychopathologies. The mechanisms for why individuals differ in punishment sensitivity are poorly understood, although recent studies of conditioned punishment in rodents highlight a key role for punishment contingency detection (Jean-Richard-Dit-Bressel et al., 2019). Here, we applied a novel ‘Planets and Pirates’ conditioned punishment task in humans, allowing us to identify the mechanisms for why individuals differ in their sensitivity to punishment. We show that punishment sensitivity is bimodally distributed in a large sample of normal participants. Sensitive and insensitive individuals equally liked reward and showed similar rates of reward-seeking. They also equally disliked punishment and did not differ in their valuation of cues that signalled punishment. However, sensitive and insensitive individuals differed profoundly in their capacity to detect and learn volitional control over aversive outcomes. Punishment insensitive individuals did not learn the instrumental contingencies, so they could not withhold behaviour that caused punishment and could not generate appropriately selective behaviours to prevent impending punishment. These differences in punishment sensitivity could not be explained by individual differences in behavioural inhibition, impulsivity, or anxiety. This bimodal punishment sensitivity and these deficits in instrumental contingency learning are identical to those dictating punishment sensitivity in non-human animals, suggesting that they are general properties of aversive learning and decision-making.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.