A Transient Dopamine Signal Represents Avoidance Value and Causally Influences the Demand to Avoid

Habit Formation and the Effect of Repeated Stress Exposures on Cognitive Flexibility Learning in Horses

Horse training exposes horses to an array of cognitive and ethological challenges. Horses are routinely required to perform behaviours that are not aligned to aspects of their ethology, which may delay learning. While horses readily form habits during training, not all of these responses are considered desirable, resulting in the horse being subject to retraining. This is a form of cognitive flexibility and is critical to the extinction of habits and the learning of new responses. It is underpinned by complex neural processes which can be impaired by chronic or repeated stress. Domestic horses may be repeatedly exposed to multiples stressors. The potential contribution of stress impairments of cognitive flexibility to apparent training failures is not well understood, however research from neuroscience can be used to understand horses’ responses to training. We trained horses to acquire habit-like responses in one of two industry-style aversive instrumental learning scenarios (moving away from the stimulus-instinctual or moving towards the stimulus-non-instinctual) and evaluated the effect of repeated stress exposures on their cognitive flexibility in a reversal task. We measured heart rate as a proxy for noradrenaline release, salivary cortisol and serum Brain Derived Neurotrophic Factor (BDNF) to infer possible neural correlates of the learning outcomes. The instinctual task which aligned with innate equine escape responses to aversive stimuli was acquired significantly faster than the non-instinctual task during both learning phases, however contrary to expectations, the repeated stress exposure did not impair the reversal learning. We report a preliminary finding that serum BDNF and salivary cortisol concentrations in horses are positively correlated. The ethological salience of training tasks and cognitive flexibility learning can significantly affect learning in horses and trainers should adapt their practices where such tasks challenge innate equine behaviour.

Overnight fasting affects avoidance learning and relief

OBJECTIVES

prolonged fasting influences threat and reward processing, two fundamental systems underpinning adaptive behaviors. In animals, overnight fasting sensitizes the mesolimbic-dopaminergic activity governing avoidance, reward, and fearextinction learning. Despite evidence that overnight fasting may also affect reward and fear learning in humans, effects on human avoidance learning have not been studied yet. Here, we examined the effects of 16 h-overnight fasting on instrumental avoidance and relief from threat omission.

METHODS

to this end, 50 healthy women were randomly assigned to a Fasting (N = 25) or a Re-feeding group (N = 25) and performed an Avoidance-Relief Task.

RESULTS

we found that fasting decreases unnecessary avoidance during signaled safety; this effect was mediated via a reduction in relief pleasantness during signaled absence of threat. A fasting-induced reduction in relief was also found during fear extinction learning.

DISCUSSION

we conclude that fasting optimizes avoidance and safety learning. Future studies should test whether these effects also hold for anxious individuals.

Altered Pain Processing Associated with Administration of Dopamine Agonist and Antagonist in Healthy Volunteers

Striatal dopamine dysfunction is associated with the altered top-down modulation of pain processing. The dopamine D2-like receptor family is a potential substrate for such effects due to its primary expression in the striatum, but evidence for this is currently lacking. Here, we investigated the effect of pharmacologically manipulating striatal dopamine D2 receptor activity on the anticipation and perception of acute pain stimuli in humans. Participants received visual cues that induced either certain or uncertain anticipation of two pain intensity levels delivered via a CO₂ laser. Rating of the pain intensity and unpleasantness was recorded. Brain activity was recorded with EEG and analysed via source localisation to investigate neural activity during the anticipation and receipt of pain. Participants completed the experiment under three conditions, control (Sodium Chloride), D2 receptor agonist (Cabergoline), and D2 receptor antagonist (Amisulpride), in a repeated-measures, triple-crossover, double-blind study. The antagonist reduced an individuals’ ability to distinguish between low and high pain following uncertain anticipation. The EEG source localisation showed that the agonist and antagonist reduced neural activations in specific brain regions associated with the sensory integration of salient stimuli during the anticipation and receipt of pain. During anticipation, the agonist reduced activity in the right mid-temporal region and the right angular gyrus, whilst the antagonist reduced activity within the right postcentral, right mid-temporal, and right inferior parietal regions. In comparison to control, the antagonist reduced activity within the insula during the receipt of pain, a key structure involved in the integration of the sensory and affective aspects of pain. Pain sensitivity and unpleasantness were not changed by D2R modulation. Our results support the notion that D2 receptor neurotransmission has a role in the top-down modulation of pain.

In trans neuregulin3-Caspr3 interaction controls DA axonal bassoon cluster development

Dopamine (DA) transmission is critical to motivation, movement, and emotion. Unlike glutamatergic and GABAergic synapses, the development of DA synapses is less understood. We show that bassoon (BSN) clusters along DA axons in the core of nucleus accumbens (NAcc) were increased in neonatal stages and reduced afterward, suggesting DA synapse elimination. Remarkably, DA neuron-specific ablating neuregulin 3 (NRG3), a protein whose levels correlate with BSN clusters, increased the clusters and impaired DA release and behaviors related to DA transmission. An unbiased screen of transmembrane proteins with the extracellular domain (ECD) of NRG3 identified Caspr3 (contactin associate-like protein 3) as a binding partner. Caspr3 was enriched in striatal medium spiny neurons (MSNs). NRG3 and Caspr3 interact in trans, which was blocked by Caspr3-ECD. Caspr3 null mice displayed phenotypes similar to those in DAT-Nrg3^f/f mice in DA axonal BSN clusters and DA transmission. Finally, in vivo disruption of the NRG3-Caspr3 interaction increased BSN clusters. Together, these results demonstrate that DA synapse development is controlled by trans interaction between NRG3 in DA neurons and Caspr3 in MSNs, identifying a novel pair of cell adhesion molecules for brain circuit wiring.

Cannabinoid Modulation of Dopamine Release During Motivation, Periodic Reinforcement, Exploratory Behavior, Habit Formation, and Attention

Motivational and attentional processes energize action sequences to facilitate evolutionary competition and promote behavioral fitness. Decades of neuropharmacology, electrophysiology and electrochemistry research indicate that the mesocorticolimbic DA pathway modulates both motivation and attention. More recently, it was realized that mesocorticolimbic DA function is tightly regulated by the brain's endocannabinoid system and greatly influenced by exogenous cannabinoids-which have been harnessed by humanity for medicinal, ritualistic, and recreational uses for 12,000 years. Exogenous cannabinoids, like the primary psychoactive component of cannabis, delta-9-tetrahydrocannabinol, produce their effects by acting at binding sites for naturally occurring endocannabinoids. The brain's endocannabinoid system consists of two G-protein coupled receptors, endogenous lipid ligands for these receptor targets, and several synthetic and metabolic enzymes involved in their production and degradation. Emerging evidence indicates that the endocannabinoid 2-arachidonoylglycerol is necessary to observe concurrent increases in DA release and motivated behavior. And the historical pharmacology literature indicates a role for cannabinoid signaling in both motivational and attentional processes. While both types of behaviors have been scrutinized under manipulation by either DA or cannabinoid agents, there is considerably less insight into prospective interactions between these two important signaling systems. This review attempts to summate the relevance of cannabinoid modulation of DA release during operant tasks designed to investigate either motivational or attentional control of behavior. We first describe how cannabinoids influence DA release and goal-directed action under a variety of reinforcement contingencies. Then we consider the role that endocannabinoids might play in switching an animal's motivation from a goal-directed action to the search for an alternative outcome, in addition to the formation of long-term habits. Finally, dissociable features of attentional behavior using both the 5-choice serial reaction time task and the attentional set-shifting task are discussed along with their distinct influences by DA and cannabinoids. We end with discussing potential targets for further research regarding DA-cannabinoid interactions within key substrates involved in motivation and attention.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

The drive for thinness: Towards a mechanistic understanding of avoidance behaviors in a non-clinical population

Fear of weight gain is a cardinal feature of eating disorders, including Anorexia Nervosa (AN). This fear motivates behaviors aimed at avoiding weight gain, such as restricting food intake. Of note, avoidance in AN is not confined to food-related items but extends to intense emotional states. Despite the presence of several forms of excessive avoidance in AN, little is known about the mechanisms underpinning avoidance behavior in AN. In the present exploratory study, we investigated whether university students with an elevated desire to avoid weight gain (as measured through self-reported Drive for Thinness, DT) show deficits in generic avoidance learning. Two-hundred and seventy-five female students filled in the Eating Disorder Inventory-II (EDI-II) and performed a food-unrelated avoidance task. Generalized and linear mixed models (GLMM) revealed that students scoring higher on the DT scale of the EDI-II showed more ineffective avoidance, suggesting a tendency for excessive avoidance in at-risk individuals for AN. Similar results might extend to other eating disorders.

Value, drug addiction and the brain

Over the years, various models have been proposed to explain the psychology and biology of drug addiction, built primarily around the habit and compulsion models. Recent research indicates drug addiction may be goal-directed, motivated by excessive valuation of drugs. Drug consumption may initially occur for the sake of pleasure but may transition to a means of escaping withdrawal, stress and negative emotions. In this hypothetical paper, we propose a value-based neurobiological model for drug addiction. We posit that during dependency, the value-based decision-making system in the brain is not inactive but has instead prioritized drugs as the reward of choice. In support of this model, we consider the role of valuation in choice, its influence on pleasure and punishment, and how valuation is contrasted in impulsive and compulsive behaviours. We then discuss the neurobiology of value, beginning with the dopaminergic system and its relationship with incentive salience before moving to brain-wide networks involved in valuation, control and prospection. These value-based neurobiological components are then integrated into the cycle of addiction as we consider the development of drug dependency from a valuation perspective. We conclude with a discussion of cognitive interventions utilizing value-based decision-making, highlighting not just advances in recalibrating the valuation system to focus on non-drug rewards, but also areas for improvement in refining this approach.

Cue-Evoked Dopamine Neuron Activity Helps Maintain but Does Not Encode Expected Value

Cue-evoked midbrain dopamine (DA) neuron activity reflects expected value, but its influence on reward assessment is unclear. In mice performing a trial-based operant task, we test if bidirectional manipulations of cue or operant-associated DA neuron activity drive learning as a result of under- or overexpectation of reward value. We target optogenetic manipulations to different components of forced trials, when only one lever is presented, and assess lever biases on choice trials in the absence of photomanipulation. Although lever biases are demonstrated to be flexible and sensitive to changes in expected value, augmentation of cue or operant-associated DA signaling does not significantly alter choice behavior, and blunting DA signaling during any component of the forced trials reduces choice trial responses on the associated lever. These data suggest cue-evoked DA helps maintain cue-value associations but does not encode expected value as to set the benchmark against which received reward is judged.

Dopaminergic Signaling in the Nucleus Accumbens Modulates Stress-Coping Strategies during Inescapable Stress

Maladaptation to stress is a critical risk factor in stress-related disorders, such as major depression and post-traumatic stress disorder (PTSD). Dopamine signaling in the nucleus accumbens (NAc) has been shown to modulate behavior by reinforcing learning and evading aversive stimuli, which are important for the survival of animals under environmental challenges such as stress. However, the mechanisms through which dopaminergic transmission responds to stressful events and subsequently regulates its downstream neuronal activity during stress remain unknown. To investigate how dopamine signaling modulates stress-coping behavior, we measured the subsecond fluctuation of extracellular dopamine concentration and pH using fast scanning cyclic voltammetry (FSCV) in the NAc, a postsynaptic target of midbrain dopaminergic neurons, in male mice engaged in a tail suspension test (TST). The results revealed a transient decrease in dopamine concentration and an increase in pH levels when the animals changed behaviors, from being immobile to struggling. Interestingly, optogenetic inhibition of dopamine release in NAc, potentiated the struggling behavior in animals under the TST. We then addressed the causal relationship of such a dopaminergic transmission with behavioral alterations by knocking out both the dopamine receptors, i.e., D1 and D2, in the NAc using viral vector-mediated genome editing. Behavioral analyses revealed that male D1 knock-out mice showed significantly more struggling bouts and longer struggling durations during the TST, while male D2 knock-out mice did not. Our results therefore indicate that D1 dopaminergic signaling in the NAc plays a pivotal role in the modulation of stress-coping behaviors in animals under tail suspension stress.SIGNIFICANCE STATEMENT The tail suspension test (TST) has been widely used as a despair-based behavioral assessment to screen the antidepressant so long. Despite its prevalence in the animal studies, the neural substrate underlying the changes of behavior during the test remains unclear. This study provides an evidence for a role of dopaminergic transmission in the modulation of stress-coping behavior during the TST, a despair test widely used to screen the antidepressants in rodents. Taking into consideration the fact that the dopamine metabolism is upregulated by almost all antidepressants, a part of which acts directly on the dopaminergic transmission, current results would uncover the molecular mechanism through which the dopaminergic signaling mediates antidepressant effect with facilitation of the recovery from the despair-like behavior in the TST.

Dopamine modulates individual differences in avoidance behavior: A pharmacological, immunohistochemical, neurochemical and volumetric investigation

Avoidance behavior is a hallmark in pathological anxiety disorders and results in impairment of daily activities. Individual differences in avoidance responses are critical in determining vulnerability or resistance to anxiety disorders. Dopaminergic activation is implicated in the processing of avoidance responses; however, the mechanisms underlying these responses are unknown. In this sense, we used a preclinical model of avoidance behavior to investigate the possibility of an intrinsic differential dopaminergic pattern between good and poor performers. The specific goal was to assess the participation of dopamine (DA) through pharmacological manipulation, and we further evaluated the effects of systemic injections of the dopaminergic receptor type 1 (D1 antagonist - SCH23390) and dopaminergic receptor type 2 (D2 antagonist - sulpiride) antagonists in the good performers. Additionally, we evaluated the effects of intra-amygdala microinjection of a D1 antagonist (SCH23390) and a D2 antagonist (sulpiride) in good performers as well as intra-amygdala microinjection of a D1 agonist (SKF38393) and D2 agonist (quinpirole) in poor performers. Furthermore, we quantified the contents of dopamine and metabolites (3,4-dihydroxyphenylacetic acid (DOPAC) and homovanillic acid (HVA)) in the amygdala, evaluated the basal levels of tyrosine hydroxylase expression (catecholamine synthesis enzyme) and measured the volume of the substantia nigra, ventral tegmental area and locus coeruleus. Our results showed that it could be possible to convert animals from good to poor performers, and vice versa, by intra-amygdala (basolateral and central nucleus) injections of D1 receptor antagonists in good performers or D2 receptor agonists in poor performers. Additionally, the good performers had lower levels of DOPAC and HVA in the amygdala, an increase in the total volume of the amygdala (AMG), substantia nigra (SN), ventral tegmental area (VTA) and locus coeruleus (LC), and an increase in the number of tyrosine hydroxylase-positive cells in SN, VTA and LC, which positively correlates with the avoidance behavior. Taken together, our data show evidence for a dopaminergic signature of avoidance performers, emphasizing the role of distinct dopaminergic receptors in individual differences in avoidance behavior based on pharmacological, immunohistochemical, neurochemical and volumetric analyses. Our findings provide a better understanding of the role of the dopaminergic system in the execution of avoidance behavior.

•

The role of dopamine in individual differences in avoidance behavior.

•

Dopamine modulates avoidance behavior.

•

Dopaminergic evidence of individual difference in avoidance behavior.

•

Good and poor avoiders distinction based on dopaminergic signature.

•

Dopaminergic signature of avoidance performers: poor versus good avoiders.

Anhedonia following mild traumatic brain injury in rats: A behavioral economic analysis of positive and negative reinforcement

Psychiatric disorders affect nearly 50% of individuals who have experienced a traumatic brain injury (TBI). Anhedonia is a major symptom of numerous psychiatric disorders and is a diagnostic criterion for depression. It has recently been appreciated that reinforcement may be separated into consummatory (hedonic), motivational and decisional components, all of which may be affected differently in disease. Although anhedonia is typically assessed using positive reinforcement, the importance of stress in psychopathology suggests the study of negative reinforcement (removal or avoidance of aversive events) may be equally important. The present study investigated positive and negative reinforcement following a rat model of mild TBI (mTBI) using lateral fluid percussion. Hedonic value and motivation for reinforcement was determined by behavioral economic analyses. Following mTBI, the hedonic value of avoiding foot shock was reduced. In contrast, the hedonic value of escaping foot shock or obtaining a sucrose pellet was not altered by mTBI. Moreover, motivation to avoid or escape foot shock or to acquire sucrose was not altered by mTBI. Our results suggest that individuals experiencing mTBI find avoidance of aversive events less reinforcing, and therefore are less apt to utilize proactive control of stress.

Cell-type and projection-specific dopaminergic encoding of aversive stimuli in addiction

Drug addiction is a major public health concern across the world for which there are limited treatment options. In order to develop new therapies to correct the behavioral deficits that result from repeated drug use, we need to understand the neural circuit dysfunction that underlies the pathophysiology of the disorder. Because the initial reinforcing effects of drugs are dependent on increases in dopamine in reward-related brain regions such as the mesolimbic dopamine pathway, a large focus of addiction research has centered on the dysregulation of this system and its control of positive reinforcement and motivation. However, in addition to the processing of positive, rewarding stimuli, there are clear deficits in the encoding and valuation of information about potential negative outcomes and how they control decision making and motivation. Further, aversive stimuli can motivate or suppress behavior depending on the context in which they are encountered. We propose a model where rewarding and aversive information guides the execution of specific motivated actions through mesocortical and mesolimbic dopamine acting on D1- and D2- receptor containing neuronal populations. Volitional drug exposure alters the processing of rewarding and aversive stimuli through remodeling of these dopaminergic circuits, causing maladaptive drug seeking, self-administration in the face of negative consequences, and drug craving. Together, this review discusses the dysfunction of the circuits controlling different types of aversive learning as well as how these guide specific discrete behaviors, and provides a conceptual framework for how they should be considered in preclinical addiction models.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

The power of price compels you: Behavioral economic insights into dopamine-based valuation of rewarding and aversively motivated behavior

The mesocorticolimbic dopamine pathway is generally considered to be a reward pathway. While shortsighted, there is a strong basis for this view of dopamine function. Here, we first describe the role of phasic dopamine release events in reward seeking. We then explain why these release events are being reconsidered as value signals and how we applied behavioral economics to confirm they play a causal role in the valuation of reward. Just because dopamine release can function as a dopamine reward value signal however, does not imply that dopamine is solely a reward molecule. Rather, mesocorticolimbic dopamine appears to mediate many adaptive behaviors, including: reward seeking, avoidance, escape and fear-associated conditioned freezing. While more studies are needed before a consensus is reached on when, where and how dopamine mediates aversively-motivated behavior, its involvement is almost irrefutable. Thus, we next describe the role dopamine plays in these ethologically-relevant defensive behaviors. We conclude by describing our recent behavioral economics results that reveal a causal role for dopamine in the valuation of avoidance.

Pavlovian-instrumental interactions in active avoidance: The bark of neutral trials

In active avoidance tasks, subjects have to learn to execute particular actions in order to avoid an aversive stimulus, such as a shock. Such paradigms pose a number of psychological and neural enigmas, and so have attracted substantial computational interest. However, the ratio of conjecture to confirmation remains high. Here, we perform a theoretical inquiry into a recent experiment by Gentry, Lee, and Roesch ('Phasic dopamine release in the rat nucleus accumbens predicts approach and avoidance performance', Nat. Commun., 7:13154) who measured phasic dopamine concentrations in the nucleus accumbens core of rats whilst they avoided shocks, acquired food, or acted to gain no programmed outcome. These last, neutral, trials turned out to be a perfect probe for the workings of avoidance, partly because of the substantial differences between subjects and sessions revealed in the experiment. We suggest a way to interpret this probe, gaining support for opponency-, safety-, and Pavlovian-influenced treatments of avoidance.