Pattern of dopamine signaling during aversive events predicts active avoidance learning

Functional connectivity between the nucleus accumbens and amygdala underlies avoidance learning during adolescence: Implications for developmental psychopathology

BACKGROUND

Reward and threat processes work together to support adaptive learning during development. Adolescence is associated with increasing approach behavior (e.g., novelty-seeking, risk-taking) but often also coincides with emerging internalizing symptoms, which are characterized by heightened avoidance behavior. Peaking engagement of the nucleus accumbens (NAcc) during adolescence, often studied in reward paradigms, may also relate to threat mechanisms of adolescent psychopathology.

METHODS

47 typically developing adolescents (9.9-22.9 years) completed an aversive learning task during functional magnetic resonance imaging, wherein visual cues were paired with an aversive sound or no sound. Task blocks involved an escapable aversively reinforced stimulus (CS+r), the same stimulus without reinforcement (CS+nr), or a stimulus that was never reinforced (CS-). Parent-reported internalizing symptoms were measured using Revised Child Anxiety and Depression Scales.

RESULTS

Functional connectivity between the NAcc and amygdala differentiated the stimuli, such that connectivity increased for the CS+r (p = .023) but not for the CS+nr and CS-. Adolescents with greater internalizing symptoms demonstrated greater positive functional connectivity for the CS- (p = .041).

CONCLUSIONS

Adolescents show heightened NAcc-amygdala functional connectivity during escape from threat. Higher anxiety and depression symptoms are associated with elevated NAcc-amygdala connectivity during safety, which may reflect poor safety versus threat discrimination.

Aversion-induced drug taking and escape behavior involve similar nucleus accumbens core dopamine signaling signatures

Dopamine release in the nucleus accumbens core (NAcC) has long been associated with the promotion of motivated behavior. However, inhibited dopamine signaling can increase behavior in certain settings, such as during drug self-administration. While aversive environmental stimuli can reduce dopamine, it is unclear whether such stimuli reliably engage this mechanism in different contexts. Here we compared the physiological and behavioral responses to the same aversive stimulus in different designs to determine if there is uniformity in the manner that aversive stimuli are encoded and promote behavior. NAcC dopamine was measured using fiber photometry in male and female rats during cocaine self-administration sessions in which an acutely aversive 90 dB white noise was intermittently presented. In a separate group of rats, aversion-induced changes in dopamine were measured in an escape design in which operant responses terminated aversive white noise. Aversive white noise significantly reduced NAcC dopamine and increased cocaine self-administration in both male and female rats. The same relationship was observed in the escape design, in which white noise reduced dopamine and promoted escape attempts. In both designs, the magnitude of the dopamine reduction predicted behavioral performance. While prior research demonstrated that pharmacologically reduced dopamine signaling can promote intake, this report demonstrates that this physiological mechanism is naturally engaged by aversive environmental stimuli and generalizable to non-drug contexts. These findings illustrate a common physiological signature in response to aversion that may promote both adaptive and maladaptive behavior.

Valence ambiguity dynamically shapes striatal dopamine heterogeneity

Adaptive decision making relies on dynamic updating of learned associations where environmental cues come to predict positive and negatively valenced stimuli, such as food or threat. Flexible cue-guided behaviors depend on a network of brain systems, including dopamine signaling in the striatum, which is critical for learning and maintenance of conditioned behaviors. Critically, it remains unclear how dopamine signaling encodes multi-valent, dynamic learning contexts, where positive and negative associations must be rapidly disambiguated. To understand this, we employed a Pavlovian discrimination paradigm, where cues predicting positive and negative outcomes were intermingled during conditioning sessions, and their meaning was serially reversed across training. We found that rats readily distinguished these cues, and updated their behavior rapidly upon valence reversal. Using fiber photometry, we recorded dopamine signaling in three major striatal subregions -,the dorsolateral striatum (DLS), the nucleus accumbens core, and the nucleus accumbens medial shell - and found heterogeneous responses to positive and negative conditioned cues and their predicted outcomes. Valence ambiguity introduced by cue reversal reshaped striatal dopamine on different timelines: nucleus accumbens core and shell signals updated more readily than those in the DLS. Together, these results suggest that striatal dopamine flexibly encodes multi-valent learning contexts, and these signals are dynamically modulated by changing contingencies to resolve ambiguity about the meaning of environmental cues.

Devaluation of response-produced safety signals reveals circuits for goal-directed versus habitual avoidance in dorsal striatum

Active avoidance responses (ARs) are instrumental behaviors that prevent harm. Adaptive ARs may contribute to active coping, whereas maladaptive avoidance habits are implicated in anxiety and obsessive-compulsive disorders. The AR learning mechanism has remained elusive, as successful avoidance trials produce no obvious reinforcer. We used a novel outcome-devaluation procedure in rats to show that ARs are positively reinforced by response-produced feedback (FB) cues that develop into safety signals during training. Males were sensitive to FB-devaluation after moderate training, but not overtraining, consistent with a transition from goal-directed to habitual avoidance. Using chemogenetics and FB-devaluation, we also show that goal-directed vs. habitual ARs depend on dorsomedial vs. dorsolateral striatum, suggesting a significant overlap between the mechanisms of avoidance and rewarded instrumental behavior. Females were insensitive to FB-devaluation due to a remarkable context-dependence of counterconditioning. However, degrading the AR-FB contingency suggests that both sexes rely on safety signals to perform goal-directed ARs.

Nucleus accumbens core single cell ensembles bidirectionally respond to experienced versus observed aversive events

Fear learning is a critical feature of survival skills among mammals. In rodents, fear learning manifests itself through direct experience of the aversive event or social transmission of aversive stimuli such as observing and acting on conspecifics' distress. The neuronal network underlying the social transmission of information largely overlaps with the brain regions that mediate behavioral responses to aversive and rewarding stimuli. In this study, we recorded single cell activity patterns of nucleus accumbens (NAc) core neurons using in vivo optical imaging of calcium transients via miniature scopes. This cutting-edge imaging methodology not only allows us to record activity patterns of individual neurons but also lets us longitudinally follow these individual neurons across time and different behavioral states. Using this approach, we identified NAc core single cell ensembles that respond to experienced and/or observed aversive stimuli. Our results showed that experienced and observed aversive stimuli evoke NAc core ensemble activity that is largely positive, with a smaller subset of negative responses. The size of the NAc single cell ensemble response was greater for experienced aversive stimuli compared to observed aversive events. Our results also revealed sex differences in the NAc core single cell ensembles responses to experience aversive stimuli, where females showed a greater accumbal response. Importantly, we found a subpopulation within the NAc core single cell ensembles that show a bidirectional response to experienced aversive stimuli versus observed aversive stimuli (i.e., negative response to experienced and positive response to observed). Our results suggest that the NAc plays a role in differentiating somatosensory experience from social observation of aversion at a single cell level. These results have important implications for psychopathologies where social information processing is maladaptive, such as autism spectrum disorders.

Multiple routes to enhanced memory for emotionally relevant events

Events associated with aversive or rewarding outcomes are prioritized in memory. This memory boost is commonly attributed to the elicited affective response, closely linked to noradrenergic and dopaminergic modulation of hippocampal plasticity. Herein we review and compare this 'affect' mechanism to an additional, recently discovered, 'prediction' mechanism whereby memories are strengthened by the extent to which outcomes deviate from expectations, that is, by prediction errors (PEs). The mnemonic impact of PEs is separate from the affective outcome itself and has a distinct neural signature. While both routes enhance memory, these mechanisms are linked to different - and sometimes opposing - predictions for memory integration. We discuss new findings that highlight mechanisms by which emotional events strengthen, integrate, and segment memory.

Dopamine release at the time of a predicted aversive outcome causally controls the trajectory and expression of conditioned behavior

Dopamine release in the nucleus accumbens (NAc) is causally linked to adaptive aversive learning, and its dysregulation is a core phenotype in anxiety and stress disorders. Here, we record NAc core dopamine during a task where mice learn to discriminate between cues signaling two types of outcomes: (1) footshock presentation and (2) footshock omission. We show that dopamine release is evoked by footshock omission. This dopamine response is largest when the omission is unexpected and decreases over learning, and artificially increasing this signal disrupts discrimination learning. Conversely, optogenetic inhibition of dopamine responses to the footshock itself impairs learning. Finally, theory-driven computational modeling suggests that these effects can be explained by dopamine signaling the perceived saliency of predicted aversive events. Together, we elucidate the role of NAc dopamine in aversive learning and offer potential avenues for understanding the neural mechanisms involved in anxiety and stress disorders.

Nucleus accumbens core single cell ensembles bidirectionally respond to experienced versus observed aversive events

Empathy is the ability to adopt others' sensory and emotional states and is an evolutionarily conserved trait among mammals. In rodents, empathy manifests itself as social modulation of aversive stimuli such as acknowledging and acting on conspecifics' distress. The neuronal network underlying social transmission of information is known to overlap with the brain regions that mediate behavioral responses to aversive and rewarding stimuli. In this study, we recorded single cell activity patterns of nucleus accumbens (NAc) core neurons using in vivo optical imaging of calcium transients via miniature scopes. This cutting-edge imaging methodology not only allows us to record activity patterns of individual neurons but also lets us longitudinally follow these individual neurons across time and different behavioral states. Using this approach, we identified NAc core single cell ensembles that respond to experienced and/or observed aversive stimuli. Our results showed that experienced and observed aversive stimuli evoke NAc core ensemble activity that is largely positive, with a smaller subset of negative responses. The size of the NAc single cell ensemble response was greater for experienced aversive stimuli compared to observed aversive events. Our results also revealed a subpopulation within the NAc core single cell ensembles that show a bidirectional response to experienced aversive stimuli versus observed aversive stimuli (i.e., negative response to experienced and positive response to observed). These results suggest that the NAc plays a role in differentiating somatosensory experience from social observation of aversion at a single cell level. This has important implications for psychopathologies where social information processing is maladaptive, such as autism spectrum disorders.

Neural mechanism of acute stress regulation by trace aminergic signalling in the lateral habenula in male mice

Stress management is necessary for vertebrate survival. Chronic stress drives depression by excitation of the lateral habenula (LHb), which silences dopaminergic neurons in the ventral tegmental area (VTA) via GABAergic neuronal projection from the rostromedial tegmental nucleus (RMTg). However, the effect of acute stress on this LHb-RMTg-VTA pathway is not clearly understood. Here, we used fluorescent in situ hybridisation and in vivo electrophysiology in mice to show that LHb aromatic L-amino acid decarboxylase-expressing neurons (D-neurons) are activated by acute stressors and suppress RMTg GABAergic neurons via trace aminergic signalling, thus activating VTA dopaminergic neurons. We show that the LHb regulates RMTg GABAergic neurons biphasically under acute stress. This study, carried out on male mice, has elucidated a molecular mechanism in the efferent LHb-RMTg-VTA pathway whereby trace aminergic signalling enables the brain to manage acute stress by preventing the hypoactivity of VTA dopaminergic neurons.

The Role of Intra-Amygdaloid Neurotensin and Dopamine Interaction in Spatial Learning and Memory

Neurotransmitter and neuromodulator neurotensin (NT) has been proved to facilitate spatial and passive avoidance learning after microinjected into the rat central nucleus of amygdala (CeA). These previous studies of our laboratory also revealed that neurotensin-1 receptor (NTS1) is involved in the mentioned actions of NT. Extensive literature confirms the interaction between neurotensinergic and dopaminergic systems, and our research group also suppose that the mesolimbic dopaminergic system (MLDS) is involved in the spatial learning and memory-facilitating effect of NT in the CeA. In the present work, NT and dopamine (DA) interaction has been examined in the Morris water maze and passive avoidance tests. Rats received 100 ng NT, 5 µg dopamine D2 receptor antagonist sulpiride in itself, sulpiride as a pretreatment before NT or vehicle solution into the CeA. NT microinjection significantly decreased target-finding latency in the Morris water maze test and significantly increased entrance latency in the passive avoidance test, as was expected based on our previous findings. The DA D2 receptor antagonist pretreatment was able to inhibit both effects of NT. The results confirm the facilitatory effect of NT on spatial learning and memory and let us conclude that these actions can be exerted via the DA D2 receptors.

Nucleus accumbens dopamine tracks aversive stimulus duration and prediction but not value or prediction error

There is active debate on the role of dopamine in processing aversive stimuli, where inferred roles range from no involvement at all, to signaling an aversive prediction error (APE). Here, we systematically investigate dopamine release in the nucleus accumbens core (NAC), which is closely linked to reward prediction errors, in rats exposed to white noise (WN, a versatile, underutilized, aversive stimulus) and its predictive cues. Both induced a negative dopamine ramp, followed by slow signal recovery upon stimulus cessation. In contrast to reward conditioning, this dopamine signal was unaffected by WN value, context valence, or probabilistic contingencies, and the WN dopamine response shifted only partially toward its predictive cue. However, unpredicted WN provoked slower post-stimulus signal recovery than predicted WN. Despite differing signal qualities, dopamine responses to simultaneous presentation of rewarding and aversive stimuli were additive. Together, our findings demonstrate that instead of an APE, NAC dopamine primarily tracks prediction and duration of aversive events.

Habit Formation and the Effect of Repeated Stress Exposures on Cognitive Flexibility Learning in Horses

Horse training exposes horses to an array of cognitive and ethological challenges. Horses are routinely required to perform behaviours that are not aligned to aspects of their ethology, which may delay learning. While horses readily form habits during training, not all of these responses are considered desirable, resulting in the horse being subject to retraining. This is a form of cognitive flexibility and is critical to the extinction of habits and the learning of new responses. It is underpinned by complex neural processes which can be impaired by chronic or repeated stress. Domestic horses may be repeatedly exposed to multiples stressors. The potential contribution of stress impairments of cognitive flexibility to apparent training failures is not well understood, however research from neuroscience can be used to understand horses’ responses to training. We trained horses to acquire habit-like responses in one of two industry-style aversive instrumental learning scenarios (moving away from the stimulus-instinctual or moving towards the stimulus-non-instinctual) and evaluated the effect of repeated stress exposures on their cognitive flexibility in a reversal task. We measured heart rate as a proxy for noradrenaline release, salivary cortisol and serum Brain Derived Neurotrophic Factor (BDNF) to infer possible neural correlates of the learning outcomes. The instinctual task which aligned with innate equine escape responses to aversive stimuli was acquired significantly faster than the non-instinctual task during both learning phases, however contrary to expectations, the repeated stress exposure did not impair the reversal learning. We report a preliminary finding that serum BDNF and salivary cortisol concentrations in horses are positively correlated. The ethological salience of training tasks and cognitive flexibility learning can significantly affect learning in horses and trainers should adapt their practices where such tasks challenge innate equine behaviour.

A zona incerta-basomedial amygdala circuit modulates aversive expectation in emotional stress-induced aversive learning deficits

A previously published study showed that stress may interfere with associative aversive learning and facilitate mood-related disorders. However, whether emotional stress alone affects aversive learning is unknown. Using three chamber-vicarious social defeat stress (3C-VSDS) model mice, we investigated the effect of emotional stress on aversive learning. An important origin of dopamine (DA) neurons, the zona incerta (ZI), is expected to be a novel target for the modulation of aversive learning. However, less is known about the circuit mechanism of ZI^DA neurons in aversive learning. Here, we subjected mice to a fear-conditioning system (FCS) and observed an increased calcium activity of ZI TH⁺ neurons in aversive expectation during the conditioning phase, especially during the late stage of the conditional stimulus (CS) when CS and unconditional stimulus (US) pairings were used. Optogenetic inhibition of ZI TH⁺ neurons at the late stage of CS disrupted conditioned fear learning in mice. We further identified a TH⁺ projection from the ZI to the basomedial amygdala (BMA) and found that optogenetic inhibition of the ZI-BMA circuit could also block aversive learning. Finally, we used 3C-VSDS mice as a model of emotional stress. We found that the 3C-VSDS model mice demonstrated reduced aversive expectation associated with ZI TH⁺ neurons in the late stage of CS and impaired aversive learning in FCS. Optogenetic activation of ZI-BMA TH⁺ projections in the late stage of CS significantly reversed the aversive FCS learning disability of 3C-VSDS model mice. These data suggest that a TH⁺ circuit from the ZI to the BMA is required for aversive expectation, both at baseline and in 3C-VSDS-induced aversive learning deficits and that this circuit is a potential target for the modulation of aversive learning. Low activity of ZI-BMA TH⁺ projections is one reason for 3C-VSDS-induced aversive learning deficits.

Different brain systems support learning from received and avoided pain during human pain-avoidance learning

Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems (N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.

A unidirectional but not uniform striatal landscape of dopamine signaling for motivational stimuli

Dopamine signals in the striatum are critical for motivated behavior. However, their regional specificity and precise information content are actively debated. Dopaminergic projections to the striatum are topographically organized. Thus, we quantified dopamine release in response to motivational stimuli and associated predictive cues in six principal striatal regions of unrestrained, behaving rats. Absolute signal size and its modulation by stimulus value and by subjective state of the animal were interregionally heterogeneous on a medial to lateral gradient. In contrast, dopamine-concentration direction of change was homogeneous across all regions: appetitive stimuli increased and aversive stimuli decreased dopamine concentration. Although cues predictive of such motivational stimuli acquired the same influence over dopamine homogeneously across all regions, dopamine-mediated prediction-error signals were restricted to the ventromedial, limbic striatum. Together, our findings demonstrate a nuanced striatal landscape of unidirectional but not uniform dopamine signals, topographically encoding distinct aspects of motivational stimuli and their prediction.

Photostimulation of Ventral Tegmental Area-Insular Cortex Dopaminergic Inputs Enhances the Salience to Consolidate Aversive Taste Recognition Memory via D1-Like Receptors

Taste memory involves storing information through plasticity changes in the neural network of taste, including the insular cortex (IC) and ventral tegmental area (VTA), a critical provider of dopamine. Although a VTA-IC dopaminergic pathway has been demonstrated, its role to consolidate taste recognition memory remains poorly understood. We found that photostimulation of dopaminergic neurons in the VTA or VTA-IC dopaminergic terminals of TH-Cre mice improves the salience to consolidate a subthreshold novel taste stimulus regardless of its hedonic value, without altering their taste palatability. Importantly, the inhibition of the D1-like receptor into the IC impairs the salience to facilitate consolidation of an aversive taste recognition memory. Finally, our results showed that VTA photostimulation improves the salience to consolidate a conditioned taste aversion memory through the D1-like receptor into the IC. It is concluded that the dopamine activity from the VTA into IC is required to increase the salience enabling the consolidation of a taste recognition memory. Notably, the D1-like receptor activity into the IC is required to consolidate both innate and learned aversive taste memories but not appetitive taste memory.

Sex Differences in Behavioral Responding and Dopamine Release during Pavlovian Learning

Learning associations between cues and rewards require the mesolimbic dopamine system. The dopamine response to cues signals differences in reward value in well trained animals. However, these value-related dopamine responses are absent during early training sessions when cues signal differences in the reward rate. These findings suggest cue-evoked dopamine release conveys differences between outcomes only after extensive training, though it is unclear whether this is unique to when cues signal differences in reward rate, or whether this is also evident when cues signal differences in other value-related parameters such as reward size. To address this, we used a Pavlovian conditioning task in which one audio cue was associated with a small reward (one pellet) and another audio cue was associated with a large reward (three pellets). We performed fast-scan cyclic voltammetry to record changes in dopamine release in the nucleus accumbens of male and female rats throughout learning. While female rats exhibited higher levels of conditioned responding, a faster latency to respond, and elevated post-reward head entries relative to male rats, there were no sex differences in the dopamine response to cues. Multiple training sessions were required before cue-evoked dopamine release signaled differences in reward size. Reward-evoked dopamine release scaled with reward size, though females displayed lower reward-evoked dopamine responses relative to males. Conditioned responding related to the decrease in the peak reward-evoked dopamine response and not to cue-evoked dopamine release. Collectively, these data illustrate sex differences in behavioral responding as well as in reward-evoked dopamine release during Pavlovian learning.

Optogenetic stimulation of dynorphinergic neurons within the dorsal raphe activate kappa opioid receptors in the ventral tegmental area and ablation of dorsal raphe prodynorphin or kappa receptors in dopamine neurons blocks stress potentiation of cocaine reward

Behavioral stress exposure increases the risk of drug-taking in individuals with substance use disorders by mechanisms involving the dynorphins, which are the endogenous neuropeptides for the kappa opioid receptor (KOR). KOR agonists have been shown to encode dysphoria, aversion, and changes in reward valuation, and kappa opioid antagonists are in clinical development for treating substance use disorders. In this study, we confirmed that KORs were expressed in dopaminergic neurons in the ventral tegmental area (VTA) of male C57BL6/J mice. Genetic ablation of KORs from dopamine neurons blocked the potentiating effects of repeated forced swim stress on cocaine conditioned place preference (CPP). KOR activation inhibited dopamine neuron GCaMP6m calcium activity in VTA during swim stress and caused a rebound enhancement during the period after stress exposure. Transient optogenetic inhibition of VTA dopamine neurons with AAV5-DIO-SwiChR was acutely aversive in a real time place preference assay and blunted cocaine CPP when inhibition was administered concurrently with cocaine conditioning. However, when inhibition preceded cocaine conditioning by 30 min, cocaine CPP was enhanced. Retrograde tracing with CAV2-DIO-ZsGreen identified a population of prodynorphin^Cre neurons in the dorsal raphe nucleus (DRN) projecting to the VTA. Optogenetic stimulation of dynorphinergic neurons within the DRN by Channelrhodopsin2 activated KOR in VTA and ablation of prodynorphin blocked stress potentiation of cocaine CPP. Together, these studies demonstrate the presence of a dynorphin/KOR midbrain circuit that projects from the DRN to VTA and is involved in altering the dynamic response of dopamine neuron activity to enhance drug reward learning.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

The Dopamine System and Automatization of Movement Sequences: A Review With Relevance for Speech and Stuttering

The last decades of research have gradually elucidated the complex functions of the dopamine system in the vertebrate brain. The multiple roles of dopamine in motor function, learning, attention, motivation, and the emotions have been difficult to reconcile. A broad and detailed understanding of the physiology of cerebral dopamine is of importance in understanding a range of human disorders. One of the core functions of dopamine involves the basal ganglia and the learning and execution of automatized sequences of movements. Speech is one of the most complex and highly automatized sequential motor behaviors, though the exact roles that the basal ganglia and dopamine play in speech have been difficult to determine. Stuttering is a speech disorder that has been hypothesized to be related to the functions of the basal ganglia and dopamine. The aim of this review was to provide an overview of the current understanding of the cerebral dopamine system, in particular the mechanisms related to motor learning and the execution of movement sequences. The primary aim was not to review research on speech and stuttering, but to provide a platform of neurophysiological mechanisms, which may be utilized for further research and theoretical development on speech, speech disorders, and other behavioral disorders. Stuttering and speech are discussed here only briefly. The review indicates that a primary mechanism for the automatization of movement sequences is the merging of isolated movements into chunks that can be executed as units. In turn, chunks can be utilized hierarchically, as building blocks of longer chunks. It is likely that these mechanisms apply also to speech, so that frequent syllables and words are produced as motor chunks. It is further indicated that the main learning principle for sequence learning is reinforcement learning, with the phasic release of dopamine as the primary teaching signal indicating successful sequences. It is proposed that the dynamics of the dopamine system constitute the main neural basis underlying the situational variability of stuttering.

Dopamine release in the nucleus accumbens core signals perceived saliency

A large body of work has aimed to define the precise information encoded by dopaminergic projections innervating the nucleus accumbens (NAc). Prevailing models are based on reward prediction error (RPE) theory, in which dopamine updates associations between rewards and predictive cues by encoding perceived errors between predictions and outcomes. However, RPE cannot describe multiple phenomena to which dopamine is inextricably linked, such as behavior driven by aversive and neutral stimuli. We combined a series of behavioral tasks with direct, subsecond dopamine monitoring in the NAc of mice, machine learning, computational modeling, and optogenetic manipulations to describe behavior and related dopamine release patterns across multiple contingencies reinforced by differentially valenced outcomes. We show that dopamine release only conforms to RPE predictions in a subset of learning scenarios but fits valence-independent perceived saliency encoding across conditions. Here, we provide an extended, comprehensive framework for accumbal dopamine release in behavioral control.

Salt as a non-caloric behavioral modifier: A review of evidence from pre-clinical studies

Though excess salt intake is well-accepted as a dietary risk factor for cardiovascular diseases, relatively little has been explored about how it impacts behavior, despite the ubiquity of salt in modern diets. Given the challenges of manipulating salt intake in humans, non-human animals provide a more tractable means for evaluating behavioral sequelae of high salt. By describing what is known about the impact of elevated salt on behavior, this review highlights how underexplored salt's behavioral effects are. Increased salt consumption in adulthood does not affect spontaneous anxiety-related behaviors or locomotor activity, nor acquisition of maze or fear tasks, but does impede expression of spatial/navigational and fear memory. Nest building is reduced by heightened salt in adults, and stress responsivity is augmented. When excess salt exposure occurs during development, and/or to parents, offspring locomotion is increased, and both spatial memory expression and social investigation are attenuated. The largely consistent findings reviewed here indicate expanded study of salt's effects will likely uncover broader behavioral implications, particularly in the scarcely studied female sex.

Dopamine release and its control over early Pavlovian learning differs between the NAc core and medial NAc shell

Dopamine neurons respond to cues to reflect the value of associated outcomes. These cue-evoked dopamine responses can encode the relative rate of reward in rats with extensive Pavlovian training. Specifically, a cue that always follows the previous reward by a short delay (high reward rate) evokes a larger dopamine response in the nucleus accumbens (NAc) core relative to a distinct cue that always follows the prior reward by a long delay (low reward rate). However, it was unclear if these reward rate dopamine signals are evident during early Pavlovian training sessions and across NAc subregions. To address this, we performed fast-scan cyclic voltammetry recordings of dopamine levels to track the pattern of cue- and reward-evoked dopamine signals in the NAc core and medial NAc shell. We identified regional differences in the progression of cue-evoked dopamine signals across training. However, the dopamine response to cues did not reflect the reward rate in either the NAc core or the medial NAc shell during early training sessions. Pharmacological experiments found that dopamine-sensitive conditioned responding emerged in the NAc core before the medial NAc shell. Together, these findings illustrate regional differences in NAc dopamine release and its control over behavior during early Pavlovian learning.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Attenuated dopamine signaling after aversive learning is restored by ketamine to rescue escape actions

Escaping aversive stimuli is essential for complex organisms, but prolonged exposure to stress leads to maladaptive learning. Stress alters neuronal activity and neuromodulatory signaling in distributed networks, modifying behavior. Here, we describe changes in dopaminergic neuron activity and signaling following aversive learning in a learned helplessness paradigm in mice. A single dose of ketamine suffices to restore escape behavior after aversive learning. Dopaminergic neuron activity in the ventral tegmental area (VTA) systematically varies across learning, correlating with future sensitivity to ketamine treatment. Ketamine’s effects are blocked by chemogenetic inhibition of dopamine signaling. Rather than directly altering the activity of dopaminergic neurons, ketamine appears to rescue dopamine dynamics through actions in the medial prefrontal cortex (mPFC). Chemogenetic activation of Drd1 receptor positive mPFC neurons mimics ketamine’s effects on behavior. Together, our data link neuromodulatory dynamics in mPFC-VTA circuits, aversive learning, and the effects of ketamine.

Over 264 million people around the world suffer from depression, according to the World Health Organization (WHO). Depression can be debilitating, and while anti-depressant drugs are available, they do not always work. A small molecule drug mainly used for anesthesia called ketamine has recently been shown to ameliorate depressive symptoms within hours, much faster than most anti-depressants. However, the molecular mechanisms behind this effect are still largely unknown.

Most anti-depressant drugs work by restoring the normal balance of dopamine and other chemical messengers in the brain. Dopamine is released by a specialized group of cells called dopaminergic neurons, and helps us make decisions by influencing a wide range of other cells in the brain. In a healthy brain, dopamine directs us to rewarding choices, while avoiding actions with negative outcomes. During depression, these dopamine signals are perturbed, resulting in reduced motivation and pleasure. But it remained unclear whether ketamine’s anti-depressant activity also relied on dopamine.

To investigate this, Wu et al. used a behavioral study called “learned helplessness” which simulates depression by putting mice in unavoidable stressful situations. Over time the mice learn that their actions do not change the outcome and eventually stop trying to escape from unpleasant situations, even if they are avoidable. The experiment showed that dopaminergic neurons in an area of the brain that is an important part of the “reward and aversion” system became less sensitive to unpleasant stimuli following learned helplessness. When the mice received ketamine, these neurons recovered after a few hours.

Individual mice also responded differently to ketamine. The most ‘resilient’, stress-resistant mice, which had distinct patterns of dopamine signaling, also responded most strongly to the drug. Genetic and chemical manipulation of dopaminergic neurons confirmed that ketamine needed intact dopamine signals to work, and revealed that it acted indirectly on dopamine dynamics via another brain region called the medial prefrontal cortex.

These results shed new light on how a promising new anti-depressant works. In the future, they may also explain why drugs like ketamine work better for some people than others, ultimately helping clinicians select the most effective treatment for individual patients.

Analysis of memory modulation by conditioned stimuli

Conditioned stimuli (CS) have multiple psychological functions that can potentially contribute to their effect on memory formation. It is generally believed that CS-induced memory modulation is primarily due to conditioned emotional responses, however, well-learned CSs not only generate the appropriate behavioral and physiological reactions required to best respond to an upcoming unconditioned stimulus (US), but they also serve as signals that the US is about to occur. Therefore, it is possible that CSs can impact memory consolidation even when their ability to elicit conditioned emotional arousal is significantly reduced. To test this, male Sprague–Dawley rats trained on a signaled active avoidance task were divided into “Avoider” and “Non-Avoider” subgroups on the basis of percentage avoidance after 6 d of training. Subgroup differences in responding to the CS complex were maintained during a test carried out in the absence of the US. Moreover, the subgroups displayed significant differences in stress-induced analgesia (hot-plate test) immediately after this test, suggesting significant subgroup differences in conditioned emotionality. Importantly, using the spontaneous object recognition task, it was found that immediate post-sample exposure to the avoidance CS complex had a similar enhancing effect on object memory in the two subgroups. Therefore, to our knowledge, this is the first study to demonstrate that a significant conditioned emotional response is not necessary for the action of a predictive CS on modulation of memory consolidation.

Acute Stress Enhances Associative Learning via Dopamine Signaling in the Ventral Lateral Striatum

Acute stress transiently increases vigilance, enhancing the detection of salient stimuli in one's environment. This increased perceptual sensitivity is thought to promote the association of rewarding outcomes with relevant cues. The mesolimbic dopamine system is critical for learning cue-reward associations. Dopamine levels in the ventral striatum are elevated following exposure to stress. Together, this suggests that the mesolimbic dopamine system could mediate the influence of acute stress on cue-reward learning. To address this possibility, we examined how a single stressful experience influenced learning in an appetitive pavlovian conditioning task. Male rats underwent an episode of restraint prior to the first conditioning session. This acute stress treatment augmented conditioned responding in subsequent sessions. Voltammetry recordings of mesolimbic dopamine levels demonstrated that acute stress selectively increased reward-evoked dopamine release in the ventral lateral striatum (VLS), but not in the ventral medial striatum. Antagonizing dopamine receptors in the VLS blocked the stress-induced enhancement of conditioned responding. Collectively, these findings illustrate that stress engages dopamine signaling in the VLS to facilitate appetitive learning.SIGNIFICANCE STATEMENT Acute stress influences learning about aversive and rewarding outcomes. Dopamine neurons are sensitive to stress and critical for reward learning. However, it is unclear whether stress regulates reward learning via dopamine signaling. Using fast-scan cyclic voltammetry as rats underwent pavlovian conditioning, we demonstrate that a single stressful experience increases reward-evoked dopamine release in the ventral lateral striatum. This enhanced dopamine signal accompanies a long-lasting increase in conditioned behavioral responding. These findings highlight that the ventral lateral striatum is a node for mediating the effect of stress on reward processing.

The Lateral Preoptic Area: A Novel Regulator of Reward Seeking and Neuronal Activity in the Ventral Tegmental Area

The lateral preoptic area (LPO) is a hypothalamic region whose function has been largely unexplored. Its direct and indirect projections to the ventral tegmental area (VTA) suggest that the LPO could modulate the activity of the VTA and the reward-related behaviors that the VTA underlies. We examined the role of the LPO on reward taking and seeking using operant self-administration of cocaine or sucrose. Rats were trained to self-administer cocaine or sucrose and then subjected to extinction, whereby responding was no longer reinforced. We tested if stimulating the LPO pharmacologically with bicuculline or chemogenetically with Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) modifies self-administration and/or seeking. In another set of experiments, we tested if manipulating the LPO influences cocaine self-administration during and after punishment. To examine the functional connectivity between the LPO and VTA, we used in vivo electrophysiology recordings in anesthetized rats. We tested if stimulating the LPO modifies the activity of GABA and dopamine neurons of the VTA. We found that stimulating the LPO reinstated cocaine and sucrose seeking behavior but had no effect on reward intake. Furthermore, both stimulating and inhibiting the LPO prevented the sustained reduction in cocaine intake seen after punishment. Finally, stimulating the LPO inhibited the activity of VTA GABA neurons while enhancing that of VTA dopamine neurons. These findings indicate that the LPO has the capacity to drive reward seeking, modulate sustained reductions in self-administration following punishment, and regulate the activity of VTA neurons. Taken together, these findings implicate the LPO as a previously overlooked member of the reward circuit.

Interfacing behavioral and neural circuit models for habit formation

Habits are an important mechanism by which organisms can automate the control of behavior to alleviate cognitive demand. However, transitions to habitual control are risky because they lead to inflexible responding in the face of change. The question of how the brain controls transitions into habit is thus an intriguing one. How do we regulate when our repeated actions become automated? When is it advantageous or disadvantageous to release actions from cognitive control? Decades of research have identified a variety of methods for eliciting habitual responding in animal models. Progress has also been made to understand which brain areas and neural circuits control transitions into habit. Here, I discuss existing research on behavioral and neural circuit models for habit formation (with an emphasis on striatal circuits), and discuss strategies for combining information from different paradigms and levels of analysis to prompt further progress in the field.