A selective role for dopamine in stimulus-reward learning

A brain-inspired algorithm that mitigates catastrophic forgetting of artificial and spiking neural networks with low computational cost

Neuromodulators in the brain act globally at many forms of synaptic plasticity, represented as metaplasticity, which is rarely considered by existing spiking (SNNs) and nonspiking artificial neural networks (ANNs). Here, we report an efficient brain-inspired computing algorithm for SNNs and ANNs, referred to here as neuromodulation-assisted credit assignment (NACA), which uses expectation signals to induce defined levels of neuromodulators to selective synapses, whereby the long-term synaptic potentiation and depression are modified in a nonlinear manner depending on the neuromodulator level. The NACA algorithm achieved high recognition accuracy with substantially reduced computational cost in learning spatial and temporal classification tasks. Notably, NACA was also verified as efficient for learning five different class continuous learning tasks with varying degrees of complexity, exhibiting a markedly mitigated catastrophic forgetting at low computational cost. Mapping synaptic weight changes showed that these benefits could be explained by the sparse and targeted synaptic modifications attributed to expectation-based global neuromodulation.

Nucleus accumbens core acetylcholine receptors modulate the balance of flexible and inflexible cue-directed motivation

Sign-tracking is a conditioned response where animals interact with reward-predictive cues due to the cues having motivational value, or incentive salience. The nucleus accumbens core (NAc) has been implicated in mediating the sign-tracking response. Additionally, acetylcholine (ACh) transmission throughout the striatum has been attributed to both incentive motivation and behavioral flexibility. Here, we demonstrate a role for NAc ACh receptors in the flexibility of sign-tracking. Sign-tracking animals were exposed to an omission contingency, in which vigorous sign-tracking was punished by reward omission. Animals rapidly adjusted their behavior, but they maintained sign-tracking in a less vigorous manner that did not cancel reward. Within this context of sign-tracking being persistent yet flexible in structure, blockade of NAc nicotinic receptors (nAChRs) led to a persistence in the initial sign-tracking response during omission followed by a period of change in the makeup of sign-tracking, whereas blockade of muscarinic receptors (mAChRs) oppositely enhanced the omission-related development of the new sign-tracking behaviors. Later, once omission learning had occurred, nAChR blockade uniquely led to reduced sign-tracking and elevated reward-directed behaviors instead. These results indicate that NAc ACh receptors have opposing roles in maintaining learned patterns of sign-tracking, with nAChRs having a special involvement in regulating the structure of the sign-tracking response.

Liraglutide restores impaired associative learning in individuals with obesity

Survival under selective pressure is driven by the ability of our brain to use sensory information to our advantage to control physiological needs. To that end, neural circuits receive and integrate external environmental cues and internal metabolic signals to form learned sensory associations, consequently motivating and adapting our behaviour. The dopaminergic midbrain plays a crucial role in learning adaptive behaviour and is particularly sensitive to peripheral metabolic signals, including intestinal peptides, such as glucagon-like peptide 1 (GLP-1). In a single-blinded, randomized, controlled, crossover basic human functional magnetic resonance imaging study relying on a computational model of the adaptive learning process underlying behavioural responses, we show that adaptive learning is reduced when metabolic sensing is impaired in obesity, as indexed by reduced insulin sensitivity (participants: N = 30 with normal insulin sensitivity; N = 24 with impaired insulin sensitivity). Treatment with the GLP-1 receptor agonist liraglutide normalizes impaired learning of sensory associations in men and women with obesity. Collectively, our findings reveal that GLP-1 receptor activation modulates associative learning in people with obesity via its central effects within the mesoaccumbens pathway. These findings provide evidence for how metabolic signals can act as neuromodulators to adapt our behaviour to our body's internal state and how GLP-1 receptor agonists work in clinics.

Dopamine transporter blockade during adolescence increases adult dopamine function, impulsivity, and aggression

Sensitive developmental periods shape neural circuits and enable adaptation. However, they also engender vulnerability to factors that can perturb developmental trajectories. An understanding of sensitive period phenomena and mechanisms separate from sensory system development is still lacking, yet critical to understanding disease etiology and risk. The dopamine system is pivotal in controlling and shaping adolescent behaviors, and it undergoes heightened plasticity during that time, such that interference with dopamine signaling can have long-lasting behavioral consequences. Here we sought to gain mechanistic insight into this dopamine-sensitive period and its impact on behavior. In mice, dopamine transporter (DAT) blockade from postnatal (P) day 22 to 41 increases aggression and sensitivity to amphetamine (AMPH) behavioral stimulation in adulthood. Here, we refined this sensitive window to P32-41 and identified increased firing of dopaminergic neurons in vitro and in vivo as a neural correlate to altered adult behavior. Aggression can result from enhanced impulsivity and cognitive dysfunction, and dopamine regulates working memory and motivated behavior. Hence, we assessed these behavioral domains and found that P32-41 DAT blockade increases impulsivity but has no effect on cognition, working memory, or motivation in adulthood. Lastly, using optogenetics to drive dopamine neurons, we find that increased VTA but not SNc dopaminergic activity mimics the increase in impulsive behavior in the Go/NoGo task observed after adolescent DAT blockade. Together our data provide insight into the developmental origins of aggression and impulsivity that may ultimately improve diagnosis, prevention, and treatment strategies for related neuropsychiatric disorders.

Effects of inter-trial interval on sign-tracking and conditioned reinforcer efficacy in female rats

Previous nonhuman studies have reported that sign-tracking to a conditioned stimulus (CS) is increased when the intertrial interval (ITI) duration is increased. Separate studies indicate that individual differences in sign-tracking (vs. goal-tracking) at a fixed ITI (and CS duration) is predictive of the conditioned reinforcer efficacy of the CS. The present study evaluates, for the first time, if increasing the ITI increases rats' sign-tracking and the conditioned reinforcing efficacy of the CS. Forty-five female rats were randomly assigned to one of three groups that completed appetitive Pavlovian training with ITIs of 14, 24, or 96 s. Subsequently, they completed tests of conditioned reinforcement. Replicating previous findings, longer ITIs increased sign-tracking to a lever-CS and, extending the literature, conditioned reinforcer efficacy of that CS was highest at the longest ITI used during Pavlovian training. Implications for behavioral interventions using conditioned reinforcement are discussed.

Changes in dorsomedial striatum activity mediate expression of goal-directed vs. habit-like cue-induced cocaine seeking

A preclinical model of cue exposure therapy, cue extinction, reduces cue-induced cocaine seeking when drug seeking is goal-directed but not habitual. Goal-directed and habitual behaviors differentially rely on the dorsomedial striatum (DMS) and dorsolateral striatum (DLS), but the effects of cue extinction on dorsal striatal responses to cue-induced drug seeking are unknown. We used fiber photometry to examine how dorsal striatal intracellular calcium and extracellular dopamine activity differs between goal-directed and habitual cue-induced cocaine seeking and how it is impacted by cue extinction. Rats trained to self-administer cocaine paired with an audiovisual cue on schedules of reinforcement that promote goal-directed or habitual cocaine seeking had different patterns of dorsal striatal calcium and dopamine responses to cue-reinforced lever presses. Cue extinction reduced calcium and dopamine responses during subsequent drug seeking in the DMS, but not in the DLS. Therefore, cue extinction may reduce goal-directed behavior through its effects on the DMS, whereas habitual behavior and the DLS are unaffected.

Past, present, and future of tools for dopamine detection

Dopamine (DA) is a critical neuromodulator involved in various brain functions. To understand how DA regulates neural circuits and behaviors in the physiological and pathological conditions, it is essential to have tools that enable the direct detection of DA dynamics in vivo. Recently, genetically encoded DA sensors based on G protein-coupled receptors revolutionized this field, as it allows us to track in vivo DA dynamic with unprecedented spatial-temporal resolution, high molecular specificity, and sub-second kinetics. In this review, we first summarize traditional DA detection methods. Then we focus on the development of genetically encoded DA sensors and feature its significance to understanding dopaminergic neuromodulation across diverse behaviors and species. Finally, we present our perspectives about the future direction of the next-generation DA sensors and extend their potential applications. Overall, this review offers a comprehensive perspective on the past, present, and future of DA detection tools, with important implications for the study of DA functions in health and disease.

Decreased Ventral Tegmental Area CB1R Signaling Reduces Sign Tracking and Shifts Cue-Outcome Dynamics in Rat Nucleus Accumbens

Sign-tracking (ST) rats show enhanced cue sensitivity before drug experience that predicts greater discrete cue-induced drug seeking compared with goal-tracking or intermediate rats. Cue-evoked dopamine in the nucleus accumbens (NAc) is a neurobiological signature of sign-tracking behaviors. Here, we examine a critical regulator of the dopamine system, endocannabinoids, which bind the cannabinoid receptor-1 (CB1R) in the ventral tegmental area (VTA) to control cue-evoked striatal dopamine levels. We use cell type-specific optogenetics, intra-VTA pharmacology, and fiber photometry to test the hypothesis that VTA CB1R receptor signaling regulates NAc dopamine levels to control sign tracking. We trained male and female rats in a Pavlovian lever autoshaping (PLA) task to determine their tracking groups before testing the effect of VTA → NAc dopamine inhibition. We found that this circuit is critical for mediating the vigor of the ST response. Upstream of this circuit, intra-VTA infusions of rimonabant, a CB1R inverse agonist, during PLA decrease lever and increase food cup approach in sign-trackers. Using fiber photometry to measure fluorescent signals from a dopamine sensor, GRABDA (AAV9-hSyn-DA2m), we tested the effects of intra-VTA rimonabant on NAc dopamine dynamics during autoshaping in female rats. We found that intra-VTA rimonabant decreased sign-tracking behaviors, which was associated with increases in NAc shell, but not core, dopamine levels during reward delivery [unconditioned stimulus (US)]. Our results suggest that CB1R signaling in the VTA influences the balance between the conditioned stimulus-evoked and US-evoked dopamine responses in the NAc shell and biases behavioral responding to cues in sign-tracking rats.SIGNIFICANCE STATEMENT Substance use disorder (SUD) is a chronically relapsing psychological disorder that affects a subset of individuals who engage in drug use. Recent research suggests that there are individual behavioral and neurobiological differences before drug experience that predict SUD and relapse vulnerabilities. Here, we investigate how midbrain endocannabinoids regulate a brain pathway that is exclusively involved in driving cue-motivated behaviors of sign-tracking rats. This work contributes to our mechanistic understanding of individual vulnerabilities to cue-triggered natural reward seeking that have relevance for drug-motivated behaviors.

The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity

Prediction errors are differences between expected and actual sensory input and are thought to be key computational signals that drive learning related plasticity. One way that prediction errors could drive learning is by activating neuromodulatory systems to gate plasticity. The catecholaminergic locus coeruleus (LC) is a major neuromodulatory system involved in neuronal plasticity in the cortex. Using two-photon calcium imaging in mice exploring a virtual environment, we found that the activity of LC axons in the cortex correlated with the magnitude of unsigned visuomotor prediction errors. LC response profiles were similar in both motor and visual cortical areas, indicating that LC axons broadcast prediction errors throughout the dorsal cortex. While imaging calcium activity in layer 2/3 of the primary visual cortex, we found that optogenetic stimulation of LC axons facilitated learning of a stimulus-specific suppression of visual responses during locomotion. This plasticity - induced by minutes of LC stimulation - recapitulated the effect of visuomotor learning on a scale that is normally observed during visuomotor development across days. We conclude that prediction errors drive LC activity, and that LC activity facilitates sensorimotor plasticity in the cortex, consistent with a role in modulating learning rates.

Towards neuromarkers for tailored smoking cessation treatments

Vulnerability to compulsive drug use stems from dysregulated activity within the neural networks that underlie reward and executive functions. Empirical evidence suggests that a) attributing high motivational salience to drug-related stimuli leads to compulsive drug seeking and b) cognitive control deficits lead to compulsive drug taking. Noninvasive neuroimaging techniques enable brain activity monitoring during affective and cognitive processing and are paving the way to precision medicine for substance use disorders. Identifying robust neuromarkers of affective and cognitive dysregulation would allow clinicians to personalize treatments by targeting individual psychophysiological vulnerabilities. However, methodological choices have biased the field toward experimental paradigms that cannot optimally assess individual differences in the motivational salience of drug-related cues and in the ability to control drug-related decisions, choices which have hindered the identification of clinically relevant neuromarkers. Here, we show that once these shortcomings are amended, replicable neuromarkers of the tendency to attribute motivational salience to drug-related cues and the ability to control drug-related decisions emerge. While we use tobacco use disorder as a model, we also show that the methodological issues highlighted here are relevant to other disorders characterized by maladaptive appetitive behaviors.

Dopamine in the dorsal bed nucleus of stria terminalis signals Pavlovian sign-tracking and reward violations

Midbrain and striatal dopamine signals have been extremely well characterized over the past several decades, yet novel dopamine signals and functions in reward learning and motivation continue to emerge. A similar characterization of real-time sub-second dopamine signals in areas outside of the striatum has been limited. Recent advances in fluorescent sensor technology and fiber photometry permit the measurement of dopamine binding correlates, which can divulge basic functions of dopamine signaling in non-striatal dopamine terminal regions, like the dorsal bed nucleus of the stria terminalis (dBNST). Here, we record GRABDA signals in the dBNST during a Pavlovian lever autoshaping task. We observe greater Pavlovian cue-evoked dBNST GRABDA signals in sign-tracking (ST) compared to goal-tracking/intermediate (GT/INT) rats and the magnitude of cue-evoked dBNST GRABDA signals decreases immediately following reinforcer-specific satiety. When we deliver unexpected rewards or omit expected rewards, we find that dBNST dopamine signals encode bidirectional reward prediction errors in GT/INT rats, but only positive prediction errors in ST rats. Since sign- and goal-tracking approach strategies are associated with distinct drug relapse vulnerabilities, we examined the effects of experimenter-administered fentanyl on dBNST dopamine associative encoding. Systemic fentanyl injections do not disrupt cue discrimination but generally potentiate dBNST dopamine signals. These results reveal multiple dBNST dopamine correlates of learning and motivation that depend on the Pavlovian approach strategy employed.

Inhibition of dopamine neurons prevents incentive value encoding of a reward cue: With revelations from deep phenotyping

The survival of an organism is dependent on their ability to respond to cues in the environment. Such cues can attain control over behavior as a function of the value ascribed to them. Some individuals have an inherent tendency to attribute reward-paired cues with incentive motivational value, or incentive salience. For these individuals, termed sign-trackers, a discrete cue that precedes reward delivery becomes attractive and desirable in its own right. Prior work suggests that the behavior of sign-trackers is dopamine-dependent, and cue-elicited dopamine in the nucleus accumbens is believed to encode the incentive value of reward cues. Here we exploited the temporal resolution of optogenetics to determine whether selective inhibition of ventral tegmental area (VTA) dopamine neurons during cue presentation attenuates the propensity to sign-track. Using male tyrosine hydroxylase (TH)-Cre Long Evans rats it was found that, under baseline conditions, ∼84% of TH-Cre rats tend to sign-track. Laser-induced inhibition of VTA dopamine neurons during cue presentation prevented the development of sign-tracking behavior, without affecting goal-tracking behavior. When laser inhibition was terminated, these same rats developed a sign-tracking response. Video analysis using DeepLabCut revealed that, relative to rats that received laser inhibition, rats in the control group spent more time near the location of the reward cue even when it was not present and were more likely to orient towards and approach the cue during its presentation. These findings demonstrate that cue-elicited dopamine release is critical for the attribution of incentive salience to reward cues.

Significance Statement

Activity of dopamine neurons in the ventral tegmental area (VTA) during cue presentation is necessary for the development of a sign-tracking, but not a goal-tracking, conditioned response in a Pavlovian task. We capitalized on the temporal precision of optogenetics to pair cue presentation with inhibition of VTA dopamine neurons. A detailed behavioral analysis with DeepLabCut revealed that cue-directed behaviors do not emerge without VTA dopamine. Importantly, however, when optogenetic inhibition is lifted, cue-directed behaviors increase, and a sign-tracking response develops. These findings confirm the necessity of VTA dopamine during cue presentation to encode the incentive value of reward cues.

Enhancement of memory consolidation by an avoidance conditioned stimulus: Modulation by the D3 receptor

Conditioned stimuli (CS) paired with foot-shock can enhance memory consolidation. Because the dopamine D3R has been implicated in mediating various responses to CSs, the current study explored its potential role in modulation of memory consolidation by an avoidance CS. Male Sprague-Dawley rats trained to avoid foot-shocks in a two-way signalled active avoidance task (8 sessions, 30 trials per session, 0.8 mA foot-shock) were pre-treated with the D3R antagonist NGB-2904 (Vehicle, 0.1 or 5 mg/kg) and exposed to the CS immediately after the sample phase of an object recognition memory task. Discrimination ratios were assessed 72 h later. Immediate, but not delayed (6 h), post-sample exposure to the CS enhanced object recognition memory and this effect was dose-dependently blocked by NGB-2904. Control experiments with the beta-noradrenergic receptor antagonist propranolol (10 or 20 mg/kg) and D2R antagonist pimozide (0.2 or 0.6 mg/kg) indicated that NGB-2904 targeted post-training memory consolidation. Exploring the pharmacological selectivity of the D3R effect, it was found that: 1) 5 mg/kg NGB-2904 blocked conditioned memory modulation produced by post-sample exposure to a "weak" CS (one day of avoidance training) and concurrent stimulation of catecholamine activity by 10 mg/kg bupropion; 2) post-sample exposure to a "weak" CS and concurrent administration of the D3R agonist 7-OH-DPAT (1 mg/kg) enhanced consolidation of object memory. Finally, because 5 mg/kg NGB-2904 had no effect on modulation by avoidance training in the presence of foot-shocks, the findings herein support the hypothesis that the D3R plays an important role in modulation of memory consolidation by CSs.

Conditioned approach behavior of SHR and SD rats during Pavlovian conditioning

Individual differences in reward-related learning are relevant to many behavioral disorders. Sensory cues that predict reward can become incentive stimuli that adaptively support behavior, or alternatively, cause maladaptive behaviors. The spontaneously hypertensive rat (SHR) expresses a genetically determined elevated sensitivity to delay of reward, and has been extensively studied as a behavioral model for attention deficit hyperactivity disorder (ADHD). We investigated reward-related learning in the SHR, comparing them to Sprague-Dawley (SD) rats as a reference strain. A standard Pavlovian conditioned approach task was used, in which a lever cue was followed by reward. Lever presses could occur while the lever was extended, but had no effect on reward delivery. The behavior of both the SHRs and the SD rats showed that they learnt that the lever cue predicted reward. However, the pattern of behavior differed between the strains. During lever cue presentation, SD rats pressed the lever more often and made fewer magazine entries than SHRs. When lever contacts that did not result in lever presses were analyzed, there was no significant difference between SHRs and SDs. These results suggest that the SHRs attributed less incentive value to the conditioned stimulus than the SD rats. During the presentation of the conditioned cue, cue directed responses are called sign tracking responses, whereas responses directed towards the food magazine are called goal tracking responses. Analysis of behavior using a standard Pavlovian conditioned approach index to quantify sign and goal tracking tendencies showed that both strains had a tendency towards goal tracking in this task. However, the SHRs showed a significantly greater goal tracking tendency than the SD rats. Taken together, these findings suggest that attribution of incentive value to reward predicting cues is attenuated in SHRs, which might explain their elevated sensitivity to delay of reward.

Nicotinic and muscarinic acetylcholine receptor antagonism dose-dependently decreases sign- but not goal-tracking behavior in male rats

RATIONALE

Acetylcholinergic antagonists have shown some promise in reducing addiction-related behaviors in both preclinical and clinical studies. However, the psychological mechanisms by which these drugs are able to affect addictive behavior remain unclear. A particular key process for the development of addiction is the attribution of incentive salience to reward-related cues, which can be specifically measured in animals using a Pavlovian conditioned approach procedure. When confronted with a lever that predicts food delivery, some rats engage with the lever directly (i.e., they sign track), indicating attribution of incentive-motivational properties to the lever itself. In contrast, others treat the lever as a predictive cue and approach the location of impending food delivery (i.e., they goal track), without treating the lever itself as a reward.

OBJECTIVES

We tested whether systemic antagonism of the either nicotinic or muscarinic acetylcholine receptors would selectively affect sign- or goal-tracking behavior, indicating a selective effect on incentive salience attribution.

METHODS

A total of 98 male Sprague Dawley rats were either given the muscarinic antagonist scopolamine (100, 50, or 10 µg/kg i.p.) or the nicotinic antagonist mecamylamine (0.3, 1.0, or 3 mg/kg i.p.) before being trained on a Pavlovian conditioned approach procedure.

RESULTS

Scopolamine dose-dependently decreased sign tracking behavior and increased goal-tracking behavior. Mecamylamine reduced sign-tracking but did not affect goal-tracking behavior.

CONCLUSIONS

Antagonism of either muscarinic or nicotinic acetylcholine receptors can reduce incentive sign-tracking behavior in male rats. This effect appears to be specifically due to a reduction in incentive salience attribution since goal-tracking either increased or was not affected by these manipulations.

The propensity to sign-track is associated with externalizing behavior and distinct patterns of reward-related brain activation in youth

Externalizing behaviors in childhood often predict impulse control disorders in adulthood; however, the underlying bio-behavioral risk factors are incompletely understood. In animals, the propensity to sign-track, or the degree to which incentive motivational value is attributed to reward cues, is associated with externalizing-type behaviors and deficits in executive control. Using a Pavlovian conditioned approach paradigm, we quantified sign-tracking in 40 healthy 9-12-year-olds. We also measured parent-reported externalizing behaviors and anticipatory neural activations to outcome-predicting cues using the monetary incentive delay fMRI task. Sign-tracking was associated with attentional and inhibitory control deficits and the degree of amygdala, but not cortical, activation during reward anticipation. These findings support the hypothesis that youth with a propensity to sign-track are prone to externalizing tendencies, with an over-reliance on subcortical cue-reactive brain systems. This research highlights sign-tracking as a promising experimental approach delineating the behavioral and neural circuitry of individuals at risk for externalizing disorders.

Neuro-Immune Modulation of Cholinergic Signaling in an Addiction Vulnerability Trait

Sign-tracking (ST) describes the propensity to approach and contact a Pavlovian reward cue. By contrast, goal-trackers (GTs) respond to such a cue by retrieving the reward. These behaviors index the presence of opponent cognitive-motivational traits, with STs exhibiting attentional control deficits, behavior dominated by incentive motivational processes, and vulnerability for addictive drug taking. Attentional control deficits in STs were previously attributed to attenuated cholinergic signaling, resulting from deficient translocation of intracellular choline transporters (CHTs) into synaptosomal plasma membrane. Here, we investigated a posttranslational modification of CHTs, poly-ubiquitination, and tested the hypothesis that elevated cytokine signaling in STs contributes to CHT modification. We demonstrated that intracellular CHTs, but not plasma membrane CHTs, are highly ubiquitinated in male and female sign-tracking rats when compared with GTs. Moreover, levels of cytokines measured in cortex and striatum, but not spleen, were higher in STs than in GTs. Activation of the innate immune system by systemic administration of the bacterial endotoxin lipopolysaccharide (LPS) elevated ubiquitinated CHT levels in cortex and striatum of GTs only, suggesting ceiling effects in STs. In spleen, LPS increased levels of most cytokines in both phenotypes. In cortex, LPS particularly robustly increased levels of the chemokines CCL2 and CXCL10. Phenotype-specific increases were restricted to GTs, again suggesting ceiling effects in STs. These results indicate that interactions between elevated brain immune modulator signaling and CHT regulation are essential components of the neuronal underpinnings of the addiction vulnerability trait indexed by sign-tracking.

Reinstatement of Pavlovian responses to alcohol cues by stress

RATIONALE

Stress may contribute to relapse to alcohol use in part by enhancing reactivity to cues previously paired with alcohol. Yet, standard models of stress-induced reinstatement generally use contingent presentations of alcohol-paired cues to reinforce instrumental behaviors, making it difficult to isolate the ability of cues to invigorate alcohol-seeking.

OBJECTIVE

Here we sought to test the impact of stress on behavioral responses to alcohol-paired cues, using a model of stress-induced reinstatement of Pavlovian conditioned approach, inspired by Nadia Chaudhri's work on context-induced reinstatement.

METHODS

Long Evans rats were trained to associate one auditory cue with delivery of alcohol or sucrose and an alternative auditory cue with no reward. Following extinction training, rats were exposed to a stressor prior to being re-exposed to the cues under extinction conditions. We assessed the effects of yohimbine, intermittent footshock and olfactory cues paired with social defeat on responses to alcohol-paired cues and the effects of yohimbine on responses to sucrose-paired cues.

RESULTS

The pharmacological stressor, yohimbine, enhanced alcohol seeking in a Pavlovian setting, but not in a cue-selective manner. Intermittent footshock and social defeat cues did not enhance alcohol seeking in this paradigm.

CONCLUSIONS

While yohimbine elicited reinstatement of reward-seeking in a Pavlovian setting, these effects may be unrelated to activation of stress systems or to interactions with specific cues.

Repeated dosing with cocaine produces strain-dependent effects on responding for conditioned reinforcement in Collaborative Cross mice

RATIONALE

Cocaine use disorder (CUD) is a highly heritable form of substance use disorder, with genetic variation accounting for a substantial proportion of the risk for transitioning from recreational use to a clinically impairing addiction. With repeated exposures to cocaine, psychomotor and incentive sensitization are observed in rodents. These phenomena are thought to model behavioral changes elicited by the drug that contribute to the progression into addiction, but little is known about how genetic variation may moderate these consequences.

OBJECTIVES

Here, we describe the use of two Collaborative Cross (CC) recombinant inbred mouse strains that either exhibit high (CC018/UncJ) or no (CC027/GeniUncJ) psychomotor sensitization in response to cocaine to measure phenotypes related to incentive sensitization after repeated cocaine exposures; given the relationship of incentive motivation to nucleus accumbens core (NAc) dopamine release and reuptake, we also assessed these neurochemical mechanisms.

METHODS

Adult male and female CC018/UncJ and CC027/GeniUncJ mice underwent Pavlovian conditioning to associate a visual cue with presentation of a palatable food reward, then received five, every-other-day injections of cocaine or vehicle. Following Pavlovian re-training, they underwent testing acquisition of a new operant response for the visual cue, now serving as a conditioned reinforcer. Subsequently, electrically evoked dopamine release was assessed using fast-scan cyclic voltammetry from acute brain slices containing the NAc.

RESULTS

While both strains acquired the Pavlovian association, only CC018/UncJ mice showed conditioned reinforcement and incentive sensitization in response to cocaine, while CC027/GeniUncJ mice did not. Voltammetry data revealed that CC018/UncJ, compared to CC027/GeniUnc, mice exhibited higher baseline dopamine release and uptake. Moreover, chronic cocaine exposure blunted tonic and phasic dopamine release in CC018/UncJ, but not CC027/GeniUncJ, mice.

CONCLUSIONS

Genetic background is a moderator of cocaine-induced neuroadaptations in mesolimbic dopamine signaling, which may contribute to both psychomotor and incentive sensitization and indicate a shared biological mechanism of variation.

Optogenetic activation of basolateral amygdala-to-nucleus accumbens core neurons promotes Pavlovian approach responses but not instrumental pursuit of reward cues

Reward-associated conditioned stimuli (CSs) can acquire predictive value, evoking conditioned approach behaviours that prepare animals to engage with forthcoming rewards. Such CSs can also acquire conditioned reinforcing value, becoming attractive and pursued. Through their conditioned effects, CSs can promote adaptive (e.g., locating food) but also maladaptive behaviours (e.g., drug use). Basolateral amygdala neurons projecting to the nucleus accumbens core (BLA→NAc core neurons) mediate the response to appetitive CSs, but the extent to which this involves effects on the predictive and/or conditioned reinforcing properties of CSs is unclear. Thus, we examined the effects of optogenetic stimulation of BLA→NAc core neurons on i) CS-triggered approach to the site of reward delivery, a Pavlovian conditioned approach response and ii) the instrumental pursuit of a CS, a measure of conditioned reinforcement. Water-restricted, adult male rats learned that a light-tone compound cue (the CS) predicts water delivery into a receptacle. Pairing optogenetic stimulation of BLA→NAc core neurons with CS presentation potentiated CS-triggered water receptacle visits. This suggests that activity in BLA→NAc core neurons promotes Pavlovian goal-approach behaviour. Next, rats could lever press for CS presentations, without water delivery. Optogenetic stimulation of BLA→NAc core neurons either during instrumental test sessions or during prior CS-water conditioning did not influence lever responding for the CS. This suggests that activity in BLA→NAc core neurons does not influence the instrumental pursuit of a water-paired CS. We conclude that activation of BLA→NAc core neurons promotes cue-induced control over behaviour by increasing conditioned goal-approach responses, without affecting the operant pursuit of reward cues.

Hydrogel-Coated Microelectrode Resists Protein Passivation of In Vivo Amperometric Sensors

Passivation of electrodes caused by nonspecific adsorption of protein can dramatically reduce sensing sensitivity and accuracy, which is a great challenge for in vivo neurochemical monitoring. However, most antipassivation strategies are not suitable to carbon fiber microelectrodes (CFMEs) for in vivo measurement, and these methods also do not work on electrochemical biosensors that fix biometric elements. In this study, we demonstrate that chitosan hydrogel-coated microelectrodes can avoid the current passivation caused by protein adsorption on the surface of carbon fiber because the chitosan hydrogel prepared by local pH gradient caused by hydrogen evolution reaction has three-dimensional networks containing large amounts of water. The highly hydrophilic three-dimensional structure of hydrogel not only forms a biocompatible interface to confine enzymes but also keeps the fast mass transfer of analytes, such as dopamine, ascorbic acid, and glucose. The consistency of the precalibration and postcalibration of the prepared sensor enables in vivo amperometric detection of both electroactive species based on their redox property and electroinactive species based on the enzyme. This study provides a simple and versatile strategy to constitute an amperometric sensor interface to resist passivation of protein adsorption in a complex biological environment such as the brain.

D2/3 Agonist during Learning Potentiates Cued Risky Choice

Impulse control and/or gambling disorders can be triggered by dopamine agonist therapies used to treat Parkinson's disease, but the cognitive and neurobiological mechanisms underlying these adverse effects are unknown. Recent data show that adding win-paired sound and light cues to the rat gambling task (rGT) potentiates risky decision-making and impulsivity via the dopamine system, and that changing dopaminergic tone has a greater influence on behavior while subjects are learning task contingencies. Dopamine agonist therapy may therefore be potentiating risk-taking by amplifying the behavioral impact of gambling-related cues on novel behavior. Here, we show that ropinirole treatment in male rats transiently increased motor impulsivity but robustly and progressively increased choice of the high-risk/high-reward options when administered during acquisition of the cued but not uncued rGT. Early in training, ropinirole increased win-stay behavior after large unlikely wins on the cued rGT, indicative of enhanced model-free learning, which mediated the drug's effect on later risk preference. Ex vivo cFos imaging showed that both chronic ropinirole and the addition of win-paired cues suppressed the activity of dopaminergic midbrain neurons. The ratio of midbrain:prefrontal cFos+ neurons was lower in animals with suboptimal choice patterns and tended to predict risk preference across all rats. Network analyses further suggested that ropinirole induced decoupling of the dopaminergic cells of the VTA and nucleus accumbens but only when win-paired cues were present. Frontostriatal activity uninformed by the endogenous dopaminergic teaching signal therefore appeared to perpetuate risky choice, and ropinirole exaggerated this disconnect in synergy with reward-paired cues.SIGNIFICANCE STATEMENT D2/3 receptor agonists, used to treat Parkinson's disease, can cause gambling disorder through an unknown mechanism. Ropinirole increased risky decision-making in rats, but only when wins were paired with casino-inspired sounds and lights. This was mediated by increased win-stay behavior after large unlikely wins early in learning, indicating enhanced model-free learning. cFos imaging showed that ropinirole suppressed activity of midbrain dopamine neurons, an effect that was mimicked by the addition of win-paired cues. The degree of risky choice rats exhibited was uniquely predicted by the ratio of midbrain dopamine:PFC activity. Depriving the PFC of the endogenous dopaminergic teaching signal may therefore drive risky decision-making on-task, and ropinirole acts synergistically with win-paired cues to amplify this.

Ventral pallidal regulation of motivated behaviors and reinforcement

The interconnected nuclei of the ventral basal ganglia have long been identified as key regulators of motivated behavior, and dysfunction of this circuit is strongly implicated in mood and substance use disorders. The ventral pallidum (VP) is a central node of the ventral basal ganglia, and recent studies have revealed complex VP cellular heterogeneity and cell- and circuit-specific regulation of reward, aversion, motivation, and drug-seeking behaviors. Although the VP is canonically considered a relay and output structure for this circuit, emerging data indicate that the VP is a central hub in an extensive network for reward processing and the regulation of motivation that extends beyond classically defined basal ganglia borders. VP neurons respond temporally faster and show more advanced reward coding and prediction error processing than neurons in the upstream nucleus accumbens, and regulate the activity of the ventral mesencephalon dopamine system. This review will summarize recent findings in the literature and provide an update on the complex cellular heterogeneity and cell- and circuit-specific regulation of motivated behaviors and reinforcement by the VP with a specific focus on mood and substance use disorders. In addition, we will discuss mechanisms by which stress and drug exposure alter the functioning of the VP and produce susceptibility to neuropsychiatric disorders. Lastly, we will outline unanswered questions and identify future directions for studies necessary to further clarify the central role of VP neurons in the regulation of motivated behaviors. Significance: Research in the last decade has revealed a complex cell- and circuit-specific role for the VP in reward processing and the regulation of motivated behaviors. Novel insights obtained using cell- and circuit-specific interrogation strategies have led to a major shift in our understanding of this region. Here, we provide a comprehensive review of the VP in which we integrate novel findings with the existing literature and highlight the emerging role of the VP as a linchpin of the neural systems that regulate motivation, reward, and aversion. In addition, we discuss the dysfunction of the VP in animal models of neuropsychiatric disorders.

Conductive Metal-Organic Framework Microelectrodes Regulated by Conjugated Molecular Wires for Monitoring of Dopamine in the Mouse Brain

Herein, we demonstrated a strategy to regulate the conductive metal-organic framework (MOF) surface, by the conjugated molecule wires for selective and sensitive determination of dopamine (DA) in the live brain. The MOFs were decorated at the carbon fiber electrode deposited by Au nanoleaves as the upper electric transducer to provide rich electrocatalytic sites for electron transfer of neurochemicals at the electrode surface, leading to greatly enhanced sensitivity for detection of neurochemicals. On the other hand, the conjugated molecular wire, 4-(thiophen-3-ylethynyl)-benzaldehyde (RP1), was synthesized and assembled as an underlying bridge to regulate the electrochemical processes at the MOF-based electrode, specifically decreasing the reaction Gibbs free energy of DA oxidation, thus selectively promoting the heterogeneous electron transfer of DA from the MOF layer to the electrode surface. Owing to the electrocatalytic activity for DA oxidation, the present microsensor exhibited high selectivity for real-time tracking of DA in a good linear relationship in the range of 0.004-0.4 μM with a detection limit of 1 nM. Eventually, this functionalized electrode was successfully applied for in vivo monitoring of DA in mouse brains with Parkinson's disease (PD) model. The results indicated that the levels of DA were obviously decreased in both acute and subacute PD models. Moreover, the level of DA strongly depended on the amount of uric acid (UA), a physiological antioxidant, which rose as the UA amount was lower than 200 mg kg^-1 but was downregulated again after treatment by a higher amount of UA.

Blockade of dopamine D3 receptor in ventral tegmental area attenuating contextual fear memory

The abnormal fear memory will lead to the onset of stress disorders, such as post-traumatic stress disorder (PTSD) and so on. Therefore, the intervention in the formation of abnormal fear memory will provide a new strategy for the prevention and treatment of PTSD. In our previous studies, we found that blockade of dopamine D3 receptor (DRD3) with highly selective antagonist YQA14 or knockout of DRD3 was able to attenuate the expression or retrieval of fear memory in PTSD animal models. However, the neurobiological mechanism of regulation of DRD3 in fear is unclear. In the present research, we clarified that DRD3 was expressed in the dopaminergic (DAergic) neurons in the ventral tegmental area (VTA). Then, we identified that microinjection of YQA14 (1 μg/0.2 μl/side) in VTA before the aversive stimuli in the training session or during days subsequent to the shock significantly meliorated the freezing behaviors in the inescapable electric foot-shock model. At last, using fiber photometry system, we found that microinjection of YQA14 in VTA promoted the dopamine neurotransmitter release in the basolateral amygdala (BLA), and pre-training YQA14 infusion in VTA lowered the increase of dopamine (DA) in BLA induced by shock during the training session or by context during the retrieval session. All above the results demonstrated that YQA14 attenuated the fear learning through the blockade of DRD3 in VTA decreasing the excitability of the projection to BLA. This study may provide new mechanisms and potential intervention targets for stress disorders with abnormal fear memory.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Genome-wide association study in a rat model of temperament identifies multiple loci for exploratory locomotion and anxiety-like traits

Common genetic factors likely contribute to multiple psychiatric diseases including mood and substance use disorders. Certain stable, heritable traits reflecting temperament, termed externalizing or internalizing, play a large role in modulating vulnerability to these disorders. To model these heritable tendencies, we selectively bred rats for high and low exploration in a novel environment [bred High Responders (bHR) vs. Low Responders (bLR)]. To identify genes underlying the response to selection, we phenotyped and genotyped 538 rats from an F₂ cross between bHR and bLR. Several behavioral traits show high heritability, including the selection trait: exploratory locomotion (EL) in a novel environment. There were significant phenotypic and genetic correlations between tests that capture facets of EL and anxiety. There were also correlations with Pavlovian conditioned approach (PavCA) behavior despite the lower heritability of that trait. Ten significant and conditionally independent loci for six behavioral traits were identified. Five of the six traits reflect different facets of EL that were captured by three behavioral tests. Distance traveled measures from the open field and the elevated plus maze map onto different loci, thus may represent different aspects of novelty-induced locomotor activity. The sixth behavioral trait, number of fecal boli, is the only anxiety-related trait mapping to a significant locus on chromosome 18 within which the Pik3c3 gene is located. There were no significant loci for PavCA. We identified a missense variant in the Plekhf1 gene on the chromosome 1:95 Mb QTL and Fancf and Gas2 as potential candidate genes that may drive the chromosome 1:107 Mb QTL for EL traits. The identification of a locomotor activity-related QTL on chromosome 7 encompassing the Pkhd1l1 and Trhr genes is consistent with our previous finding of these genes being differentially expressed in the hippocampus of bHR vs. bLR rats. The strong heritability coupled with identification of several loci associated with exploratory locomotion and emotionality provide compelling support for this selectively bred rat model in discovering relatively large effect causal variants tied to elements of internalizing and externalizing behaviors inherent to psychiatric and substance use disorders.

Mesolimbic dopamine adapts the rate of learning from action

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions^1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction⁴; however, so far there has been little consideration of how direct policy learning might inform our understanding⁵. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning⁶.

Sleep-mediated regulation of reward circuits: implications in substance use disorders

Our modern society suffers from both pervasive sleep loss and substance abuse-what may be the indications for sleep on substance use disorders (SUDs), and could sleep contribute to the individual variations in SUDs? Decades of research in sleep as well as in motivated behaviors have laid the foundation for us to begin to answer these questions. This review is intended to critically summarize the circuit, cellular, and molecular mechanisms by which sleep influences reward function, and to reveal critical challenges for future studies. The review also suggests that improving sleep quality may serve as complementary therapeutics for treating SUDs, and that formulating sleep metrics may be useful for predicting individual susceptibility to SUDs and other reward-associated psychiatric diseases.

Viral Tracing Confirms Paranigral Ventral Tegmental Area Dopaminergic Inputs to the Interpeduncular Nucleus Where Dopamine Release Encodes Motivated Exploration

Midbrain dopaminergic (DAergic) neurons of the ventral tegmental area (VTA) are engaged by rewarding stimuli and encode reward prediction error to update goal-directed learning. However, recent data indicate that VTA DAergic neurons are functionally heterogeneous with emerging roles in aversive signaling, salience, and novelty, based in part on anatomic location and projection, highlighting a need to functionally characterize the repertoire of VTA DAergic efferents in motivated behavior. Previous work identifying a mesointerpeduncular circuit consisting of VTA DAergic neurons projecting to the interpeduncular nucleus (IPN), a midbrain area implicated in aversion, anxiety-like behavior, and familiarity, has recently come into question. To verify the existence of this circuit, we combined presynaptic targeted and retrograde viral tracing in the dopamine transporter-Cre mouse line. Consistent with previous reports, synaptic tracing revealed that axon terminals from the VTA innervate the caudal IPN; whereas, retrograde tracing revealed DAergic VTA neurons, predominantly in the paranigral region, project to the nucleus accumbens shell, as well as the IPN. To test whether functional DAergic neurotransmission exists in the IPN, we expressed the genetically encoded DA sensor, dLight 1.2, in the IPN of C57BL/6J mice and measured IPN DA signals in vivo during social and anxiety-like behavior using fiber photometry. We observed an increase in IPN DA signal during social investigation of a novel but not familiar conspecific and during exploration of the anxiogenic open arms of the elevated plus maze. Together, these data confirm VTA DAergic neuron projections to the IPN and implicate this circuit in encoding motivated exploration.

Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues

The ability to learn Pavlovian associations from environmental cues predicting positive outcomes is critical for survival, motivating adaptive behaviours. This cued-motivated behaviour depends on the nucleus accumbens (NAc). NAc output activity mediated by spiny projecting neurons (SPNs) is regulated by dopamine, but also by cholinergic interneurons (CINs), which can release acetylcholine and glutamate via the activity of the vesicular acetylcholine transporter (VAChT) or the vesicular glutamate transporter (VGLUT3), respectively. Here we investigated behavioural and neurochemical changes in mice performing a touchscreen Pavlovian approach task by recording dopamine, acetylcholine, and calcium dynamics from D1- and D2-SPNs using fibre photometry in control, VAChT or VGLUT3 mutant mice to understand how these signals cooperate in the service of approach behaviours toward reward-predicting cues. We reveal that NAc acetylcholine-dopaminergic signalling is continuously updated to regulate striatal output underlying the acquisition of Pavlovian approach learning toward reward-predicting cues.

Future Newborns with Opioid-Induced Neonatal Abstinence Syndrome (NAS) Could Be Assessed with the Genetic Addiction Risk Severity (GARS) Test and Potentially Treated Using Precision Amino-Acid Enkephalinase Inhibition Therapy (KB220) as a Frontline Modality Instead of Potent Opioids

In this nonsystematic review and opinion, including articles primarily selected from PubMed, we examine the pharmacological and nonpharmacological treatments of neonatal abstinence syndrome (NAS) in order to craft a reasonable opinion to help forge a paradigm shift in the treatment and prevention of primarily opioid-induced NAS. Newborns of individuals who use illicit and licit substances during pregnancy are at risk for withdrawal, also known as NAS. In the US, the reported prevalence of NAS has increased from 4.0 per 1000 hospital births in 2010 to 7.3 per 1000 hospital births in 2017, which is an 82% increase. The management of NAS is varied and involves a combination of nonpharmacologic and pharmacologic therapy. The preferred first-line pharmacological treatment for NAS is opioid therapy, specifically morphine, and the goal is the short-term improvement in NAS symptomatology. Nonpharmacological therapies are individualized and typically focus on general care measures, the newborn-parent/caregiver relationship, the environment, and feeding. When used appropriately, nonpharmacologic therapies can help newborns with NAS avoid or reduce the amount of pharmacologic therapy required and the length of hospitalization. In addition, genetic polymorphisms of the catechol-o-methyltransferase (COMT) and mu-opioid receptor (OPRM1) genes appear to affect the length of stay and the need for pharmacotherapy in newborns with prenatal opioid exposure. Therefore, based on this extensive literature and additional research, this team of coauthors suggests that, in the future, in addition to the current nonpharmacological therapies, patients with opioid-induced NAS should undergo genetic assessment (i.e., the genetic addiction risk severity (GARS) test), which can subsequently be used to guide DNA-directed precision amino-acid enkephalinase inhibition (KB220) therapy as a frontline modality instead of potent opioids.

The Role of Intra-Amygdaloid Neurotensin and Dopamine Interaction in Spatial Learning and Memory

Neurotransmitter and neuromodulator neurotensin (NT) has been proved to facilitate spatial and passive avoidance learning after microinjected into the rat central nucleus of amygdala (CeA). These previous studies of our laboratory also revealed that neurotensin-1 receptor (NTS1) is involved in the mentioned actions of NT. Extensive literature confirms the interaction between neurotensinergic and dopaminergic systems, and our research group also suppose that the mesolimbic dopaminergic system (MLDS) is involved in the spatial learning and memory-facilitating effect of NT in the CeA. In the present work, NT and dopamine (DA) interaction has been examined in the Morris water maze and passive avoidance tests. Rats received 100 ng NT, 5 µg dopamine D2 receptor antagonist sulpiride in itself, sulpiride as a pretreatment before NT or vehicle solution into the CeA. NT microinjection significantly decreased target-finding latency in the Morris water maze test and significantly increased entrance latency in the passive avoidance test, as was expected based on our previous findings. The DA D2 receptor antagonist pretreatment was able to inhibit both effects of NT. The results confirm the facilitatory effect of NT on spatial learning and memory and let us conclude that these actions can be exerted via the DA D2 receptors.

The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors

The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.

Nucleus accumbens dopamine tracks aversive stimulus duration and prediction but not value or prediction error

There is active debate on the role of dopamine in processing aversive stimuli, where inferred roles range from no involvement at all, to signaling an aversive prediction error (APE). Here, we systematically investigate dopamine release in the nucleus accumbens core (NAC), which is closely linked to reward prediction errors, in rats exposed to white noise (WN, a versatile, underutilized, aversive stimulus) and its predictive cues. Both induced a negative dopamine ramp, followed by slow signal recovery upon stimulus cessation. In contrast to reward conditioning, this dopamine signal was unaffected by WN value, context valence, or probabilistic contingencies, and the WN dopamine response shifted only partially toward its predictive cue. However, unpredicted WN provoked slower post-stimulus signal recovery than predicted WN. Despite differing signal qualities, dopamine responses to simultaneous presentation of rewarding and aversive stimuli were additive. Together, our findings demonstrate that instead of an APE, NAC dopamine primarily tracks prediction and duration of aversive events.

Glutathione in the nucleus accumbens regulates motivation to exert reward-incentivized effort

Emerging evidence is implicating mitochondrial function and metabolism in the nucleus accumbens in motivated performance. However, the brain is vulnerable to excessive oxidative insults resulting from neurometabolic processes, and whether antioxidant levels in the nucleus accumbens contribute to motivated performance is not known. Here, we identify a critical role for glutathione (GSH), the most important endogenous antioxidant in the brain, in motivation. Using proton magnetic resonance spectroscopy at ultra-high field in both male humans and rodent populations, we establish that higher accumbal GSH levels are highly predictive of better, and particularly, steady performance over time in effort-related tasks. Causality was established in in vivo experiments in rats that, first, showed that downregulating GSH levels through micro-injections of the GSH synthesis inhibitor buthionine sulfoximine in the nucleus accumbens impaired effort-based reward-incentivized performance. In addition, systemic treatment with the GSH precursor N-acetyl-cysteine increased accumbal GSH levels in rats and led to improved performance, potentially mediated by a cell-type-specific shift in glutamatergic inputs to accumbal medium spiny neurons. Our data indicate a close association between accumbal GSH levels and an individual’s capacity to exert reward-incentivized effort over time. They also suggest that improvement of accumbal antioxidant function may be a feasible approach to boost motivation.

Prelimbic cortex neural encoding dynamically tracks expected outcome value

Animals must modify their behavior based on updated expected outcomes in a changing environment. Prelimbic cortex (PrL) neural encoding during learning predicts, and is necessary for, appropriately altering behavior based on a new expected outcome value following devaluation. We aimed to determine how PrL neural activity encodes reward predictive cues after the expected outcome value of those cues is decreased following conditioned taste aversion. In one post-devaluation session, rats were tested under extinction to determine their ability to alter their behavior to the expected outcome values (i.e., extinction test). In a second post-devaluation session, rats were tested with the newly devalued outcome delivered so that the rats experienced the updated outcome value within the session (i.e., re-exposure test). We found that PrL neural encoding of the cue associated with the devalued reward predicted the ability of rats to suppress behavior in the extinction test session, but not in the re-exposure test session. While all rats were able to successfully devalue the outcome during conditioned taste aversion, a subset of rats continued to consume the devalued outcome in the re-exposure test session. We found differential patterns of PrL neural encoding in the population of rats that did not avoid the devalued outcome during the re-exposure test compared to the rats that successfully avoided the devalued outcome. Our findings suggest that PrL neural encoding dynamically tracks expected outcome values, and differential neural encoding in the PrL to reward predictive cues following expected outcome value changes may contribute to distinct behavioral phenotypes.

The impact of caloric availability on eating behavior and ultra-processed food reward

The food environment has changed rapidly and dramatically in the last 50 years. While industrial food processing has increased the safety and stability of the food supply, a rapid expansion in the scope and scale of food processing in the 1980's has resulted in a market dominated by ultra-processed foods. Here, we use the NOVA definition of category 4 ultra-processed foods (UPFs) as they make up around 58% of total calories consumed in the US and 66% of calories in US children. UPFs are formulated from ingredients with no or infrequent culinary use, contain additives, and have a long shelf-life, spending long periods in contact with packaging materials, allowing for the absorption of compounds from those materials. The full implications of this dietary shift to UPFs on human health and disease outcomes are difficult, if not impossible, to quantify. However, UPF consumption is linked with various forms of cancer, increased cardiovascular disease, and increased all-cause mortality. Understanding food choice is, therefore, a critical problem in health research. Although many factors influence food choice, here we focus on the properties of the foods themselves. UPFs are generally treated as food, not as the highly refined, industrialized substances that they are, whose properties and components must be studied. Here, we examine one property of UPFs, that they deliver useable calories rapidly as a potential factor driving UPF overconsumption. First, we explore evidence that UPFs deliver calories more rapidly. Next, we examine the role of the gut-brain axis and its interplay with canonical reward systems, and last, we describe how speed affects both basic learning processes and drugs of abuse.

Electrophysiological responses to emotional and cocaine cues reveal individual neuroaffective profiles in cocaine users

Smokers with stronger neuroaffective responses to drug-related cues compared to nondrug-related pleasant images (C > P) are more vulnerable to compulsive smoking than individuals with the opposite brain reactivity profile (P > C). However, it is unknown if these neurobehavioral profiles exist in individuals abusing other drugs. We tested whether individuals with cocaine use disorder (CUD) show similar neuroaffective profiles to smokers. We also monitored eye movements to assess attentional bias toward cues and we further performed exploratory analyses on demographics, personality, and drug use between profiles. Participants with CUD (n = 43) viewed pleasant, unpleasant, cocaine, and neutral images while we recorded electroencephalogram. For each picture category, we computed the amplitude of the late positive potential (LPP), an event-related potential component that reflects motivational relevance. k-means clustering classified participants based on their LPP responses. In line with what has been observed in smokers, clustering participants using LPP responses revealed the presence of two groups: one with larger LPPs to pleasant images compared to cocaine images (P > C) and one group with larger LPPs to cocaine images compared to pleasant images (C > P). Individuals with the C > P reactivity profile also had higher attentional bias toward drug cues. The two groups did not differ on demographic and drug use characteristics, however individuals with the C > P profile reported lower distress tolerance, higher anhedonia, and higher posttraumatic stress symptoms compared to the P > C group. This is the first study to report the presence of these neuroaffective profiles in individuals with CUD, indicating that this pattern may cut across addiction populations. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Phosphorylation Signals Downstream of Dopamine Receptors in Emotional Behaviors: Association with Preference and Avoidance

Dopamine regulates emotional behaviors, including rewarding and aversive behaviors, through the mesolimbic dopaminergic pathway, which projects dopamine neurons from the ventral tegmental area to the nucleus accumbens (NAc). Protein phosphorylation is critical for intracellular signaling pathways and physiological functions, which are regulated by neurotransmitters in the brain. Previous studies have demonstrated that dopamine stimulated the phosphorylation of intracellular substrates, such as receptors, ion channels, and transcription factors, to regulate neuronal excitability and synaptic plasticity through dopamine receptors. We also established a novel database called KANPHOS that provides information on phosphorylation signals downstream of monoamines identified by our kinase substrate screening methods, including dopamine, in addition to those reported in the literature. Recent advances in proteomics techniques have enabled us to clarify the mechanisms through which dopamine controls rewarding and aversive behaviors through signal pathways in the NAc. In this review, we discuss the intracellular phosphorylation signals regulated by dopamine in these two emotional behaviors.

Critical periods when dopamine controls behavioral responding during Pavlovian learning

RATIONALE

Learning the association between rewards and predictive cues is critical for appetitive behavioral responding. The mesolimbic dopamine system is thought to play an integral role in establishing these cue-reward associations. The dopamine response to cues can signal differences in reward value, though this emerges only after significant training. This suggests that the dopamine system may differentially regulate behavioral responding depending on the phase of training.

OBJECTIVES

The purpose of this study was to determine whether antagonizing dopamine receptors elicited different effects on behavior depending on the phase of training or the type of Pavlovian task.

METHODS

Separate groups of male rats were trained on Pavlovian tasks in which distinct audio cues signaled either differences in reward size or differences in reward rate. The dopamine receptor antagonist flupenthixol was systemically administered prior to either the first ten sessions of training (acquisition phase) or the second ten sessions of training (expression phase), and we monitored the effect of these manipulations for an additional ten training sessions.

RESULTS

We identified acute effects of dopamine receptor antagonism on conditioned responding, the latency to respond, and post-reward head entries in both Pavlovian tasks. Interestingly, dopamine receptor antagonism during the expression phase produced persistent deficits in behavioral responding only in rats trained on the reward size Pavlovian task.

CONCLUSIONS

Together, our results illustrate that dopamine's control over behavior in Pavlovian tasks depends upon one's prior training experience and the information signaled by the cues.

The Arousal-motor Hypothesis of Dopamine Function: Evidence that Dopamine Facilitates Reward Seeking in Part by Maintaining Arousal

Dopamine facilitates approach to reward via its actions on dopamine receptors in the nucleus accumbens. For example, blocking either D1 or D2 dopamine receptors in the accumbens reduces the proportion of reward-predictive cues to which rats respond with cued approach. Recent evidence indicates that accumbens dopamine also promotes wakefulness and arousal, but the relationship between dopamine's roles in arousal and reward seeking remains unexplored. Here, we show that the ability of systemic or intra-accumbens injections of the D1 antagonist SCH23390 to reduce cued approach to reward depends on the animal's state of arousal. Handling the animal, a manipulation known to increase arousal, was sufficient to reverse the behavioral effects of the antagonist. In addition, SCH23390 reduced spontaneous locomotion and increased time spent in sleep postures, both consistent with reduced arousal, but also increased time spent immobile in postures inconsistent with sleep. In contrast, the ability of the D2 antagonist haloperidol to reduce cued approach was not reversible by handling. Haloperidol reduced spontaneous locomotion but did not increase sleep postures, instead increasing immobility in non-sleep postures. We place these results in the context of the extensive literature on dopamine's contributions to behavior, and propose the arousal-motor hypothesis. This novel synthesis, which proposes that two main functions of dopamine are to promote arousal and facilitate motor behavior, accounts both for our findings and many previous behavioral observations that have led to disparate and conflicting conclusions.

Activation of mesocorticolimbic dopamine projections initiates cue-induced reinstatement of reward seeking in mice

Drug addiction is characterized by relapse when addicts are re-exposed to drug-associated environmental cues, but the neural mechanisms underlying cue-induced relapse are unclear. In the present study we investigated the role of a specific dopaminergic (DA) pathway from ventral tegmental area (VTA) to nucleus accumbens core (NAcore) in mouse cue-induced relapse. Optical intracranial self-stimulation (oICSS) was established in DAT-Cre transgenic mice. We showed that optogenetic excitation of DA neurons in the VTA or their projection terminals in NAcore, NAshell or infralimbic prefrontal cortex (PFC-IL) was rewarding. Furthermore, activation of the VTA-NAcore pathway alone was sufficient and necessary to induce reinstatement of oICSS. In cocaine self-administration model, cocaine-associated cues activated VTA DA neurons as assessed by intracellular GCaMP signals. Cue-induced reinstatement of cocaine-seeking was triggered by optogenetic stimulation of the VTA-NAcore pathway, and inhibited by chemogenetic inhibition of this pathway. Together, these results demonstrate that cue-induced reinstatement of reward seeking is in part mediated by activation of the VTA-NAcore DA pathway.

Fast-Scanning Potential-Gated Organic Electrochemical Transistors for Highly Sensitive Sensing of Dopamine in Living Rat Brain

Developing techniques for the highly sensitive assay of neurotransmitters is essential for understanding physiological and pathological processes. Here, we demonstrate a fast-scanning potential (FSP)-gated organic electrochemical transistor (OECT): for the highly sensitive sensing of dopamine (DA) in a living rat brain. The configuration combines the selectivity of fast-scan cyclic voltammetry (FSCV) with the high sensitivity of an OECT. The combined use of FSP as a gating mode and transconductance (g_m ) as a sensing parameter further improve the sensing performance in terms of sensitivity, limit of detection, reproducibility, and stability. The FSP-OECT exhibits a sensitivity of 0.899 S M^-1 and a low limit of detection down to 5 nM and was validated for in vivo monitoring of the basal level and electrically stimulated release of DA.

A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning

A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.

Mesoaccumbal Dopamine Heterogeneity: What Do Dopamine Firing and Release Have to Do with It?

Ventral tegmental area (VTA) dopamine (DA) neurons are often thought to uniformly encode reward prediction errors. Conversely, DA release in the nucleus accumbens (NAc), the prominent projection target of these neurons, has been implicated in reinforcement learning, motivation, aversion, and incentive salience. This contrast between heterogeneous functions of DA release versus a homogeneous role for DA neuron activity raises numerous questions regarding how VTA DA activity translates into NAc DA release. Further complicating this issue is increasing evidence that distinct VTA DA projections into defined NAc subregions mediate diverse behavioral functions. Here, we evaluate evidence for heterogeneity within the mesoaccumbal DA system and argue that frameworks of DA function must incorporate the precise topographic organization of VTA DA neurons to clarify their contribution to health and disease.

Cortical dopamine reduces the impact of motivational biases governing automated behaviour

Motivations shape our behaviour: the promise of reward invigorates, while in the face of punishment, we hold back. Abnormalities of motivational processing are implicated in clinical disorders characterised by excessive habits and loss of top-down control, notably substance and behavioural addictions. Striatal and frontal dopamine have been hypothesised to play complementary roles in the respective generation and control of these motivational biases. However, while dopaminergic interventions have indeed been found to modulate motivational biases, these previous pharmacological studies used regionally non-selective pharmacological agents. Here, we tested the hypothesis that frontal dopamine controls the balance between Pavlovian, bias-driven automated responding and instrumentally learned action values. Specifically, we examined whether selective enhancement of cortical dopamine either (i) enables adaptive suppression of Pavlovian control when biases are maladaptive; or (ii) non-specifically modulates the degree of bias-driven automated responding. Healthy individuals (n = 35) received the catechol-o-methyltransferase (COMT) inhibitor tolcapone in a randomised, double-blind, placebo-controlled cross-over design, and completed a motivational Go NoGo task known to elicit motivational biases. In support of hypothesis (ii), tolcapone globally decreased motivational bias. Specifically, tolcapone improved performance on trials where the bias was unhelpful, but impaired performance in bias-congruent conditions. These results indicate a non-selective role for cortical dopamine in the regulation of motivational processes underpinning top-down control over automated behaviour. The findings have direct relevance to understanding neurobiological mechanisms underpinning addiction and obsessive-compulsive disorders, as well as highlighting a potential trans-diagnostic novel mechanism to address such symptoms.