Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry

Effects of psychedelic, DOI, on nucleus accumbens dopamine signaling to predictable rewards and cues in rats

Psychedelics produce lasting therapeutic responses in neuropsychiatric diseases suggesting they may disrupt entrenched associations and catalyze learning. Here, we examine psychedelic 5-HT2A/2C agonist, DOI, effects on dopamine signaling in the nucleus accumbens (NAc) core, a region extensively linked to reward learning, motivation, and drug-seeking. We measure phasic dopamine transients following acute DOI administration in rats during well learned Pavlovian tasks in which sequential cues predict rewards. We find that DOI (0.0-1.2 mg/kg, i.p.) increases dopamine signals, photometrically measured using GRABDA optical sensor, to rewards and proximal reward cues, but not to the distal cues that predict these events. We determine that the elevated dopamine produced by DOI to reward cues occurs independently of DOI-induced changes in reward value. The increased dopamine associated with predictable reward cues and rewards supports DOI-induced increases in prediction error signaling. These findings lay a foundation for developing psychedelic strategies aimed at engaging error-driven learning mechanisms to disrupt entrenched associations or produce new associations.

Role of dopamine in reward expectation and predictability during execution of action sequences

Reward-associated cues serve different functions depending on whether they precede or terminate action sequences. Cues that precede action sequences and signal opportunity for reward could serve as GO signals to initiate the sequence, whereas sequence termination cues could serve as response feedback by signaling reward delivery. Reward expectation during sequence execution depends on these cues and might condition whether behavior is habitual or goal-directed. However, it remains unknown how sequence initiation and termination cues differentially affect reward expectation and contribute to habit learning. Further, while mesolimbic dopamine plays a key role in cue-induced reward expectation and sequence learning, how dynamic changes in dopamine signals differ depending on the response strategy is unclear. Here, we determined how mesolimbic DA signals change over training during cue-mediated sequence learning, depending on the type of cue and the nature of behavioral control. We found sequence initiation and termination cues differentially affect reward expectation during action sequences, with the termination cue contributing to habit and automaticity. Distinct response strategies induced by sequence initiation and termination cues induced differential changes in mesolimbic DA signals that captured variations in reward expectation along sequence execution. Notably, habit-like behavior induced by the sequence termination cue was associated with a rapid shift in DA signals from reward retrieval to the cue. This habit-like behavior was reflected in behavioral inflexibility and attenuated DA reward prediction error signals. Finally, using optogenetics, we provide evidence that phasic DA activity elicited by the sequence termination cue is critical for the development of habit-like behavior.

Reward-driven cerebellar climbing fiber activity influences both neural and behavioral learning

The cerebellum plays a key role in motor coordination and learning. In contrast with classical supervised learning models, recent work has revealed that CFs can signal reward-predictive information in some behaviors. This raises the question of whether CFs may also operate according to principles similar to those described by reinforcement learning models. To test how CFs operate during reward-guided behavior, and evaluate the role of reward-related CF activity in learning, we have measured CF responses in Purkinje cells of the lateral cerebellum during a Pavlovian task using 2-photon calcium imaging. Specifically, we have performed multi-stimulus experiments to determine whether CF activity meets the requirements of a reward prediction error (rPE) signal for transfer from an unexpected reward to a reward-predictive cue. We find that once CF activity is transferred to a conditioned stimulus, and there is no longer a response to reward, CFs cannot generate learned responses to a second conditioned stimulus that carries the same reward prediction. In addition, by expressing the inhibitory opsin GtACR2 in neurons of the inferior olive, and optically inhibiting these neurons across behavioral training at the time of unexpected reward, we find that the transfer of CF signals to the conditioned stimulus is impaired. Moreover, this optogenetic inhibition also impairs learning, resulting in a deficit in anticipatory lick timing. Together, these results indicate that CF signals can exhibit several characteristics in common with rPEs during reinforcement learning, and that the cerebellum can harness these learning signals to generate accurately timed motor behavior.

Cerebellar climbing fibers signal flexible, rapidly adapting reward predictions

Classical models of cerebellar computation posit that climbing fibers (CFs) operate according to supervised learning rules, correcting movements by signaling the occurrence of motor errors. However, recent findings suggest that in some behaviors, CF activity can exhibit features that resemble the instructional signals necessary for reinforcement learning, namely reward prediction errors (rPEs). Despite these initial observations, many key properties of reward-related CF responses remain unclear, thus limiting our understanding of how they operate to guide cerebellar learning. Here, we have measured the postsynaptic responses of CFs onto cerebellar Purkinje cells using two-photon calcium imaging to test how they respond to learned stimuli that either do or do not predict reward. We find that CFs can develop generalized responses to similar cues of the same modality, regardless of whether they are reward predictive. However, this generalization depends on temporal context, and does not extend across sensory modalities. Further, learned CF responses are flexible, and can be rapidly updated according to new reward contingencies. Together these results suggest that CFs can generate learned, reward-predictive responses that flexibly adapt to the current environment in a context-sensitive manner.

Dopamine transmission at D1 and D2 receptors in the nucleus accumbens contributes to the expression of incubation of cocaine craving

Relapse represents a consistent clinical problem for individuals with substance use disorder. In the incubation of craving model of persistent craving and relapse, cue-induced drug seeking progressively intensifies or "incubates" during the first weeks of abstinence from drug self-administration and then remains high for months. Previously, we and others have demonstrated that expression of incubated cocaine craving requires strengthening of excitatory synaptic transmission in the nucleus accumbens core (NAcc). However, despite the importance of dopaminergic signaling in the NAcc for motivated behavior, little is known about the role that dopamine (DA) plays in the incubation of cocaine craving. Here we used fiber photometry to measure DA transients in the NAcc of male and female rats during cue-induced seeking tests conducted in early abstinence from cocaine self-administration, prior to incubation, and late abstinence, after incubation of craving has plateaued. We observed DA transients time-locked to cue-induced responding but their magnitude did not differ significantly when measured during early versus late abstinence seeking tests. Next, we tested for a functional role of these DA transients by injecting DA receptor antagonists into the NAcc just before the cue-induced seeking test. Blockade of either D1 or D2 DA receptors reduced cue-induced cocaine seeking after but not before incubation. We found no main effect of sex or significant interaction of sex with other factors in our experiments. These results suggest that DA contributes to incubated cocaine seeking but the emergence of this role reflects changes in postsynaptic responsiveness to DA rather than presynaptic alterations.

Ventral tegmental area dopamine neural activity switches simultaneously with rule representations in the prefrontal cortex and hippocampus

Multiple brain regions need to coordinate activity to support cognitive flexibility and behavioral adaptation. Neural activity in both the hippocampus (HPC) and prefrontal cortex (PFC) is known to represent spatial context and is sensitive to reward and rule alterations. Midbrain dopamine (DA) activity is key in reward seeking behavior and learning. There is abundant evidence that midbrain DA modulates HPC and PFC activity. However, it remains underexplored how these networks engage dynamically and coordinate temporally when animals must adjust their behavior according to changing reward contingencies. In particular, is there any relationship between DA reward prediction change during rule switching, and rule representation changes in PFC and CA1? We addressed these questions using simultaneous recording of neuronal population activity from the hippocampal area CA1, PFC and ventral tegmental area (VTA) in male TH-Cre rats performing two spatial working memory tasks with frequent rule switches in blocks of trials. CA1 and PFC ensembles showed rule-specific activity both during maze running and at reward locations, with PFC rule coding more consistent across animals compared to CA1. Optogenetically tagged VTA DA neuron firing activity responded to and predicted reward outcome. We found that the correct prediction in DA emerged gradually over trials after rule-switching in coordination with transitions in PFC and CA1 ensemble representations of the current rule after a rule switch, followed by behavioral adaptation to the correct rule sequence. Therefore, our study demonstrates a crucial temporal coordination between the rule representation in PFC/CA1, the dopamine reward signal and behavioral strategy.

The secondary visual cortex mediated the enhancement of associative learning on methamphetamine self-administration behaviors

RATIONALE

Methamphetamine addiction is a persistent and intractable pathological learning and memory, whereas no approved therapeutics is available. However, few attentions have been paid to how associative learning participates in the formation of intractable memory related to drug addiction OBJECTIVES AND METHODS: To investigate the role of associative learning in methamphetamine addiction and the underlying neurobiological mechanism, methamphetamine self-administration, oral sucrose self-administration, chemogenetic neuromanipulation, and fiber photometry in mice were performed in this study.

RESULTS

We reported that associative learning increased methamphetamine-induced self-administration, but not oral sucrose self-administration. In addition, the enhancement of methamphetamine-induced self-administration was independent of more methamphetamine consumption, and remained with higher drug-taking and motivation in the absence of visual cues, suggesting the direct effects of the associative learning that enhanced methamphetamine-induced self-administration. Moreover, chemogenetic inactivation of the secondary visual cortex (V2) reduced the enhancement of the drug-taking induced by associative learning but did not alter sucrose-taking. Further fiber photometry of V2 neurons demonstrated that methamphetamine-associative learning elicits V2 neuron excitation, and sucrose-associative learning elicits V2 neuron inhibition.

CONCLUSIONS

Therefore, this study reveals the neurobiological mechanism of V2 excitability underlying how associative learning participates in the formation of intractable memory related to drug addiction, and gives evidence to support V2 as a promising target for stimulation therapy for methamphetamine addiction.

Forward genetic screen of the C. elegans million mutation library reveals essential, cell-autonomous contributions of BBSome proteins to dopamine signaling

The nematode Caenorhabditis elegans is well known for its ability to support forward genetic screens to identify molecules involved in neuronal viability and signaling. The proteins involved in C. elegans dopamine (DA) regulation are highly conserved across evolution, with prior work demonstrating that the model can serve as an efficient platform to identify novel genes involved in disease-associated processes. To identify novel players in DA signaling, we took advantage of a recently developed library of pre-sequenced mutant nematodes arising from the million mutation project (MMP) to identify strains that display the DA-dependent swimming-induced-paralysis phenotype (Swip). Our screen identified novel mutations in the dopamine transporter encoding gene dat-1, whose loss was previously used to identify the Swip phenotype, as well as multiple genes with previously unknown connections to DA signaling. Here, we present our isolation and characterization of one of these genes, bbs-1, previously linked to the function of primary cilia in worms and higher organisms, including humans, and where loss-of-function mutations result in a human disorder known as Bardet-Biedl syndrome. Our studies of C. elegans BBS-1 protein, as well as other proteins that are known to be assembled into a higher order complex (the BBSome) reveal that functional or structural disruption of this complex leads to exaggerated C. elegans DA signaling to produce Swip via a cell-autonomous mechanism. We provide evidence that not only does the proper function of cilia in C. elegans DA neurons support normal swimming behavior, but also that bbs-1 maintains normal levels of DAT-1 trafficking or function via a RHO-1 and SWIP-13/MAPK-15 dependent pathway where mutants may contribute to Swip independent of altered ciliary function. Together, these studies demonstrate novel contributors to DA neuron function in the worm and demonstrate the utility and efficiency of forward genetic screens using the MMP library.

Frontal Norepinephrine Represents a Threat Prediction Error Under Uncertainty

BACKGROUND

To adapt to threats in the environment, animals must predict them and engage in defensive behavior. While the representation of a prediction error signal for reward has been linked to dopamine, a neuromodulatory prediction error for aversive learning has not been identified.

METHODS

We measured and manipulated norepinephrine release during threat learning using optogenetics and a novel fluorescent norepinephrine sensor.

RESULTS

We found that norepinephrine response to conditioned stimuli reflects aversive memory strength. When delays between auditory stimuli and footshock are introduced, norepinephrine acts as a prediction error signal. However, temporal difference prediction errors do not fully explain norepinephrine dynamics. To explain noradrenergic signaling, we used an updated reinforcement learning model with uncertainty about time and found that it explained norepinephrine dynamics across learning and variations in temporal and auditory task structure.

CONCLUSIONS

Norepinephrine thus combines cognitive and affective information into a predictive signal and links time with the anticipation of danger.

A biophysical model for dopamine modulating working memory through reward system in obsessive-compulsive disorder

Dopamine modulates working memory in the prefrontal cortex (PFC) and is crucial for obsessive-compulsive disorder (OCD). However, the mechanism is unclear. Here we establish a biophysical model of the effect of dopamine (DA) in PFC to explain the mechanism of how high dopamine concentrations induce persistent neuronal activities with the network plunging into a deep, stable attractor state. The state develops a defect in working memory and tends to obsession and compulsion. Weakening the reuptake of dopamine acts on synaptic plasticity according to Hebbian learning rules and reward learning, which in turn affects the strength of neuronal synaptic connections, resulting in the tendency of compulsion and learned obsession. In addition, we elucidate the potential mechanisms of dopamine antagonists in OCD, indicating that dopaminergic drugs might be available for treatment, even if the abnormality is a consequence of glutamate hypermetabolism rather than dopamine. The theory highlights the significance of early intervention and behavioural therapies for obsessive-compulsive disorder. It potentially offers new approaches to dopaminergic pharmacotherapy and psychotherapy for OCD patients.

Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states

Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making.

Highlights

Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders.

eTOC

Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Nucleus accumbens sub-regions experience distinct dopamine release responses following acute and chronic morphine exposure

It is well established that dopamine neurons of the ventral tegmental area (VTA) play a critical role in reward and aversion as well as pathologies including drug dependence and addiction. The distinct effects of acute and chronic opioid exposure have been previously characterized at VTA synapses. Recent work suggests that distinct VTA projections that target the medial and lateral shell of the nucleus accumbens (NAc), may play opposing roles in modulating behavior. It is possible that these two anatomically and functionally distinct pathways also have disparate roles in opioid reward, tolerance, and withdrawal in the brain. In this study we monitored dopamine release in the medial or lateral shell of the NAc of male mice during a week-long morphine treatment paradigm. We measured dopamine release in response to an intravenous morphine injection both acutely and following a week of repeated morphine. We also measured dopamine in response to a naloxone injection both prior to and following repeated morphine treatment. Morphine induced a transient increase in dopamine in the medial NAc shell that was much larger than the slower rise observed in the lateral shell. Surprisingly, chronic morphine treatment induced a sensitization of the medial dopamine response to morphine that opposed a diminished response observed in the saline-treated control group. This study expands on our current understanding of the medial NAc shell as hub of opioid-induced dopamine fluctuation. It also highlights the need for future opioid studies to appreciate the heterogeneity of dopamine neurons.

Significance Statement

The social and economic consequences of the opioid epidemic are tragic and far-reaching. Yet, opioids are indisputably necessary in clinical settings where they remain the most useful treatment for severe pain. To combat this crisis, we must improve our understanding of opioid function in the brain, particularly the neural mechanisms that underlie opioid dependence and addictive behaviors. This study uses fiber photometry to examine dopamine changes that occur in response to repeated morphine, and morphine withdrawal, at multiple stages of a longitudinal opioid-dependence paradigm. We reveal key differences in how dopamine levels respond to opioid administration in distinct sub-regions of the ventral striatum and lay a foundation for future opioid research that appreciates our contemporary understanding of the dopamine system.

Dopamine transmission at D1 and D2 receptors in the nucleus accumbens contributes to the expression of incubation of cocaine craving

Relapse represents a consistent clinical problem for individuals with substance use disorder. In the incubation of craving model of persistent craving and relapse, cue-induced drug seeking progressively intensifies or 'incubates' during the first weeks of abstinence from drug self-administration and then remains high for months. Previously, we and others have demonstrated that expression of incubated cocaine craving requires strengthening of excitatory synaptic transmission in the nucleus accumbens core (NAcc). However, despite the importance of dopaminergic signaling in the NAcc for motivated behavior, little is known about the role that dopamine (DA) plays in the incubation of cocaine craving. Here we used fiber photometry to measure DA transients in the NAcc of male and female rats during cue-induced seeking tests conducted in early abstinence from cocaine self-administration, prior to incubation, and late abstinence, after incubation of craving has plateaued. We observed DA transients time-locked to cue-induced responding but their magnitude did not differ significantly when measured during early versus late abstinence seeking tests. Next, we tested for a functional role of these DA transients by injecting DA receptor antagonists into the NAcc just before the cue-induced seeking test. Blockade of either D1 or D2 DA receptors reduced cue-induced cocaine seeking after but not before incubation. We found no main effect of sex in our experiments. These results suggest that DA contributes to incubated cocaine seeking but the emergence of this role reflects changes in postsynaptic responsiveness to DA rather than presynaptic alterations.

Ventral tegmental area interneurons revisited: GABA and glutamate projection neurons make local synapses

The ventral tegmental area (VTA) contains projection neurons that release the neurotransmitters dopamine, GABA, and/or glutamate from distal synapses. VTA also contains GABA neurons that synapse locally on to VTA dopamine neurons, synapses widely credited to a population of so-called VTA interneurons. Interneurons in cortex, striatum, and elsewhere have well-defined morphological features, physiological properties, and molecular markers, but such features have not been clearly described in VTA. Indeed, there is scant evidence that local and distal synapses originate from separate populations of VTA GABA neurons. In this study we tested whether several markers expressed in non-dopamine VTA neurons are selective markers of interneurons, defined as neurons that synapse locally but not distally. Challenging previous assumptions, we found that VTA neurons genetically defined by expression of parvalbumin, somatostatin, neurotensin, or mu-opioid receptor project to known VTA targets including nucleus accumbens, ventral pallidum, lateral habenula, and prefrontal cortex. Moreover, we provide evidence that VTA GABA and glutamate projection neurons make functional inhibitory or excitatory synapses locally within VTA. These findings suggest that local collaterals of VTA projection neurons could mediate functions prior attributed to VTA interneurons. This study underscores the need for a refined understanding of VTA connectivity to explain how heterogeneous VTA circuits mediate diverse functions related to reward, motivation, or addiction.

Generalized cue reactivity in dopamine neurons after opioids

Cue reactivity is the maladaptive neurobiological and behavioral response upon exposure to drug cues and is a major driver of relapse. The leading hypothesis is that dopamine release by addictive drugs represents a persistently positive reward prediction error that causes runaway enhancement of dopamine responses to drug cues, leading to their pathological overvaluation compared to non-drug reward alternatives. However, this hypothesis has not been directly tested. Here we developed Pavlovian and operant procedures to measure firing responses, within the same dopamine neurons, to drug versus natural reward cues, which we found to be similarly enhanced compared to cues predicting natural rewards in drug-naïve controls. This enhancement was associated with increased behavioral reactivity to the drug cue, suggesting that dopamine release is still critical to cue reactivity, albeit not as previously hypothesized. These results challenge the prevailing hypothesis of cue reactivity, warranting new models of dopaminergic function in drug addiction, and provide critical insights into the neurobiology of cue reactivity with potential implications for relapse prevention.

Assessing midbrain neuromelanin and its relationship to reward learning in anorexia nervosa: Stage 1 of a registered report

INTRODUCTION

Anorexia nervosa (AN) is a debilitating and potentially chronic eating disorder, characterized by low hedonic drive toward food, which has been linked with perturbations in both reward processing and dopaminergic activity. Neuromelanin-sensitive magnetic resonance imaging (MRI) is an emerging method to index midbrain neuromelanin-a by-product of dopaminergic synthesis. The assessment of midbrain neuromelanin, and its association with AN psychopathology and reward-related processes, may provide critical insights into reward circuit function in AN.

METHODS

This study will incorporate neuromelanin-sensitive MRI into an existing study of appetitive conditioning in those with AN. Specifically, those with acute and underweight AN (N = 30), those with weight-restored AN (N = 30), and age-matched healthy controls (N = 30) will undergo clinical assessment of current and previous psychopathology, in addition to structural neuromelanin-sensitive MRI, diffusion MRI, and functional MRI (fMRI) during appetitive conditioning.

CONCLUSION

This study will be among the first to interrogate midbrain neuromelanin in AN-a disorder characterized by altered dopaminergic activity. Results will help establish whether abnormalities in the midbrain synthesis of dopamine are evident in those with AN and are associated with symptomatic behavior and reduced ability to experience pleasure and reward.

Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types

The ventral pallidum (VP) contains GABA and glutamate neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the mechanisms by which VP cell types shape VTA activity and drive behavior. Here, we found that both VP GABA and glutamate neurons were activated during approach to reward or by delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine and glutamate neurons. Remarkably, stimulation-evoked activation was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP glutamate neurons activated VTA GABA, as well as dopamine and glutamate neurons, despite driving aversion. However, VP glutamate neurons evoked dopamine in aversion-associated ventromedial nucleus accumbens (NAc), but reduced dopamine release in reward-associated dorsomedial NAc. These findings show how heterogeneous VP projections to VTA can be engaged to shape approach and avoidance behaviors.

Stimulation of VTA dopamine inputs to LH upregulates orexin neuronal activity in a DRD2-dependent manner

Dopamine and orexins (hypocretins) play important roles in regulating reward-seeking behaviors. It is known that hypothalamic orexinergic neurons project to dopamine neurons in the ventral tegmental area (VTA), where they can stimulate dopaminergic neuronal activity. Although there are reciprocal connections between dopaminergic and orexinergic systems, whether and how dopamine regulates the activity of orexin neurons is currently not known. Here we implemented an opto-Pavlovian task in which mice learn to associate a sensory cue with optogenetic dopamine neuron stimulation to investigate the relationship between dopamine release and orexin neuron activity in the lateral hypothalamus (LH). We found that dopamine release can be evoked in LH upon optogenetic stimulation of VTA dopamine neurons and is also naturally evoked by cue presentation after opto-Pavlovian learning. Furthermore, orexin neuron activity could also be upregulated by local stimulation of dopaminergic terminals in the LH in a way that is partially dependent on dopamine D2 receptors (DRD2). Our results reveal previously unknown orexinergic coding of reward expectation and unveil an orexin-regulatory axis mediated by local dopamine inputs in the LH.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.

Glutamate inputs send prediction error of reward, but not negative value of aversive stimuli, to dopamine neurons

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation. Notably, whereas glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli into more positive responses, whereas excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

Medial prefrontal dopamine dynamics reflect allocation of selective attention

The mesocortical dopamine system is comprised of midbrain dopamine neurons that predominantly innervate the medial prefrontal cortex (mPFC) and exert a powerful neuromodulatory influence over this region 1,2 . mPFC dopamine activity is thought to be critical for fundamental neurobiological processes including valence coding and decision-making 3,4 . Despite enduring interest in this pathway, the stimuli and conditions that engage mPFC dopamine release have remained enigmatic due to inherent limitations in conventional methods for dopamine monitoring which have prevented real-time in vivo observation 5 . Here, using a fluorescent dopamine sensor enabling time-resolved recordings of cortical dopamine activity in freely behaving mice, we reveal the coding properties of this system and demonstrate that mPFC dopamine dynamics conform to a selective attention signal. Contrary to the long-standing theory that mPFC dopamine release preferentially encodes aversive and stressful events 6-8 , we observed robust dopamine responses to both appetitive and aversive stimuli which dissipated with increasing familiarity irrespective of stimulus intensity. We found that mPFC dopamine does not evolve as a function of learning but displays striking temporal precedence with second-to-second changes in behavioral engagement, suggesting a role in allocation of attentional resources. Systematic manipulation of attentional demand revealed that quieting of mPFC dopamine signals the allocation of attentional resources towards an expected event which, upon detection triggers a sharp dopamine transient marking the transition from decision-making to action. The proposed role of mPFC dopamine as a selective attention signal is the first model based on direct observation of time-resolved dopamine dynamics and reconciles decades of competing theories.

D1 and D2 medium spiny neurons in the nucleus accumbens core have distinct and valence-independent roles in learning

At the core of value-based learning is the nucleus accumbens (NAc). D1- and D2-receptor-containing medium spiny neurons (MSNs) in the NAc core are hypothesized to have opposing valence-based roles in behavior. Using optical imaging and manipulation approaches in mice, we show that neither D1 nor D2 MSNs signal valence. D1 MSN responses were evoked by stimuli regardless of valence or contingency. D2 MSNs were evoked by both cues and outcomes, were dynamically changed with learning, and tracked valence-free prediction error at the population and individual neuron level. Finally, D2 MSN responses to cues were necessary for associative learning. Thus, D1 and D2 MSNs work in tandem, rather than in opposition, by signaling specific properties of stimuli to control learning.

Ventral hippocampal cholecystokinin interneurons gate contextual reward memory

Associating contexts with rewards depends on hippocampal circuits, with local inhibitory interneurons positioned to play an important role in shaping activity. Here, we demonstrate that the encoding of context-reward memory requires a ventral hippocampus (vHPC) to nucleus accumbens (NAc) circuit that is gated by cholecystokinin (CCK) interneurons. In a sucrose conditioned place preference (CPP) task, optogenetically inhibiting vHPC-NAc terminals impaired the acquisition of place preference. Transsynaptic rabies tracing revealed vHPC-NAc neurons were monosynaptically innervated by CCK interneurons. Using intersectional genetic targeting of CCK interneurons, ex vivo optogenetic activation of CCK interneurons increased GABAergic transmission onto vHPC-NAc neurons, while in vivo optogenetic inhibition of CCK interneurons increased cFos in these projection neurons. Notably, CCK interneuron inhibition during sucrose CPP learning increased time spent in the sucrose-associated location, suggesting enhanced place-reward memory. Our findings reveal a previously unknown hippocampal microcircuit crucial for modulating the strength of contextual reward learning.

Prediction error in dopamine neurons during associative learning

Dopamine neurons have long been thought to facilitate learning by broadcasting reward prediction error (RPE), a teaching signal used in machine learning, but more recent work has advanced alternative models of dopamine's computational role. Here, I revisit this critical issue and review new experimental evidences that tighten the link between dopamine activity and RPE. First, I introduce the recent observation of a gradual backward shift of dopamine activity that had eluded researchers for over a decade. I also discuss several other findings, such as dopamine ramping, that were initially interpreted to conflict but later found to be consistent with RPE. These findings improve our understanding of neural computation in dopamine neurons.

Transcriptional Dysregulation of Cholesterol Synthesis Underlies Hyposensitivity to GABA in the Ventral Tegmental Area During Acute Alcohol Withdrawal

BACKGROUND

The ventral tegmental area (VTA) is a dopaminergic brain area that is critical in the development and maintenance of addiction. During withdrawal from chronic ethanol exposure, the response of VTA neurons to GABA (gamma-aminobutyric acid) is reduced through an epigenetically regulated mechanism. In the current study, a whole-genome transcriptomic approach was used to investigate the underlying molecular mechanism of GABA hyposensitivity in the VTA during withdrawal after chronic ethanol exposure.

METHODS

We performed RNA sequencing of the VTA of Sprague Dawley male rats withdrawn for 24 hours from a chronic ethanol diet as well as sequencing of the VTA of control rats fed the Lieber-DeCarli diet. RNA sequencing data were analyzed using weighted gene coexpression network analysis to identify modules that contained coexpressed genes. Validation was performed with quantitative polymerase chain reaction, gas chromatography-mass spectrometry, and electrophysiological assays.

RESULTS

Pathway and network analysis of weighted gene coexpression network analysis module 1 revealed a significant downregulation of genes associated with the cholesterol synthesis pathway. Consistent with this association, VTA cholesterol levels were significantly decreased during withdrawal. Chromatin immunoprecipitation indicated a decrease in levels of acetylated H3K27 at the transcriptional control regions of these genes. Electrophysiological studies in VTA slices demonstrated that GABA hyposensitivity during withdrawal was normalized by addition of exogenous cholesterol. In addition, inhibition of cholesterol synthesis produced GABA hyposensitivity, which was reversed by adding exogenous cholesterol to VTA slices.

CONCLUSIONS

These results suggest that decreased expression of cholesterol synthesis genes may regulate GABA hyposensitivity of VTA neurons during alcohol withdrawal. Increasing cholesterol levels in the brain may be a novel avenue for therapeutic intervention to reverse detrimental effects of chronic alcohol exposure.

Altered neural activity in the reward-related circuit associated with anhedonia in mild to moderate Major Depressive Disorder

BACKGROUND

Anhedonia is a significant predictor of disease progression and treatment outcomes in Major Depressive Disorder (MDD), linked to reward network dysfunctions. However, understanding of its underlying neural mechanisms remains limited. This study aimed to investigate the brain functional mechanisms underlying MDD with anhedonia using resting-state functional magnetic resonance imaging (rs-fMRI).

METHODS

The Snaith-Hamilton Pleasure Scale (SHAPS) was used to evaluation MDD with anhedonia (anMDD) and non-anhedonia MDD (non-anMDD). Forty-eight patients with anMDD, Forty-four patients with non-anMDD, and Fifty healthy controls (HCs) were enrolled for the fMRI scans. A seed-based functional connectivity (FC) method was employed to explore reward network abnormalities.

RESULTS

anMDD patients exhibited lower FC values in Ventral Striatum (VS), right lateral Ventral Tegmental Area (VTA_R), left Thalamus (THA_L), and higher FC values in Ventromedial Prefrontal Cortex (vmPFC), left Anterior Insula (AI_L), and Presupplementary Motor Area (Pre-SMA) compared to HCs. Comparing anMDD to non-anMDD, significant differences were observed in FC values of VS, vmPFC, Pre-SMA, and THA_L regions. Correlation analysis revealed positive correlations between FC values of VS_R and NAc_R, as well as THA_L and Cerebellum_Crus1_L, with SHAPS scores. Negative correlations were observed between FC values of Pre-SMA and the right caudate, and between vmPFC and Frontal_Mid_Orb_L, and SHAPS scores.

CONCLUSION

Both anMDD and non-anMDD groups demonstrated abnormal FCs in the reward network. These findings indicate distinct roles of reward-related circuits in the two subtypes, contributing to a refined understanding of depression phenotypes and potential directions for targeted interventions.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Reduced neural encoding of utility prediction errors in cocaine addiction

Influential accounts of addiction posit alterations in adaptive behavior driven by deficient dopaminergic prediction errors (PEs), signaling the discrepancy between actual and expected reward. Dopamine neurons encode these error signals in subjective terms, calibrated by individual risk preferences, as "utility" PEs. It remains unclear, however, whether people with drug addiction have PE deficits or their computational source. Here, using an analogous task to prior single-unit studies with known expectancies, we show that fMRI-measured PEs similarly reflect utility PEs. Relative to control participants, people with chronic cocaine addiction demonstrate reduced utility PEs in the dopaminoceptive ventral striatum, with similar trends in orbitofrontal cortex. Dissecting this PE signal into its subcomponent terms attributed these reductions to weaker striatal responses to received reward/utility, whereas suppression of activity with reward expectation was unchanged. These findings support that addiction may fundamentally disrupt PE signaling and reveal an underappreciated role for perceived reward value in this mechanism.

Quantitative and qualitative sex difference in habenula-induced inhibition of midbrain dopamine neurons in the rat

Introduction

Clinically relevant sex differences have been noted in a number of affective, behavioral, cognitive, and neurological health disorders. Midbrain dopamine neurons are implicated in several of these same disorders and consequently are under investigation for their potential role in the manifestation of these sex differences. The lateral habenula exerts significant inhibitory control over dopamine neuronal firing, yet little is known about sex differences in this particular neurocircuit.

Methods

We performed in vivo, single unit, extracellular recordings of dopamine neurons in female and male anesthetized rats in response to single pulse stimulation of the lateral habenula. In addition, we assessed baseline firing properties of lateral habenula neurons and, by immunochemical means, assessed the distribution of estrogen receptor alpha cells in the lateral habenula.

Results

Habenula-induced inhibition of dopamine neuronal firing is reduced in female rats relative to male rats. In addition, male rats had a higher prevalence of rebound excitation. Furthermore, the firing pattern of lateral habenula neurons was less variable in female rats, and female rats had a higher density of estrogen receptor alpha positive cells in the lateral habenula.

Discussion

We found that the dopamine neuronal response to habenular stimulation is both qualitatively and quantitatively different in female and male rats. These novel findings together with reports in the contemporary literature lead us to posit that the sex difference in dopamine inhibition seen here relate to differential firing properties of lateral habenula neurons resulting from the presence of sex hormones. Further work is needed to test this hypothesis, which may have implications for understanding the etiology of several mental health disorders including depression, schizophrenia, and addiction.

Glutamate inputs send prediction error of reward but not negative value of aversive stimuli to dopamine neurons

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs) but the mechanisms underlying RPE computation, particularly contributions of different neurotransmitters, remain poorly understood. Here we used a genetically-encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons. We found that glutamate inputs exhibit virtually all of the characteristics of RPE, rather than conveying a specific component of RPE computation such as reward or expectation. Notably, while glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli toward more positive responses, while excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

Contexts facilitate dynamic value encoding in the mesolimbic dopamine system

Adaptive behavior in a dynamic environment often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical test to theories of dopamine's function in learning, motivation, and motor control. Yet how dopamine neurons are involved in the revaluation of cues when the world changes to alter our behavior remains unclear. Here we make use of pharmacology, in vivo electrophysiology, fiber photometry, and optogenetics to resolve the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward-seeking. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by sucrose reward by exploiting the prior, non-overlapping presentation of a separate discrete cue - an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery. As a result, in this task we were able to dissociate the average value of the conditioned stimulus from its immediate expected value on a trial-to-trial basis. Both the activity of ventral tegmental area dopamine neurons and dopamine signaling in the nucleus accumbens were essential for rats to successfully update behavioral responding in response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward. Downstream of dopamine release, we found that single neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together these results reveal a novel mechanism within the mesolimbic dopamine system for the rapid revaluation of motivation.

Reward prediction-errors weighted by cue salience produces addictive behaviours in simulations, with asymmetrical learning and steeper delay discounting

Dysfunction in learning and motivational systems are thought to contribute to addictive behaviours. Previous models have suggested that dopaminergic roles in learning and motivation could produce addictive behaviours through pharmacological manipulations that provide excess dopaminergic signalling towards these learning and motivational systems. Redish (2004) suggested a role based on dopaminergic signals of value prediction error, while (Zhang et al., 2009) suggested a role based on dopaminergic signals of motivation. However, both models present significant limitations. They do not explain the reduced sensitivity to drug-related costs/negative consequences, the increased impulsivity generally found in people with a substance use disorder, craving behaviours, and non-pharmacological dependence, all of which are key hallmarks of addictive behaviours. Here, we propose a novel mathematical definition of salience, that combines aspects of dopamine's role in both learning and motivation within the reinforcement learning framework. Using a single parameter regime, we simulated addictive behaviours that the (Zhang et al., 2009; Redish, 2004) models also produce but we went further in simulating the downweighting of drug-related negative prediction-errors, steeper delay discounting of drug rewards, craving behaviours and aspects of behavioural/non-pharmacological addictions. The current salience model builds on our recently proposed conceptual theory that salience modulates internal representation updating and may contribute to addictive behaviours by producing misaligned internal representations (Kalhan et al., 2021). Critically, our current mathematical model of salience argues that the seemingly disparate learning and motivational aspects of dopaminergic functioning may interact through a salience mechanism that modulates internal representation updating.

Reward expectations direct learning and drive operant matching in Drosophila

Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein's operant matching law. Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear. Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards. We then built a model of the fly mushroom body to explain each fly's sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both. However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.

The Initial Efficacy of Stand-Alone DBT Skills Training for Treating Impulsivity Among Individuals With Alcohol and Other Substance Use Disorders

Impulsivity is considered a core feature of substance use disorders (SUDs), including personological (i.e., negative urgency, positive urgency, lack of premeditation) and neuropsychological (i.e., cognitive and motor disinhibition, impulsive choice) dimensions. Dialectical Behavior Therapy Skills Training (DBT-ST) as a stand-alone treatment is an effective intervention for alcohol use disorder (AUD) and other SUDs. However, there are no studies that have investigated changes in impulsivity levels during a DBT-ST program, especially testing the therapeutic effects of DBT skills. Twenty-nine patients with AUD and other SUDs were admitted to a 3-month DBT-ST program. Self-report (i.e., UPPS-P) and computerized neuropsychological (i.e., Attentional Network test; Go/No-Go task; Iowa Gambling Task) measures of impulsivity were administered at the beginning and end of the DBT-ST. Distress tolerance (DTS), mindfulness (MAAS, FFMQ) and emotion regulation (DERS) were also assessed pre- and post-intervention. The study included two age- and gender-matched control groups: (a) untreated patients with SUDs (N = 29); (b) healthy controls (HCs) (N = 29). Twenty-four (82.7%) patients concluded the DBT-ST program. Emotion-based forms of impulsivity significantly improved during the program. At the end of treatment, impulsivity levels were significantly lower than those of untreated patients with SUDs and they were not significantly different from HCs. Cognitive disinhibition significantly decreased during the treatment. The improvement in impulsivity was explained by pre- posttreatment changes in distress tolerance, mindfulness and emotion regulation. Motor disinhibition did not improve during the treatment. These findings supported the initial efficacy of the DBT-ST program for addressing different features of impulsivity among individuals with AUD and other SUDs. Future follow-up studies should demonstrate the role of impulsivity domains in long-term relapse prevention.

Role of dopamine D1 receptor in the modulation of memory consolidation by passive and self-administered heroin and associated conditioned stimuli

It has been proposed that opiates modulate memory consolidation, but recent work has indicated that this effect may be mediated by how the drug is experienced (i.e., passive injections vs. self-administration). Because the dopamine (DA) D1 receptor is involved in processing of learning signals and attribution of salience to events experienced by an organism, two studies in male Sprague-Dawley rats tested the effect of blocking this receptor on modulation of memory consolidation by passive and self-administered heroin, in addition to conditioned memory modulation by heroin-paired cues. Using the object location memory task, Study 1 employed SCH23390 (0, 0.05, 0.10 mg/kg, SC) to modulate enhancement of memory consolidation induced by post-training injections of heroin (1 mg/kg, SC) as well as by exposure to the environment paired with heroin injections (6 pairings, 1 h each, 1 mg/kg). Study 2 was conducted in rats that could self-administer heroin (0.05 mg/kg/infusion, IV) and tested whether SCH23390 (0 and 0.1 mg/kg, SC) could prevent memory modulation induced by a change in schedule of self-administration (from fixed to variable ratio). It was found that while repeated passive injections of heroin retained their enhancing effect on memory, when self-administered, heroin enhanced consolidation of object location memory only at the beginning of self-administration and after a change in schedule. Importantly, SCH23390 blocked memory modulation by heroin when passively administered and when the drug was self-administered on a novel schedule. SCH23390 also blocked conditioned memory modulation induced by post-training exposure to heroin-paired cues. Taken together, these results suggest that modulation of memory consolidation by unconditioned and conditioned opiate reinforcers involve a D1-dependent mechanism of salience attribution linked to the anticipation of drug effects.

Dopamine transporter blockade during adolescence increases adult dopamine function, impulsivity, and aggression

Sensitive developmental periods shape neural circuits and enable adaptation. However, they also engender vulnerability to factors that can perturb developmental trajectories. An understanding of sensitive period phenomena and mechanisms separate from sensory system development is still lacking, yet critical to understanding disease etiology and risk. The dopamine system is pivotal in controlling and shaping adolescent behaviors, and it undergoes heightened plasticity during that time, such that interference with dopamine signaling can have long-lasting behavioral consequences. Here we sought to gain mechanistic insight into this dopamine-sensitive period and its impact on behavior. In mice, dopamine transporter (DAT) blockade from postnatal (P) day 22 to 41 increases aggression and sensitivity to amphetamine (AMPH) behavioral stimulation in adulthood. Here, we refined this sensitive window to P32-41 and identified increased firing of dopaminergic neurons in vitro and in vivo as a neural correlate to altered adult behavior. Aggression can result from enhanced impulsivity and cognitive dysfunction, and dopamine regulates working memory and motivated behavior. Hence, we assessed these behavioral domains and found that P32-41 DAT blockade increases impulsivity but has no effect on cognition, working memory, or motivation in adulthood. Lastly, using optogenetics to drive dopamine neurons, we find that increased VTA but not SNc dopaminergic activity mimics the increase in impulsive behavior in the Go/NoGo task observed after adolescent DAT blockade. Together our data provide insight into the developmental origins of aggression and impulsivity that may ultimately improve diagnosis, prevention, and treatment strategies for related neuropsychiatric disorders.

Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types

The ventral pallidum (VP) contains GABA and glutamate (Glut) neurons projecting to ventral tegmental area (VTA) whose stimulation drives approach and avoidance, respectively. Yet little is known about the cell-type-specific mechanisms by which VP projections to VTA drive behavior. Here, we found that both VP GABA and Glut neurons were activated during approach to reward or delivery of an aversive stimulus. Stimulation of VP GABA neurons inhibited VTA GABA, but activated dopamine (DA) and glutamate neurons. Remarkably, this cell-type-specific recruitment was behavior-contingent such that VTA recruitment was inhibited when evoked by the subject's own action. Conversely, VP Glut neurons activated VTA GABA, as well as DA and Glut neurons, despite driving aversion. However, VP Glut neurons evoked DA in reward-associated ventromedial nucleus accumbens (NAc), but reduced DA in aversion-associated dorsomedial NAc. These findings show how heterogeneous VP cell types can engage VTA cell types to shape approach and avoidance behaviors.

Food reinforcement architecture: A framework for impulsive and compulsive overeating and food abuse

Few reward-based theories address key drivers of susceptibility to food cues and consumption beyond fullness. Decision-making and habit formation are governed by reinforcement-based learning processes that, when overstimulated, can drive unregulated hedonically motivated overeating. Here, a model food reinforcement architecture is proposed that uses fundamental concepts in reinforcement and decision-making to identify maladaptive eating habits that can lead to obesity. This model is unique in that it identifies metabolic drivers of reward and incorporates neuroscience, computational decision-making, and psychology to map overeating and obesity. Food reinforcement architecture identifies two paths to overeating: a propensity for hedonic targeting of food cues contributing to impulsive overeating and lack of satiation that contributes to compulsive overeating. A combination of those paths will result in a conscious and subconscious drive to overeat independent of negative consequences, leading to food abuse and/or obesity. Use of this model to identify aberrant reinforcement learning processes and decision-making systems that can serve as markers of overeating risk may provide an opportunity for early intervention in obesity.

Opioid withdrawal: role in addiction and neural mechanisms

Withdrawal from opioids involves a negative affective state that promotes maintenance of drug-seeking behavior and relapse. As such, understanding the neurobiological mechanisms underlying withdrawal from opioid drugs is critical as scientists and clinicians seek to develop new treatments and therapies. In this review, we focus on the neural systems known to mediate the affective and somatic signs and symptoms of opioid withdrawal, including the mesolimbic dopaminergic system, basolateral amygdala, extended amygdala, and brain and hormonal stress systems. Evidence from preclinical studies suggests that these systems are altered following opioid exposure and that these changes mediate behavioral signs of negative affect such as aversion and anxiety during withdrawal. Adaptations in these systems also parallel the behavioral and psychological features of opioid use disorder (OUD), highlighting the important role of withdrawal in the development of addictive behavior. Implications for relapse and treatment are discussed as well as promising avenues for future research, with the hope of promoting continued progress toward characterizing neural contributors to opioid withdrawal and compulsive opioid use.

Behavioral pattern separation and cognitive flexibility are enhanced in a mouse model of increased lateral entorhinal cortex-dentate gyrus circuit activity

Behavioral pattern separation and cognitive flexibility are essential cognitive abilities that are disrupted in many brain disorders. A better understanding of the neural circuitry involved in these abilities will open paths to treatment. In humans and mice, discrimination and adaptation rely on the integrity of the hippocampal dentate gyrus (DG) which receives glutamatergic input from the entorhinal cortex (EC), including the lateral EC (LEC). An inducible increase of EC-DG circuit activity improves simple hippocampal-dependent associative learning and increases DG neurogenesis. Here, we asked if the activity of LEC fan cells that directly project to the DG (LEC → DG neurons) regulates the relatively more complex hippocampal-dependent abilities of behavioral pattern separation or cognitive flexibility. C57BL/6J male mice received bilateral LEC infusions of a virus expressing shRNA TRIP8b, an auxiliary protein of an HCN channel or a control virus (SCR shRNA). Prior work shows that 4 weeks post-surgery, TRIP8b mice have more DG neurogenesis and greater activity of LEC → DG neurons compared to SCR shRNA mice. Here, 4 weeks post-surgery, the mice underwent testing for behavioral pattern separation and reversal learning (touchscreen-based location discrimination reversal [LDR]) and innate fear of open spaces (elevated plus maze [EPM]) followed by quantification of new DG neurons (doublecortin-immunoreactive cells [DCX+] cells). There was no effect of treatment (SCR shRNA vs. TRIP8b) on performance during general touchscreen training, LDR training, or the 1st days of LDR testing. However, in the last days of LDR testing, the TRIP8b shRNA mice had improved pattern separation (reached the first reversal more quickly and had more accurate discrimination) compared to the SCR shRNA mice, specifically when the load on pattern separation was high (lit squares close together or "small separation"). The TRIP8b shRNA mice were also more cognitively flexible (achieved more reversals) compared to the SCR shRNA mice in the last days of LDR testing. Supporting a specific influence on cognitive behavior, the SCR shRNA and TRIP8b shRNA mice did not differ in total distance traveled or in time spent in the closed arms of the EPM. Supporting an inducible increase in LEC-DG activity, DG neurogenesis was increased. These data indicate that the TRIP8b shRNA mice had better pattern separation and reversal learning and more neurogenesis compared to the SCR shRNA mice. This study advances fundamental and translational neuroscience knowledge relevant to two cognitive functions critical for adaptation and survival-behavioral pattern separation and cognitive flexibility-and suggests that the activity of LEC → DG neurons merits exploration as a therapeutic target to normalize dysfunctional DG behavioral output.

Dopaminergic prediction errors in the ventral tegmental area reflect a multithreaded predictive model

Dopamine neuron activity is tied to the prediction error in temporal difference reinforcement learning models. These models make significant simplifying assumptions, particularly with regard to the structure of the predictions fed into the dopamine neurons, which consist of a single chain of timepoint states. Although this predictive structure can explain error signals observed in many studies, it cannot cope with settings where subjects might infer multiple independent events and outcomes. In the present study, we recorded dopamine neurons in the ventral tegmental area in such a setting to test the validity of the single-stream assumption. Rats were trained in an odor-based choice task, in which the timing and identity of one of several rewards delivered in each trial changed across trial blocks. This design revealed an error signaling pattern that requires the dopamine neurons to access and update multiple independent predictive streams reflecting the subject's belief about timing and potentially unique identities of expected rewards.

Computational approaches to modeling gambling behaviour: Opportunities for understanding disordered gambling

Computational modeling has become an important tool in neuroscience and psychiatry research to provide insight into the cognitive processes underlying normal and pathological behavior. There are two modeling frameworks, reinforcement learning (RL) and drift diffusion modeling (DDM), that are well-developed in cognitive science, and have begun to be applied to Gambling Disorder. RL models focus on explaining how an agent uses reward to learn about the environment and make decisions based on outcomes. The DDM is a binary choice framework that breaks down decision making into psychologically meaningful components based on choice reaction time analyses. Both approaches have begun to yield insight into aspects of cognition that are important for, but not unique to, gambling, and thus relevant to the development of Gambling Disorder. However, these approaches also oversimplify or neglect various aspects of decision making seen in real-world gambling behavior. Gambling Disorder presents an opportunity for 'bespoke' modeling approaches to consider these neglected components. In this review, we discuss studies that have used RL and DDM frameworks to investigate some of the key cognitive components in gambling and Gambling Disorder. We also include an overview of Bayesian models, a methodology that could be useful for more tailored modeling approaches. We highlight areas in which computational modeling could enable progression in the investigation of the cognitive mechanisms relevant to gambling.

Insights into the mechanisms underlying opioid use disorder and potential treatment strategies

Opioid use disorder is a worldwide societal problem and public health burden. Strategies for treating opioid use disorder can be divided into those that target the opioid receptor system and those that target non-opioid receptor systems, including the dopamine and glutamate receptor systems. Currently, the clinical drugs used to treat opioid use disorder include the opioid receptor agonists methadone and buprenorphine, which are limited by their abuse liability, and the opioid receptor antagonist naltrexone, which is limited by poor compliance. Therefore, the development of effective medications with lower abuse liability and better potential for compliance is urgently needed. Based on recent advances in the understanding of the neurobiological mechanisms underlying opioid use disorder, potential treatment strategies and targets have emerged. This review focuses on the progress made in identifying potential targets and developing medications to treat opioid use disorder, including progress made by our laboratory, and provides insights for future medication development. LINKED ARTICLES: This article is part of a themed issue on Advances in Opioid Pharmacology at the Time of the Opioid Epidemic. To view the other articles in this section visit http://onlinelibrary.wiley.com/doi/10.1111/bph.v180.7/issuetoc.

Homeostatic Reinforcement Theory Accounts for Sodium Appetitive State- and Taste-Dependent Dopamine Responding

Seeking and consuming nutrients is essential to survival and the maintenance of life. Dynamic and volatile environments require that animals learn complex behavioral strategies to obtain the necessary nutritive substances. While this has been classically viewed in terms of homeostatic regulation, recent theoretical work proposed that such strategies result from reinforcement learning processes. This theory proposed that phasic dopamine (DA) signals play a key role in signaling potentially need-fulfilling outcomes. To examine links between homeostatic and reinforcement learning processes, we focus on sodium appetite as sodium depletion triggers state- and taste-dependent changes in behavior and DA signaling evoked by sodium-related stimuli. We find that both the behavior and the dynamics of DA signaling underlying sodium appetite can be accounted for by a homeostatically regulated reinforcement learning framework (HRRL). We first optimized HRRL-based agents to sodium-seeking behavior measured in rodents. Agents successfully reproduced the state and the taste dependence of behavioral responding for sodium as well as for lithium and potassium salts. We then showed that these same agents account for the regulation of DA signals evoked by sodium tastants in a taste- and state-dependent manner. Our models quantitatively describe how DA signals evoked by sodium decrease with satiety and increase with deprivation. Lastly, our HRRL agents assigned equal preference for sodium versus the lithium containing salts, accounting for similar behavioral and neurophysiological observations in rodents. We propose that animals use orosensory signals as predictors of the internal impact of the consumed good and our results pose clear targets for future experiments. In sum, this work suggests that appetite-driven behavior may be driven by reinforcement learning mechanisms that are dynamically tuned by homeostatic need.

A computational model of learning flexible navigation in a maze by layout-conforming replay of place cells

Recent experimental observations have shown that the reactivation of hippocampal place cells (PC) during sleep or wakeful immobility depicts trajectories that can go around barriers and can flexibly adapt to a changing maze layout. However, existing computational models of replay fall short of generating such layout-conforming replay, restricting their usage to simple environments, like linear tracks or open fields. In this paper, we propose a computational model that generates layout-conforming replay and explains how such replay drives the learning of flexible navigation in a maze. First, we propose a Hebbian-like rule to learn the inter-PC synaptic strength during exploration. Then we use a continuous attractor network (CAN) with feedback inhibition to model the interaction among place cells and hippocampal interneurons. The activity bump of place cells drifts along paths in the maze, which models layout-conforming replay. During replay in sleep, the synaptic strengths from place cells to striatal medium spiny neurons (MSN) are learned by a novel dopamine-modulated three-factor rule to store place-reward associations. During goal-directed navigation, the CAN periodically generates replay trajectories from the animal's location for path planning, and the trajectory leading to a maximal MSN activity is followed by the animal. We have implemented our model into a high-fidelity virtual rat in the MuJoCo physics simulator. Extensive experiments have demonstrated that its superior flexibility during navigation in a maze is due to a continuous re-learning of inter-PC and PC-MSN synaptic strength.

Developmental treatments with Δ9- tetrahydrocannabinol and the MAGL inhibitor JZL184 persistently alter adult cocaine conditioning in contrasting ways

Using a songbird, zebra finches, as a developmental drug abuse model we found previously that cannabinoid agonists administered during the sensorimotor period of vocal learning (50-75 days of age) persistently alter song patterns and cocaine responsiveness in adulthood. However, these effects were not produced in adults exposed to similar treatment regimens. Currently, we have used the MAGL inhibitor, JZL184, to test whether enhanced endocannabinoid signaling may similarly alter cocaine responsiveness. We found that, as expected and consistent with prior results, repeated developmental (but not adult) treatments with Δ⁹-tetrahydrocannabinol (THC, 3 mg/kg QD IM) resulted in increased time spent in cocaine-paired chambers. Unexpectedly and in contrast, repeated developmental JZL184 (4 mg/kg QD IM) treatments decreased time spent in cocaine-conditioned chambers. That is, young finches repeatedly treated with JZL184 avoided cocaine-paired chambers later in adulthood, while similar development treatments with THC had the opposite effect. To begin to identify brain regions that may underly this differential responsiveness we used c-Fos expression as a marker of neuronal activity. Differences in c-Fos expression patterns following placement of cocaine-conditioned finches into vehicle- vs. cocaine-paired chambers suggest distinct involvement of circuits through striatal and amygdaloid regions in respective effects of THC and JZL184. Results demonstrate that, like exogenous cannabinoid exposure, inhibition of MAGL activity during late post-natal development persistently alters behavior in adulthood. Contrasting effects of THC vs. MAGL inhibition with JZL184 suggests the latter alters development of brain regions to favor promotion of aversive rather than appetitive cocaine responsiveness later in adulthood.

Behavioral pattern separation and cognitive flexibility are enhanced in a mouse model of increased lateral entorhinal cortex-dentate gyrus circuit activity

Behavioral pattern separation and cognitive flexibility are essential cognitive abilities which are disrupted in many brain disorders. Better understanding of the neural circuitry involved in these abilities will open paths to treatment. In humans and mice, discrimination and adaptation rely on integrity of the hippocampal dentate gyrus (DG) which both receive glutamatergic input from the entorhinal cortex (EC), including the lateral EC (LEC). Inducible increase of EC-DG circuit activity improves simple hippocampal-dependent associative learning and increases DG neurogenesis. Here we asked if the activity of LEC fan cells that directly project to the DG (LEC➔DG neurons) regulates behavioral pattern separation or cognitive flexibility. C57BL6/J male mice received bilateral LEC infusions of a virus expressing shRNA TRIP8b, an auxiliary protein of an HCN channel or a control virus (SCR shRNA); this approach increases the activity of LEC➔DG neurons. Four weeks later, mice underwent testing for behavioral pattern separation and reversal learning (touchscreen-based Location Discrimination Reversal [LDR] task) and innate fear of open spaces (elevated plus maze [EPM]) followed by counting of new DG neurons (doublecortin-immunoreactive cells [DCX+] cells). TRIP8b and SCR shRNA mice performed similarly in general touchscreen training and LDR training. However, in late LDR testing, TRIP8b shRNA mice reached the first reversal more quickly and had more accurate discrimination vs. SCR shRNA mice, specifically when pattern separation was challenging (lit squares close together or "small separation"). Also, TRIP8b shRNA mice achieved more reversals in late LDR testing vs. SCR shRNA mice. Supporting a specific influence on cognitive behavior, SCR shRNA and TRIP8b shRNA mice did not differ in total distance traveled or in time spent in the closed arms of the EPM. Supporting an inducible increase in LEC-DG activity, DG neurogenesis was increased. These data indicate TRIP8b shRNA mice had better pattern separation and reversal learning and more neurogenesis vs. SCR shRNA mice. This work advances fundamental and translational neuroscience knowledge relevant to two cognitive functions critical for adaptation and survival - behavioral pattern separation and cognitive flexibility - and suggests the activity of LEC➔DG neurons merits exploration as a therapeutic target to normalize dysfunctional DG behavioral output.

Whole-brain tracking of cocaine and sugar rewards processing

Natural rewards, such as food, and sex are appetitive stimuli available for animals in their natural environment. Similarly, addictive rewards such as drugs of abuse possess strong, positive valence, but their action relies on their pharmacological properties. Nevertheless, it is believed that both of these kinds of rewards activate similar brain circuitry. The present study aimed to discover which parts of the brain process the experience of natural and addictive rewards. To holistically address this question, we used a single-cell whole-brain imaging approach to find patterns of activation for acute and prolonged sucrose and cocaine exposure. We analyzed almost 400 brain structures and created a brain-wide map of specific, c-Fos-positive neurons engaged by these rewards. Acute but not prolonged sucrose exposure triggered a massive c-Fos expression throughout the brain. Cocaine exposure on the other hand potentiated c-Fos expression with prolonged use, engaging more structures than sucrose treatment. The functional connectivity analysis unraveled an increase in brain modularity after the initial exposure to both types of rewards. This modularity was increased after repeated cocaine, but not sucrose, intake. To check whether discrepancies between the processing of both types of rewards can be found on a cellular level, we further studied the nucleus accumbens, one of the most strongly activated brain structures by both sucrose and cocaine experience. We found a high overlap between natural and addictive rewards on the level of c-Fos expression. Electrophysiological measurements of cellular correlates of synaptic plasticity revealed that natural and addictive rewards alike induce the accumulation of silent synapses. These results strengthen the hypothesis that in the nucleus accumbens drugs of abuse cause maladaptive neuronal plasticity in the circuitry that typically processes natural rewards.