Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

Effects of psychedelic, DOI, on nucleus accumbens dopamine signaling to predictable rewards and cues in rats

Psychedelics produce lasting therapeutic responses in neuropsychiatric diseases suggesting they may disrupt entrenched associations and catalyze learning. Here, we examine psychedelic 5-HT2A/2C agonist, DOI, effects on dopamine signaling in the nucleus accumbens (NAc) core, a region extensively linked to reward learning, motivation, and drug-seeking. We measure phasic dopamine transients following acute DOI administration in rats during well learned Pavlovian tasks in which sequential cues predict rewards. We find that DOI (0.0-1.2 mg/kg, i.p.) increases dopamine signals, photometrically measured using GRABDA optical sensor, to rewards and proximal reward cues, but not to the distal cues that predict these events. We determine that the elevated dopamine produced by DOI to reward cues occurs independently of DOI-induced changes in reward value. The increased dopamine associated with predictable reward cues and rewards supports DOI-induced increases in prediction error signaling. These findings lay a foundation for developing psychedelic strategies aimed at engaging error-driven learning mechanisms to disrupt entrenched associations or produce new associations.

Cross-species translational paradigms for assessing positive valence system as defined by the RDoC matrix

Functions associated with processing reward-related information are fundamental drivers of motivation, learning, and goal-directed behavior. Such functions have been classified as the positive valence system under the Research Domain and Criteria (RDoC) criteria and are negatively impacted across a range of psychiatric disorders and mental illnesses. The positive valence system is composed of three comprehensive categories containing related but dissociable functions that are organized into either Reward Responsiveness, Reward Learning, or Reward Valuation. The presence of overlapping behavioral dysfunction across diagnostic mental disorders is in-part what motivated the RDoC initiative, which emphasized that the study of mental illness focus on investigating relevant behavior and cognitive functions and their underlying mechanisms, rather than separating efforts on diagnostic categories (i.e., transdiagnostic). Moreover, the RDoC approach is well-suited for preclinical neuroscience research, as the rise in genetic toolboxes and associated neurotechnologies enables researchers to probe specific cellular targets with high specificity. Thus, there is an opportunity to dissect whether behaviors and cognitive functions are supported by shared or distinct neural mechanisms. For preclinical research to effectively inform our understandings of human behavior however, the cognitive and behavioral paradigms should have predictive, neurobiological, and pharmacological predictive validity to the human test. Touchscreen-based testing systems provide a further advantage for this endeavor enabling tasks to be presented to animals using the same media and task design as in humans. Here, we outline the primary categories of the positive valence system and review the work that has been done cross-species to investigate the neurobiology and neurochemistry underlying reward-related functioning. Additionally, we provide clinical tasks outlined by RDoC, along with validity and/or need for further validation for analogous rodent paradigms with a focus on implementing the touchscreen-based cognitive testing systems.

Dopamine transients encode reward prediction errors independent of learning rates

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented by corticostriatal synaptic weights, which are updated by dopamine-dependent plasticity. This suggests that dopamine release reflects the product of the learning rate and RPE. Here, we characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc) in a volatile environment. Using a task with semi-observable states offering different rewards, we find that rats adjust how quickly they initiate trials across states using RPEs. Computational modeling and behavioral analyses show that learning rates are higher following state transitions and scale with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encodes RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

Wideband ratiometric measurement of tonic and phasic dopamine release in the striatum

Reward learning, cognition, and motivation are supported by changes in neurotransmitter levels across multiple timescales. Current measurement technologies for various neuromodulators (such as dopamine and serotonin) do not bridge timescales of fluctuations, limiting the ability to define the behavioral significance, regulation, and relationship between fast (phasic) and slow (tonic) dynamics. To help resolve longstanding debates about the behavioral significance of dopamine across timescales, we developed a novel quantification strategy, augmenting extensively used carbon-fiber Fast Scan Cyclic Voltammetry (FSCV). We iteratively engineered the FSCV scan sequence to rapidly modify electrode sensitivity within a sampling window and applied ratiometric analysis for wideband dopamine measurement. This allowed us to selectively eliminate artifacts unrelated to electrochemical detection (i.e., baseline drift), overcoming previous limitations that precluded wideband dopamine detection from milliseconds to hours. We extensively characterize this approach in vitro, validate performance in vivo with simultaneous microdialysis, and deploy this technique to measure wideband dopamine changes across striatal regions under pharmacological, optogenetic, and behavioral manipulations. We demonstrate that our approach can extend to additional analytes, including serotonin and pH, providing a robust platform to assess the contributions of multi-timescale neuromodulator fluctuations to cognition, learning, and motivation.

Value modulation of self-defeating impulsivity

BACKGROUND

Impulse control is a critical aspect of cognitive functioning. Intuitively, whether an action is executed prematurely depends on its associated reward, yet the link between value and impulsivity remains poorly understood. Three frameworks for impulsivity offer contrasting views: impulsive behavior may be valuable because it is associated with hidden internal reward (e.g., reduction of mental effort). Alternatively, it can emerge from exploration, which is disadvantageous in the short term, but can yield long-term benefits. Finally, impulsivity may reflect Pavlovian bias, an inherent tendency that occurs even when its outcome is negative.

METHODS

To test these hypotheses, we trained seventeen male mice to withhold licking while anticipating variable rewards. We then measured and optogenetically manipulated dopamine release in the ventral striatum.

RESULTS

We found that higher reward magnitudes correlated with increased impulsivity. This behavior was well explained by a Pavlovian-bias model. Furthermore, we observed negative dopamine signals during premature licking, suggesting that in this task, impulsivity is not merely an unsuccessful attempt at obtaining a reward. Rather, it is a failure to overcome the urge to act prematurely despite knowledge of the negative consequences of such impulsive action.

CONCLUSION

Our findings underscore the integral role value plays in regulating impulsivity and suggest that the dopaminergic system influences impulsivity through the mediation of value learning.

Estrogenic control of reward prediction errors and reinforcement learning

Gonadal hormones act throughout the brain 1 , and neuropsychiatric disorders vary in symptom severity over the reproductive cycle, pregnancy, and perimenopause 2-4 . Yet how hormones influence cognitive processes is unclear. Exogenous 17 β -estradiol modulates dopamine signaling in the nucleus accumbens core (NAcc) 5,6 , which instantiates reward prediction errors (RPEs) for reinforcement learning 7-16 . Here we show that endogenous 17 β -estradiol enhances RPEs and sensitivity to previous rewards by reducing dopamine reuptake proteins in the NAcc. Rats performed a task with different reward states; they adjusted how quickly they initiated trials across states, balancing effort against expected rewards. NAcc dopamine reflected RPEs that predicted and causally influenced initiation times. Elevated endogenous 17 β -estradiol increased sensitivity to reward states by enhancing dopaminergic RPEs in the NAcc. Proteomics revealed reduced dopamine transporter expression. Finally, knockdown of midbrain estrogen receptors suppressed reinforcement learning. 17 β -estradiol therefore controls RPEs via dopamine reuptake, mechanistically revealing how hormones influence neural dynamics for motivation and learning.

Dopamine transmission at D1 and D2 receptors in the nucleus accumbens contributes to the expression of incubation of cocaine craving

Relapse represents a consistent clinical problem for individuals with substance use disorder. In the incubation of craving model of persistent craving and relapse, cue-induced drug seeking progressively intensifies or "incubates" during the first weeks of abstinence from drug self-administration and then remains high for months. Previously, we and others have demonstrated that expression of incubated cocaine craving requires strengthening of excitatory synaptic transmission in the nucleus accumbens core (NAcc). However, despite the importance of dopaminergic signaling in the NAcc for motivated behavior, little is known about the role that dopamine (DA) plays in the incubation of cocaine craving. Here we used fiber photometry to measure DA transients in the NAcc of male and female rats during cue-induced seeking tests conducted in early abstinence from cocaine self-administration, prior to incubation, and late abstinence, after incubation of craving has plateaued. We observed DA transients time-locked to cue-induced responding but their magnitude did not differ significantly when measured during early versus late abstinence seeking tests. Next, we tested for a functional role of these DA transients by injecting DA receptor antagonists into the NAcc just before the cue-induced seeking test. Blockade of either D1 or D2 DA receptors reduced cue-induced cocaine seeking after but not before incubation. We found no main effect of sex or significant interaction of sex with other factors in our experiments. These results suggest that DA contributes to incubated cocaine seeking but the emergence of this role reflects changes in postsynaptic responsiveness to DA rather than presynaptic alterations.

Reward timescale controls the rate of behavioural and dopaminergic learning

Learning the causes of rewards is necessary for survival. Thus, it is critical to understand the mechanisms of such a vital biological process. Cue-reward learning is controlled by mesolimbic dopamine and improves with spacing of cue-reward pairings. However, whether a mathematical rule governs such improvements in learning rate, and if so, whether a unifying mechanism captures this rule and dopamine dynamics during learning remain unknown. Here, we investigate the behavioral, algorithmic, and dopaminergic mechanisms governing cue-reward learning rate. Across a range of conditions in mice, we show a strong, mathematically proportional relationship between both behavioral and dopaminergic learning rates and the duration between rewards. Due to this relationship, removing up to 19 out of 20 cue-reward pairings over a fixed duration has no influence on overall learning. These findings are explained by a dopamine-based model of retrospective learning, thereby providing a unified account of the biological mechanisms of learning.

Dopaminergic Neurons in Zona Incerta Drives Appetitive Self-Grooming

Dopaminergic (DA) neurons are known to play a key role in controlling behaviors. While DA neurons in other brain regions are extensively characterized, those in zona incerta (ZITH or A13) receive much less attention and their function remains to be defined. Here it is shown that optogenetic stimulation of these neurons elicited intensive self-grooming behaviors and promoted place preference, which can be enhanced by training but cannot be converted into contextual memory. Interestingly, the same stimulation increased DA release to periaqueductal grey (PAG) neurons and local PAG antagonism of DA action reduced the elicited self-grooming. In addition, A13 neurons increased their activity in response to various external stimuli and during natural self-grooming episodes. Finally, monosynaptic retrograde tracing showed that the paraventricular hypothalamus represents one of the major upstream brain regions to A13 neurons. Taken together, these results reveal that A13 neurons are one of the brain sites that promote appetitive self-grooming involving DA release to the PAG.

Distinct cholinergic circuits underlie discrete effects of reward on attention

Attention and reward are functions that are critical for the control of behavior, and massive multi-region neural systems have evolved to support the discrete computations associated with each. Previous research has also identified that attention and reward interact, though our understanding of the neural mechanisms that mediate this interplay is incomplete. Here, we review the basic neuroanatomy of attention, reward, and cholinergic systems. We then examine specific contexts in which attention and reward computations interact. Building on this work, we propose two discrete neural circuits whereby acetylcholine, released from cell groups located in different parts of the brain, mediates the impact of stimulus-reward associations as well as motivation on attentional control. We conclude by examining these circuits as a potential shared loci of dysfunction across diseases states associated with deficits in attention and reward.

Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning

Dopamine release in the nucleus accumbens core (NAcC) is generally considered to be a proxy for phasic firing of the ventral tegmental area dopamine (VTADA) neurons. Thus, dopamine release in NAcC is hypothesized to reflect a unitary role in reward prediction error signaling. However, recent studies reveal more diverse roles of dopamine neurons, which support an emerging idea that dopamine regulates learning differently in distinct circuits. To understand whether the NAcC might regulate a unique component of learning, we recorded dopamine release in NAcC while male rats performed a backward conditioning task where a reward is followed by a neutral cue. We used this task because we can delineate different components of learning, which include sensory-specific inhibitory and general excitatory components. Furthermore, we have shown that VTADA neurons are necessary for both the specific and general components of backward associations. Here, we found that dopamine release in NAcC increased to the reward across learning while reducing to the cue that followed as it became more expected. This mirrors the dopamine prediction error signal seen during forward conditioning and cannot be accounted for temporal-difference reinforcement learning. Subsequent tests allowed us to dissociate these learning components and revealed that dopamine release in NAcC reflects the general excitatory component of backward associations, but not their sensory-specific component. These results emphasize the importance of examining distinct functions of different dopamine projections in reinforcement learning.

Dopamine transients encode reward prediction errors independent of learning rates

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc). We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

The nucleus accumbens in reward and aversion processing: insights and implications

The nucleus accumbens (NAc), a central component of the brain's reward circuitry, has been implicated in a wide range of behaviors and emotional states. Emerging evidence, primarily drawing from recent rodent studies, suggests that the function of the NAc in reward and aversion processing is multifaceted. Prolonged stress or drug use induces maladaptive neuronal function in the NAc circuitry, which results in pathological conditions. This review aims to provide comprehensive and up-to-date insights on the role of the NAc in motivated behavior regulation and highlights areas that demand further in-depth analysis. It synthesizes the latest findings on how distinct NAc neuronal populations and pathways contribute to the processing of opposite valences. The review examines how a range of neuromodulators, especially monoamines, influence the NAc's control over various motivational states. Furthermore, it delves into the complex underlying mechanisms of psychiatric disorders such as addiction and depression and evaluates prospective interventions to restore NAc functionality.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Spatially organized striatum-wide acetylcholine dynamics for the learning and extinction of Pavlovian cues and actions

Striatal acetylcholine (ACh) has been linked to behavioral flexibility. A key component of flexibility is down-regulating responding as valued cues and actions become decoupled from positive outcomes. We used array fiber photometry in mice to investigate how ACh release across the striatum evolves during learning and extinction of Pavlovian associations. Changes in multi-phasic release to cues and consummatory actions were bi-directional and region-specific. Following extinction, increases in cue-evoked ACh release emerged in the anterior dorsal striatum (aDS) which preceded a down-regulation of anticipatory behavior. Silencing ACh release from cholinergic interneurons in the aDS blocked behavioral extinction. Dopamine release dipped below baseline for down-shifted cues, but glutamate input onto cholinergic interneurons did not change, suggesting an intrastriatal mechanism for the emergence of ACh increases. Our large-scale mapping of striatal ACh dynamics during learning pinpoints region-specific elevations in ACh release positioned to down-regulate behavior during extinction, a central feature of flexible behavior.

Value-Driven Adaptations of Mesolimbic Dopamine Release Are Governed by Both Model-Based and Model-Free Mechanisms

The magnitude of dopamine signals elicited by rewarding events and their predictors is updated when reward value changes. It is actively debated how readily these dopamine signals adapt and whether adaptation aligns with model-free or model-based reinforcement-learning principles. To investigate this, we trained male rats in a pavlovian-conditioning paradigm and measured dopamine release in the nucleus accumbens core in response to food reward (unconditioned stimulus) and reward-predictive conditioned stimuli (CS), both before and after reward devaluation, induced via either sensory-specific or nonspecific satiety. We demonstrate that (1) such devaluation reduces CS-induced dopamine release rapidly, without additional pairing of CS with devalued reward and irrespective of whether the devaluation was sensory-specific or nonspecific. In contrast, (2) reward devaluation did not decrease food reward-induced dopamine release. Surprisingly, (3) postdevaluation reconditioning, by additional pairing of CS with devalued reward, rapidly reinstated CS-induced dopamine signals to predevaluation levels. Taken together, we identify distinct, divergent adaptations in dopamine-signal magnitude when reward value is decreased: CS dopamine diminishes but reinstates fast, whereas reward dopamine is resistant to change. This implies that, respective to abovementioned findings, (1) CS dopamine may be governed by a model-based mechanism and (2) reward dopamine by a model-free one, where (3) the latter may contribute to swift reinstatement of the former. However, changes in CS dopamine were not selective for sensory specificity of reward devaluation, which is inconsistent with model-based processes. Thus, mesolimbic dopamine signaling incorporates both model-free and model-based mechanisms and is not exclusively governed by either.

Dopamine transmission at D1 and D2 receptors in the nucleus accumbens contributes to the expression of incubation of cocaine craving

Relapse represents a consistent clinical problem for individuals with substance use disorder. In the incubation of craving model of persistent craving and relapse, cue-induced drug seeking progressively intensifies or 'incubates' during the first weeks of abstinence from drug self-administration and then remains high for months. Previously, we and others have demonstrated that expression of incubated cocaine craving requires strengthening of excitatory synaptic transmission in the nucleus accumbens core (NAcc). However, despite the importance of dopaminergic signaling in the NAcc for motivated behavior, little is known about the role that dopamine (DA) plays in the incubation of cocaine craving. Here we used fiber photometry to measure DA transients in the NAcc of male and female rats during cue-induced seeking tests conducted in early abstinence from cocaine self-administration, prior to incubation, and late abstinence, after incubation of craving has plateaued. We observed DA transients time-locked to cue-induced responding but their magnitude did not differ significantly when measured during early versus late abstinence seeking tests. Next, we tested for a functional role of these DA transients by injecting DA receptor antagonists into the NAcc just before the cue-induced seeking test. Blockade of either D1 or D2 DA receptors reduced cue-induced cocaine seeking after but not before incubation. We found no main effect of sex in our experiments. These results suggest that DA contributes to incubated cocaine seeking but the emergence of this role reflects changes in postsynaptic responsiveness to DA rather than presynaptic alterations.

Reelin marks cocaine-activated striatal ensembles, promotes neuronal excitability, and regulates cocaine reward

Drugs of abuse activate defined neuronal ensembles in brain reward structures such as the nucleus accumbens (NAc), which are thought to promote the enduring synaptic, circuit, and behavioral consequences of drug exposure. While the molecular and cellular effects arising from experience with drugs like cocaine are increasingly well understood, the mechanisms that sculpt NAc ensemble participation are largely unknown. Here, we leveraged unbiased single-nucleus transcriptional profiling to identify expression of the secreted glycoprotein Reelin (encoded by the Reln gene) as a marker of cocaine-activated neuronal ensembles within the rat NAc. Multiplexed in situ detection confirmed selective expression of the immediate early gene Fos in Reln+ neurons after cocaine experience, and also revealed enrichment of Reln mRNA in Drd1 + medium spiny neurons (MSNs) in both the rat and human brain. Using a novel CRISPR interference strategy enabling selective Reln knockdown in the adult NAc, we observed altered expression of genes linked to calcium signaling, emergence of a transcriptional trajectory consistent with loss of cocaine sensitivity, and a striking decrease in MSN intrinsic excitability. At the behavioral level, loss of Reln prevented cocaine locomotor sensitization, abolished cocaine place preference memory, and decreased cocaine self-administration behavior. Together, these results identify Reelin as a critical mechanistic link between ensemble participation and cocaine-induced behavioral adaptations.

A Cocaine-Activated Ensemble Exerts Increased Control Over Behavior While Decreasing in Size

BACKGROUND

Substance use disorder is characterized by long-lasting changes in reward-related brain regions, such as the nucleus accumbens. Previous work has shown that cocaine exposure induces plasticity in broad, genetically defined cell types in the nucleus accumbens; however, in response to a stimulus, only a small percentage of neurons are transcriptionally active-termed an ensemble. Here, we identify an Arc-expressing neuronal ensemble that has a unique trajectory of recruitment and causally controls drug self-administration after repeated, but not acute, cocaine exposure.

METHODS

Using Arc-CreERT2 transgenic mice, we expressed transgenes in Arc+ ensembles activated by cocaine exposure (either acute [1 × 10 mg/kg intraperitoneally] or repeated [10 × 10 mg/kg intraperitoneally]). Using genetic, optical, and physiological recording and manipulation strategies, we assessed the contribution of these ensembles to behaviors associated with substance use disorder.

RESULTS

Repeated cocaine exposure reduced the size of the ensemble while simultaneously increasing its control over behavior. Neurons within the repeated cocaine ensemble were hyperexcitable, and their optogenetic excitation was sufficient for reinforcement. Finally, lesioning the repeated cocaine, but not the acute cocaine, ensemble blunted cocaine self-administration. Thus, repeated cocaine exposure reduced the size of the ensemble while simultaneously increasing its contributions to drug reinforcement.

CONCLUSIONS

We showed that repeated, but not acute, cocaine exposure induced a physiologically distinct ensemble characterized by the expression of the immediate early gene Arc, which was uniquely capable of modulating reinforcement behavior.

Mesolimbic Neural Response Dynamics Predict Future Individual Alcohol Drinking in Mice

BACKGROUND

Individual variability in response to rewarding stimuli is a striking but understudied phenomenon. The mesolimbic dopamine system is critical in encoding the reinforcing properties of both natural reward and alcohol; however, how innate or baseline differences in the response dynamics of this circuit define individual behavior and shape future vulnerability to alcohol remain unknown.

METHODS

Using naturalistic behavioral assays, a voluntary alcohol drinking paradigm, in vivo fiber photometry, in vivo electrophysiology, and chemogenetics, we investigated how differences in mesolimbic neural circuit activity contribute to the individual variability seen in reward processing and, by proxy, alcohol drinking.

RESULTS

We first characterized heterogeneous behavioral and neural responses to natural reward and defined how these baseline responses predicted future individual alcohol-drinking phenotypes in male mice. We then determined spontaneous ventral tegmental area dopamine neuron firing profiles associated with responses to natural reward that predicted alcohol drinking. Using a dual chemogenetic approach, we mimicked specific mesolimbic dopamine neuron firing activity before or during voluntary alcohol drinking to link unique neurophysiological profiles to individual phenotype. We show that hyperdopaminergic individuals exhibit a lower neuronal response to both natural reward and alcohol that predicts lower levels of alcohol consumption in the future.

CONCLUSIONS

These findings reveal unique, circuit-specific neural signatures that predict future individual vulnerability or resistance to alcohol and expand the current knowledge base on how some individuals are able to titrate their alcohol consumption whereas others go on to engage in unhealthy alcohol-drinking behaviors.

Dopamine and acetylcholine correlations in the nucleus accumbens depend on behavioral task states

Dopamine in the nucleus accumbens ramps up as animals approach desired goals. These ramps have received intense scrutiny because they seem to violate long-held hypotheses on dopamine function. Furthermore, it has been proposed that they are driven by local acetylcholine release, i.e., that they are mechanistically separate from dopamine signals related to reward prediction errors. Here, we tested this hypothesis by simultaneously recording accumbal dopamine and acetylcholine signals in rats executing a task involving motivated approach. Contrary to recent reports, we found that dopamine ramps were not coincidental with changes in acetylcholine. Instead, we found that acetylcholine could be positively, negatively, or uncorrelated with dopamine depending on whether the task phase was determined by a salient cue, reward prediction error, or active approach, respectively. Our results suggest that accumbal dopamine and acetylcholine are largely independent but may combine to engage different postsynaptic mechanisms depending on the behavioral task states.

Dopamine neuron activity evoked by sucrose and sucrose-predictive cues is augmented by peripheral and central manipulations of glucose availability

Food deprivation drives eating through multiple signals and circuits. Decreased glucose availability (i.e., cytoglucopenia) drives eating and also increases the value of sucrose. Ventral tegmental area (VTA) dopamine neurons (DANs) contribute to the evaluation of taste stimuli, but their role in integrating glucoprivic signals remains unknown. We monitored VTA DAN activity via Cre-dependent expression of a calcium indicator with in vivo fibre photometry. In ad libitum fed rats, intraoral sucrose evoked a phasic increase in DAN activity. To manipulate glucose availability, we administered (intraperitoneal, lateral or fourth ventricular) the antiglycolytic agent 5-thio-D-glucose (5TG), which significantly augmented the phasic DAN activity to sucrose. 5TG failed to alter DAN activity to water or saccharin, suggesting the response was selective for caloric stimuli. 5TG enhancement of sucrose-evoked DAN activity was stronger after fourth ventricular administration, suggesting a critical node of action within the hindbrain. As 5TG also increases blood glucose, in a separate study, we used peripheral insulin, which stimulates eating, to decrease blood glucose-which was associated with increased DAN activity to intraoral sucrose. DAN activity developed to a cue predictive of intraoral sucrose. While 5TG augmented cue-evoked DAN activity, its action was most potent when delivered to the lateral ventricle. Together, the studies point to central glucose availability as a key modulator of phasic DAN activity to food and food-cues. As glucose sensing neurons are known to populate the hypothalamus and brainstem, results suggest differential modulation of cue-evoked and sucrose-evoked DAN activity.

Phasic dopamine signals are reduced in the spontaneously hypertensive rat and increased by methylphenidate

The spontaneously hypertensive rat (SHR) is a selectively bred animal strain that is frequently used to model attention-deficit hyperactivity disorder (ADHD) because of certain genetically determined behavioural characteristics. To test the hypothesis that the characteristically altered response to positive reinforcement in SHRs may be due to altered phasic dopamine response to reward, we measured phasic dopamine signals in the SHRs and Sprague Dawley (SD) rats using in vivo fast-scan cyclic voltammetry. The effects of the dopamine reuptake inhibitor, methylphenidate, on these signals were also studied. Phasic dopamine signals during the pairing of a sensory cue with electrical stimulation of midbrain dopamine neurons were significantly smaller in the SHRs than in the SD rats. Over repeated pairings, the dopamine response to the sensory cue increased, whereas the response to the electrical stimulation of dopamine neurons decreased, similarly in both strains. However, the final amplitude of the response to the sensory cue after pairing was significantly smaller in SHRs than in the SD rats. Methylphenidate increased responses to sensory cues to a significantly greater extent in the SHRs than in the SD rats, due largely to differences in the low dose effect. At a higher dose, methylphenidate increased responses to sensory cues and electrical stimulation similarly in SHRs and SD rats. The smaller dopamine responses may explain the reduced salience of reward-predicting cues previously reported in the SHR, whereas the action of methylphenidate on the cue response suggests a potential mechanism for the therapeutic effects of low-dose methylphenidate in ADHD.

Dopamine transients follow a striatal gradient of reward time horizons

Animals make predictions to guide their behavior and update those predictions through experience. Transient increases in dopamine (DA) are thought to be critical signals for updating predictions. However, it is unclear how this mechanism handles a wide range of behavioral timescales-from seconds or less (for example, if singing a song) to potentially hours or more (for example, if hunting for food). Here we report that DA transients in distinct rat striatal subregions convey prediction errors based on distinct time horizons. DA dynamics systematically accelerated from ventral to dorsomedial to dorsolateral striatum, in the tempo of spontaneous fluctuations, the temporal integration of prior rewards and the discounting of future rewards. This spectrum of timescales for evaluative computations can help achieve efficient learning and adaptive motivation for a broad range of behaviors.

Striatal dopamine signals reflect perceived cue-action-outcome associations in mice

Striatal dopamine drives associative learning by acting as a teaching signal. Much work has focused on simple learning paradigms, including Pavlovian and instrumental learning. However, higher cognition requires that animals generate internal concepts of their environment, where sensory stimuli, actions and outcomes become flexibly associated. Here, we performed fiber photometry dopamine measurements across the striatum of male mice as they learned cue-action-outcome associations based on implicit and changing task rules. Reinforcement learning models of the behavioral and dopamine data showed that rule changes lead to adjustments of learned cue-action-outcome associations. After rule changes, mice discarded learned associations and reset outcome expectations. Cue- and outcome-triggered dopamine signals became uncoupled and dependent on the adopted behavioral strategy. As mice learned the new association, coupling between cue- and outcome-triggered dopamine signals and task performance re-emerged. Our results suggest that dopaminergic reward prediction errors reflect an agent's perceived locus of control.

Augmenting glutamatergic, but not dopaminergic, activity in the nucleus accumbens shell disrupts responding to a discrete alcohol cue in an alcohol context

Discrete alcohol cues and contexts are relapse triggers for people with alcohol use disorder exerting particularly powerful control over behaviour when they co-occur. Here, we investigated the neural substrates subserving the capacity for alcohol-associated contexts to elevate responding to an alcohol-predictive conditioned stimulus (CS). Specifically, rats were trained in a distinct 'alcohol context' to respond by entering a fluid port during a discrete auditory CS that predicted the delivery of alcohol and were familiarized with a 'neutral context' wherein alcohol was never available. When conditioned CS responding was tested by presenting the CS without alcohol, we found that augmenting glutamatergic activity in the nucleus accumbens (NAc) shell by microinfusing α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) reduced responding to an alcohol CS in an alcohol, but not neutral, context. Further, AMPA microinfusion robustly affected behaviour, attenuating the number, duration and latency of CS responses selectively in the alcohol context. Although dopaminergic inputs to the NAc shell were previously shown to be necessary for CS responding in an alcohol context, here, chemogenetic excitation of ventral tegmental area (VTA) dopamine neurons and their inputs to the NAc shell did not affect CS responding. Critically, chemogenetic excitation of VTA dopamine neurons affected feeding behaviour and elevated c-fos immunoreactivity in the VTA and NAc shell, validating the chemogenetic approach. These findings enrich our understanding of the substrates underlying Pavlovian responding for alcohol and reveal that the capacity for contexts to modulate responding to discrete alcohol cues is delicately underpinned by the NAc shell.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.

Striatal and Behavioral Responses to Reward Vary by Socioeconomic Status in Adolescents

Disparities in socioeconomic status (SES) lead to unequal access to financial and social support. These disparities are believed to influence reward sensitivity, which in turn are hypothesized to shape how individuals respond to and pursue rewarding experiences. However, surprisingly little is known about how SES shapes reward sensitivity in adolescence. Here, we investigated how SES influenced adolescent responses to reward, both in behavior and the striatum-a brain region that is highly sensitive to reward. We examined responses to both immediate reward (tracked by phasic dopamine) and average reward rate fluctuations (tracked by tonic dopamine) as these distinct signals independently shape learning and motivation. Adolescents (n = 114; 12-14 years; 58 female) performed a gambling task during functional magnetic resonance imaging. We manipulated trial-by-trial reward and loss outcomes, leading to fluctuations between periods of reward scarcity and abundance. We found that a higher reward rate hastened behavioral responses, and increased guess switching, consistent with the idea that reward abundance increases response vigor and exploration. Moreover, immediate reward reinforced previously rewarding decisions (win-stay, lose-switch) and slowed responses (postreward pausing), particularly when rewards were scarce. Notably, lower-SES adolescents slowed down less after rare rewards than higher-SES adolescents. In the brain, striatal activations covaried with the average reward rate across time and showed greater activations during rewarding blocks. However, these striatal effects were diminished in lower-SES adolescents. These findings show that the striatum tracks reward rate fluctuations, which shape decisions and motivation. Moreover, lower SES appears to attenuate reward-driven behavioral and brain responses.

Medial prefrontal dopamine dynamics reflect allocation of selective attention

The mesocortical dopamine system is comprised of midbrain dopamine neurons that predominantly innervate the medial prefrontal cortex (mPFC) and exert a powerful neuromodulatory influence over this region 1,2 . mPFC dopamine activity is thought to be critical for fundamental neurobiological processes including valence coding and decision-making 3,4 . Despite enduring interest in this pathway, the stimuli and conditions that engage mPFC dopamine release have remained enigmatic due to inherent limitations in conventional methods for dopamine monitoring which have prevented real-time in vivo observation 5 . Here, using a fluorescent dopamine sensor enabling time-resolved recordings of cortical dopamine activity in freely behaving mice, we reveal the coding properties of this system and demonstrate that mPFC dopamine dynamics conform to a selective attention signal. Contrary to the long-standing theory that mPFC dopamine release preferentially encodes aversive and stressful events 6-8 , we observed robust dopamine responses to both appetitive and aversive stimuli which dissipated with increasing familiarity irrespective of stimulus intensity. We found that mPFC dopamine does not evolve as a function of learning but displays striking temporal precedence with second-to-second changes in behavioral engagement, suggesting a role in allocation of attentional resources. Systematic manipulation of attentional demand revealed that quieting of mPFC dopamine signals the allocation of attentional resources towards an expected event which, upon detection triggers a sharp dopamine transient marking the transition from decision-making to action. The proposed role of mPFC dopamine as a selective attention signal is the first model based on direct observation of time-resolved dopamine dynamics and reconciles decades of competing theories.

An excitatory projection from the basal forebrain to the ventral tegmental area that underlies anorexia-like phenotypes

Maladaptation in balancing internal energy needs and external threat cues may result in eating disorders. However, brain mechanisms underlying such maladaptations remain elusive. Here, we identified that the basal forebrain (BF) sends glutamatergic projections to glutamatergic neurons in the ventral tegmental area (VTA) in mice. Glutamatergic neurons in both regions displayed correlated responses to various stressors. Notably, in vivo manipulation of BF terminals in the VTA revealed that the glutamatergic BF → VTA circuit reduces appetite, increases locomotion, and elicits avoidance. Consistently, activation of VTA glutamatergic neurons reduced body weight, blunted food motivation, and caused hyperactivity with behavioral signs of anxiety, all hallmarks of typical anorexia symptoms. Importantly, activation of BF glutamatergic terminals in the VTA reduced dopamine release in the nucleus accumbens. Collectively, our results point to overactivation of the glutamatergic BF → VTA circuit as a potential cause of anorexia-like phenotypes involving reduced dopamine release.

Nucleus accumbens dopamine release reflects the selective nature of pair bonds

In monogamous species, prosocial behaviors directed toward partners are dramatically different from those directed toward unknown individuals and potential threats. Dopamine release in the nucleus accumbens has a well-established role in social reward and motivation, but how this mechanism may be engaged to drive the highly divergent social behaviors directed at a partner or unfamiliar conspecific remains unknown. Using monogamous prairie voles, we first employed receptor pharmacology in partner preference and social operant tasks to show that dopamine is critical for the appetitive drive for social interaction but not for low-effort, unconditioned consummatory behaviors. We then leveraged the subsecond temporal resolution of the fluorescent biosensor, GRABDA, to ask whether differential dopamine release might distinguish between partner and novel social access and interaction. We found that partner seeking, anticipation, and interaction resulted in more accumbal dopamine release than the same events directed toward a novel vole. Further, partner-associated dopamine release decreased after prolonged partner separation. Our results are consistent with a model in which dopamine signaling plays a prominent role in the appetitive aspects of social interactions. Within this framework, differences in partner- and novel-associated dopamine release reflect the selective nature of pair bonds and may drive the partner- and novel-directed social behaviors that reinforce and cement bonds over time. This provides a potential mechanism by which highly conserved reward systems can enable selective, species-appropriate social behaviors.

Dynamics Learning Rate Bias in Pigeons: Insights from Reinforcement Learning and Neural Correlates

Research in reinforcement learning indicates that animals respond differently to positive and negative reward prediction errors, which can be calculated by assuming learning rate bias. Many studies have shown that humans and other animals have learning rate bias during learning, but it is unclear whether and how the bias changes throughout the entire learning process. Here, we recorded the behavior data and the local field potentials (LFPs) in the striatum of five pigeons performing a probabilistic learning task. Reinforcement learning models with and without learning rate biases were used to dynamically fit the pigeons' choice behavior and estimate the option values. Furthemore, the correlation between the striatal LFPs power and the model-estimated option values was explored. We found that the pigeons' learning rate bias shifted from negative to positive during the learning process, and the striatal Gamma (31 to 80 Hz) power correlated with the option values modulated by dynamic learning rate bias. In conclusion, our results support the hypothesis that pigeons employ a dynamic learning strategy in the learning process from both behavioral and neural aspects, providing valuable insights into reinforcement learning mechanisms of non-human animals.

Prediction error in dopamine neurons during associative learning

Dopamine neurons have long been thought to facilitate learning by broadcasting reward prediction error (RPE), a teaching signal used in machine learning, but more recent work has advanced alternative models of dopamine's computational role. Here, I revisit this critical issue and review new experimental evidences that tighten the link between dopamine activity and RPE. First, I introduce the recent observation of a gradual backward shift of dopamine activity that had eluded researchers for over a decade. I also discuss several other findings, such as dopamine ramping, that were initially interpreted to conflict but later found to be consistent with RPE. These findings improve our understanding of neural computation in dopamine neurons.

Impact of movie and video game elements on tic manifestation in children

BACKGROUND AND PURPOSE

Children in developed countries spend a significant portion of their waking hours engaging with audiovisual content and video games. The impact of media consumption on children's health and well-being has been widely studied, including its effects on tic disorders. Previous studies have shown that tic frequency can both increase and decrease during activities like gaming and television watching, resulting in mixed findings.

METHODS

To better understand the impact of audiovisual media on tics, we conducted a fine-grained tic manifestation analysis. We focused on the effects of the impact of a movie scene with suspensful elements and a video game designed to heighten anticipation, thought to stimulate phasic and striatal dopamine release. We closely monitored tic frequency throuhghout these experiences based on moment-to-moment tic annotation. The study included 20 participants (19 males aged 7-16) diagnosed with tic disorders (Yale Global Tic Severity Scale≥8), and we tested the replicability of our findings with an independent group of 36 children (15 females, aged 7-15) with tic disorders.

RESULTS

During film viewing, we observed significant synchronization in the temporal tic patterns of various individuals despite diversity in their tic profiles. Furthermore, employing a video game developed for our study, we found that tic frequency increases during anticipation of a pending reward. This finding was replicated in a second experiment with an independent cohort.

CONCLUSIONS

Our results indicate that tic frequency is affected by media elements in the short-term, and call for further investigation of the long-term impacts of exposure to such tic triggers.

Using rodent data to elucidate dopaminergic mechanisms of ADHD: Implications for human personality

An altered behavioral response to positive reinforcement has been proposed to be a core deficit in attention deficit hyperactivity disorder (ADHD). The spontaneously hypertensive rat (SHR), a congenic animal strain, displays a similarly altered response to reinforcement. The presence of this genetically determined phenotype in a rodent model allows experimental investigation of underlying neural mechanisms. Behaviorally, the SHR displays increased preference for immediate reinforcement, increased sensitivity to individual instances of reinforcement relative to integrated reinforcement history, and a steeper delay of reinforcement gradient compared to other rat strains. The SHR also shows less development of incentive to approach sensory stimuli, or cues, that predict reward after repeated cue-reward pairing. We consider the underlying neural mechanisms for these characteristics. It is well known that midbrain dopamine neurons are initially activated by unexpected reward and gradually transfer their responses to reward-predicting cues. This finding has inspired the dopamine transfer deficit (DTD) hypothesis, which predicts certain behavioral effects that would arise from a deficient transfer of dopamine responses from actual rewards to reward-predicting cues. We argue that the DTD predicts the altered responses to reinforcement seen in the SHR and individuals with ADHD. These altered responses to reinforcement in turn predict core symptoms of ADHD. We also suggest that variations in the degree of dopamine transfer may underlie variations in personality dimensions related to altered reinforcement sensitivity. In doing so, we highlight the value of rodent models to the study of human personality.

Levodopa reduces consumption of multiple classes of addictive substances in rats

Dopamine transmission is implicated in aberrant behaviors associated with substance use disorders. Previous research revealed a causal link between excessive drug consumption and the loss of dopamine signaling to stimuli associated with psychostimulant use. The emerging change in dopamine signaling is specific to stimuli associated with the substance rather than the pharmacological properties of the drug itself. Because the change in dopamine signaling was specific to the associated stimuli and not the pharmacological properties of the substance, we examined if treatment with the dopamine precursor, l-DOPA, alters alcohol and opioid self-administration. Therefore, we trained rats to orally self-administer ethanol or the synthetic opioid fentanyl and found that treating animals with l-DOPA significantly reduced consumption of both alcohol and fentanyl. These data suggest dopamine signaling has a vital role in mediating the amount of drug animals will voluntarily take, across multiple classes of drugs. Importantly, these data are preclinical demonstrations of l-DOPA being utilized as a harm reducing treatment in substance use disorders.

Prefrontal Dopamine in Flexible Adaptation to Environmental Changes: A Game for Two Players

Deficits in cognitive flexibility have been characterized in affective, anxiety, and neurodegenerative disorders. This paper reviews data, mainly from studies on animal models, that support the existence of a cortical-striatal brain circuit modulated by dopamine (DA), playing a major role in cognitive/behavioral flexibility. Moreover, we reviewed clinical findings supporting misfunctioning of this circuit in Parkinson's disease that could be responsible for some important non-motoric symptoms. The reviewed findings point to a role of catecholaminergic transmission in the medial prefrontal cortex (mpFC) in modulating DA's availability in the nucleus accumbens (NAc), as well as a role of NAc DA in modulating the motivational value of natural and conditioned stimuli. The review section is accompanied by a preliminary experiment aimed at testing weather the extinction of a simple Pavlovian association fosters increased DA transmission in the mpFC and inhibition of DA transmission in the NAc.

Neuronal implementation of the temporal difference learning algorithm in the midbrain dopaminergic system

The temporal difference learning (TDL) algorithm has been essential to conceptualizing the role of dopamine in reinforcement learning (RL). Despite its theoretical importance, it remains unknown whether a neuronal implementation of this algorithm exists in the brain. Here, we provide an interpretation of the recently described signaling properties of ventral tegmental area (VTA) GABAergic neurons and show that a circuitry of these neurons implements the TDL algorithm. Specifically, we identified the neuronal mechanism of three key components of the TDL model: a sustained state value signal encoded by an afferent input to the VTA, a temporal differentiation circuit formed by two types of VTA GABAergic neurons the combined output of which computes momentary reward prediction (RP) as the derivative of the state value, and the computation of reward prediction errors (RPEs) in dopamine neurons utilizing the output of the differentiation circuit. Using computational methods, we also show that this mechanism is optimally adapted to the biophysics of RPE signaling in dopamine neurons, mechanistically links the emergence of conditioned reinforcement to RP, and can naturally account for the temporal discounting of reinforcement. Elucidating the implementation of the TDL algorithm may further the investigation of RL in biological and artificial systems.

Contexts facilitate dynamic value encoding in the mesolimbic dopamine system

Adaptive behavior in a dynamic environment often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical test to theories of dopamine's function in learning, motivation, and motor control. Yet how dopamine neurons are involved in the revaluation of cues when the world changes to alter our behavior remains unclear. Here we make use of pharmacology, in vivo electrophysiology, fiber photometry, and optogenetics to resolve the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward-seeking. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by sucrose reward by exploiting the prior, non-overlapping presentation of a separate discrete cue - an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery. As a result, in this task we were able to dissociate the average value of the conditioned stimulus from its immediate expected value on a trial-to-trial basis. Both the activity of ventral tegmental area dopamine neurons and dopamine signaling in the nucleus accumbens were essential for rats to successfully update behavioral responding in response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward. Downstream of dopamine release, we found that single neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together these results reveal a novel mechanism within the mesolimbic dopamine system for the rapid revaluation of motivation.

Acute inhibition of hunger-sensing AgRP neurons promotes context-specific learning in mice

OBJECTIVE

An environmental context, which reliably predicts food availability, can increase the appetitive food drive within the same environment context. However, hunger is required for the development of such a context-induced feeding (CIF) response, suggesting the neural circuits sensitive to hunger link an internal energy state with a particular environment context. Since Agouti related peptide (AgRP) neurons are activated by energy deficit, we hypothesised that AgRP neurons are both necessary and sufficient to drive CIF.

METHODS

To examine the role of AgRP neurons in the CIF process, we used fibre photometry with GCaMP7f, chemogenetic activation of AgRP neurons, as well as optogenetic control of AgRP neurons to facilitate acute temporal control not permitted with chemogenetics.

RESULTS

A CIF response at test was only observed when mice were fasted during context training and AgRP population activity at test showed an attenuated inhibitory response to food, suggesting increased food-seeking and/or decreased satiety signalling drives the increased feeding response at test. Intriguingly, chemogenetic activation of AgRP neurons during context training did not increase CIF, suggesting precise temporal firing properties may be required. Indeed, termination of AgRP neuronal photostimulation during context training (ON-OFF in context), in the presence or absence of food, increased CIF. Moreover, photoinhibition of AgRP neurons during context training in fasted mice was sufficient to drive a subsequent CIF in the absence of food.

CONCLUSIONS

Our results suggest that AgRP neurons regulate the acquisition of CIF when the acute inhibition of AgRP activity is temporally matched to context exposure. These results establish acute AgRP inhibition as a salient neural event underscoring the effect of hunger on associative learning.

Reframing dopamine: A controlled controller at the limbic-motor interface

Pavlovian influences notoriously interfere with operant behaviour. Evidence suggests this interference sometimes coincides with the release of the neuromodulator dopamine in the nucleus accumbens. Suppressing such interference is one of the targets of cognitive control. Here, using the examples of active avoidance and omission behaviour, we examine the possibility that direct manipulation of the dopamine signal is an instrument of control itself. In particular, when instrumental and Pavlovian influences come into conflict, dopamine levels might be affected by the controlled deployment of a reframing mechanism that recasts the prospect of possible punishment as an opportunity to approach safety, and the prospect of future reward in terms of a possible loss of that reward. We operationalize this reframing mechanism and fit the resulting model to rodent behaviour from two paradigmatic experiments in which accumbens dopamine release was also measured. We show that in addition to matching animals' behaviour, the model predicts dopamine transients that capture some key features of observed dopamine release at the time of discriminative cues, supporting the idea that modulation of this neuromodulator is amongst the repertoire of cognitive control strategies.

Distinct sub-second dopamine signaling in dorsolateral striatum measured by a genetically-encoded fluorescent sensor

The development of genetically encoded dopamine sensors such as dLight has provided a new approach to measuring slow and fast dopamine dynamics both in brain slices and in vivo, possibly enabling dopamine measurements in areas like the dorsolateral striatum (DLS) where previously such recordings with fast-scan cyclic voltammetry (FSCV) were difficult. To test this, we first evaluated dLight photometry in mouse brain slices with simultaneous FSCV and found that both techniques yielded comparable results, but notable differences in responses to dopamine transporter inhibitors, including cocaine. We then used in vivo fiber photometry with dLight in mice to examine responses to cocaine in DLS. We also compared dopamine responses during Pavlovian conditioning across the striatum. We show that dopamine increases were readily detectable in DLS and describe transient dopamine kinetics, as well as slowly developing signals during conditioning. Overall, our findings indicate that dLight photometry is well suited to measuring dopamine dynamics in DLS.

Improved dual-color GRAB sensors for monitoring dopaminergic activity in vivo

Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a vast network of dopaminergic projections. To fully dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent GPCR activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity, and signal-to-noise properties with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in freely moving mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala, and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.

Disentangling the role of NAc D1 and D2 cells in hedonic eating

Overeating is driven by both the hedonic component ('liking') of food, and the motivation ('wanting') to eat it. The nucleus accumbens (NAc) is a key brain center implicated in these processes, but how distinct NAc cell populations encode 'liking' and 'wanting' to shape overconsumption remains unclear. Here, we probed the roles of NAc D1 and D2 cells in these processes using cell-specific recording and optogenetic manipulation in diverse behavioral paradigms that disentangle reward traits of 'liking' and 'wanting' related to food choice and overeating in healthy mice. Medial NAc shell D2 cells encoded experience-dependent development of 'liking', while D1 cells encoded innate 'liking' during the first food taste. Optogenetic control confirmed causal links of D1 and D2 cells to these aspects of 'liking'. In relation to 'wanting', D1 and D2 cells encoded and promoted distinct aspects of food approach: D1 cells interpreted food cues while D2 cells also sustained food-visit-length that facilitates consumption. Finally, at the level of food choice, D1, but not D2, cell activity was sufficient to switch food preference, programming subsequent long-lasting overconsumption. By revealing complementary roles of D1 and D2 cells in consumption, these findings assign neural bases to 'liking' and 'wanting' in a unifying framework of D1 and D2 cell activity.

Frontal tDCS reduces alcohol relapse rates by increasing connections from left dorsolateral prefrontal cortex to addiction networks

BACKGROUND

Brain-based interventions are needed to address persistent relapse in alcohol use disorder (AUD). Neuroimaging evidence suggests higher frontal connectivity as well as higher within-network connectivity of theoretically defined addiction networks are associated with reduced relapse rates and extended abstinence during follow-up periods.

OBJECTIVE

/Hypothesis: A longitudinal randomized double-blind sham-controlled clinical trial investigated whether a non-invasive neuromodulation intervention delivered during early abstinence can (i) modulate connectivity of addiction networks supporting abstinence and (ii) improve relapse rates.

HYPOTHESES

Active transcranial direct current stimulation (tDCS) will (i) increase connectivity of addiction networks known to support abstinence and (ii) reduce relapse rates.

METHODS

Short-term abstinent AUD participants (n = 60) were assigned to 5 days of either active tDCS or sham during cognitive training. Causal discovery analysis (CDA) examined the directional influence from left dorsolateral prefrontal cortex (LDLPFC, stimulation site) to addiction networks that support abstinence.

RESULTS

Active tDCS had an effect on the average strength of CDA-determined connectivity from LDLPFC to the incentive salience and negative emotionality addiction networks - increasing in the active tDCS group only. Active tDCS had an effect on relapse rates following the intervention, with lower probability of relapse in the active tDCS vs. sham. Active tDCS showed an unexpected sex-dependent effect on relapse rates.

CONCLUSION

Our results suggest that LDLPFC stimulation delivered during early abstinence has an effect on addiction networks supporting abstinence and on relapse rates. The unexpected sex-dependent neuromodulation effects need to be further examined in larger clinical trials.

Decreased Ventral Tegmental Area CB1R Signaling Reduces Sign Tracking and Shifts Cue-Outcome Dynamics in Rat Nucleus Accumbens

Sign-tracking (ST) rats show enhanced cue sensitivity before drug experience that predicts greater discrete cue-induced drug seeking compared with goal-tracking or intermediate rats. Cue-evoked dopamine in the nucleus accumbens (NAc) is a neurobiological signature of sign-tracking behaviors. Here, we examine a critical regulator of the dopamine system, endocannabinoids, which bind the cannabinoid receptor-1 (CB1R) in the ventral tegmental area (VTA) to control cue-evoked striatal dopamine levels. We use cell type-specific optogenetics, intra-VTA pharmacology, and fiber photometry to test the hypothesis that VTA CB1R receptor signaling regulates NAc dopamine levels to control sign tracking. We trained male and female rats in a Pavlovian lever autoshaping (PLA) task to determine their tracking groups before testing the effect of VTA → NAc dopamine inhibition. We found that this circuit is critical for mediating the vigor of the ST response. Upstream of this circuit, intra-VTA infusions of rimonabant, a CB1R inverse agonist, during PLA decrease lever and increase food cup approach in sign-trackers. Using fiber photometry to measure fluorescent signals from a dopamine sensor, GRABDA (AAV9-hSyn-DA2m), we tested the effects of intra-VTA rimonabant on NAc dopamine dynamics during autoshaping in female rats. We found that intra-VTA rimonabant decreased sign-tracking behaviors, which was associated with increases in NAc shell, but not core, dopamine levels during reward delivery [unconditioned stimulus (US)]. Our results suggest that CB1R signaling in the VTA influences the balance between the conditioned stimulus-evoked and US-evoked dopamine responses in the NAc shell and biases behavioral responding to cues in sign-tracking rats.SIGNIFICANCE STATEMENT Substance use disorder (SUD) is a chronically relapsing psychological disorder that affects a subset of individuals who engage in drug use. Recent research suggests that there are individual behavioral and neurobiological differences before drug experience that predict SUD and relapse vulnerabilities. Here, we investigate how midbrain endocannabinoids regulate a brain pathway that is exclusively involved in driving cue-motivated behaviors of sign-tracking rats. This work contributes to our mechanistic understanding of individual vulnerabilities to cue-triggered natural reward seeking that have relevance for drug-motivated behaviors.

Distinct subpopulations of D1 medium spiny neurons exhibit unique transcriptional responsiveness to cocaine

Drugs of abuse increase extracellular concentrations of dopamine in the nucleus accumbens (NAc), resulting in transcriptional alterations that drive long-lasting cellular and behavioral adaptations. While decades of research have focused on the transcriptional mechanisms by which drugs of abuse influence neuronal physiology and function, few studies have comprehensively defined NAc cell type heterogeneity in transcriptional responses to drugs of abuse. Here, we used single nucleus RNA-seq (snRNA-seq) to characterize the transcriptome of over 39,000 NAc cells from male and female adult Sprague-Dawley rats following acute or repeated cocaine experience. This dataset identified 16 transcriptionally distinct cell populations, including two populations of medium spiny neurons (MSNs) that express the Drd1 dopamine receptor (D1-MSNs). Critically, while both populations expressed classic marker genes of D1-MSNs, only one population exhibited a robust transcriptional response to cocaine. Validation of population-selective transcripts using RNA in situ hybridization revealed distinct spatial compartmentalization of these D1-MSN populations within the NAc. Finally, analysis of published NAc snRNA-seq datasets from non-human primates and humans demonstrated conservation of MSN subtypes across rat and higher order mammals, and further highlighted cell type-specific transcriptional differences across the NAc and broader striatum. These results highlight the utility in using snRNA-seq to characterize both cell type heterogeneity and cell type-specific responses to cocaine and provides a useful resource for cross-species comparisons of NAc cell composition.

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement

Reward seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement, and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations in male and female rats, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit, whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is nonscalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies.SIGNIFICANCE STATEMENT Dopamine neurons in the midbrain are essential for learning, motivation, and movement. Here we describe in detail the ability of VTA and SNc dopamine neurons to generate instrumental reinforcement, a process where an agent learns about actions they can emit to earn reward. While rats will avidly work and learn to respond for activation of VTA and SNc dopamine neurons, we find that only VTA dopamine neurons imbue actions and their associated cues with motivational value that spur continued pursuit of reward. Our data support a hypothesis that VTA and SNc dopamine neurons engage distinct psychological processes that have consequences for our understanding of these neurons in health and disease.

Habitual daily intake of a sweet and fatty snack modulates reward processing in humans

Western diets rich in fat and sugar promote excess calorie intake and weight gain; however, the underlying mechanisms are unclear. Despite a well-documented association between obesity and altered brain dopamine function, it remains elusive whether these alterations are (1) pre-existing, increasing the individual susceptibility to weight gain, (2) secondary to obesity, or (3) directly attributable to repeated exposure to western diet. To close this gap, we performed a randomized, controlled study (NCT05574660) with normal-weight participants exposed to a high-fat/high-sugar snack or a low-fat/low-sugar snack for 8 weeks in addition to their regular diet. The high-fat/high-sugar intervention decreased the preference for low-fat food while increasing brain response to food and associative learning independent of food cues or reward. These alterations were independent of changes in body weight and metabolic parameters, indicating a direct effect of high-fat, high-sugar foods on neurobehavioral adaptations that may increase the risk for overeating and weight gain.

Dopamine error signal to actively cope with lack of expected reward

To obtain more of a particular uncertain reward, animals must learn to actively overcome the lack of reward and adjust behavior to obtain it again. The neural mechanisms underlying such coping with reward omission remain unclear. Here, we developed a task in rats to monitor active behavioral switch toward the next reward after no reward. We found that some dopamine neurons in the ventral tegmental area exhibited increased responses to unexpected reward omission and decreased responses to unexpected reward, following the opposite responses of the well-known dopamine neurons that signal reward prediction error (RPE). The dopamine increase reflected in the nucleus accumbens correlated with behavioral adjustment to actively overcome unexpected no reward. We propose that these responses signal error to actively cope with lack of expected reward. The dopamine error signal thus cooperates with the RPE signal, enabling adaptive and robust pursuit of uncertain reward to ultimately obtain more reward.