Striatal dopamine signals are region specific and temporally stable across action-sequence habit formation

Striosomes Target Nigral Dopamine-Containing Neurons via Direct-D1 and Indirect-D2 Pathways Paralleling Classic Direct-Indirect Basal Ganglia Systems

Balanced activity of canonical direct D1 and indirect D2 basal ganglia pathways is considered a core requirement for normal movement, and their imbalance is an etiologic factor in movement and neuropsychiatric disorders. We present evidence for a conceptually equivalent pair of direct-D1 and indirect-D2 pathways that arise from striatal projection neurons (SPNs) of the striosome compartment rather than from SPNs of the matrix, as do the canonical pathways. These S-D1 and S-D2 striosomal pathways target substantia nigra dopamine-containing neurons instead of basal ganglia motor output nuclei. They modulate movement oppositely to the modulation by the canonical pathways: S-D1 is inhibitory and S-D2 is excitatory. The S-D1 and S-D2 circuits likely influence motivation for learning and action, complementing and reorienting canonical pathway modulation. A major conceptual reformulation of the classic direct-indirect pathway model of basal ganglia function is needed, as well as reconsideration of the effects of D2-targeting therapeutic drugs.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Value-Driven Adaptations of Mesolimbic Dopamine Release Are Governed by Both Model-Based and Model-Free Mechanisms

The magnitude of dopamine signals elicited by rewarding events and their predictors is updated when reward value changes. It is actively debated how readily these dopamine signals adapt and whether adaptation aligns with model-free or model-based reinforcement-learning principles. To investigate this, we trained male rats in a pavlovian-conditioning paradigm and measured dopamine release in the nucleus accumbens core in response to food reward (unconditioned stimulus) and reward-predictive conditioned stimuli (CS), both before and after reward devaluation, induced via either sensory-specific or nonspecific satiety. We demonstrate that (1) such devaluation reduces CS-induced dopamine release rapidly, without additional pairing of CS with devalued reward and irrespective of whether the devaluation was sensory-specific or nonspecific. In contrast, (2) reward devaluation did not decrease food reward-induced dopamine release. Surprisingly, (3) postdevaluation reconditioning, by additional pairing of CS with devalued reward, rapidly reinstated CS-induced dopamine signals to predevaluation levels. Taken together, we identify distinct, divergent adaptations in dopamine-signal magnitude when reward value is decreased: CS dopamine diminishes but reinstates fast, whereas reward dopamine is resistant to change. This implies that, respective to abovementioned findings, (1) CS dopamine may be governed by a model-based mechanism and (2) reward dopamine by a model-free one, where (3) the latter may contribute to swift reinstatement of the former. However, changes in CS dopamine were not selective for sensory specificity of reward devaluation, which is inconsistent with model-based processes. Thus, mesolimbic dopamine signaling incorporates both model-free and model-based mechanisms and is not exclusively governed by either.

Rethinking dopamine-guided action sequence learning

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.

Synchronous Measurements of Extracellular Action Potentials and Neurochemical Activity with Carbon Fiber Electrodes in Nonhuman Primates

Measuring the dynamic relationship between neuromodulators, such as dopamine, and neuronal action potentials is imperative to understand how these fundamental modes of neural signaling interact to mediate behavior. We developed methods to measure concurrently dopamine and extracellular action potentials (i.e., spikes) in monkeys. Standard fast-scan cyclic voltammetric (FSCV) electrochemical (EChem) and electrophysiological (EPhys) recording systems are combined and used to collect spike and dopamine signals, respectively, from an array of carbon fiber (CF) sensors implanted in the monkey striatum. FSCV requires the application of small voltages at the implanted sensors to measure redox currents generated from target molecules, such as dopamine. These applied voltages create artifacts at neighboring EPhys measurement sensors which may lead to misclassification of these signals as physiological spikes. Therefore, simple automated temporal interpolation algorithms were designed to remove these artifacts and enable accurate spike extraction. We validated these methods using simulated artifacts and demonstrated an average spike recovery rate of 84.5%. We identified and discriminated cell type-specific units in the monkey striatum that were shown to correlate to specific behavioral task parameters related to reward size and eye movement direction. Synchronously recorded spike and dopamine signals displayed contrasting relations to the task variables, suggesting a complex relationship between these two modes of neural signaling. Future application of our methods will help advance our understanding of the interactions between neuromodulator signaling and neuronal activity, to elucidate more detailed mechanisms of neural circuitry and plasticity mediating behaviors in health and in disease.

A role for the dorsolateral striatum in prospective action control

The dorsolateral striatum (DLS) is important for performing actions persistently, even when it becomes suboptimal, reflecting a function that is reflexive and habitual. However, there are also ways in which persistent behaviors can result from a more prospective, planning mode of behavior. To help tease apart these possibilities for DLS function, we trained animals to perform a lever press for reward and then inhibited the DLS in key test phases: as the task shifted from a 1-press to a 3-press rule (upshift), as the task was maintained, as the task shifted back to the one-press rule (downshift), and when rewards came independent of pressing. During DLS inhibition, animals always favored their initially learned strategy to press just once, particularly so during the free-reward period. DLS inhibition surprisingly changed performance speed bidirectionally depending on the task shifts. DLS inhibition thus encouraged habitual behavior, suggesting it could normally help adapt to changing conditions.

What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Contribution of the retrosplenial cortex to route selection in a complex maze

The retrosplenial cortex (RSC) is a region involved in navigation. In this study, we investigated the role of the RSC in navigation in a large-scale environment where the destination is not visible from the current location. We used a large maze where the routes could be freely designed by inserting and removing plates. In Experiment 1, rats learned a specific route in the maze and then were tested with a shortcut route in addition to the learned route. The rats with RSC lesions utilized the shortcut faster than those in the control group. In Experiment 2, rats were initially trained to follow a specific route, and subsequently, we tested the effects of a small change in the environment on their route-following behavior. In the test, the rats with RSC lesions demonstrated more errors than those in the control group. This suggests that lesions in the RSC make navigation to a goal unstable. These findings suggest that the RSC may be involved in the ability to perform appropriate behavior at a segment on a learned route in a large-scale environment, which drives habitually following the learned route.

Dopamine across timescales and cell types: Relevance for phenotypes in Parkinson's disease progression

Dopamine neurons in the substantia nigra pars compacta (SNc) synthesize and release dopamine, a critical neurotransmitter for movement and learning. SNc dopamine neurons degenerate in Parkinson's Disease (PD), causing a host of motor and non-motor symptoms. Here, we review recent conceptual advances in our basic understanding of the dopamine system - including our rapidly advancing knowledge of dopamine neuron heterogeneity - with special attention to their importance for understanding PD. In PD patients, dopamine neuron degeneration progresses from lateral SNc to medial SNc, suggesting clinically relevant heterogeneity in dopamine neurons. With technical advances in dopamine system interrogation, we can understand the relevance of this heterogeneity for PD progression and harness it to develop new treatments.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Stimulus control of habits: Evidence for both stimulus specificity and devaluation insensitivity in a dual-response task

Goal-directed and habitual actions are clearly defined by their associative relations. Whereas goal-directed control can be confirmed via tests of outcome devaluation and contingency-degradation sensitivity, a comparable criterion for positively detecting habits has not been established. To confirm habitual responding, a test of control by the stimulus-response association is required while also ruling out goal-directed control. Here we describe an approach to developing such a test in rats using two discriminative stimuli that set the occasion for two different responses that then earn the same outcome. Performance was insensitive to outcome devaluation and showed stimulus-response specificity, indicative of stimulus-controlled behavior. The reliance of stimulus-response associations was further supported by a lack of sensitivity during the single extinction test session used here. These results demonstrate that two concurrently trained responses can come under habitual control when they share a common outcome. By reducing the ability of one stimulus to signal its corresponding response-outcome association, we found evidence for goal-directed control that can be dissociated from habits. Overall, these experiments provide evidence that tests assessing specific stimulus-response associations can be used to investigate habits.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

Antagonism of D2 receptors via raclopride ameliorates amphetamine-induced associative learning deficits in male mice

Dopamine levels in the dorsomedial striatum (DMS) are highly dynamic and are thought to underly the encoding of action-outcome associations. Although it is known that amphetamine disrupts the learning that is required for goal-directed action, the role of D1 and D2 receptors in this process has not been established. In this study, we examined the role of D1 and D2 receptor antagonists on learning in response to amphetamine. We used the outcome-specific devaluation task to examine goal-directed action in male C57BL6/J mice treated systemically with either a D1 antagonist (SCH-23990; 0.01 mg/kg) or a D2 antagonist (raclopride; 0.5 mg/kg) and then administered amphetamine (1 mg/kg). The mice were injected repeatedly throughout the instrumental training phase of the task to assess the impact on the learning of action-outcomes, and the subsequent choice test assessing performance of goal-directed action was conducted drug free. Effects of chronic drug administration on locomotor behaviour was assessed before and after the choice test. Treatment during learning with either amphetamine, or the D1 or D2 antagonists, impaired the subsequent performance of goal-directed action. The amphetamine-induced impairment in goal-directed action was reversed in mice treated with raclopride, but not when treated with SCH-23990. By contrast, amphetamine-induced hyperactivity was reversed in mice treated with SCH-23990, but not in mice treated with raclopride. Taken together, these data support the role of a balance of dopamine receptor signalling after amphetamine treatment. While overall D1 receptor availability is necessary to promote learning, in a state of elevated dopamine, modifying D2 receptor function can ameliorate learning deficits.

Distinct sub-second dopamine signaling in dorsolateral striatum measured by a genetically-encoded fluorescent sensor

The development of genetically encoded dopamine sensors such as dLight has provided a new approach to measuring slow and fast dopamine dynamics both in brain slices and in vivo, possibly enabling dopamine measurements in areas like the dorsolateral striatum (DLS) where previously such recordings with fast-scan cyclic voltammetry (FSCV) were difficult. To test this, we first evaluated dLight photometry in mouse brain slices with simultaneous FSCV and found that both techniques yielded comparable results, but notable differences in responses to dopamine transporter inhibitors, including cocaine. We then used in vivo fiber photometry with dLight in mice to examine responses to cocaine in DLS. We also compared dopamine responses during Pavlovian conditioning across the striatum. We show that dopamine increases were readily detectable in DLS and describe transient dopamine kinetics, as well as slowly developing signals during conditioning. Overall, our findings indicate that dLight photometry is well suited to measuring dopamine dynamics in DLS.

Revealing abrupt transitions from goal-directed to habitual behavior

A fundamental tenet of animal behavior is that decision-making involves multiple 'controllers.' Initially, behavior is goal-directed, driven by desired outcomes, shifting later to habitual control, where cues trigger actions independent of motivational state. Clark Hull's question from 1943 still resonates today: "Is this transition abrupt, or is it gradual and progressive?"1 Despite a century-long belief in gradual transitions, this question remains unanswered2,3 as current methods cannot disambiguate goal-directed versus habitual control in real-time. Here, we introduce a novel 'volitional engagement' approach, motivating animals by palatability rather than biological need. Offering less palatable water in the home cage4,5 reduced motivation to 'work' for plain water in an auditory discrimination task when compared to water-restricted animals. Using quantitative behavior and computational modeling6, we found that palatability-driven animals learned to discriminate as quickly as water-restricted animals but exhibited state-like fluctuations when responding to the reward-predicting cue-reflecting goal-directed behavior. These fluctuations spontaneously and abruptly ceased after thousands of trials, with animals now always responding to the reward-predicting cue. In line with habitual control, post-transition behavior displayed motor automaticity, decreased error sensitivity (assessed via pupillary responses), and insensitivity to outcome devaluation. Bilateral lesions of the habit-related dorsolateral striatum7 blocked transitions to habitual behavior. Thus, 'volitional engagement' reveals spontaneous and abrupt transitions from goal-directed to habitual behavior, suggesting the involvement of a higher-level process that arbitrates between the two.

Spontaneous behaviour is structured by reinforcement without explicit reward

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.

In vivo overexpression of synaptogyrin-3 promotes striatal synaptic dopamine uptake in LRRK2R1441G mutant mouse model of Parkinson's disease

BACKGROUND

Leucine-rich repeat kinase 2 (LRRK2) mutation is a common genetic risk factor of Parkinson's disease (PD). Presynaptic dysfunction is an early pathogenic event associated with dopamine (DA) dysregulation in striatum of the brain. DA uptake activity of DA uptake transporter (DAT) affects synaptic plasticity and motor and non-motor behavior. Synaptogyrin-3 (SYNGR3) is part of the synaptogyrin family, especially abundant in brain. Previous in vitro studies demonstrated interaction between SYNGR3 and DAT. Reduced SYNGR3 expression was observed in human PD brains with unclear reasons.

METHODS

Here, we further explored whether inducing SYNGR3 expression can influence (i) cellular DA uptake using differentiated human SH-SY5Y neuronal cells, (ii) striatal synaptosomal DA uptake in a mutant LRRK2^R1441G  knockin mouse model of PD, and (iii) innate rodent behavior using the marble burying test.

RESULTS

Young LRRK2 mutant mice exhibited significantly lower SYNGR3 levels in striatum compared to age-matched wild-type (WT) controls, resembling level in aged WT mice. SYNGR3 is spatially co-localized with DAT at striatal presynaptic terminals, visualized by immuno-gold transmission electron microscopy and immunohistochemistry. Their protein-protein interaction was confirmed by co-immunoprecipitation. Transient overexpression of SYNGR3 in differentiated SH-SY5Y cells increased cellular DA uptake activity without affecting total DAT levels. Inducing SYNGR3 overexpression by adeno-associated virus-7 (AAV7) injection in vivo into striatum increased ex vivo synaptosomal DA uptake in LRRK2 mutant mice and improved their innate marble burying behavior.

CONCLUSION

Brain SYNGR3 expression may be an important determinant to striatal DA homeostasis and synaptic function. Our preliminary behavioral test showed improved innate behavior after SYNGR3 overexpression in LRRK2 mutant mice, advocating further studies to determine the influence of SYNGR3 in the pathophysiology of DA neurons in PD.

Task history dictates how the dorsolateral striatum controls action strategy and vigor

The dorsolateral striatum (DLS) is linked to the learning and honing of action routines. However, the DLS is also important for performing behaviors that have been successful in the past. The learning function can be thought of as prospective, helping to plan ongoing actions to be efficient and often optimal. The performance function is more retrospective, helping the animal continue to behave in a way that had worked previously. How the DLS manages this all is curious. What happens when a learned behavior becomes sub-optimal due to environment changes. In this case, the prospective function of the DLS would cause animals to (adaptively) learn and plan more optimal actions. In contrast, the retrospective function would cause animals to (maladaptively) favor the old behavior. Here we find that, during a change in learned task rules, DLS inhibition causes animals to adjust less rapidly to the new task (and to behave less vigorously) in a 'maladaptive' way. Yet, when the task is changed back to the initially learned rules, DLS inhibition instead causes a rapid and vigorous adjustment of behavior in an 'adaptive' way. These results show that inhibiting the DLS biases behavior towards initially acquired strategies, implying a more retrospective outlook in action selection when the DLS is offline. Thus, an active DLS could encourage planning and learning action routines more prospectively. Moreover, the DLS control over behavior can appear to be either advantageous/flexible or disadvantageous/inflexible depending on task context, and its control over vigor can change depending on task context.

Significant Statement

Basal ganglia networks aid behavioral learning (a prospective planning function) but also favor the use of old behaviors (a retrospective performance function), making it unclear what happens when learned behaviors become suboptimal. Here we inhibit the dorsolateral striatum (DLS) as animals encounter a change in task rules, and again when they shift back to those learned task rules. DLS inhibition reduces adjustment to new task rules (and reduces behavioral vigor), but it increases adjustment back to the initially learned task rules later (and increases vigor). Thus, in both cases, DLS inhibition favored the use of the initially learned behavioral strategy, which could appear either maladaptive or adaptive. We suggest that the DLS might promote a prospective orientation of action control.

Mesolimbic dopamine release conveys causal associations

Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system-a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.

Lack of action monitoring as a prerequisite for habitual and chunked behavior: Behavioral and neural correlates

We previously reported the rapid development of habitual behavior in a discrete-trials instrumental task in which lever insertion and retraction act as reward-predictive cues delineating sequence execution. Here we asked whether lever cues or performance variables reflective of skill and automaticity might account for habitual behavior in male rats. Behavior in the discrete-trials habit-promoting task was compared with two task variants lacking the sequence-delineating cues of lever extension and retraction. We find that behavior is under goal-directed control in absence of sequence-delineating cues but not in their presence, and that skilled performance does not predict goal-directed vs. habitual behavior. Neural activity recordings revealed an engagement of dorsolateral striatum and a disengagement of dorsomedial striatum during the sequence execution of the habit-promoting task, specifically. Together, these results indicate that sequence delineation cues promote habit and differential engagement of striatal subregions during instrumental responding, a pattern that may reflect cue-elicited behavioral chunking.

Male DAT Val559 Mice Exhibit Compulsive Behavior under Devalued Reward Conditions Accompanied by Cellular and Pharmacological Changes

Identified across multiple psychiatric disorders, the dopamine (DA) transporter (DAT) Ala559Val substitution triggers non-vesicular, anomalous DA efflux (ADE), perturbing DA neurotransmission and behavior. We have shown that DAT Val559 mice display a waiting impulsivity and changes in cognitive performance associated with enhanced reward motivation. Here, utilizing a within-subject, lever-pressing paradigm designed to bias the formation of goal-directed or habitual behavior, we demonstrate that DAT Val559 mice modulate their nose poke behavior appropriately to match context, but demonstrate a perseverative checking behavior. Although DAT Val559 mice display no issues with the cognitive flexibility required to acquire and re-learn a visual pairwise discrimination task, devaluation of reward evoked habitual reward seeking in DAT Val559 mutants in operant tasks regardless of reinforcement schedule. The direct DA agonist apomorphine also elicits locomotor stereotypies in DAT Val559, but not WT mice. Our observation that dendritic spine density is increased in the dorsal medial striatum (DMS) of DAT Val559 mice speaks to an imbalance in striatal circuitry that might underlie the propensity of DAT Val559 mutants to exhibit compulsive behaviors when reward is devalued. Thus, DAT Val559 mice represent a model for dissection of how altered DA signaling perturbs circuits that normally balance habitual and goal-directed behaviors.

Nucleus accumbens dopamine tracks aversive stimulus duration and prediction but not value or prediction error

There is active debate on the role of dopamine in processing aversive stimuli, where inferred roles range from no involvement at all, to signaling an aversive prediction error (APE). Here, we systematically investigate dopamine release in the nucleus accumbens core (NAC), which is closely linked to reward prediction errors, in rats exposed to white noise (WN, a versatile, underutilized, aversive stimulus) and its predictive cues. Both induced a negative dopamine ramp, followed by slow signal recovery upon stimulus cessation. In contrast to reward conditioning, this dopamine signal was unaffected by WN value, context valence, or probabilistic contingencies, and the WN dopamine response shifted only partially toward its predictive cue. However, unpredicted WN provoked slower post-stimulus signal recovery than predicted WN. Despite differing signal qualities, dopamine responses to simultaneous presentation of rewarding and aversive stimuli were additive. Together, our findings demonstrate that instead of an APE, NAC dopamine primarily tracks prediction and duration of aversive events.

Making habits measurable beyond what they are not: A focus on associative dual-process models

Habits are the subject of intense international research. Under the associative dual-process model the outcome devaluation paradigm has been used extensively to classify behaviours as being either goal-directed (sensitive to shifts in the value of associated outcomes) or habitual (triggered by stimuli without anticipation of consequences). This has proven to be a useful framework for studying the neurobiology of habit and relevance of habits in clinical psychopathology. However, in recent years issues have been raised about this rather narrow definition of habits in comparison to habitual behaviour experienced in the real world. Specifically, defining habits as the absence of goal-directed control, the very specific set-ups required to demonstrate habit experimentally and the lack of direct evidence for habits as stimulus-response behaviours are viewed as problematic. In this review paper we address key critiques that have been raised about habit research within the framework of the associative dual-process model. We then highlight novel research approaches studying different features of habits with methods that expand beyond traditional paradigms.

A unidirectional but not uniform striatal landscape of dopamine signaling for motivational stimuli

SignificanceAlthough it is undisputed that striatal dopamine plays a prominent role in motivated behavior and learning, the precise information conveyed by dopamine signals as such is under active debate. For a long time, the idea dominated that dopamine encodes a reward prediction error and that this signal is broadcast uniformly throughout the brain. However, here, we capture dopamine dynamics across many striatal regions and demonstrate that dopamine release is, regionally, extremely heterogeneous and that a reward prediction error-like signal is predominantly found in the relatively small limbic domain of the striatum. Another striking organizing principle is that stimulus valence directs dopamine concentration homogeneously across all regions (i.e., appetitive stimuli increase dopamine and aversive stimuli decrease dopamine).

Dopamine signaling in the dorsomedial striatum promotes compulsive behavior

Compulsive behavior is a defining feature of disorders such as substance use disorders. Current evidence suggests that corticostriatal circuits control the expression of established compulsions, but little is known about the mechanisms regulating the development of compulsions. We hypothesized that dopamine, a critical modulator of striatal synaptic plasticity, could control alterations in corticostriatal circuits leading to the development of compulsions (defined here as continued reward seeking in the face of punishment). We used dual-site fiber photometry to measure dopamine axon activity in the dorsomedial striatum (DMS) and the dorsolateral striatum (DLS) as compulsions emerged. Individual variability in the speed with which compulsions emerged was predicted by DMS dopamine axon activity. Amplifying this dopamine signal accelerated animals' transitions to compulsion, whereas inhibition delayed it. In contrast, amplifying DLS dopamine signaling had no effect on the emergence of compulsions. These results establish DMS dopamine signaling as a key controller of the development of compulsive reward seeking.