Chemogenetic inhibition in the dorsal striatum reveals regional specificity of direct and indirect pathway control of action sequencing

Rethinking dopamine-guided action sequence learning

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.

Changes in dorsomedial striatum activity during expression of goal-directed vs. habit-like cue-induced cocaine seeking

A preclinical model of cue exposure therapy, cue extinction, reduces cue-induced cocaine seeking that is goal-directed but not habit-like. Goal-directed and habitual behaviors differentially rely on the dorsomedial striatum (DMS) and dorsolateral striatum (DLS), but the effects of cue extinction on dorsal striatal responses to cue-induced drug seeking are unknown. We used fiber photometry in rats trained to self-administer cocaine paired with an audiovisual cue to examine how dorsal striatal intracellular calcium and extracellular dopamine activity differs between goal-directed and habit-like cue-induced cocaine seeking and how it is impacted by cue extinction. After minimal fixed-ratio training, rats showed enhanced DMS and DLS calcium responses to cue-reinforced compared to unreinforced lever presses. After rats were trained on goal-promoting fixed ratio schedules or habit-promoting second-order schedules of reinforcement, different patterns of dorsal striatal calcium and dopamine responses to cue-reinforced lever presses emerged. Rats trained on habit-promoting second-order schedules showed reduced DMS calcium responses and enhanced DLS dopamine responses to cue-reinforced lever presses. Cue extinction reduced calcium responses during subsequent drug seeking in the DMS, but not in the DLS. Therefore, cue extinction may reduce goal-directed behavior through its effects on the DMS, whereas habit-like behavior and the DLS are unaffected.

Adaptation of sequential action benefits from timing variability related to lateral basal ganglia circuitry

Streamlined action sequences must remain flexible should stable contingencies in the environment change. By combining analyses of behavioral structure with a circuit-specific manipulation in mice, we report on a relationship between action timing variability and successful adaptation that relates to post-synaptic targets of primary motor cortical (M1) projections to dorsolateral striatum (DLS). In a two-lever instrumental task, mice formed successful action sequences by, first, establishing action scaffolds and, second, smoothly extending action duration to adapt to increased task requirements. Interruption of DLS neurons in M1 projection territories altered this process, evoking higher-rate actions that were more stereotyped in their timing, reducing opportunities for success. Based on evidence from neuronal tracing experiments, we propose that DLS neurons in M1 projection territories supply action timing variability to facilitate adaptation, a function that may involve additional downstream subcortical processing relating to collateralization of descending motor pathways to multiple basal ganglia centers.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Changes in dorsomedial striatum activity mediate expression of goal-directed vs. habit-like cue-induced cocaine seeking

A preclinical model of cue exposure therapy, cue extinction, reduces cue-induced cocaine seeking when drug seeking is goal-directed but not habitual. Goal-directed and habitual behaviors differentially rely on the dorsomedial striatum (DMS) and dorsolateral striatum (DLS), but the effects of cue extinction on dorsal striatal responses to cue-induced drug seeking are unknown. We used fiber photometry to examine how dorsal striatal intracellular calcium and extracellular dopamine activity differs between goal-directed and habitual cue-induced cocaine seeking and how it is impacted by cue extinction. Rats trained to self-administer cocaine paired with an audiovisual cue on schedules of reinforcement that promote goal-directed or habitual cocaine seeking had different patterns of dorsal striatal calcium and dopamine responses to cue-reinforced lever presses. Cue extinction reduced calcium and dopamine responses during subsequent drug seeking in the DMS, but not in the DLS. Therefore, cue extinction may reduce goal-directed behavior through its effects on the DMS, whereas habitual behavior and the DLS are unaffected.

Striatal indirect pathway mediates exploration via collicular competition

The ability to suppress actions that lead to a negative outcome and explore alternative actions is necessary for optimal decision making. Although the basal ganglia have been implicated in these processes^1-5, the circuit mechanisms underlying action selection and exploration remain unclear. Here, using a simple lateralized licking task, we show that indirect striatal projection neurons (iSPN) in the basal ganglia contribute to these processes through modulation of the superior colliculus (SC). Optogenetic activation of iSPNs suppresses contraversive licking and promotes ipsiversive licking. Activity in lateral superior colliculus (lSC), a region downstream of the basal ganglia, is necessary for task performance and predicts lick direction. Furthermore, iSPN activation suppresses ipsilateral lSC, but surprisingly excites contralateral lSC, explaining the emergence of ipsiversive licking. Optogenetic inactivation reveals inter-collicular competition whereby each hemisphere of the superior colliculus inhibits the other, thus allowing the indirect pathway to disinhibit the contralateral lSC and trigger licking. Finally, inactivating iSPNs impairs suppression of devalued but previously rewarded licking and reduces exploratory licking. Our results reveal that iSPNs engage the competitive interaction between lSC hemispheres to trigger a motor action and suggest a general circuit mechanism for exploration during action selection.

Chemogenetics drives paradigm change in the investigation of behavioral circuits and neural mechanisms underlying drug action

Recent developments in chemogenetic approaches to the investigation of brain function have ushered in a paradigm change in the strategy for drug and behavior research and clinical drug-based medications. As the nature of the drug action is based on humoral regulation, it is a challenge to identify the neuronal mechanisms responsible for the expression of certain targeted behavior induced by drug application. The development of chemogenetic approaches has allowed researchers to control neural activities in targeted neurons through a toolbox, including engineered G protein-coupled receptors or ligand-gated ion channels together with exogenously inert synthetic ligands. This review provides a brief overview of the chemogenetics toolbox with an emphasis on the DREADDs (Designer Receptors Exclusively Activated by Designer Drugs) technique used in rodent models, which is applicable to the investigation of how specific neural circuits regulate behavioral processes. The use of chemogenetics has had a significant impact on basic neuroscience for a better understanding of the relationships between brain activity and the expression of behaviors with cell- and circuit-specific orders. Furthermore, chemogenetics is potentially a useful tool to deconstruct the neuropathological mechanisms of mental diseases and its regulation by drug, and provide us with transformative therapeutics with medication. We also review recent findings in the use of chemogenetic techniques to uncover functional circuit connections of serotonergic neurons in rodent models.

Striatal Direct Pathway Targets Npas1+ Pallidal Neurons

The classic basal ganglia circuit model asserts a complete segregation of the two striatal output pathways. Empirical data argue that, in addition to indirect-pathway striatal projection neurons (iSPNs), direct-pathway striatal projection neurons (dSPNs) innervate the external globus pallidus (GPe). However, the functions of the latter were not known. In this study, we interrogated the organization principles of striatopallidal projections and their roles in full-body movement in mice (both males and females). In contrast to the canonical motor-promoting response of dSPNs in the dorsomedial striatum (DMSdSPNs), optogenetic stimulation of dSPNs in the dorsolateral striatum (DLSdSPNs) suppressed locomotion. Circuit analyses revealed that dSPNs selectively target Npas1+ neurons in the GPe. In a chronic 6-hydroxydopamine lesion model of Parkinson's disease, the dSPN-Npas1+ projection was dramatically strengthened. As DLSdSPN-Npas1+ projection suppresses movement, the enhancement of this projection represents a circuit mechanism for the hypokinetic symptoms of Parkinson's disease that has not been previously considered. In sum, our results suggest that dSPN input to the GPe is a critical circuit component that is involved in the regulation of movement in both healthy and parkinsonian states.SIGNIFICANCE STATEMENT In the classic basal ganglia model, the striatum is described as a divergent structure: it controls motor and adaptive functions through two segregated, opposing output streams. However, the experimental results that show the projection from direct-pathway neurons to the external pallidum have been largely ignored. Here, we showed that this striatopallidal subpathway targets a select subset of neurons in the external pallidum and is motor-suppressing. We found that this subpathway undergoes changes in a Parkinson's disease model. In particular, our results suggest that the increase in strength of this subpathway contributes to the slowness or reduced movements observed in Parkinson's disease.

Orbitofrontal-striatal potentiation underlies cocaine-induced hyperactivity

Psychomotor stimulants increase dopamine levels in the striatum and promote locomotion; however, their effects on striatal pathway function in vivo remain unclear. One model that has been proposed to account for these motor effects suggests that stimulants drive hyperactivity via activation and inhibition of direct and indirect pathway striatal neurons, respectively. Although this hypothesis is consistent with the cellular actions of dopamine receptors and received support from optogenetic and chemogenetic studies, it has been rarely tested with in vivo recordings. Here, we test this model and observe that cocaine increases the activity of both pathways in the striatum of awake mice. These changes are linked to a dopamine-dependent cocaine-induced strengthening of upstream orbitofrontal cortex (OFC) inputs to the dorsomedial striatum (DMS) in vivo. Finally, depressing OFC-DMS pathway with a high frequency stimulation protocol in awake mice over-powers the cocaine-induced potentiation of OFC-DMS pathway and attenuates the expression of locomotor sensitization, directly linking OFC-DMS potentiation to cocaine-induced hyperactivity.