Neuronal correlates of instrumental learning in the dorsal striatum

Changes in dorsomedial striatum activity during expression of goal-directed vs. habit-like cue-induced cocaine seeking

A preclinical model of cue exposure therapy, cue extinction, reduces cue-induced cocaine seeking that is goal-directed but not habit-like. Goal-directed and habitual behaviors differentially rely on the dorsomedial striatum (DMS) and dorsolateral striatum (DLS), but the effects of cue extinction on dorsal striatal responses to cue-induced drug seeking are unknown. We used fiber photometry in rats trained to self-administer cocaine paired with an audiovisual cue to examine how dorsal striatal intracellular calcium and extracellular dopamine activity differs between goal-directed and habit-like cue-induced cocaine seeking and how it is impacted by cue extinction. After minimal fixed-ratio training, rats showed enhanced DMS and DLS calcium responses to cue-reinforced compared to unreinforced lever presses. After rats were trained on goal-promoting fixed ratio schedules or habit-promoting second-order schedules of reinforcement, different patterns of dorsal striatal calcium and dopamine responses to cue-reinforced lever presses emerged. Rats trained on habit-promoting second-order schedules showed reduced DMS calcium responses and enhanced DLS dopamine responses to cue-reinforced lever presses. Cue extinction reduced calcium responses during subsequent drug seeking in the DMS, but not in the DLS. Therefore, cue extinction may reduce goal-directed behavior through its effects on the DMS, whereas habit-like behavior and the DLS are unaffected.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Changes in dorsomedial striatum activity mediate expression of goal-directed vs. habit-like cue-induced cocaine seeking

A preclinical model of cue exposure therapy, cue extinction, reduces cue-induced cocaine seeking when drug seeking is goal-directed but not habitual. Goal-directed and habitual behaviors differentially rely on the dorsomedial striatum (DMS) and dorsolateral striatum (DLS), but the effects of cue extinction on dorsal striatal responses to cue-induced drug seeking are unknown. We used fiber photometry to examine how dorsal striatal intracellular calcium and extracellular dopamine activity differs between goal-directed and habitual cue-induced cocaine seeking and how it is impacted by cue extinction. Rats trained to self-administer cocaine paired with an audiovisual cue on schedules of reinforcement that promote goal-directed or habitual cocaine seeking had different patterns of dorsal striatal calcium and dopamine responses to cue-reinforced lever presses. Cue extinction reduced calcium and dopamine responses during subsequent drug seeking in the DMS, but not in the DLS. Therefore, cue extinction may reduce goal-directed behavior through its effects on the DMS, whereas habitual behavior and the DLS are unaffected.

Huntingtin recruits KIF1A to transport synaptic vesicle precursors along the mouse axon to support synaptic transmission and motor skill learning

Neurotransmitters are released at synapses by synaptic vesicles (SVs), which originate from SV precursors (SVPs) that have traveled along the axon. Because each synapse maintains a pool of SVs, only a small fraction of which are released, it has been thought that axonal transport of SVPs does not affect synaptic function. Here, studying the corticostriatal network both in microfluidic devices and in mice, we find that phosphorylation of the Huntingtin protein (HTT) increases axonal transport of SVPs and synaptic glutamate release by recruiting the kinesin motor KIF1A. In mice, constitutive HTT phosphorylation causes SV over-accumulation at synapses, increases the probability of SV release, and impairs motor skill learning on the rotating rod. Silencing KIF1A in these mice restored SV transport and motor skill learning to wild-type levels. Axonal SVP transport within the corticostriatal network thus influences synaptic plasticity and motor skill learning.

Learning under social versus nonsocial uncertainty: A meta-analytic approach

Much of the uncertainty that clouds our understanding of the world springs from the covert values and intentions held by other people. Thus, it is plausible that specialized mechanisms that compute learning signals under uncertainty of exclusively social origin operate in the brain. To test this hypothesis, we scoured academic databases for neuroimaging studies involving learning under uncertainty, and performed a meta‐analysis of brain activation maps that compared learning in the face of social versus nonsocial uncertainty. Although most of the brain activations associated with learning error signals were shared between social and nonsocial conditions, we found some evidence for functional segregation of error signals of exclusively social origin during learning in limited regions of ventrolateral prefrontal cortex and insula. This suggests that most behavioral adaptations to navigate social environments are reused from frontal and subcortical areas processing generic value representation and learning, but that a specialized circuitry might have evolved in prefrontal regions to deal with social context representation and strategic action.

Striatum expresses region-specific plasticity consistent with distinct memory abilities

The striatum mediates two learning modalities: goal-directed behavior in dorsomedial (DMS) and habits in dorsolateral (DLS) striata. The synaptic bases of these learnings are still elusive. Indeed, while ample research has described DLS plasticity, little remains known about DMS plasticity and its involvement in procedural learning. Here, we find symmetric and asymmetric anti-Hebbian spike-timing-dependent plasticity (STDP) in DMS and DLS, respectively, with opposite plasticity dominance upon increasing corticostriatal activity. During motor-skill learning, plasticity is engaged in DMS and striatonigral DLS neurons only during early learning stages, whereas striatopallidal DLS neurons are mobilized only during late phases. With a mathematical modeling approach, we find that symmetric anti-Hebbian STDP favors memory flexibility, while asymmetric anti-Hebbian STDP favors memory maintenance, consistent with memory processes at play in procedural learning.

Neurofeedback Training Protocol Based on Selecting Distinctive Features to Treat or Reduce ADHD Symptoms

Recent research has shown that electroencephalography (EEG) theta/beta ratio (TBR) in cases with attention deficit hyperactivity disorder (ADHD) has thus far been reported lower than that in healthy individuals. Accordingly, utilizing EEG-TBR as a biomarker to diagnose ADHD has been called into question. Besides, employing known protocol to reduce EEG-TBR in the vertex (Cz) channel to treat ADHD via neurofeedback (NFB) has been doubted. The present study was to propose a new NFB treatment protocol to manage ADHD using EEG signals from 30 healthy controls and 30 children with ADHD through an attention-based task and to calculate relative power in their different frequency bands. Then, the most significant distinguishing features of EEG signals from both groups were determined via a genetic algorithm (GA). The results revealed that EEG-TBR values in children with ADHD were lower compared with those in healthy peers; however, such a difference was not statistically significant. Likewise, inhibiting alpha band activity and enhancing delta one in F7 or T5 channels was proposed as a new NFB treatment protocol for ADHD. No significant increase in EEG-TBR in the Cz channel among children with ADHD casts doubt on the effectiveness of using EEG-TBR inhibitory protocols in the Cz channel. Consequently, it was proposed to apply the new protocol along with reinforced beta-band activity to treat or reduce ADHD symptoms.

Optogenetic stimulation of striatal patches modifies habit formation and inhibits dopamine release

Habits are inflexible behaviors that develop after extensive repetition, and overreliance on habits is a hallmark of many pathological states. The striatum is involved in the transition from flexible to inflexible responding, and interspersed throughout the striatum are patches, or striosomes, which make up ~15% of the volume of the striatum relative to the surrounding matrix compartment. Previous studies have suggested that patches are necessary for normal habit formation, but it remains unknown exactly how patches contribute to habit formation and expression. Here, using optogenetics, we stimulated striatal patches in Sepw1-NP67 mice during variable interval training (VI60), which is used to establish habitual responding. We found that activation of patches at reward retrieval resulted in elevated responding during VI60 training by modifying the pattern of head entry and pressing. Further, this optogenetic manipulation reduced subsequent responding following reinforcer devaluation, suggesting modified habit formation. However, patch stimulation did not generally increase extinction rates during a subsequent extinction probe, but did result in a small 'extinction burst', further suggesting goal-directed behavior. On the other hand, this manipulation had no effect in omission trials, where mice had to withhold responses to obtain rewards. Finally, we utilized fast-scan cyclic voltammetry to investigate how patch activation modifies evoked striatal dopamine release and found that optogenetic activation of patch projections to the substantia nigra pars compacta (SNc) is sufficient to suppress dopamine release in the dorsal striatum. Overall, this work provides novel insight into the role of the patch compartment in habit formation, and provides a potential mechanism for how patches modify habitual behavior by exerting control over dopamine signaling.

Opposing roles for striatonigral and striatopallidal neurons in dorsolateral striatum in consolidating new instrumental actions

Comparatively little is known about how new instrumental actions are encoded in the brain. Using whole-brain c-Fos mapping, we show that neural activity is increased in the anterior dorsolateral striatum (aDLS) of mice that successfully learn a new lever-press response to earn food rewards. Post-learning chemogenetic inhibition of aDLS disrupts consolidation of the new instrumental response. Similarly, post-learning infusion of the protein synthesis inhibitor anisomycin into the aDLS disrupts consolidation of the new response. Activity of D1 receptor-expressing medium spiny neurons (D1-MSNs) increases and D2-MSNs activity decreases in the aDLS during consolidation. Chemogenetic inhibition of D1-MSNs in aDLS disrupts the consolidation process whereas D2-MSN inhibition strengthens consolidation but blocks the expression of previously learned habit-like responses. These findings suggest that D1-MSNs in the aDLS encode new instrumental actions whereas D2-MSNs oppose this new learning and instead promote expression of habitual actions.

Experience-related enhancements in striatal temporal encoding

Temporal control of action is key for a broad range of behaviors and is disrupted in human diseases such as Parkinson's disease and schizophrenia. A brain structure that is critical for temporal control is the dorsal striatum. Experience and learning can influence dorsal striatal neuronal activity, but it is unknown how these neurons change with experience in contexts which require precise temporal control of movement. We investigated this question by recording from medium spiny neurons (MSNs) via dorsal striatal microelectrode arrays in mice as they gained experience controlling their actions in time. We leveraged an interval timing task optimized for mice which required them to "switch" response ports after enough time had passed without receiving a reward. We report three main results. First, we found that time-related ramping activity and response-related activity increased with task experience. Second, temporal decoding by MSN ensembles improved with experience and was predominantly driven by time-related ramping activity. Finally, we found that a subset of MSNs had differential modulation on error trials. These findings enhance our understanding of dorsal striatal temporal processing by demonstrating how MSN ensembles can evolve with experience. Our results can be linked to temporal habituation and illuminate striatal flexibility during interval timing, which may be relevant for human disease.

Frontal Beta Activity in the Meta-Intention of Children With Attention Deficit Hyperactivity Disorder

Children with attention deficit hyperactivity disorder (ADHD) have high theta and low beta activity in the frontal lobe. The higher the theta/beta ratio, the lower the level of central nervous system (CNS) cortical arousal. However, there is seldom evidence between electroencephalograms (EEGs) and the patient's intentionality to regulate the cortical activity of executive attention tasks. We investigated whether children with ADHD intended to improve their performance in executive attention tasks and whether that increased their brain activity. Fifty-one children with ADHD (ADHD) and 51 typical developing (TD) children were investigated using focused attention (FA) and search attention (SA) tasks and a simultaneous EEG. The children were then regrouped as faster (ADHD-F, TD-F) and slower (ADHD-S, TD-S) depending on reaction time (RT). Quantitative EEGs of frontal lobe theta and beta activity at frontal F3, F4, and Fz were used. Twenty-eight (54.9%) ADHD children were regrouped as ADHD-S and 14 (27.5%) as TD-S. The ADHD-S group, however, had poorer FA and SA performance than the other 3 groups did: fewer correct answers, more frequent impulsive and missing errors, and higher RT variations. There were no significant differences in theta activity, but the TD-S group had higher beta activity than the ADHD-S group did. We conclude that the ADHD-F and ADHD-S groups had different attention processes. beta activity did not increase in the ADHD-S group, and their executive attention performance in the FA and SA tests was poor. It seems ADHD-S had poor meta-intention function. The frontal beta activity might be a feasible training target of neurofeedback in ADHD-S patients.

Animal Models for OCD Research

OCD has lagged behind other psychiatric illnesses in the identification of molecular treatment targets, due in part to a lack of significant findings in genome-wide association studies. However, while progress in this area is being made, OCD's symptoms of obsessions, compulsions, and anxiety can be deconstructed into distinct neural functions that can be dissected in animal models. Studies in rodents and non-human primates have highlighted the importance of cortico-basal ganglia-thalamic circuits in OCD pathophysiology, and emerging studies in human post-mortem brain tissue point to glutamatergic synapse abnormalities as a potential cellular substrate for observed dysfunctional behaviors. In addition, accumulated evidence points to a potential role for neuromodulators including serotonin and dopamine in both OCD pathology and treatment. Here, we review current efforts to use animal models for the identification of molecules, cell types, and circuits relevant to OCD pathophysiology. We start by describing features of OCD that can be modeled in animals, including circuit abnormalities and genetic findings. We then review different strategies that have been used to study OCD using animal model systems, including transgenic models, circuit manipulations, and dissection of OCD-relevant neural constructs. Finally, we discuss how these findings may ultimately help to develop new treatment strategies for OCD and other related disorders.

Dorsolateral striatum engagement during reversal learning

Most experimental preparations demonstrate a role for dorsolateral striatum (DLS) in stimulus-response, but not outcome-based, learning. Here, we assessed DLS involvement in a touchscreen-based reversal task requiring mice to update choice following a change in stimulus-reward contingencies. In vivo single-unit recordings in the DLS showed reversal produced a population-level shift from excited to inhibited neuronal activity prior to choices being made. The larger the shift, the faster mice reversed. Furthermore, optogenetic photosilencing DLS neurons during choice increased early reversal errors. These findings suggest dynamic DLS engagement may facilitate reversal, possibly by signaling a change in contingencies to other striatal and cortical regions.

The orbitofrontal cortex, food intake and obesity

Obesity is a major health challenge facing many people throughout the world. Increased consumption of palatable, high-caloric foods is one of the major drivers of obesity. Both orexigenic and anorexic states have been thoroughly reviewed elsewhere; here, we focus on the cognitive control of feeding in the context of obesity, and how the orbitofrontal cortex (OFC) is implicated, based on data from preclinical and clinical research. The OFC is important in decision-making and has been heavily researched in neuropsychiatric illnesses such as addiction and obsessive–compulsive disorder. However, activity in the OFC has only recently been described in research into food intake, obesity and eating disorders. The OFC integrates sensory modalities such as taste, smell and vision, and it has dense reciprocal projections into thalamic, midbrain and striatal regions to fine-tune decision-making. Thus, the OFC may be anatomically and functionally situated to play a critical role in the etiology and maintenance of excess feeding behaviour. We propose that the OFC serves as an integrative hub for orchestrating motivated feeding behaviour and suggest how its neurobiology and functional output might be altered in the obese state.

Open-source raspberry Pi-based operant box for translational behavioral testing in rodents

BACKGROUND

Rodents have been used for decades to probe neural circuits involved in behavior. Increasingly, attempts have been developed to standardize training paradigms across labs; and to use visual/auditory paradigms that can be also tested in humans. Commercially available systems are expensive and thus do not scale easily, and are not optimized for electrophysiology.

NEW METHOD

Using the rich open-source technology built around Raspberry Pi, we were able to develop an inexpensive (<$1000) visual-screen based operant chamber with electrophysiological and optogenetic compatibility. The chamber is operated within MATLAB/Simulink, a commonly used scientific programming language allowing for rapid customization.

RESULTS

Here, we describe and provide all relevant details needed to develop and produce these chambers, and show examples of behavior and electrophysiology data collected using these chambers. We also include all of the tools needed to allow readers to build and develop their own boxes (CAD models for 3D printing and laser-cutting; PCB-board design; all bill of materials for required parts and supplies, and some examples of Simulink models to operate the boxes).

COMPARISON WITH EXISTING METHODS

The new boxes are far more cost-effective than commercially available environments and allow for the combination of automated behavioral testing with electrophysiological read-outs with high temporal precision.

CONCLUSION

These open-source boxes can be used for labs interested in developing high-throughput visual/auditory behavioral assays for ∼ 10th the cost of commercial systems.

A biophysical model of striatal microcircuits suggests gamma and beta oscillations interleaved at delta/theta frequencies mediate periodicity in motor control

Striatal oscillatory activity is associated with movement, reward, and decision-making, and observed in several interacting frequency bands. Local field potential recordings in rodent striatum show dopamine- and reward-dependent transitions between two states: a "spontaneous" state involving β (∼15-30 Hz) and low γ (∼40-60 Hz), and a state involving θ (∼4-8 Hz) and high γ (∼60-100 Hz) in response to dopaminergic agonism and reward. The mechanisms underlying these rhythmic dynamics, their interactions, and their functional consequences are not well understood. In this paper, we propose a biophysical model of striatal microcircuits that comprehensively describes the generation and interaction of these rhythms, as well as their modulation by dopamine. Building on previous modeling and experimental work suggesting that striatal projection neurons (SPNs) are capable of generating β oscillations, we show that networks of striatal fast-spiking interneurons (FSIs) are capable of generating δ/θ (ie, 2 to 6 Hz) and γ rhythms. Under simulated low dopaminergic tone our model FSI network produces low γ band oscillations, while under high dopaminergic tone the FSI network produces high γ band activity nested within a δ/θ oscillation. SPN networks produce β rhythms in both conditions, but under high dopaminergic tone, this β oscillation is interrupted by δ/θ-periodic bursts of γ-frequency FSI inhibition. Thus, in the high dopamine state, packets of FSI γ and SPN β alternate at a δ/θ timescale. In addition to a mechanistic explanation for previously observed rhythmic interactions and transitions, our model suggests a hypothesis as to how the relationship between dopamine and rhythmicity impacts motor function. We hypothesize that high dopamine-induced periodic FSI γ-rhythmic inhibition enables switching between β-rhythmic SPN cell assemblies representing the currently active motor program, and thus that dopamine facilitates movement in part by allowing for rapid, periodic shifts in motor program execution.

Temporal dynamics of Arc/Arg3.1 expression in the dorsal striatum during acquisition and consolidation of a motor skill in mice

Region- and pathway-specific plasticity within striatal circuits is critically involved in the acquisition and long-term retention of a new motor skill as it becomes automatized. However, the molecular substrates contributing to this plasticity remain unclear. Here, we examined the expression of the activity-regulated cytoskeleton-associated protein (Arc) in the associative or dorsomedial striatum (DMS) and the sensorimotor or dorsolateral striatum (DLS), as well as in striatonigral and striatopallidal neurons, during different skill learning phases in the accelerating rotarod task. We found that Arc was mainly expressed in the DMS during early motor learning and progressively increased in the DLS during gradual motor skill consolidation. Moreover, Arc was preferentially expressed in striatopallidal neurons early in training and gradually increased in striatonigral neurons later in training. These data demonstrate that in the dorsal striatum, the expression of Arc exhibits a region- and cell-specific transfer during the learning of a motor skill, suggesting a link between striatal Arc expression and motor skill learning in mice.

Striatal and hippocampal contributions to flexible navigation in rats and humans

The hippocampus has been firmly established as playing a crucial role in flexible navigation. Recent evidence suggests that dorsal striatum may also play an important role in such goal-directed behaviour in both rodents and humans. Across recent studies, activity in the caudate nucleus has been linked to forward planning and adaptation to changes in the environment. In particular, several human neuroimaging studies have found the caudate nucleus tracks information traditionally associated with that by the hippocampus. In this brief review, we examine this evidence and argue the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour. We highlight that future research should explore the following: (1) Investigate neural responses during spatial navigation via a biophysically plausible framework explained by reinforcement learning models and (2) Observe the interaction between cortical areas and both the dorsal striatum and hippocampus during flexible navigation.

No substantial change in the balance between model-free and model-based control via training on the two-step task

Human decisions can be habitual or goal-directed, also known as model-free (MF) or model-based (MB) control. Previous work suggests that the balance between the two decision systems is impaired in psychiatric disorders such as compulsion and addiction, via overreliance on MF control. However, little is known whether the balance can be altered through task training. Here, 20 healthy participants performed a well-established two-step task that differentiates MB from MF control, across five training sessions. We used computational modelling and functional near-infrared spectroscopy to assess changes in decision-making and brain hemodynamic over time. Mixed-effects modelling revealed overall no substantial changes in MF and MB behavior across training. Although our behavioral and brain findings show task-induced changes in learning rates, these parameters have no direct relation to either MF or MB control or the balance between the two systems, and thus do not support the assumption of training effects on MF or MB strategies. Our findings indicate that training on the two-step paradigm in its current form does not support a shift in the balance between MF and MB control. We discuss these results with respect to implications for restoring the balance between MF and MB control in psychiatric conditions.

Psychiatric conditions such as compulsion or addiction are associated with an overreliance on habitual, or model-free, decision-making. Goal-directed, or model-based, decision-making may protect against such overreliance. We therefore asked whether model-free control could be reduced, and model-based control strengthened, via task training. We used the well-characterized two-step task that differentiates model-based from model-free actions. Our results suggest that training on the current form of the two-step task does not support a shift in the balance between model-free and model-based strategies. Factors such as devaluation, demotivation or automatization during training may play a role in the missing emergence of a training effect. Future studies could adapt the two-step task so as to separate such factors from decision-making strategies.

Neural substrates of the influence of emotional cues on cognitive control in risk-taking adolescents

Adolescence is a period characterised by increases in risk-taking. This behaviour has been associated with an imbalance in the integration of the networks involved in cognitive control and motivational processes. We examined whether the influence of emotional cues on cognitive control differs between adolescents who show high or low levels of risk-taking behaviour. Participants who scored especially high or low on a risky decision task were subsequently administered an emotional go/no-go fMRI task comprising angry, happy and calm faces. Both groups showed decreased cognitive control when confronted with appetitive and aversive emotional cues. Activation in the inferior frontal gyrus (IFG) increased in line with the cognitive control demands of the task. Though the risk taking groups did not differ in their behavioural performance, functional connectivity analyses revealed the dorsal striatum plays a more central role in the processing of cognitive control in high than low risk-takers. Overall, these findings suggest that variance in fronto-striatal circuitry may underlie individual differences in risk-taking behaviour.

Dorsolateral Striatum Engagement Interferes with Early Discrimination Learning

In current models, learning the relationship between environmental stimuli and the outcomes of actions involves both stimulus-driven and goal-directed systems, mediated in part by the DLS and DMS, respectively. However, though these models emphasize the importance of the DLS in governing actions after extensive experience has accumulated, there is growing evidence of DLS engagement from the onset of training. Here, we used in vivo photosilencing to reveal that DLS recruitment interferes with early touchscreen discrimination learning. We also show that the direct output pathway of the DLS is preferentially recruited and causally involved in early learning and find that silencing the normal contribution of the DLS produces plasticity-related alterations in a PL-DMS circuit. These data provide further evidence suggesting that the DLS is recruited in the construction of stimulus-elicited actions that ultimately automate behavior and liberate cognitive resources for other demands, but with a cost to performance at the outset of learning.

Synchronization in the prefrontal-striatal circuit tracks behavioural choice in a go-no-go task in rats

Rodent striatum is involved in sensory-motor transformations and reward-related learning. Lesion studies suggest dorsolateral striatum, dorsomedial striatum and nucleus accumbens underlie stimulus-response transformations, goal-directed behaviour and reward expectation, respectively. In addition, prefrontal inputs likely control these functions. Here, we set out to study how reward-driven behaviour is mediated by the coordinated activity of these structures in the intact brain. We implemented a discrimination task requiring rats to either respond or suppress responding on a lever after the presentation of auditory cues in order to obtain rewards. Single unit activity in the striatal subregions and pre-limbic cortex was recorded using tetrode arrays. Striatal units showed strong onset responses to auditory cues paired with an opportunity to obtain reward. Cue-onset responses in both striatum and cortex were significantly modulated by previous errors suggesting a role of these structures in maintaining appropriate motivation or action selection during ongoing behaviour. Furthermore, failure to respond to the reward-paired tones was associated with higher pre-trial coherence among striatal subregions and between cortex and striatum suggesting a task-negative corticostriatal network whose activity may be suppressed to enable processing of reward-predictive cues. Our findings highlight that coordinated activity in a distributed network including both pre-limbic cortex and multiple striatal regions underlies reward-related decisions.

Local or Not Local: Investigating the Nature of Striatal Theta Oscillations in Behaving Rats

In the cortex and hippocampus, neuronal oscillations of different frequencies can be observed in local field potentials (LFPs). LFPs oscillations in the theta band (6–10 Hz) have also been observed in the dorsolateral striatum (DLS) of rodents, mostly during locomotion, and have been proposed to mediate behaviorally-relevant interactions between striatum and cortex (or between striatum and hippocampus). However, it is unclear if these theta oscillations are generated in the striatum. To address this issue, we recorded LFPs and spiking activity in the DLS of rats performing a running sequence on a motorized treadmill. We observed an increase in rhythmical activity of the LFP in the theta-band during run compared to rest periods. However, several observations suggest that these oscillations are mainly generated outside of the striatum. First, theta oscillations disappeared when LFPs were rereferenced against a striatal recording electrode and the imaginary coherence between LFPs recorded at different locations within the striatum was null. Second, 8% of the recorded neurons had their spiking activity phase-locked to the theta rhythm. Third, Granger causality analyses between LFPs simultaneously recorded in the cortex and the striatum revealed that the interdependence between these two signals in the theta range was mostly accounted for by a common external source. The most parsimonious interpretation of these results is that theta oscillations observed in striatal LFPs are largely contaminated by volume-conducted signals. We propose that striatal LFPs are not optimal proxies of network dynamics in the striatum and should be interpreted with caution.

Parvalbumin Interneurons Modulate Striatal Output and Enhance Performance during Associative Learning

The prevailing view is that striatal parvalbumin (PV)-positive interneurons primarily function to downregulate medium spiny projection neuron (MSN) activity via monosynaptic inhibitory signaling. Here, by combining in vivo neural recordings and optogenetics, we unexpectedly find that both suppressing and over-activating PV cells attenuates spontaneous MSN activity. To account for this, we find that, in addition to monosynaptic coupling, PV-MSN interactions are mediated by a competing disynaptic inhibitory circuit involving a variety of neuropeptide Y-expressing interneurons. Next we use optogenetic and chemogenetic approaches to show that dorsolateral striatal PV interneurons influence the initial expression of reward-conditioned responses but that their contribution to performance declines with experience. Consistent with this, we observe with large-scale recordings in behaving animals that the relative contribution of PV cells on MSN activity diminishes with training. Together, this work provides a possible mechanism by which PV interneurons modulate striatal output and selectively enhance performance early in learning.

The Medial Orbitofrontal Cortex Regulates Sensitivity to Outcome Value

An essential component of goal-directed decision-making is the ability to maintain flexible responding based on the value of a given reward, or "reinforcer." The medial orbitofrontal cortex (mOFC), a subregion of the ventromedial prefrontal cortex, is uniquely positioned to regulate this process. We trained mice to nose poke for food reinforcers and then stimulated this region using CaMKII-driven Gs-coupled designer receptors exclusively activated by designer drugs (DREADDs). In other mice, we silenced the neuroplasticity-associated neurotrophin brain-derived neurotrophic factor (BDNF). Activation of Gs-DREADDs increased behavioral sensitivity to reinforcer devaluation, whereas Bdnf knockdown blocked sensitivity. These changes were accompanied by modifications in breakpoint ratios in a progressive ratio task, and they were recapitulated in Bdnf(+/-)mice. Replacement of BDNF selectively in the mOFC in Bdnf(+/-)mice rescued behavioral deficiencies, as well as phosphorylation of extracellular-signal regulated kinase 1/2 (ERK1/2). Thus, BDNF expression in the mOFC is both necessary and sufficient for the expression of typical effort allocation relative to an anticipated reinforcer. Additional experiments indicated that expression of the immediate-early gene c-fos was aberrantly elevated in the Bdnf(+/-)dorsal striatum, and BDNF replacement in the mOFC normalized expression. Also, systemic administration of an MAP kinase kinase inhibitor increased breakpoint ratios, whereas the addition of discrete cues bridging the response-outcome contingency rescued breakpoints in Bdnf(+/-)mice. We argue that BDNF-ERK1/2 in the mOFC is a key regulator of "online" goal-directed action selection.

SIGNIFICANCE STATEMENT

Goal-directed response selection often involves predicting the consequences of one's actions and the value of potential payoffs. Lesions or chemogenetic inactivation of the medial orbitofrontal cortex (mOFC) in rats induces failures in retrieving outcome identity memories (Bradfield et al., 2015), suggesting that the healthy mOFC serves to access outcome value information when it is not immediately observable and thereby guide goal-directed decision-making. Our findings suggest that the mOFC also bidirectionally regulates effort allocation for a given reward and that expression of the neurotrophin BDNF in the mOFC is both necessary and sufficient for mice to sustain stable representations of reinforcer value.

Neuronal activity in dorsomedial and dorsolateral striatum under the requirement for temporal credit assignment

To investigate neural processes underlying temporal credit assignment in the striatum, we recorded neuronal activity in the dorsomedial and dorsolateral striatum (DMS and DLS, respectively) of rats performing a dynamic foraging task in which a choice has to be remembered until its outcome is revealed for correct credit assignment. Choice signals appeared sequentially, initially in the DMS and then in the DLS, and they were combined with action value and reward signals in the DLS when choice outcome was revealed. Unlike in conventional dynamic foraging tasks, neural signals for chosen value were elevated in neither brain structure. These results suggest that dynamics of striatal neural signals related to evaluating choice outcome might differ drastically depending on the requirement for temporal credit assignment. In a behavioral context requiring temporal credit assignment, the DLS, but not the DMS, might be in charge of updating the value of chosen action by integrating choice, action value, and reward signals together.

Reappraising striatal D1- and D2-neurons in reward and aversion

The striatum has been involved in complex behaviors such as motor control, learning, decision-making, reward and aversion. The striatum is mainly composed of medium spiny neurons (MSNs), typically divided into those expressing dopamine receptor D1, forming the so-called direct pathway, and those expressing D2 receptor (indirect pathway). For decades it has been proposed that these two populations exhibit opposing control over motor output, and recently, the same dichotomy has been proposed for valenced behaviors. Whereas D1-MSNs mediate reinforcement and reward, D2-MSNs have been associated with punishment and aversion. In this review we will discuss pharmacological, genetic and optogenetic studies that indicate that there is still controversy to what concerns the role of striatal D1- and D2-MSNs in this type of behaviors, highlighting the need to reconsider the early view that they mediate solely opposing aspects of valenced behaviour.

Regionally distinct phasic dopamine release patterns in the striatum during reversal learning

Striatal dopamine (DA) plays a central role in reward-related learning and behavioral adaptation to changing environments. Recent studies suggest that rather than being broadcast as a uniform signal throughout the entire region, DA release dynamics diverge between different striatal regions. In a previous study, we showed that phasic DA release patterns in the ventromedial striatum (VMS) rapidly adapt during reversal learning. However, it is unknown how DA dynamics in the dorsolateral striatum (DLS) are modulated during such adaptive behavior. Here, we used fast-scan cyclic voltammetry to measure phasic DA release in the DLS during spatial reversal learning. In the DLS, we observed minor DA release after the onset of a visual cue signaling reward availability, followed by more pronounced DA release during more proximal reward cues (e.g., lever extension) and execution of the operant response (i.e., lever press), both in rewarded and non-rewarded trials. These release dynamics (minor DA after onset of the predictive visual cue, prominent DA during the operant response) did not change significantly during or following a reversal of response-reward contingencies. Notably, the DA increase to the lever press did not reflect a general signal related to the initiation of any motivated motor response, as we did not observe DA release when rats initiated nose pokes into the food receptacle during inter-trial intervals. This suggests that DA release in the DLS occurs selectively during the initiation and execution of a learned operant response. Together with our previous results obtained in the VMS, these findings reveal distinct phasic DA release patterns during adaptation of established behavior in DLS and VMS. The VMS DA signal, which is highly sensitive to reversal of response-reward contingences, may provide a teaching signal to guide reward-related learning and facilitate behavioral adaptation, whereas DLS DA may reflect a 'response execution signal' largely independent of outcome, that may be involved in initiation and energizing of operant behavior.

Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum

Evaluating outcomes of behavior is a central function of the striatum. In circuits engaging the dorsomedial striatum, sensitivity to goal value is accentuated during learning, whereas outcome sensitivity is thought to be minimal in the dorsolateral striatum and its habit-related corticostriatal circuits. However, a distinct population of projection neurons in the dorsolateral striatum exhibits selective sensitivity to rewards. Here, we evaluated the outcome-related signaling in such neurons as rats performed an instructional T-maze task for two rewards. As the rats formed maze-running habits and then changed behavior after reward devaluation, we detected outcome-related spike activity in 116 units out of 1,479 recorded units. During initial training, nearly equal numbers of these units fired preferentially either after rewarded runs or after unrewarded runs, and the majority were responsive at only one of two reward locations. With overtraining, as habits formed, firing in nonrewarded trials almost disappeared, and reward-specific firing declined. Thus error-related signaling was lost, and reward signaling became generalized. Following reward devaluation, in an extinction test, postgoal activity was nearly undetectable, despite accurate running. Strikingly, when rewards were then returned, postgoal activity reappeared and recapitulated the original early response pattern, with nearly equal numbers responding to rewarded and unrewarded runs and to single rewards. These findings demonstrate that outcome evaluation in the dorsolateral striatum is highly plastic and tracks stages of behavioral exploration and exploitation. These signals could be a new target for understanding compulsive behaviors that involve changes to dorsal striatum function.

Corticostriatal circuitry and habitual ethanol seeking

The development of alcohol-use disorders is thought to involve a transition from casual alcohol use to uncontrolled alcohol-seeking behavior. This review will highlight evidence suggesting that the shift toward inflexible alcohol seeking that occurs across the development of addiction consists, in part, of a progression from goal-directed to habitual behaviors. This shift in "response strategy" is thought to be largely regulated by corticostriatal network activity. Indeed, specific neuroanatomical substrates within the prefrontal cortex and the striatum have been identified as playing opposing roles in the expression of actions and habits. A majority of the research on the neurobiology of habitual behavior has focused on non-drug reward seeking. Here, we will highlight recent research identifying corticostriatal structures that regulate the expression of habitual alcohol seeking and a comparison will be made when possible to findings for non-drug rewards.

Neuroscience of learning and memory for addiction medicine: from habit formation to memory reconsolidation

Identifying effective pharmacological treatments for addictive disorders has remained an elusive goal. Many different classes of drugs have shown some efficacy in preclinical models, but the number of effective clinical therapeutics has remained stubbornly low. The persistence of drug use and the high frequency of relapse is at least partly attributable to the enduring ability of environmental stimuli associated with drug use to maintain behavioral patterns of drug use and induce craving during abstinence. We propose that stimuli associated with drug use exert such powerful control over behavior through the development of abnormally strong memories, and their ability to initiate subconscious sequences of motor actions (habits) that promote uncontrolled drug use. In this chapter, we will review the evidence suggesting that drugs of abuse strengthen associations with cues in the environment and facilitate habit formation. We will also discuss potential mechanisms for disrupting memories associated with drug use to help improve treatments for addiction.

Distinct neural representation in the dorsolateral, dorsomedial, and ventral parts of the striatum during fixed- and free-choice tasks

The striatum is a major input site of the basal ganglia, which play an essential role in decision making. Previous studies have suggested that subareas of the striatum have distinct roles: the dorsolateral striatum (DLS) functions in habitual action, the dorsomedial striatum (DMS) in goal-directed actions, and the ventral striatum (VS) in motivation. To elucidate distinctive functions of subregions of the striatum in decision making, we systematically investigated information represented by phasically active neurons in DLS, DMS, and VS. Rats performed two types of choice tasks: fixed- and free-choice tasks. In both tasks, rats were required to perform nose poking to either the left or right hole after cue-tone presentation. A food pellet was delivered probabilistically depending on the presented cue and the selected action. The reward probability was fixed in fixed-choice task and varied in a block-wise manner in free-choice task. We found the following: (1) when rats began the tasks, a majority of VS neurons increased their firing rates and information regarding task type and state value was most strongly represented in VS; (2) during action selection, information of action and action values was most strongly represented in DMS; (3) action-command information (action representation before action selection) was stronger in the fixed-choice task than in the free-choice task in both DLS and DMS; and (4) action-command information was strongest in DLS, particularly when the same choice was repeated. We propose a hypothesis of hierarchical reinforcement learning in the basal ganglia to coherently explain these results.

Characterizing the associative content of brain structures involved in habitual and goal-directed actions in humans: a multivariate FMRI study

While there is accumulating evidence for the existence of distinct neural systems supporting goal-directed and habitual action selection in the mammalian brain, much less is known about the nature of the information being processed in these different brain regions. Associative learning theory predicts that brain systems involved in habitual control, such as the dorsolateral striatum, should contain stimulus and response information only, but not outcome information, while regions involved in goal-directed action, such as ventromedial and dorsolateral prefrontal cortex and dorsomedial striatum, should be involved in processing information about outcomes as well as stimuli and responses. To test this prediction, human participants underwent fMRI while engaging in a binary choice task designed to enable the separate identification of these different representations with a multivariate classification analysis approach. Consistent with our predictions, the dorsolateral striatum contained information about responses but not outcomes at the time of an initial stimulus, while the regions implicated in goal-directed action selection contained information about both responses and outcomes. These findings suggest that differential contributions of these regions to habitual and goal-directed behavioral control may depend in part on basic differences in the type of information that these regions have access to at the time of decision making.

Dopamine D1 receptor blockade impairs alcohol seeking without reducing dorsal striatal activation to cues of alcohol availability

INTRODUCTION

Alcohol-associated cues activate both ventral and dorsal striatum in functional brain imaging studies of heavy drinkers. In rodents, alcohol-associated cues induce changes in neuronal firing frequencies and increase dopamine release in ventral striatum, but the impact of alcohol-associated cues on neuronal activity in dorsal striatum is unclear. We previously reported phasic changes in action potential frequency in the dorsomedial and dorsolateral striatum after cues that signaled alcohol availability, prompting approach behavior.

METHODS

We investigated the hypothesis that dopamine transmission modulates these phasic firing changes. Rats were trained to self-administer alcohol, and neuronal activity was monitored with extracellular electrophysiology during "anticipatory" cues that signaled the start of the operant session. Sessions were preceded by systemic administration of the D1-type dopamine receptor antagonist SCH23390 (0, 10, and 20 μg/kg).

RESULTS

SCH23390 significantly decreased firing rates during the 60 s prior to cue onset without reducing phasic excitations immediately following the cues. While neuronal activation to cues might be expected to initiate behavioral responses, in this study alcohol seeking was reduced despite the presence of dorsal striatal excitations to alcohol cues.

CONCLUSIONS

These data suggest that D1 receptor antagonism reduces basal firing rates in the dorsal striatum and modulates the ability of neuronal activation to "anticipatory" cues to initiate alcohol seeking in rats with an extensive history of alcohol self-administration.

A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface

A computational model yields new insights into the bewildering complexity of cortico-striatal plasticity and its rationale for supporting operant learning.

Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem—action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

A key component of survival is the ability to learn which actions, in what contexts, yield useful and rewarding outcomes. Actions are encoded in the brain in the cortex but, as many actions are possible at any one time, there needs to be a mechanism to select which one is to be performed. This problem of action selection is mediated by a set of nuclei known as the basal ganglia, which receive convergent “action requests” from all over the cortex and select the one that is currently most important. Working out which is most important is determined by the strength of the input from each action request: the stronger the connection, the more important that action. Understanding learning thus requires understanding how that strength is changed by the outcome of each action. We built a computational model that demonstrates how the brain's internal signal for outcome (carried by the neurotransmitter dopamine) changes the strength of these cortical connections to learn the selection of rewarded actions, and the suppression of unrewarded ones. Our model shows how several known signals in the brain work together to shape the influence of cortical inputs to the basal ganglia at the interface between our actions and their outcomes.

Neural processing of reward in adolescent rodents

•

The adolescent brain processes rewards differently than in adults.

•

These differences occur even when behavior is similar between age groups.

•

DS was the locus of substantial developmental differences in reward activity.

•

Surprisingly, differences were not as pronounced in VS.

•

These differences may have implications for adolescent psychiatric vulnerability.

Immaturities in adolescent reward processing are thought to contribute to poor decision making and increased susceptibility to develop addictive and psychiatric disorders. Very little is known; however, about how the adolescent brain processes reward. The current mechanistic theories of reward processing are derived from adult models. Here we review recent research focused on understanding of how the adolescent brain responds to rewards and reward-associated events. A critical aspect of this work is that age-related differences are evident in neuronal processing of reward-related events across multiple brain regions even when adolescent rats demonstrate behavior similar to adults. These include differences in reward processing between adolescent and adult rats in orbitofrontal cortex and dorsal striatum. Surprisingly, minimal age related differences are observed in ventral striatum, which has been a focal point of developmental studies. We go on to discuss the implications of these differences for behavioral traits affected in adolescence, such as impulsivity, risk-taking, and behavioral flexibility. Collectively, this work suggests that reward-evoked neural activity differs as a function of age and that regions such as the dorsal striatum that are not traditionally associated with affective processing in adults may be critical for reward processing and psychiatric vulnerability in adolescents.

Habitual alcohol seeking: modeling the transition from casual drinking to addiction

The transition from goal-directed actions to habitual ethanol seeking models the development of addictive behavior that characterizes alcohol use disorders. The progression to habitual ethanol-seeking behavior occurs more rapidly than for natural rewards, suggesting that ethanol may act on habit circuit to drive the loss of behavioral flexibility. This review will highlight recent research that has focused on the formation and expression of habitual ethanol seeking, and the commonalities and distinctions between ethanol and natural reward-seeking habits, with the goal of highlighting important, understudied research areas that we believe will lead toward the development of novel treatment and prevention strategies for uncontrolled drinking.

Differential entrainment and learning-related dynamics of spike and local field potential activity in the sensorimotor and associative striatum

Parallel cortico-basal ganglia loops are thought to have distinct but interacting functions in motor learning and habit formation. In rats, the striatal projection neuron populations (MSNs) in the dorsolateral and dorsomedial striatum, respectively corresponding to sensorimotor and associative regions of the striatum, exhibit contrasting dynamics as rats acquire T-maze tasks (Thorn et al., 2010). Here, we asked whether these patterns could be related to the activity of local interneuron populations in the striatum and to the local field potential activity recorded simultaneously in the corresponding regions. We found that dorsolateral and dorsomedial striatal fast-spiking interneurons exhibited task-specific and training-related dynamics consistent with those of corresponding MSN populations. Moreover, both MSNs and interneuron populations in both regions became entrained to theta-band (5-12 Hz) frequencies during task acquisition. However, the predominant entrainment frequencies were different for the sensorimotor and associative zones. Dorsolateral striatal neurons became entrained mid-task to oscillations centered ∼ 5 Hz, whereas simultaneously recorded neurons in the dorsomedial region became entrained to higher frequency (∼ 10 Hz) rhythms. These region-specific patterns of entrainment evolved dynamically with the development of region-specific patterns of interneuron and MSN activity, indicating that, with learning, these two striatal regions can develop different frequency-modulated circuit activities in parallel. We suggest that such differential entrainment of sensorimotor and associative neuronal populations, acquired through learning, could be critical for coordinating information flow throughout each trans-striatal network while simultaneously enabling nearby components of the separate networks to operate independently.

Investigating habits: strategies, technologies and models

Understanding habits at a biological level requires a combination of behavioral observations and measures of ongoing neural activity. Theoretical frameworks as well as definitions of habitual behaviors emerging from classic behavioral research have been enriched by new approaches taking account of the identification of brain regions and circuits related to habitual behavior. Together, this combination of experimental and theoretical work has provided key insights into how brain circuits underlying action-learning and action-selection are organized, and how a balance between behavioral flexibility and fixity is achieved. New methods to monitor and manipulate neural activity in real time are allowing us to have a first look “under the hood” of a habit as it is formed and expressed. Here we discuss ideas emerging from such approaches. We pay special attention to the unexpected findings that have arisen from our own experiments suggesting that habitual behaviors likely require the simultaneous activity of multiple distinct components, or operators, seen as responsible for the contrasting dynamics of neural activity in both cortico-limbic and sensorimotor circuits recorded concurrently during different stages of habit learning. The neural dynamics identified thus far do not fully meet expectations derived from traditional models of the structure of habits, and the behavioral measures of habits that we have made also are not fully aligned with these models. We explore these new clues as opportunities to refine an understanding of habits.

The role of the striatum in social behavior

Where and how does the brain code reward during social behavior? Almost all elements of the brain's reward circuit are modulated during social behavior. The striatum in particular is activated by rewards in social situations. However, its role in social behavior is still poorly understood. Here, we attempt to review its participation in social behaviors of different species ranging from voles to humans. Human fMRI experiments show that the striatum is reliably active in relation to others' rewards, to reward inequity and also while learning about social agents. Social contact and rearing conditions have long-lasting effects on behavior, striatal anatomy and physiology in rodents and primates. The striatum also plays a critical role in pair-bond formation and maintenance in monogamous voles. We review recent findings from single neuron recordings showing that the striatum contains cells that link own reward to self or others' actions. These signals might be used to solve the agency-credit assignment problem: the question of whose action was responsible for the reward. Activity in the striatum has been hypothesized to integrate actions with rewards. The picture that emerges from this review is that the striatum is a general-purpose subcortical region capable of integrating social information into coding of social action and reward.

Dopamine overdose hypothesis: evidence and clinical implications

About a half a century has passed since dopamine was identified as a neurotransmitter, and it has been several decades since it was established that people with Parkinson's disease receive motor symptom relief from oral levodopa. Despite the evidence that levodopa can reduce motor symptoms, there has been a developing body of literature that dopaminergic therapy can improve cognitive functions in some patients but make them worse in others. Over the past two decades, several laboratories have shown that dopaminergic medications can impair the action of intact neural structures and impair the behaviors associated with these structures. In this review, we consider the evidence that has accumulated in the areas of reversal learning, motor sequence learning, and other cognitive tasks. The purported inverted-U shaped relationship between dopamine levels and performance is complex and includes many contributory factors. The regional striatal topography of nigrostriatal denervation is a critical factor, as supported by multimodal neuroimaging studies. A patient's individual genotype will determine the relative baseline position on this inverted-U curve. Dopaminergic pharmacotherapy and individual gene polymorphisms can affect the mesolimbic and prefrontal cortical dopaminergic functions in a comparable, inverted-U dose-response relationship. Depending on these factors, a patient can respond positively or negatively to levodopa when performing reversal learning and motor sequence learning tasks. These tasks may continue to be relevant as our society moves to increased technological demands of a digital world that requires newly learned motor sequences and adaptive behaviors to manage daily life activities.

Cytoskeletal determinants of stimulus-response habits

Both humans and rodents can learn to associate specific actions with their outcomes, but with repeated performance or exposure to pathological stimuli, such as drugs of abuse, behaviors assume stimulus-elicited, or "habitual," qualities. Psychostimulants remodel dorsal striatal neurons, critical determinants of decision-making strategies, but cytoskeletal mechanisms associated with drug-induced habit formation are largely unknown. We first show that cocaine can bias decision-making strategies toward stimulus-response habits by interfering with learning about the predictive relationship between a response and its outcome. In the dorsomedial, but not ventral, striatum, cocaine decreases PSD95 expression and phosphorylation of cortactin, a cytoskeletal regulator that interacts with, and is phophorylated by, the Abl2 (Arg) kinase. Based on this pattern, we inhibited Abl-family kinase signaling in the dorsomedial striatum, impairing new response-outcome learning. Consistent with evidence that the dorsomedial striatum promotes response-outcome decision-making while the dorsolateral compartment promotes stimulus-response habits, inhibition of Abl-family kinases in the dorsolateral striatum reinstates goal sensitivity in over-trained "habitual" mice. These findings provide a structural mechanism by which even acute exposure to drugs of abuse can reorganize behavioral response strategies and promote outcome-insensitive stimulus-response habits.

A dual operator view of habitual behavior reflecting cortical and striatal dynamics

Habits are notoriously difficult to break and, if broken, are usually replaced by new routines. To examine the neural basis of these characteristics, we recorded spike activity in cortical and striatal habit sites as rats learned maze tasks. Overtraining induced a shift from purposeful to habitual behavior. This shift coincided with the activation of neuronal ensembles in the infralimbic neocortex and the sensorimotor striatum, which became engaged simultaneously but developed changes in spike activity with distinct time courses and stability. The striatum rapidly acquired an action-bracketing activity pattern insensitive to reward devaluation but sensitive to running automaticity. A similar pattern developed in the upper layers of the infralimbic cortex, but it formed only late during overtraining and closely tracked habit states. Selective optogenetic disruption of infralimbic activity during overtraining prevented habit formation. We suggest that learning-related spiking dynamics of both striatum and neocortex are necessary, as dual operators, for habit crystallization.

Dorsomedial and dorsolateral striatum exhibit distinct phasic neuronal activity during alcohol self-administration in rats

The development of alcoholism may involve a shift from goal-directed to habitual drinking. These action control systems are distinct in the dorsal striatum, with the dorsomedial striatum (DMS) important for goal-directed behavior and the dorsolateral striatum (DLS) required for habit formation. Goal-directed behavior can be modeled in rats with a fixed ratio (FR) reinforcement schedule, while a variable interval (VI) schedule promotes habitual behavior (e.g. insensitivity to contingency degradation). Using extracellular recordings from chronically implanted electrodes, we investigated how DMS and DLS neurons encoded lever-press responses and conditioned cues during operant alcohol self-administration in these two models. In rats self-administering 10% alcohol on an FR schedule, the DMS neuronal population showed increased firing at the onset of start-of-session stimuli. During self-administration, the most prominent phasic firing patterns in the DMS occurred at the time of reinforcement and reinforcement-associated cues, while the most prominent phasic activity in the DLS surrounded the lever response. Neural recordings from an additional cohort of rats trained on a VI schedule revealed a similar pattern of results; however, phasic changes in firing were smaller and differences between the medial and lateral dorsal striatum were less marked. In summary, the DMS and DLS exhibited overlapping but specialized phasic firing patterns: DMS excitations were typically time-locked to reinforcement, while DLS excitations were generally associated with lever responses. Furthermore, the regional specificities and magnitudes of phasic firing differed between reinforcement schedules, which may reflect differences in behavioral flexibility, reward expectancy and the action sequences required to procure reinforcement.

The orbitofrontal cortex regulates outcome-based decision-making via the lateral striatum

The orbitofrontal cortex (oPFC) sends substantial projections to the ventrolateral striatum and aspects of the nucleus accumbens that are, functionally, poorly understood. This is despite probable cortico-striatal involvement in multiple diseases such as addiction and obsessive-compulsive disorder. Here we surgically disconnected the oPFC from the ventrolateral striatum using unilateral asymmetric lesions in mice and classified instrumental decision-making strategies. Mice with symmetric lesions that spared one oPFC-striatal network served as controls. As a complementary approach, we selectively knocked down Brain-derived neurotrophic factor (Bdnf) bilaterally in the oPFC and ascertained behavioral and neurobiological consequences within the downstream striatum. oPFC-striatal disconnection and oPFC Bdnf knockdown blocked sensitivity to outcome-predictive relationships in both food-reinforced and cocaine-associated settings. Bdnf knockdown simultaneously regulated striatal BDNF expression, and striatal c-Fos predicted sensitivity to action-outcome associative contingencies. Previous evidence strongly implicates the dorsolateral striatum in stimulus-response habit formation. Our findings thus provide novel evidence for functional compartmentalisation within the lateral striatum, with the dorsal compartment subserving classical stimulus-response habit systems and a ventral compartment coordinating outcome-based decision-making via oPFC interactions. This compartmentalisation may apply to both 'natural', as in the case of food-reinforced behavior, and 'pathological', as in the case of cocaine-seeking, contexts.

Encoding by synchronization in the primate striatum

Information is encoded in the nervous system through the discharge and synchronization of single neurons. The striatum, the input stage of the basal ganglia, is divided into three territories: the putamen, the caudate, and the ventral striatum, all of which converge onto the same motor pathway. This parallel organization suggests that there are multiple and competing systems in the basal ganglia network controlling behavior. To explore which mechanism(s) enables the different striatal domains to encode behavioral events and to control behavior, we compared the neural activity of phasically active neurons [medium spiny neurons (MSNs), presumed projection neurons] and tonically active neurons (presumed cholinergic interneurons) across striatal territories from monkeys during the performance of a well practiced task. Although neurons in all striatal territories displayed similar spontaneous discharge properties and similar temporal modulations of their discharge rates to the behavioral events, their correlation structure was profoundly different. The distributions of signal and noise correlation of pairs of putamen MSNs were strongly shifted toward positive correlations and these two measures were correlated. In contrast, MSN pairs in the caudate and ventral striatum displayed symmetrical, near-zero signal and noise correlation distributions. Furthermore, only putamen MSN pairs displayed different noise correlation dynamics to rewarding versus neutral/aversive cues. Similarly, the noise correlation between tonically active neuron pairs was stronger in the putamen than in the caudate. We suggest that the level of synchronization of the neuronal activity and its temporal dynamics differentiate the striatal territories and may thus account for the different roles that striatal domains play in behavioral control.

Signals for previous goal choice persist in the dorsomedial, but not dorsolateral striatum of rats

The cortico-basal ganglia network has been proposed to consist of parallel loops serving distinct functions. However, it is still uncertain how the content of processed information varies across different loops and how it is related to the functions of each loop. We investigated this issue by comparing neuronal activity in the dorsolateral (sensorimotor) and dorsomedial (associative) striatum, which have been linked to habitual and goal-directed action selection, respectively, in rats performing a dynamic foraging task. Both regions conveyed significant neural signals for the animal's goal choice and its outcome. Moreover, both regions conveyed similar levels of neural signals for action value before the animal's goal choice and chosen value after the outcome of the animal's choice was revealed. However, a striking difference was found in the persistence of neural signals for the animal's chosen action. Signals for the animal's goal choice persisted in the dorsomedial striatum until the outcome of the animal's next goal choice was revealed, whereas they dissipated rapidly in the dorsolateral striatum. These persistent choice signals might be used for causally linking temporally discontiguous responses and their outcomes in the dorsomedial striatum, thereby contributing to its role in goal-directed action selection.

Differential reward coding in the subdivisions of the primate caudate during an oculomotor task

The basal ganglia play a pivotal role in reward-oriented behavior. The striatum, an input channel of the basal ganglia, is composed of subdivisions that are topographically connected with different cortical and subcortical areas. To test whether reward information is differentially processed in the different parts of the striatum, we compared reward-related neuronal activity along the dorsolateral-ventromedial axis in the caudate nucleus of monkeys performing an asymmetrically rewarded oculomotor task. In a given block, a target in one position was associated with a large reward, whereas the other target was associated with a small reward. The target position-reward value contingency was switched between blocks. We found the following: (1) activity that reflected the block-wise reward contingency emerged before the appearance of a visual target, and it was more prevalent in the dorsal, rather than central and ventral, caudate; (2) activity that was positively related to the reward size of the current trial was evident, especially after reward delivery, and it was more prevalent in the ventral and central, rather than dorsal, caudate; and (3) activity that was modulated by the memory of the outcomes of the previous trials was evident in the dorsal and central caudate. This multiple reward information, together with the target-direction information, was represented primarily by individual caudate neurons, and the different reward information was represented in caudate subpopulations with distinct electrophysiological properties, e.g., baseline firing and spike width. These results suggest parallel processing of different reward information by the basal ganglia subdivisions defined by extrinsic connections and intrinsic properties.

Advanced Parkinson's disease effect on goal-directed and habitual processes involved in visuomotor associative learning

The present behavioral study re-addresses the question of habit learning in Parkinson's disease (PD). Patients were early onset, non-demented, dopa-responsive, candidates for surgical treatment, similar to those we found earlier as suffering greater dopamine depletion in the putamen than in the caudate nucleus. The task was the same conditional associative learning task as that used previously in monkeys and healthy humans to unveil the striatum involvement in habit learning. Sixteen patients and 20 age- and education-matched healthy control subjects learned sets of 3 visuo-motor associations between complex patterns and joystick displacements during two testing sessions separated by a few hours. We distinguished errors preceding vs. following the first correct response to compare patients' performance during the earliest phase of learning dominated by goal-directed actions with that observed later on, when responses start to become habitual. The disease significantly retarded both learning phases, especially in patients under 60 years of age. However, only the late phase deficit was disease severity-dependent and persisted on the second testing session. These findings provide the first corroboration in Parkinson patients of two ideas well-established in the animal literature. The first is the idea that associating visual stimuli to motor acts is a form of habit learning that engages the striatum. It is confirmed here by the global impairment in visuo-motor learning induced by PD. The second idea is that goal-directed behaviors are predominantly caudate-dependent whereas habitual responses are primarily putamen-dependent. At the advanced PD stages tested here, dopamine depletion is greater in the putamen than in the caudate nucleus. Accordingly, the late phase of learning corresponding to the emergence of habitual responses was more vulnerable to the disease than the early phase dominated by goal-directed actions.