Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum

Striosomes Target Nigral Dopamine-Containing Neurons via Direct-D1 and Indirect-D2 Pathways Paralleling Classic Direct-Indirect Basal Ganglia Systems

Balanced activity of canonical direct D1 and indirect D2 basal ganglia pathways is considered a core requirement for normal movement, and their imbalance is an etiologic factor in movement and neuropsychiatric disorders. We present evidence for a conceptually equivalent pair of direct-D1 and indirect-D2 pathways that arise from striatal projection neurons (SPNs) of the striosome compartment rather than from SPNs of the matrix, as do the canonical pathways. These S-D1 and S-D2 striosomal pathways target substantia nigra dopamine-containing neurons instead of basal ganglia motor output nuclei. They modulate movement oppositely to the modulation by the canonical pathways: S-D1 is inhibitory and S-D2 is excitatory. The S-D1 and S-D2 circuits likely influence motivation for learning and action, complementing and reorienting canonical pathway modulation. A major conceptual reformulation of the classic direct-indirect pathway model of basal ganglia function is needed, as well as reconsideration of the effects of D2-targeting therapeutic drugs.

A role for the dorsolateral striatum in prospective action control

The dorsolateral striatum (DLS) is important for performing actions persistently, even when it becomes suboptimal, reflecting a function that is reflexive and habitual. However, there are also ways in which persistent behaviors can result from a more prospective, planning mode of behavior. To help tease apart these possibilities for DLS function, we trained animals to perform a lever press for reward and then inhibited the DLS in key test phases: as the task shifted from a 1-press to a 3-press rule (upshift), as the task was maintained, as the task shifted back to the one-press rule (downshift), and when rewards came independent of pressing. During DLS inhibition, animals always favored their initially learned strategy to press just once, particularly so during the free-reward period. DLS inhibition surprisingly changed performance speed bidirectionally depending on the task shifts. DLS inhibition thus encouraged habitual behavior, suggesting it could normally help adapt to changing conditions.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

Anticipatory attractors, functional neurochemistry and "Throw & Catch" mechanisms as illustrations of constructivism

This review explores several rarely discussed examples illustrating constructivism principles, generative and selective features of neuronal regulation of behaviour. First, the review highlights Walter Freeman's experiments and mathematical analysis that uncovered the existence of anticipatory attractors, i.e. non-random dynamical patterns in neurodynamics. Since Freeman's work did not extend to neurochemistry, this paper then points to the proposed earlier neurochemical framework summarizing the managerial roles of monoaminergic, cholinergic and opioid receptor systems likely contributing to anticipatory attractors in line with functional constructivism. As a third example, neurochemistry's evidence points to the "Throw & Catch" (T&C) principle in neurodynamics. This principle refers to the pro-active, neurochemically expensive, massive but topical increase of potentials ("Throw") within electrodynamics and neurotransmission in the brain whenever there is an uncertainty in selection of degrees of freedom (DFs). The T&C also underlines the relay-like processes during the selection of DFs. The "Throw" works as an internally generated "flashlight" that, contrarily to the expectations of entropy reduction, increases entropy and variance observed in processes related to orientation and action-formation. The discussed examples highlight the deficiency of structures-oriented projects and excitation-inhibition concepts in neuroscience. The neural regulation of behaviour appears to be a fluid, constructive process, constantly upgrading the choice of behavioural DFs, to ensure the compatibility between the environmental and individual's individuals' needs and capacities.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Revealing abrupt transitions from goal-directed to habitual behavior

A fundamental tenet of animal behavior is that decision-making involves multiple 'controllers.' Initially, behavior is goal-directed, driven by desired outcomes, shifting later to habitual control, where cues trigger actions independent of motivational state. Clark Hull's question from 1943 still resonates today: "Is this transition abrupt, or is it gradual and progressive?"1 Despite a century-long belief in gradual transitions, this question remains unanswered2,3 as current methods cannot disambiguate goal-directed versus habitual control in real-time. Here, we introduce a novel 'volitional engagement' approach, motivating animals by palatability rather than biological need. Offering less palatable water in the home cage4,5 reduced motivation to 'work' for plain water in an auditory discrimination task when compared to water-restricted animals. Using quantitative behavior and computational modeling6, we found that palatability-driven animals learned to discriminate as quickly as water-restricted animals but exhibited state-like fluctuations when responding to the reward-predicting cue-reflecting goal-directed behavior. These fluctuations spontaneously and abruptly ceased after thousands of trials, with animals now always responding to the reward-predicting cue. In line with habitual control, post-transition behavior displayed motor automaticity, decreased error sensitivity (assessed via pupillary responses), and insensitivity to outcome devaluation. Bilateral lesions of the habit-related dorsolateral striatum7 blocked transitions to habitual behavior. Thus, 'volitional engagement' reveals spontaneous and abrupt transitions from goal-directed to habitual behavior, suggesting the involvement of a higher-level process that arbitrates between the two.

Early life adversity impaired dorsal striatal synaptic transmission and behavioral adaptability to appropriate action selection in a sex-dependent manner

Early life adversity (ELA) is a major health burden in the United States, with 62% of adults reporting at least one adverse childhood experience. These experiences during critical stages of brain development can perturb the development of neural circuits that mediate sensory cue processing and behavioral regulation. Recent studies have reported that ELA impaired the maturation of dendritic spines on neurons in the dorsolateral striatum (DLS) but not in the dorsomedial striatum (DMS). The DMS and DLS are part of two distinct corticostriatal circuits that have been extensively implicated in behavioral flexibility by regulating and integrating action selection with the reward value of those actions. To date, no studies have investigated the multifaceted effects of ELA on aspects of behavioral flexibility that require alternating between different action selection strategies or higher-order cognitive processes, and the underlying synaptic transmission in corticostriatal circuitries. To address this, we employed whole-cell patch-clamp electrophysiology to assess the effects of ELA on synaptic transmission in the DMS and DLS. We also investigated the effects of ELA on the ability to update action control in response to outcome devaluation in an instrumental learning paradigm and reversal of action-outcome contingency in a water T-maze paradigm. At the circuit level, ELA decreased corticostriatal glutamate transmission in male but not in female mice. Interestingly, in DMS, glutamate transmission is decreased in male ELA mice, but increased in female ELA mice. ELA impaired the ability to update action control in response to reward devaluation in a context that promotes goal-directedness in male mice and induced deficits in reversal learning. Overall, our findings demonstrate the sex- and region-dependent effects of ELA on behavioral flexibility and underlying corticostriatal glutamate transmission. By establishing a link between ELA and circuit mechanisms underlying behavioral flexibility, our findings will begin to identify novel molecular mechanisms that can represent strategies for treating behavioral inflexibility in individuals who experienced early life traumatic incidents.

Rapid onset of functional tic-like behaviors among adolescent girls-Minnesota, September-November 2021

Background

On October 15, 2021, the Minnesota Department of Health began investigating a school cluster of students experiencing tic-like behaviors thought to be related to recent COVID-19. The objective of this report is to describe the investigation, key findings, and public health recommendations.

Methods

Affected students and proxies were interviewed with a standardized questionnaire including validated depression and anxiety screens.

Results

Eight students had tic-like behaviors lasting >24 h after initial report with onset during September 26-October 30, 2021. All eight students were females aged 15-17 years. All students either had a history of depression or anxiety or scored as having more than minimal anxiety or depression on validated screens. Four students previously had confirmed COVID-19: the interval between prior COVID-19 and tic symptom onset varied from more than a year prior to tic symptom onset to at the time of tic symptom onset.

Conclusion

The onset of tic-like behaviors at one school in Minnesota appeared to be related more to underlying mental health conditions than recent COVID-19. These findings highlight the need to better understand functional tic-like behaviors and adolescent mental health.

Task history dictates how the dorsolateral striatum controls action strategy and vigor

The dorsolateral striatum (DLS) is linked to the learning and honing of action routines. However, the DLS is also important for performing behaviors that have been successful in the past. The learning function can be thought of as prospective, helping to plan ongoing actions to be efficient and often optimal. The performance function is more retrospective, helping the animal continue to behave in a way that had worked previously. How the DLS manages this all is curious. What happens when a learned behavior becomes sub-optimal due to environment changes. In this case, the prospective function of the DLS would cause animals to (adaptively) learn and plan more optimal actions. In contrast, the retrospective function would cause animals to (maladaptively) favor the old behavior. Here we find that, during a change in learned task rules, DLS inhibition causes animals to adjust less rapidly to the new task (and to behave less vigorously) in a 'maladaptive' way. Yet, when the task is changed back to the initially learned rules, DLS inhibition instead causes a rapid and vigorous adjustment of behavior in an 'adaptive' way. These results show that inhibiting the DLS biases behavior towards initially acquired strategies, implying a more retrospective outlook in action selection when the DLS is offline. Thus, an active DLS could encourage planning and learning action routines more prospectively. Moreover, the DLS control over behavior can appear to be either advantageous/flexible or disadvantageous/inflexible depending on task context, and its control over vigor can change depending on task context.

Significant Statement

Basal ganglia networks aid behavioral learning (a prospective planning function) but also favor the use of old behaviors (a retrospective performance function), making it unclear what happens when learned behaviors become suboptimal. Here we inhibit the dorsolateral striatum (DLS) as animals encounter a change in task rules, and again when they shift back to those learned task rules. DLS inhibition reduces adjustment to new task rules (and reduces behavioral vigor), but it increases adjustment back to the initially learned task rules later (and increases vigor). Thus, in both cases, DLS inhibition favored the use of the initially learned behavioral strategy, which could appear either maladaptive or adaptive. We suggest that the DLS might promote a prospective orientation of action control.

Analytic Background in the Neuroscience of the Potential Project "Hippocrates"

This paper reviews the principles identified in analytic neuroscience that could be used in the setup of an international project, "Hippocrates" (H-project), named after the author of the endocrine theory of temperaments. The H-project can aim to summarize the findings in functional neurochemistry of consistent behavioural patterns (CBPs) in health (such as temperament traits) and psychopathology (symptoms of psychiatric disorders); to have systematically structured neurochemical investigations; to have an analysis of CBPs that include all ranges of behavioural histories and to have these modules complemented by regional contrasts related to climate, diets and other bio-environmental factors. The review highlights the benefits of constructivism and illustrates the contrast between constructivism and current approaches in terms of analytic and methodological aspects. (1) "Where" the neurochemical biomarkers should be measured: the review expands the range of needed measurements to out-of-brain systems, including environmental factors, and explores the concept of Specialized Extended Phenotype. (2) "What" should be measured but is missing: the review points to the need for measurement of the "Throw & Catch" neurochemical relays; behavioural and neuronal events contributing to the consistency of the CBPs but not documented in measurements. (3) Structuring the H-project's setup: the paper briefly describes a proposed earlier neurochemical framework, Functional Ensemble of Temperament that that accommodates the neurochemical continuum between temperament and symptoms of psychiatric disorders. This framework is in line with documented "Throw & Catch" neurochemical relays and can also be used to organize data about the personal and professional history of an individual.

Transition from predictable to variable motor cortex and striatal ensemble patterning during behavioral exploration

Animals can capitalize on invariance in the environment by learning and automating highly consistent actions; however, they must also remain flexible and adapt to environmental changes. It remains unclear how primary motor cortex (M1) can drive precise movements, yet also support behavioral exploration when faced with consistent errors. Using a reach-to-grasp task in rats, along with simultaneous electrophysiological monitoring in M1 and dorsolateral striatum (DLS), we find that behavioral exploration to overcome consistent task errors is closely associated with tandem increases in M1 and DLS neural variability; subsequently, consistent ensemble patterning returns with convergence to a new successful strategy. We also show that compared to reliably patterned intracranial microstimulation in M1, variable stimulation patterns result in significantly greater movement variability. Our results thus indicate that motor and striatal areas can flexibly transition between two modes, reliable neural pattern generation for automatic and precise movements versus variable neural patterning for behavioral exploration.

It is not fully understood how behavioral flexibility is established in the context of automatic performance of a complex motor skill. Here the authors show that corticostriatal activity can flexibly transition between two modes during a reach to-grasp task in rats: reliable neural pattern generation for precise, automatic movements versus variable neural patterning for behavioral exploration.

Opposing Roles of the Dorsolateral and Dorsomedial Striatum in the Acquisition of Skilled Action Sequencing in Rats

The shift in control from dorsomedial to dorsolateral striatum during skill and habit formation has been well established, but whether striatal subregions orchestrate this shift cooperatively or competitively remains unclear. Cortical inputs have also been implicated in the shift toward automaticity, but it is unknown whether they mirror their downstream striatal targets across this transition. We addressed these questions using a five step heterogeneous action sequencing task in male rats that is optimally performed by automated chains of actions. By optimizing automatic habitual responding, we discovered that loss of function in the dorsomedial striatum accelerated sequence acquisition. In contrast, loss of function in the dorsolateral striatum impeded acquisition of sequencing, demonstrating functional opposition within the striatum. Unexpectedly, the mPFC was not involved; however, the lateral orbitofrontal cortex was critical. These results shift current theories about striatal control of behavior to a model of competitive opposition, where the dorsomedial striatum interferes with the development of dorsolateral-striatum dependent behavior.SIGNIFICANCE STATEMENT We provide the most direct evidence to date that the dorsomedial and dorsolateral striatum compete for control in the acquisition of habitual action sequences. The dorsolateral striatum was critical for sequencing behavior, but loss of dorsomedial striatum function enhanced acquisition. In addition, we found that the mPFC was not required for the formation of automated actions. Using a task that optimizes habitual responding, we demonstrate that the arbitration of dorsomedial and dorsolateral control is not modulated by medial prefrontal cortical activity. However, we find evidence for the role of the lateral orbitofrontal cortex in action sequencing. These results have implications for our understanding of how habits and skills form.

Regulation of Voluntary Physical Activity Behavior: A Review of Evidence Involving Dopaminergic Pathways in the Brain

Physical activity leads to well-established health benefits. Current efforts to enhance physical activity have targeted mainly socioeconomic factors. However, despite these efforts, only a small number of adults engage in regular physical activity to the point of meeting current recommendations. Evidence collected in rodent models and humans establish a strong central nervous system component that regulates physical activity behavior. In particular, dopaminergic pathways in the central nervous system are among the best-characterized biological mechanisms to date with respect to regulating reward, motivation, and habit formation, which are critical for establishing regular physical activity. Herein, we discuss evidence for a role of brain dopamine in the regulation of voluntary physical activity behavior based on selective breeding and pharmacological studies in rodents, as well as genetic studies in both rodents and humans. While these studies establish a role of dopamine and associated mechanisms in the brain in the regulation of voluntary physical activity behavior, there is clearly need for more research on the underlying biology involved in motivation for physical activity and the formation of a physical activity habit. Such knowledge at the basic science level may ultimately be translated into better strategies to enhance physical activity levels within the society.

Striatal Astrocytes Shape Behavioral Flexibility via Regulation of the Glutamate Transporter EAAT2

BACKGROUND

Striatal circuits must be modulated for behavioral flexibility, the ability to adapt to environmental changes. Striatal astrocytes contribute to circuit neuromodulation by controlling the activity of ambient neurotransmitters. In particular, extracellular glutamate levels are tightly controlled by the astrocytic glutamate transporter EAAT2, influencing synaptic functioning and neural network activity. However, it remains unclear if EAAT2 responds to environmental cues to specifically shape action control.

METHODS

To investigate the relationship between behavioral flexibility and experience-dependent regulation of EAAT2 expression in the dorsal striatum, mice were trained on an instrumental task. We manipulated EAAT2 expression using chemogenetic activation of astrocytic Gq signaling or in vivo morpholinos and determined the ability to adapt to novel environmental contingencies.

RESULTS

The loss of behavioral flexibility with task overtraining is associated with the upregulation of EAAT2, which results in enhanced glutamate clearance and altered modulation of glutamatergic neurotransmission in the lateral part of the dorsal striatum. Interfering with EAAT2 upregulation in this striatal area preserves behavioral flexibility.

CONCLUSIONS

Astrocytes are emerging as critical regulators of striatal functions. This work demonstrates that plasticity of EAAT2 expression in the lateral part of the dorsal striatum shapes behavior, thus providing novel mechanistic insights into how flexibility in action control is regulated.

Potential roles of the rodent medial prefrontal cortex in conflict resolution between multiple decision-making systems

Mammalian decision-making is mediated by the interaction of multiple, neurally and computationally separable decision systems. Having multiple systems requires a mechanism to manage conflict and converge onto the selection of singular actions. A long history of evidence has pointed to the prefrontal cortex as a central component in processing the interactions between distinct decision systems and resolving conflicts among them. In this chapter we review four theories of how that interaction might occur and identify how the medial prefrontal cortex in the rodent may be involved in each theory. We then present experimental predictions implied by the neurobiological data in the context of each theory as a starting point for future investigation of medial prefrontal cortex and decision-making.

The effect of instruction on motor skill learning

Many motor skills are learned with the help of instructions. In the context of complex motor sequences, instructions often break down the movement into chunks that can then be practiced in isolation. Thus, instructions shape an initial cognitive representation of the skill, which in turn guides practice. Are there ways of breaking up a motor sequence that are better than others? If participants are instructed in a way that hinders performance, how much practice does it take to overcome the influence of the instruction? To answer these questions, we used a paradigm in which participants were asked to perform finger sequences as fast and accurately as possible on a keyboard-like device. In the initial phases of training, participants had to explicitly remember and practice two- or three-digit chunks. These chunks were then combined to form seven 11-digit sequences that participants practiced for the remainder of the study. Each sequence was broken up into chunks in a way such that the instruction was either aligned or misaligned with the basic execution-level constraints. We found that misaligned chunk instruction led to an initial performance deficit compared with the aligned chunk instruction. Overall, instructions still influenced the temporal pattern of performance after 10 days of subsequent training, with shorter interpress intervals within a chunk compared with between chunks. However, for the misaligned instructed sequences, this temporal pattern was altered more rapidly, such that participants could overcome the induced performance deficit in the last week. At the end of training, participants found idiosyncratic, but interindividually stable, ways of performing each sequence.NEW & NOTEWORTHY Instructions often break down motor sequences into smaller parts, such that they can be more easily remembered. Here, we show that different ways of breaking down a finger sequence can subsequently lead to better or worse performance. The initial instruction still influenced the temporal performance pattern after 10 days of practice. The results demonstrate that the initial cognitive representation of a motor skill strongly influences how a skill is learned and performed.

Rationalization of emotion is also rational

Cushman seeks to explain rationalization in terms of fundamental mental processes, and he hypotheses a selected-for function: information exchange between "rational" and "non-rational" processes in the brain. While this is plausible, his account overlooks the importance - and information value - of rationalizing the emotions of ourselves and others. Incorporating such rationalization would help explain the effectiveness of rationalization and its connection with valuation, as well as raise a challenge to his way of bifurcating "rational" and "non-rational" processes.

Interfacing behavioral and neural circuit models for habit formation

Habits are an important mechanism by which organisms can automate the control of behavior to alleviate cognitive demand. However, transitions to habitual control are risky because they lead to inflexible responding in the face of change. The question of how the brain controls transitions into habit is thus an intriguing one. How do we regulate when our repeated actions become automated? When is it advantageous or disadvantageous to release actions from cognitive control? Decades of research have identified a variety of methods for eliciting habitual responding in animal models. Progress has also been made to understand which brain areas and neural circuits control transitions into habit. Here, I discuss existing research on behavioral and neural circuit models for habit formation (with an emphasis on striatal circuits), and discuss strategies for combining information from different paradigms and levels of analysis to prompt further progress in the field.

Cognitive Neuroscience of Eating Disorders

Anorexia nervosa and bulimia nervosa are characterized by severely restricted intake, binge eating, and compensatory behaviors like self-induced vomiting. The neurobiological underpinnings of these maladaptive behaviors are poorly understood, but the application of cognitive neuroscience and neuroimaging to eating disorders has begun to elucidate their pathophysiology. Specifically, this review focuses on 3 areas that suggest paths forward: reward, cognitive and behavioral control, and decision making. Understanding the brain-based mechanisms that promote and maintain these often chronic symptoms could guide the development of new and more effective treatments.

Unique contributions of parvalbumin and cholinergic interneurons in organizing striatal networks during movement

Striatal parvalbumin (PV) and cholinergic interneurons (CHIs) are poised to play major roles in behavior by coordinating the networks of medium spiny cells that relay motor output. However, the small numbers and scattered distribution of these cells have hindered direct assessment of their contribution to activity in networks of medium spiny neurons (MSNs) during behavior. Here, we build on recent improvements in single-cell calcium imaging combined with optogenetics to test the capacity of PVs and CHIs to affect MSN activity and behavior in mice engaged in voluntary locomotion. We find that PVs and CHIs have unique effects on MSN activity and dissociable roles in supporting movement. PV cells facilitate movement by refining the activation of MSN networks responsible for movement execution. CHIs, in contrast, synchronize activity within MSN networks to signal the end of a movement bout. These results provide new insights into the striatal network activity that supports movement.

Thinking Outside the Box (and Arrow): Current Themes in Striatal Dysfunction in Movement Disorders

The basal ganglia are an intricately connected assembly of subcortical nuclei, forming the core of an adaptive network connecting cortical and thalamic circuits. For nearly three decades, researchers and medical practitioners have conceptualized how the basal ganglia circuit works, and how its pathology underlies motor disorders such as Parkinson's and Huntington's diseases, using what is often referred to as the "box-and-arrow model": a circuit diagram showing the broad strokes of basal ganglia connectivity and the pathological increases and decreases in the weights of specific connections that occur in disease. While this model still has great utility and has led to groundbreaking strategies to treat motor disorders, our evolving knowledge of basal ganglia function has made it clear that this classic model has several shortcomings that severely limit its predictive and descriptive abilities. In this review, we will focus on the striatum, the main input nucleus of the basal ganglia. We describe recent advances in our understanding of the rich microcircuitry and plastic capabilities of the striatum, factors not captured by the original box-and-arrow model, and provide examples of how such advances inform our current understanding of the circuit pathologies underlying motor disorders.

Molecular mechanisms underlying striatal synaptic plasticity: relevance to chronic alcohol consumption and seeking

The striatum, the input structure of the basal ganglia, is a major site of learning and memory for goal-directed actions and habit formation. Spiny projection neurons of the striatum integrate cortical, thalamic, and nigral inputs to learn associations, with cortico-striatal synaptic plasticity as a learning mechanism. Signaling molecules implicated in synaptic plasticity are altered in alcohol withdrawal, which may contribute to overly strong learning and increased alcohol seeking and consumption. To understand how interactions among signaling molecules produce synaptic plasticity, we implemented a mechanistic model of signaling pathways activated by dopamine D1 receptors, acetylcholine receptors, and glutamate. We use our novel, computationally efficient simulator, NeuroRD, to simulate stochastic interactions both within and between dendritic spines. Dopamine release during theta burst and 20-Hz stimulation was extrapolated from fast-scan cyclic voltammetry data collected in mouse striatal slices. Our results show that the combined activity of several key plasticity molecules correctly predicts the occurrence of either LTP, LTD, or no plasticity for numerous experimental protocols. To investigate spatial interactions, we stimulate two spines, either adjacent or separated on a 20-μm dendritic segment. Our results show that molecules underlying LTP exhibit spatial specificity, whereas 2-arachidonoylglycerol exhibits a spatially diffuse elevation. We also implement changes in NMDA receptors, adenylyl cyclase, and G protein signaling that have been measured following chronic alcohol treatment. Simulations under these conditions suggest that the molecular changes can predict changes in synaptic plasticity, thereby accounting for some aspects of alcohol use disorder.

A Basal Ganglia Circuit Sufficient to Guide Birdsong Learning

Learning vocal behaviors, like speech and birdsong, is thought to rely on continued performance evaluation. Whether candidate performance evaluation circuits in the brain are sufficient to guide vocal learning is not known. Here, we test the sufficiency of VTA projections to the vocal basal ganglia in singing zebra finches, a songbird species that learns to produce a complex and stereotyped multi-syllabic courtship song during development. We optogenetically manipulate VTA axon terminals in singing birds contingent on how the pitch of an individual song syllable is naturally performed. We find that optical inhibition and excitation of VTA terminals are each sufficient to reliably guide learned changes in song. Inhibition and excitation have opponent effects on future performances of targeted song syllables, consistent with positive and negative reinforcement of performance outcomes. These findings define a central role for reinforcement mechanisms in learning vocalizations and demonstrate minimal circuit elements for learning vocal behaviors. VIDEO ABSTRACT.

Habitual nicotine-seeking in rats following limited training

RATIONALE AND OBJECTIVES

A potential reason that cigarette smoking can persist despite multiple quit attempts is that repeated voluntary nicotine intake may facilitate a transition from goal-directed to habitual behavioral control. Although accelerated habit formation for self-administered ethanol or cocaine has been previously demonstrated, this phenomenon has not been extensively studied with nicotine. We therefore examined the liability of nicotine self-administration to become habitual, while also examining that of orally consumed saccharin as an experimental control.

METHODS

Under fixed ratio 1 (FR-1) schedules, male Sprague-Dawley rats (n = 8-11/group) lever-pressed for intravenous (IV) nicotine (30 μg/kg/infusion) for 10 consecutive days, while also lever-pressing for saccharin solution (0.1% w/v, 0.19 mL/delivery) in separate operant sessions. In experiment 1, either nicotine or saccharin was devalued by pairing with the aversive agent lithium chloride (LiCl; 0.15 M, 14.1 mL/kg) prior to extinction and reacquisition testing. In experiment 2, the contingency between lever pressing and delivery of either nicotine or saccharin was degraded in six sessions, followed by extinction testing.

RESULTS

LiCl pairings selectively reduced responding for nicotine (-35% from control) and saccharin (-48%) in reacquisition testing, indicating that both rewards were effectively devalued. During extinction testing, saccharin-seeking responses were reduced by both manipulations (devaluation -30%, degradation -79%), suggesting that responding for saccharin was goal-directed. In contrast, nicotine-seeking responses were not significantly affected by either manipulation (devaluation -4%, degradation -21%), suggesting that responding for nicotine was habitually driven.

CONCLUSIONS

Operant responding for IV nicotine may rapidly come under habitual control, potentially contributing to the tenacity of tobacco use.

Gambling disorder: an integrative review of animal and human studies

Gambling disorder (GD), previously called pathological gambling and classified as an impulse control disorder in DSM-III and DSM-IV, has recently been reclassified as an addictive disorder in the DSM-5. It is widely recognized as an important public health problem associated with substantial personal and social costs, high rates of psychiatric comorbidity, poor physical health, and elevated suicide rates. A number of risk factors have been identified, including some genetic polymorphisms. Animal models have been developed in order to study the underlying neural basis of GD. Here, we discuss recent advances in our understanding of the risk factors, disease course, and pathophysiology. A focus on a phenotype-based dissection of the disorder is included in which known neural correlates from animal and human studies are reviewed. Finally, current treatment approaches are discussed, as well as future directions for GD research.

Learning new sequential stepping patterns requires striatal plasticity during the earliest phase of acquisition

Animals including humans execute motor behavior to reach their goals. For this purpose, they must choose correct strategies according to environmental conditions and shape many parameters of their movements, including their serial order and timing. To investigate the neurobiology underlying such skills, we used a multi-sensor equipped, motor-driven running wheel with adjustable sequences of foothold pegs on which mice ran to obtain water reward. When the peg patterns changed from a familiar pattern to a new pattern, the mice had to learn and implement new locomotor strategies in order to receive reward. We found that the accuracy of stepping and the achievement of water reward improved with the new learning after changes in the peg-pattern, and c-Fos expression levels assayed after the first post-switch session were high in both dorsolateral striatum and motor cortex, relative to post-switch plateau levels. Combined in situ hybridization and immunohistochemistry of striatal sections demonstrated that both enkephalin-positive (indirect pathway) neurons and substance P-positive (direct pathway) neurons were recruited specifically after the pattern switches, as were interneurons expressing neuronal nitric oxide synthase. When we blocked N-methyl-D-aspartate (NMDA) receptors in the dorsolateral striatum by injecting the NMDA receptor antagonist, D-2-amino-5-phosphonopentanoic acid (AP5), we found delays in early post-switch improvement in performance. These findings suggest that the dorsolateral striatum is activated on detecting shifts in environment to adapt motor behavior to the new context via NMDA-dependent plasticity, and that this plasticity may underlie forming and breaking skills and habits as well as to behavioral difficulties in clinical disorders.

Habit formation

Habits, both good ones and bad ones, are pervasive in animal behavior. Important frameworks have been developed to understand habits through psychological and neurobiological studies. This work has given us a rich understanding of brain networks that promote habits, and has also helped us to understand what constitutes a habitual behavior as opposed to a behavior that is more flexible and prospective. Mounting evidence from studies using neural recording methods suggests that habit formation is not a simple process. We review this evidence and take the position that habits could be sculpted from multiple dissociable changes in neural activity. These changes occur across multiple brain regions and even within single brain regions. This strategy of classifying components of a habit based on different brain signals provides a potentially useful new way to conceive of disorders that involve overly fixed behaviors as arising from different potential dysfunctions within the brain's habit network.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Unmasking the Effects of L-DOPA on Rapid Dopamine Signaling with an Improved Approach for Nafion Coating Carbon-Fiber Microelectrodes

L-DOPA has been the gold standard for symptomatic treatment of Parkinson's disease. However, its efficacy wanes over time as motor complications develop. Very little is known about how L-DOPA therapy affects the dynamics of fluctuating dopamine concentrations in the striatum on a rapid time scale (seconds). Electrochemical studies investigating the effects of L-DOPA treatment on electrically evoked dopamine release have reported conflicting results with significant variability. We hypothesize that the uncertainty in the electrochemical data is largely due to electrode fouling caused by polymerization of L-DOPA and endogenous catecholamines on the electrode surface. Thus, we have systematically optimized the procedure for fabricating cylindrical, Nafion-coated, carbon-fiber microelectrodes. This has enabled rapid and reliable detection of L-DOPA's effects on striatal dopamine signaling in intact rat brain using fast-scan cyclic voltammetry. An acute dose of 5 mg/kg L-DOPA had no significant effect on dopamine dynamics, demonstrating the highly efficient regulatory mechanisms at work in the intact brain. In contrast, administration of 200 mg/kg L-DOPA significantly increased the amplitude of evoked dopamine release by ∼200%. Overall, this work describes a reliable tool that allows a better measure of L-DOPA augmented dopamine release in vivo, measured using fast-scan cyclic voltammetry. It provides a methodology that improves the stability and performance of the carbon-fiber microelectrode when studying the molecular mechanisms underlying L-DOPA therapy and also promises to benefit a wide variety of studies because Nafion is so commonly used in electroanalytical chemistry.

Dorsolateral neostriatum contribution to incentive salience: opioid or dopamine stimulation makes one reward cue more motivationally attractive than another

Pavlovian cues for rewards can become attractive incentives: approached and 'wanted' as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here it was reported that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individualized fashion, at the expense of a competing cue. In an autoshaping (sign-tracking vs. goal-tracking) paradigm, microinjection of the mu opioid receptor agonist (DAMGO) or dopamine indirect agonist (amphetamine) in the DLS of sign-tracker individuals selectively enhanced their sign-tracking attraction toward the reward-predictive lever cue. By contrast, DAMGO or amphetamine in the DLS of goal-trackers selectively enhanced prepotent attraction toward the reward-proximal cue of sucrose dish. Amphetamine also enhanced goal-tracking in some sign-tracker individuals (if they ever defected to the dish even once). That DLS enhancement of cue attraction was due to stronger motivation, not stronger habits, was suggested by: (i) sign-trackers flexibly followed their cue to a new location when the lever was suddenly moved after DLS DAMGO microinjection; and (ii) DAMGO in the DLS also made sign-trackers work harder on a new instrumental nose-poke response required to earn presentations of their Pavlovian lever cue (instrumental conditioned reinforcement). Altogether, the current results suggest that DLS circuitry can enhance the incentive salience of a Pavlovian reward cue, selectively making that cue a stronger motivational magnet.